【1】MusicAOG: an Energy-Based Model for Learning and Sampling a Hierarchical Representation of Symbolic Music标题：MusicAOG：一种基于能量的符号音乐分层表示学习和采样模型链接：https://arxiv.org/abs/2401.02678作者：Yikai Qian,Tianle Wang,Xinyi Tong,Xin Jin,Duo Xu,Bo Zheng,Tiezheng Ge,Feng Yu,Song-Chun Zhu摘要：为了解决人工音乐智能的可解释性和可推广性的挑战，本文介绍了一种新的符号表示，它融合了不同传统和粒度的显式和隐式音乐信息。该模型采用分层和或图表示，采用节点和边来封装广泛的音乐元素，包括结构，纹理，节奏和和声。这种层次化的方法扩展了音乐在不同尺度上的表现力。这种表示作为基于能量的模型的基础，通过依赖于最小最大熵原理的灵活算法框架，为学习音乐概念而专门定制。利用一个适应大都会黑斯廷斯采样技术，该模型能够细粒度控制音乐的生成。一个全面的实证评估，对比这种新的方法与现有的方法，表现出相当大的进步，在可解释性和可控性。这项研究标志着音乐分析，作曲和计算音乐学领域的重大贡献。摘要：In addressing the challenge of interpretability and generalizability of artificial music intelligence, this paper introduces a novel symbolic representation that amalgamates both explicit and implicit musical information across diverse traditions and granularities. Utilizing a hierarchical and-or graph representation, the model employs nodes and edges to encapsulate a broad spectrum of musical elements, including structures, textures, rhythms, and harmonies. This hierarchical approach expands the representability across various scales of music. This representation serves as the foundation for an energy-based model, uniquely tailored to learn musical concepts through a flexible algorithm framework relying on the minimax entropy principle. Utilizing an adapted Metropolis-Hastings sampling technique, the model enables fine-grained control over music generation. A comprehensive empirical evaluation, contrasting this novel approach with existing methodologies, manifests considerable advancements in interpretability and controllability. This study marks a substantial contribution to the fields of music analysis, composition, and computational musicology.【2】 Gradient weighting for speaker verification in extremely low Signal-to-Noise Ratio标题：极低信噪比下说话人确认的梯度加权方法链接：https://arxiv.org/abs/2401.02626作者：Yi Ma,Kong Aik Lee,Ville Hautamäki,Meng Ge,Haizhou Li备注：Accepted by ICASSP 2024摘要：说话人确认受到背景噪声的影响，特别是在信噪比（SNR）低于0 dB的情况下。很难在不引入有害伪影的情况下抑制噪声，这对说话人确认产生不利影响。我们提出了一种称为梯度加权（Grad-W）的机制，它在预测过程中动态识别和减少伪影噪声。该机制基于梯度指示模型正在关注输入的哪些部分的属性。具体来说，当扬声器网络专注于去噪话语中的区域而不是干净的对应物时，我们认为它是伪噪声，并在优化增强过程中为该区域分配更高的权重。我们通过训练增强模型和对增强后的语音进行说话人确认来验证它。实验结果表明，我们的方法有效地减少了伪噪声，提高了说话人确认在不同的SNR水平。摘要：Speaker verification is hampered by background noise, particularly at extremely low Signal-to-Noise Ratio (SNR) under 0 dB. It is difficult to suppress noise without introducing unwanted artifacts, which adversely affects speaker verification. We proposed the mechanism called Gradient Weighting (Grad-W), which dynamically identifies and reduces artifact noise during prediction. The mechanism is based on the property that the gradient indicates which parts of the input the model is paying attention to. Specifically, when the speaker network focuses on a region in the denoised utterance but not on the clean counterpart, we consider it artifact noise and assign higher weights for this region during optimization of enhancement. We validate it by training an enhancement model and testing the enhanced utterance on speaker verification. The experimental results show that our approach effectively reduces artifact noise, improving speaker verification across various SNR levels.【3】 Towards Weakly Supervised Text-to-Audio Grounding标题：走向弱监督的文本到音频的基础链接：https://arxiv.org/abs/2401.02584作者：Xuenan Xu,Ziyang Ma,Mengyue Wu,Kai Yu摘要：文本-音频接地（TAG）任务旨在预测自然语言描述的声音事件的起始和终止。这个任务可以促进应用程序，如多模态信息检索。本文重点研究弱监督文本到音频接地（WSTAG），其中声音事件的帧级注释不可用，只能利用整个音频片段的标题进行训练。WSTAG在对大型音频文本数据集的可扩展性方面优于强监督方法。本文研究了两种WSTAG框架：句子级和短语级。首先，我们分析了在以前的WSTAG方法中使用的均值池的局限性，并研究了不同的池化策略的效果。然后，我们提出短语级WSTAG使用音频片段和短语之间的匹配标签进行训练。提出了先进的负采样策略和自我监督，以提高弱标签的准确性，并提供伪强标签。实验结果表明，我们的系统显着优于以前的WSTAG SOTA。最后，我们进行了大量的实验来分析几个因素对短语级WSTAG的影响。代码和模型可在https://github.com/wsntxxn/TextToAudioGrounding上获得。摘要：Text-to-audio grounding (TAG) task aims to predict the onsets and offsets of sound events described by natural language. This task can facilitate applications such as multimodal information retrieval. This paper focuses on weakly-supervised text-to-audio grounding (WSTAG), where frame-level annotations of sound events are unavailable, and only the caption of a whole audio clip can be utilized for training. WSTAG is superior to strongly-supervised approaches in its scalability to large audio-text datasets. Two WSTAG frameworks are studied in this paper: sentence-level and phrase-level. First, we analyze the limitations of mean pooling used in the previous WSTAG approach and investigate the effects of different pooling strategies. We then propose phrase-level WSTAG to use matching labels between audio clips and phrases for training. Advanced negative sampling strategies and self-supervision are proposed to enhance the accuracy of the weak labels and provide pseudo strong labels. Experimental results show that our system significantly outperforms the previous WSTAG SOTA. Finally, we conduct extensive experiments to analyze the effects of several factors on phrase-level WSTAG. The code and model is available at https://github.com/wsntxxn/TextToAudioGrounding.【4】 Siamese Residual Neural Network for Musical Shape Evaluation in Piano Performance Assessment标题：连体残差神经网络在钢琴演奏评价中的音乐形态评价链接：https://arxiv.org/abs/2401.02566作者：Xiaoquan Li,Stephan Weiss,Yijun Yan,Yinhe Li,Jinchang Ren,John Soraghan,Ming Gong备注：X.Li, S.Weiss, Y.Yan, Y.Li, J.Ren, J.Soraghan, M.Gong,"Siamese residual neural network for musical shape evaluation in piano performance assessment" in Proc. of the 31st European Signal Processing Conference, Helsinki, Finland摘要：音乐形态的理解和识别在音乐教育和演奏评价中起着重要的作用。为了简化时间和成本密集型的音乐形状评估，在本文中，我们探讨了如何应用人工智能（AI）驱动的模型。将音乐形态评价看作一个分类问题，提出了一种轻量级的连体残差神经网络（S-ResNN）来自动识别音乐形态。为了评估所提出的方法在钢琴音乐形态评价的背景下，我们已经产生了一个新的数据集，包含4116音乐作品来自147钢琴预备练习，并在28类音乐形态。实验结果表明，S-ResNN在查准率、查全率和F1得分方面明显优于一些基准方法。摘要：Understanding and identifying musical shape plays an important role in music education and performance assessment. To simplify the otherwise time- and cost-intensive musical shape evaluation, in this paper we explore how artificial intelligence (AI) driven models can be applied. Considering musical shape evaluation as a classification problem, a light-weight Siamese residual neural network (S-ResNN) is proposed to automatically identify musical shapes. To assess the proposed approach in the context of piano musical shape evaluation, we have generated a new dataset, containing 4116 music pieces derived by 147 piano preparatory exercises and performed in 28 categories of musical shapes. The experimental results show that the S-ResNN significantly outperforms a number of benchmark methods in terms of the precision, recall and F1 score.【5】 A unified multichannel far-field speech recognition system: combining neural beamforming with attention based end-to-end model标题：一种统一的多通道远场语音识别系统：神经波束形成和基于注意力的端到端模型相结合链接：https://arxiv.org/abs/2401.02673作者：Dongdi Zhao,Jianbo Ma,Lu Lu,Jinke Li,Xuan Ji,Lei Zhu,Fuming Fang,Ming Liu,Feijun Jiang摘要：远场语音识别是一项具有挑战性的任务，传统上使用信号处理波束形成来攻击噪声和干扰问题。但由于严重依赖环境假设，其性能通常受到限制。在本文中，我们提出了一个统一的多通道远场语音识别系统，结合了神经波束形成和基于变换的听，拼写，参加（LAS）语音识别系统，它扩展了端到端的语音识别系统进一步包括语音增强。然后，这种框架被联合训练以优化感兴趣的最终目标。具体而言，因子化复线性投影（fCLP）已被采用来形成神经波束形成。然后比较了几种组合观看方向的池化策略，以找到最佳方法。此外，源方向的信息也被集成在波束形成中，以探索源方向作为先验的有用性，这通常是可用的，特别是在多模态场景中。在不同的麦克风阵列几何形状进行实验，以评估对麦克风阵列的间距变化的鲁棒性。大型内部数据库被用来评估所提出的框架的有效性和所提出的方法实现了19.26%的改善时，与一个强大的基线。摘要：Far-field speech recognition is a challenging task that conventionally uses signal processing beamforming to attack noise and interference problem. But the performance has been found usually limited due to heavy reliance on environmental assumption. In this paper, we propose a unified multichannel far-field speech recognition system that combines the neural beamforming and transformer-based Listen, Spell, Attend (LAS) speech recognition system, which extends the end-to-end speech recognition system further to include speech enhancement. Such framework is then jointly trained to optimize the final objective of interest. Specifically, factored complex linear projection (fCLP) has been adopted to form the neural beamforming. Several pooling strategies to combine look directions are then compared in order to find the optimal approach. Moreover, information of the source direction is also integrated in the beamforming to explore the usefulness of source direction as a prior, which is usually available especially in multi-modality scenario. Experiments on different microphone array geometry are conducted to evaluate the robustness against spacing variance of microphone array. Large in-house databases are used to evaluate the effectiveness of the proposed framework and the proposed method achieve 19.26\% improvement when compared with a strong baseline.【6】 Some clues to build a sound analysis relevant to hearing标题：建立与听力相关的声音分析的一些线索链接：https://arxiv.org/abs/2401.02463作者：Laurent Millot备注：None摘要：音乐家或音响工程师在研究实验室中用于声音合成的分析工具可能相当不同。通过对这些工具的假设和局限性的讨论，我们可以提出一个第一工具，它尽可能与所有声音演员相关和通用，其主要目标是：人们必须能够听到分析的每一个元素，因为听觉是最终的参考工具。这个工具也应该使用，在未来，重新调查的声音（或声学）的定义的基础上，最近的一些作品乐器建模，语音生产和扬声器设计。将给出音频插图。论文6041在音频工程学会第116届会议上发表，柏林，2004年摘要：Analysis tools used in research laboratories, for sound synthesis, by musicians or sound engineers can be rather different. Discussion of the assumptions and of the limitations of these tools permits to propose a first tool as relevant and versatile as possible for all the sound actors with a major aim: one must be able to listen to each element of the analysis because hearing is the final reference tool. This tool should also be used, in the future, to reinvestigate the definition of sound (or Acoustics) on the basis of some recent works on musical instrument modeling, speech production and loudspeakers design. Audio illustrations will be given.Paper 6041 presented at the 116th Convention of the Audio Engineering Society, Berlin, 2004eess.AS音频处理【1】 Pheme: Efficient and Conversational Speech Generation标题：Pheme：高效的会话语音生成链接：https://arxiv.org/abs/2401.02839作者：Paweł Budzianowski,Taras Sereda,Tomasz Cichy,Ivan Vulić摘要：近年来，语音生成已经取得了显着的进步，现在实现了一次性生成能力，通常与真实的人类声音几乎无法区分。将语音生成方面的这些进步与大型语言模型相结合，可能会彻底改变广泛的应用。然而，某些应用，如辅助会话系统，需要自然和会话语音生成工具，也有效地实时操作。目前最先进的模型，如VALL-E和SoundStorm，由分层神经音频编解码器提供支持，需要大型神经组件和广泛的训练数据才能良好工作。相比之下，MQTTS旨在构建更紧凑的会话TTS模型，同时利用较小规模的真实会话语音数据。然而，其自回归性质产生高推理延迟，从而限制了其实时使用。为了缓解当前最先进的TTS模型的局限性，同时利用它们的优势，在这项工作中，我们引入了Pheme模型系列，1）提供紧凑但高性能的模型，2）允许并行语音生成3）自然会话语音，4）它可以在较小规模的会话数据上有效地训练，将数据需求削减10倍以上，但仍与自回归TTS模型的质量相匹配。我们还表明，通过简单的教师-学生蒸馏，我们可以在预先训练的Pheme检查点之上实现单扬声器设置的语音质量的显着改善，仅依赖于由更大的教师模型生成的合成语音。音频样本和预训练模型可在线获取。摘要：In recent years, speech generation has seen remarkable progress, now achieving one-shot generation capability that is often virtually indistinguishable from real human voice. Integrating such advancements in speech generation with large language models might revolutionize a wide range of applications. However, certain applications, such as assistive conversational systems, require natural and conversational speech generation tools that also operate efficiently in real time. Current state-of-the-art models like VALL-E and SoundStorm, powered by hierarchical neural audio codecs, require large neural components and extensive training data to work well. In contrast, MQTTS aims to build more compact conversational TTS models while capitalizing on smaller-scale real-life conversational speech data. However, its autoregressive nature yields high inference latency and thus limits its real-time usage. In order to mitigate the current limitations of the state-of-the-art TTS models while capitalizing on their strengths, in this work we introduce the Pheme model series that 1) offers compact yet high-performing models, 2) allows for parallel speech generation of 3) natural conversational speech, and 4) it can be trained efficiently on smaller-scale conversational data, cutting data demands by more than 10x but still matching the quality of the autoregressive TTS models. We also show that through simple teacher-student distillation we can meet significant improvements in voice quality for single-speaker setups on top of pretrained Pheme checkpoints, relying solely on synthetic speech generated by much larger teacher models. Audio samples and pretrained models are available online.【2】 A unified multichannel far-field speech recognition system: combining neural beamforming with attention based end-to-end model标题：一种统一的多通道远场语音识别系统：神经波束形成和基于注意力的端到端模型相结合链接：https://arxiv.org/abs/2401.02673作者：Dongdi Zhao,Jianbo Ma,Lu Lu,Jinke Li,Xuan Ji,Lei Zhu,Fuming Fang,Ming Liu,Feijun Jiang摘要：远场语音识别是一项具有挑战性的任务，传统上使用信号处理波束形成来攻击噪声和干扰问题。但由于严重依赖环境假设，其性能通常受到限制。在本文中，我们提出了一个统一的多通道远场语音识别系统，结合了神经波束形成和基于变换的听，拼写，参加（LAS）语音识别系统，它扩展了端到端的语音识别系统进一步包括语音增强。然后，这种框架被联合训练以优化感兴趣的最终目标。具体而言，因子化复线性投影（fCLP）已被采用来形成神经波束形成。然后比较了几种组合观看方向的池化策略，以找到最佳方法。此外，源方向的信息也被集成在波束形成中，以探索源方向作为先验的有用性，这通常是可用的，特别是在多模态场景中。在不同的麦克风阵列几何形状进行实验，以评估对麦克风阵列的间距变化的鲁棒性。大型内部数据库被用来评估所提出的框架的有效性和所提出的方法实现了19.26%的改善时，与一个强大的基线。摘要：Far-field speech recognition is a challenging task that conventionally uses signal processing beamforming to attack noise and interference problem. But the performance has been found usually limited due to heavy reliance on environmental assumption. In this paper, we propose a unified multichannel far-field speech recognition system that combines the neural beamforming and transformer-based Listen, Spell, Attend (LAS) speech recognition system, which extends the end-to-end speech recognition system further to include speech enhancement. Such framework is then jointly trained to optimize the final objective of interest. Specifically, factored complex linear projection (fCLP) has been adopted to form the neural beamforming. Several pooling strategies to combine look directions are then compared in order to find the optimal approach. Moreover, information of the source direction is also integrated in the beamforming to explore the usefulness of source direction as a prior, which is usually available especially in multi-modality scenario. Experiments on different microphone array geometry are conducted to evaluate the robustness against spacing variance of microphone array. Large in-house databases are used to evaluate the effectiveness of the proposed framework and the proposed method achieve 19.26\% improvement when compared with a strong baseline.【3】 Some clues to build a sound analysis relevant to hearing标题：建立与听力相关的声音分析的一些线索链接：https://arxiv.org/abs/2401.02463作者：Laurent Millot备注：None摘要：音乐家或音响工程师在研究实验室中用于声音合成的分析工具可能相当不同。通过对这些工具的假设和局限性的讨论，我们可以提出一个第一工具，它尽可能与所有声音演员相关和通用，其主要目标是：人们必须能够听到分析的每一个元素，因为听觉是最终的参考工具。这个工具也应该使用，在未来，重新调查的声音（或声学）的定义的基础上，最近的一些作品乐器建模，语音生产和扬声器设计。将给出音频插图。论文6041在音频工程学会第116届会议上发表，柏林，2004年摘要：Analysis tools used in research laboratories, for sound synthesis, by musicians or sound engineers can be rather different. Discussion of the assumptions and of the limitations of these tools permits to propose a first tool as relevant and versatile as possible for all the sound actors with a major aim: one must be able to listen to each element of the analysis because hearing is the final reference tool. This tool should also be used, in the future, to reinvestigate the definition of sound (or Acoustics) on the basis of some recent works on musical instrument modeling, speech production and loudspeakers design. Audio illustrations will be given.Paper 6041 presented at the 116th Convention of the Audio Engineering Society, Berlin, 2004【4】 Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networks标题：基于单词混淆网络的语境学习实现ASR稳健的口语理解链接：https://arxiv.org/abs/2401.02921作者：Kevin Everson,Yile Gu,Huck Yang,Prashanth Gurunath Shivakumar,Guan-Ting Lin,Jari Kolehmainen,Ivan Bulyko,Ankur Gandhe,Shalini Ghosh,Wael Hamza,Hung-yi Lee,Ariya Rastrow,Andreas Stolcke备注：Accepted to ICASSP 2024摘要：在口语理解（SLU）领域，许多自然语言理解（NLU）方法已经通过向大型语言模型（LLM）提供转录语音而不是传统的书面文本来适应。在现实世界的场景中，在输入到LLM中之前，自动语音识别（ASR）系统生成输出转录假设，其中固有的错误可以降低后续SLU任务。在这里，我们介绍了一种方法，利用ASR系统的格输出，而不是仅仅依赖于顶部的假设，旨在封装语音歧义和增强SLU的结果。我们的上下文学习实验，包括口语问答和意图分类，强调了LLM的弹性嘈杂的语音成绩单的帮助下，从格子的单词混淆网络，弥合SLU性能之间的差距使用顶部ASR假设和甲骨文上限。此外，我们深入研究了LLM对不同ASR性能条件的鲁棒性，并仔细研究了证明最有影响力的上下文学习方面。摘要：In the realm of spoken language understanding (SLU), numerous natural language understanding (NLU) methodologies have been adapted by supplying large language models (LLMs) with transcribed speech instead of conventional written text. In real-world scenarios, prior to input into an LLM, an automated speech recognition (ASR) system generates an output transcript hypothesis, where inherent errors can degrade subsequent SLU tasks. Here we introduce a method that utilizes the ASR system's lattice output instead of relying solely on the top hypothesis, aiming to encapsulate speech ambiguities and enhance SLU outcomes. Our in-context learning experiments, covering spoken question answering and intent classification, underline the LLM's resilience to noisy speech transcripts with the help of word confusion networks from lattices, bridging the SLU performance gap between using the top ASR hypothesis and an oracle upper bound. Additionally, we delve into the LLM's robustness to varying ASR performance conditions and scrutinize the aspects of in-context learning which prove the most influential.【5】 MusicAOG: an Energy-Based Model for Learning and Sampling a Hierarchical Representation of Symbolic Music标题：MusicAOG：一种基于能量的符号音乐分层表示学习和采样模型链接：https://arxiv.org/abs/2401.02678作者：Yikai Qian,Tianle Wang,Xinyi Tong,Xin Jin,Duo Xu,Bo Zheng,Tiezheng Ge,Feng Yu,Song-Chun Zhu摘要：为了解决人工音乐智能的可解释性和可推广性的挑战，本文介绍了一种新的符号表示，它融合了不同传统和粒度的显式和隐式音乐信息。该模型采用分层和或图表示，采用节点和边来封装广泛的音乐元素，包括结构，纹理，节奏和和声。这种层次化的方法扩展了音乐在不同尺度上的表现力。这种表示作为基于能量的模型的基础，通过依赖于最小最大熵原理的灵活算法框架，为学习音乐概念而专门定制。利用一个适应大都会黑斯廷斯采样技术，该模型能够细粒度控制音乐的生成。一个全面的实证评估，对比这种新的方法与现有的方法，表现出相当大的进步，在可解释性和可控性。这项研究标志着音乐分析，作曲和计算音乐学领域的重大贡献。摘要：In addressing the challenge of interpretability and generalizability of artificial music intelligence, this paper introduces a novel symbolic representation that amalgamates both explicit and implicit musical information across diverse traditions and granularities. Utilizing a hierarchical and-or graph representation, the model employs nodes and edges to encapsulate a broad spectrum of musical elements, including structures, textures, rhythms, and harmonies. This hierarchical approach expands the representability across various scales of music. This representation serves as the foundation for an energy-based model, uniquely tailored to learn musical concepts through a flexible algorithm framework relying on the minimax entropy principle. Utilizing an adapted Metropolis-Hastings sampling technique, the model enables fine-grained control over music generation. A comprehensive empirical evaluation, contrasting this novel approach with existing methodologies, manifests considerable advancements in interpretability and controllability. This study marks a substantial contribution to the fields of music analysis, composition, and computational musicology.【6】 Gradient weighting for speaker verification in extremely low Signal-to-Noise Ratio标题：极低信噪比下说话人确认的梯度加权方法链接：https://arxiv.org/abs/2401.02626作者：Yi Ma,Kong Aik Lee,Ville Hautamäki,Meng Ge,Haizhou Li备注：Accepted by ICASSP 2024摘要：说话人确认受到背景噪声的影响，特别是在信噪比（SNR）低于0 dB的情况下。很难在不引入有害伪影的情况下抑制噪声，这对说话人确认产生不利影响。我们提出了一种称为梯度加权（Grad-W）的机制，它在预测过程中动态识别和减少伪影噪声。该机制基于梯度指示模型正在关注输入的哪些部分的属性。具体来说，当扬声器网络专注于去噪话语中的区域而不是干净的对应物时，我们认为它是伪噪声，并在优化增强过程中为该区域分配更高的权重。我们通过训练增强模型和对增强后的语音进行说话人确认来验证它。实验结果表明，我们的方法有效地减少了伪噪声，提高了说话人确认在不同的SNR水平。摘要：Speaker verification is hampered by background noise, particularly at extremely low Signal-to-Noise Ratio (SNR) under 0 dB. It is difficult to suppress noise without introducing unwanted artifacts, which adversely affects speaker verification. We proposed the mechanism called Gradient Weighting (Grad-W), which dynamically identifies and reduces artifact noise during prediction. The mechanism is based on the property that the gradient indicates which parts of the input the model is paying attention to. Specifically, when the speaker network focuses on a region in the denoised utterance but not on the clean counterpart, we consider it artifact noise and assign higher weights for this region during optimization of enhancement. We validate it by training an enhancement model and testing the enhanced utterance on speaker verification. The experimental results show that our approach effectively reduces artifact noise, improving speaker verification across various SNR levels.【7】 Towards Weakly Supervised Text-to-Audio Grounding标题：走向弱监督的文本到音频的基础链接：https://arxiv.org/abs/2401.02584作者：Xuenan Xu,Ziyang Ma,Mengyue Wu,Kai Yu摘要：文本-音频接地（TAG）任务旨在预测自然语言描述的声音事件的起始和终止。这个任务可以促进应用程序，如多模态信息检索。本文重点研究弱监督文本到音频接地（WSTAG），其中声音事件的帧级注释不可用，只能利用整个音频片段的标题进行训练。WSTAG在对大型音频文本数据集的可扩展性方面优于强监督方法。本文研究了两种WSTAG框架：句子级和短语级。首先，我们分析了在以前的WSTAG方法中使用的均值池的局限性，并研究了不同的池化策略的效果。然后，我们提出短语级WSTAG使用音频片段和短语之间的匹配标签进行训练。提出了先进的负采样策略和自我监督，以提高弱标签的准确性，并提供伪强标签。实验结果表明，我们的系统显着优于以前的WSTAG SOTA。最后，我们进行了大量的实验来分析几个因素对短语级WSTAG的影响。代码和模型可在https://github.com/wsntxxn/TextToAudioGrounding上获得。摘要：Text-to-audio grounding (TAG) task aims to predict the onsets and offsets of sound events described by natural language. This task can facilitate applications such as multimodal information retrieval. This paper focuses on weakly-supervised text-to-audio grounding (WSTAG), where frame-level annotations of sound events are unavailable, and only the caption of a whole audio clip can be utilized for training. WSTAG is superior to strongly-supervised approaches in its scalability to large audio-text datasets. Two WSTAG frameworks are studied in this paper: sentence-level and phrase-level. First, we analyze the limitations of mean pooling used in the previous WSTAG approach and investigate the effects of different pooling strategies. We then propose phrase-level WSTAG to use matching labels between audio clips and phrases for training. Advanced negative sampling strategies and self-supervision are proposed to enhance the accuracy of the weak labels and provide pseudo strong labels. Experimental results show that our system significantly outperforms the previous WSTAG SOTA. Finally, we conduct extensive experiments to analyze the effects of several factors on phrase-level WSTAG. The code and model is available at https://github.com/wsntxxn/TextToAudioGrounding.【8】 Siamese Residual Neural Network for Musical Shape Evaluation in Piano Performance Assessment标题：连体残差神经网络在钢琴演奏评价中的音乐形态评价链接：https://arxiv.org/abs/2401.02566作者：Xiaoquan Li,Stephan Weiss,Yijun Yan,Yinhe Li,Jinchang Ren,John Soraghan,Ming Gong备注：X.Li, S.Weiss, Y.Yan, Y.Li, J.Ren, J.Soraghan, M.Gong,"Siamese residual neural network for musical shape evaluation in piano performance assessment" in Proc. of the 31st European Signal Processing Conference, Helsinki, Finland摘要：音乐形态的理解和识别在音乐教育和演奏评价中起着重要的作用。为了简化时间和成本密集型的音乐形状评估，在本文中，我们探讨了如何应用人工智能（AI）驱动的模型。将音乐形态评价看作一个分类问题，提出了一种轻量级的连体残差神经网络（S-ResNN）来自动识别音乐形态。为了评估所提出的方法在钢琴音乐形态评价的背景下，我们已经产生了一个新的数据集，包含4116音乐作品来自147钢琴预备练习，并在28类音乐形态。实验结果表明，S-ResNN在查准率、查全率和F1得分方面明显优于一些基准方法。摘要：Understanding and identifying musical shape plays an important role in music education and performance assessment. To simplify the otherwise time- and cost-intensive musical shape evaluation, in this paper we explore how artificial intelligence (AI) driven models can be applied. Considering musical shape evaluation as a classification problem, a light-weight Siamese residual neural network (S-ResNN) is proposed to automatically identify musical shapes. To assess the proposed approach in the context of piano musical shape evaluation, we have generated a new dataset, containing 4116 music pieces derived by 147 piano preparatory exercises and performed in 28 categories of musical shapes. The experimental results show that the S-ResNN significantly outperforms a number of benchmark methods in terms of the precision, recall and F1 score.机器翻译由腾讯交互翻译提供，仅供参考
每日语音速递
