今日论文合集：cs.SD语音7篇，eess.AS音频处理8篇。
cs.SD语音

【1】 Adapting Meter Tracking Models to Latin American Music

标题：调整节拍跟踪模式以适应拉美音乐链接：http://arxiv.org/pdf/2304.07186v1

作者：Lucas S. Maia,Martín Rocamora,Luiz W. P. Biscainho,Magdalena Fuentes机构：PEECOPPE, Federal University of Rio de Janeiro, Brazil, FING, Universidad de la República, Uruguay, MARL-IDM, New York University, United States备注：Accepted at ISMIR 2022. This version was made after a bug fix in the code, which lead to minor modifications in the results (updated in Figure 1 and Table 1). The paper's conclusions remain unchanged摘要：近年来，随着深度学习方法的引入，节拍和下拍跟踪模型得到了显着改进。然而，尽管有这些改进，仍然存在一些挑战。特别是，现有的模型，以代表性不足的音乐传统的MIR的适应通常是同义词收集和注释大量的数据，这是不切实际的和耗时的。迁移学习、数据增强和微调技术已经在相关任务中得到了非常成功的应用，并且被认为可以缓解这一瓶颈。此外，在研究这些音乐传统时，模型不需要推广到多种主流音乐流派，而是在更受约束的同质条件下表现良好。在这项工作中，我们调查简单而有效的策略，以适应节拍和强拍跟踪模型，以两种不同的拉丁美洲音乐传统，并分析这些适应在现实世界中的应用程序的数据和计算要求的可行性。与普遍的看法相反，我们的研究结果表明，只需花几分钟时间注释一部分数据并在标准CPU机器中训练模型，就可以实现良好的性能，所需的精确资源量取决于任务和数据集的复杂性。摘要：Beat and downbeat tracking models have improved significantly in recent years with the introduction of deep learning methods. However, despite these improvements, several challenges remain. Particularly, the adaptation of available models to underrepresented music traditions in MIR is usually synonymous with collecting and annotating large amounts of data, which is impractical and time-consuming. Transfer learning, data augmentation, and fine-tuning techniques have been used quite successfully in related tasks and are known to alleviate this bottleneck. Furthermore, when studying these music traditions, models are not required to generalize to multiple mainstream music genres but to perform well in more constrained, homogeneous conditions. In this work, we investigate simple yet effective strategies to adapt beat and downbeat tracking models to two different Latin American music traditions and analyze the feasibility of these adaptations in real-world applications concerning the data and computational requirements. Contrary to common belief, our findings show it is possible to achieve good performance by spending just a few minutes annotating a portion of the data and training a model in a standard CPU machine, with the precise amount of resources needed depending on the task and the complexity of the dataset.

【2】 On Data Sampling Strategies for Training Neural Network Speech Separation Models标题：训练神经网络语音分离模型的数据采样策略研究链接：http://arxiv.org/pdf/2304.07142v1

作者：William Ravenscroft,Stefan Goetze,Thomas Hain机构：Department of Computer Science, The University of Sheffield, Sheffield, United Kingdom备注：Submitted to EUSIPCO 2023摘要：语音分离是多说话人信号处理的一个重要领域。深度神经网络（DNN）模型在许多语音分离基准测试中表现最佳。其中一些模型可能需要花费大量时间来训练，并且具有较高的内存要求。以前的工作已经提出了缩短训练示例来解决这些问题，但这对模型性能的影响尚未得到很好的理解。在这项工作中，应用这些训练信号长度（TSL）限制的影响被分析用于两个语音分离模型：SepFormer，一个Transformer模型，Conv-TasNet，一个卷积模型。WJS 0 - 2 Mix、WHAMR和Libri 2 Mix数据集在信号长度分布及其对训练效率的影响方面进行了分析。结果表明，对于特定的分布，应用特定的TSL限值会产生更好的性能。这被示出主要是由于随机采样的波形的开始索引，从而导致更独特的训练的例子。使用4.42s的TSL限制和动态混合（DM）训练的SepFormer模型被示出为与使用DM和无限信号长度训练的性能最佳的SepFormer模型相匹配。此外，4.42s的TSL限制导致WHAMR的训练时间减少44%。摘要：Speech separation remains an important area of multi-speaker signal processing. Deep neural network (DNN) models have attained the best performance on many speech separation benchmarks. Some of these models can take significant time to train and have high memory requirements. Previous work has proposed shortening training examples to address these issues but the impact of this on model performance is not yet well understood. In this work, the impact of applying these training signal length (TSL) limits is analysed for two speech separation models: SepFormer, a transformer model, and Conv-TasNet, a convolutional model. The WJS0-2Mix, WHAMR and Libri2Mix datasets are analysed in terms of signal length distribution and its impact on training efficiency. It is demonstrated that, for specific distributions, applying specific TSL limits results in better performance. This is shown to be mainly due to randomly sampling the start index of the waveforms resulting in more unique examples for training. A SepFormer model trained using a TSL limit of 4.42s and dynamic mixing (DM) is shown to match the best-performing SepFormer model trained with DM and unlimited signal lengths. Furthermore, the 4.42s TSL limit results in a 44% reduction in training time with WHAMR.

【3】 Tempo vs. Pitch: understanding self-supervised tempo estimation标题：节奏与音高：理解自我监督的节奏估计链接：http://arxiv.org/pdf/2304.06868v1

作者：Giovana Morais,Matthew E. P. Davies,Marcelo Queiroz,Magdalena Fuentes机构：Computer Science Dept., University of S˜ao Paulo, S˜ao Paulo, Brazil, SiriusXM, USA, MARL-IDM, New York University, New York, USA备注：5 pages, 3 figures, published on 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing摘要：自我监督方法通过解决不需要人工生成标签的借口任务来学习表示，从而减轻了对耗时注释的需求。这些方法已经应用于计算机视觉、自然语言处理、环境声音分析，并且最近应用于音乐信息检索，例如用于音调估计。特别是在音乐的背景下，关于这些模型在不同数据分布方面的脆弱性以及如何减轻这些脆弱性的见解很少。在本文中，我们探讨这些问题，通过解剖一个自我监督模型的音高估计适应节奏估计，通过严格的实验与合成数据。具体来说，我们研究了自监督节奏估计的输入表示和数据分布之间的关系。摘要：Self-supervision methods learn representations by solving pretext tasks that do not require human-generated labels, alleviating the need for time-consuming annotations. These methods have been applied in computer vision, natural language processing, environmental sound analysis, and recently in music information retrieval, e.g. for pitch estimation. Particularly in the context of music, there are few insights about the fragility of these models regarding different distributions of data, and how they could be mitigated. In this paper, we explore these questions by dissecting a self-supervised model for pitch estimation adapted for tempo estimation via rigorous experimentation with synthetic data. Specifically, we study the relationship between the input representation and data distribution for self-supervised tempo estimation.

【4】 Level generation for rhythm VR games标题：节奏VR游戏的关卡生成链接：http://arxiv.org/pdf/2304.06809v1

作者：Mariia Rizhko机构：Supervised by Andrii Boichuk, Lviv, Spring备注：Bachelor's thesis, 35 pages, 25 figures摘要：Ragnarock是一款虚拟现实（VR）节奏游戏，在游戏中你扮演一名维京船长参加长船比赛。有了两个锤子，任务是粉碎传入的符文与史诗维京音乐同步。符文由玩家可以手动创建的节拍图定义。创建节拍图需要几个小时。这项工作旨在自动化节拍图创建的过程，也称为学习编排的任务。作业分为三部分：确定节拍的定时（动作放置）、确定与所选节拍连接的符文应该放置在空间中的何处（动作选择）以及web应用程序创建。对于动作放置的第一任务，使用从音乐记录中提取主导局部脉冲（PLP）信息。这种方法允许学习应该放置在何处以及放置多少节拍。对于动作选择的第二个任务，使用递归神经网络（RNN），特别是门控递归单元（GRU）来学习节拍序列及其模式，以便能够重新创建这些规则并接收全新的水平。最后一项任务是为非技术人员构建一个解决方案，任务是将第一部分和第二部分的结果组合到一个Web应用程序中，以便于使用。对于这项任务，前端使用JavaScript和React构建，后端使用python和FastAPI构建。摘要：Ragnarock is a virtual reality (VR) rhythm game in which you play a Viking captain competing in a longship race. With two hammers, the task is to crush the incoming runes in sync with epic Viking music. The runes are defined by a beat map which the player can manually create. The creation of beat maps takes hours. This work aims to automate the process of beat map creation, also known as the task of learning to choreograph. The assignment is broken down into three parts: determining the timing of the beats (action placement), determining where in space the runes connected with the chosen beats should be placed (action selection) and web-application creation. For the first task of action placement, extraction of predominant local pulse (PLP) information from music recordings is used. This approach allows to learn where and how many beats are supposed to be placed. For the second task of action selection, Recurrent Neural Networks (RNN) are used, specifically Gated recurrent unit (GRU) to learn sequences of beats, and their patterns to be able to recreate those rules and receive completely new levels. Then the last task was to build a solution for non-technical players, the task was to combine the results of the first and the second parts into a web application for easy use. For this task the frontend was built using JavaScript and React and the backend - python and FastAPI.

【5】 HCAM -- Hierarchical Cross Attention Model for Multi-modal Emotion Recognition标题：HCAM--多通道情绪识别的分层交叉注意模型链接：http://arxiv.org/pdf/2304.06910v1

作者：Soumya Dutta,Sriram Ganapathy机构：Indian Instituteof Science备注：11 pages, 6 figures, submitted to IEEE TASLP摘要：由于情感表达的多模态性质，会话中的情感识别具有挑战性。我们提出了一个层次交叉注意模型（HCAM）的方法，多模态情感识别使用的组合经常性和共同注意神经网络模型。模型的输入由两种模态组成，i）通过可学习的wav 2 vec方法处理的音频数据，以及ii）使用来自Transformers的双向编码器表示（BERT）模型表示的文本数据。音频和文本表示使用一组具有自我注意力的双向递归神经网络层来处理，该神经网络层将给定对话中的每个话语转换为固定维度的嵌入。为了将上下文知识和跨两种模态的信息结合起来，使用共同关注层来组合音频和文本嵌入，该共同关注层尝试对与情感识别的任务相关的话语级嵌入进行加权。音频层、文本层以及多模态共同注意层中的神经网络参数被分层地训练用于情感分类任务。我们在三个已建立的数据集上进行实验，即IEMOCAP，MELD和CMU-MOSI，在那里我们说明了所提出的模型比其他基准有显着改进，并有助于在所有这些数据集上实现最先进的结果。摘要：Emotion recognition in conversations is challenging due to the multi-modal nature of the emotion expression. We propose a hierarchical cross-attention model (HCAM) approach to multi-modal emotion recognition using a combination of recurrent and co-attention neural network models. The input to the model consists of two modalities, i) audio data, processed through a learnable wav2vec approach and, ii) text data represented using a bidirectional encoder representations from transformers (BERT) model. The audio and text representations are processed using a set of bi-directional recurrent neural network layers with self-attention that converts each utterance in a given conversation to a fixed dimensional embedding. In order to incorporate contextual knowledge and the information across the two modalities, the audio and text embeddings are combined using a co-attention layer that attempts to weigh the utterance level embeddings relevant to the task of emotion recognition. The neural network parameters in the audio layers, text layers as well as the multi-modal co-attention layers, are hierarchically trained for the emotion classification task. We perform experiments on three established datasets namely, IEMOCAP, MELD and CMU-MOSI, where we illustrate that the proposed model improves significantly over other benchmarks and helps achieve state-of-art results on all these datasets.

【6】Efficient Sequence Transduction by Jointly Predicting Tokens and Durations标题：联合预测标记和持续时间的高效序列转导链接：http://arxiv.org/pdf/2304.06795v1

作者：Hainan Xu,Fei Jia,Somshubra Majumdar,He Huang,Shinji Watanabe,Boris Ginsburg机构：USA 2Carnegie Mellon University摘要：本文介绍了一种新的令牌和持续时间转换器（TDT）结构的序列到序列的任务。TDT通过联合预测令牌及其持续时间（即由发射的令牌覆盖的输入帧的数量）来扩展传统的RNN换能器架构。这是通过使用具有两个输出的联合网络来实现的，这两个输出被独立地归一化以生成令牌和持续时间上的分布。在推断期间，TDT模型可以跳过由预测持续时间输出引导的输入帧，这使得它们比逐帧处理编码器输出的常规换能器显著更快。TDT模型在不同的序列转导任务上实现了比常规转导器更好的准确性和显著更快的推断。用于语音识别的TDT模型实现了更高的准确性，推理速度比RNN传感器快2.82倍。用于语音翻译的TDT模型在MUST-C测试中与传统换能器相比实现了超过1 BLEU的绝对增益，其推理速度提高了2.27倍。在语音意图分类和槽填充任务中，TDT模型将意图准确度提高到传统换能器的1%（绝对值）以上，同时运行速度提高了1.28倍。摘要：This paper introduces a novel Token-and-Duration Transducer (TDT) architecture for sequence-to-sequence tasks. TDT extends conventional RNN-Transducer architectures by jointly predicting both a token and its duration, i.e. the number of input frames covered by the emitted token. This is achieved by using a joint network with two outputs which are independently normalized to generate distributions over tokens and durations. During inference, TDT models can skip input frames guided by the predicted duration output, which makes them significantly faster than conventional Transducers which process the encoder output frame by frame. TDT models achieve both better accuracy and significantly faster inference than conventional Transducers on different sequence transduction tasks. TDT models for Speech Recognition achieve better accuracy and up to 2.82X faster inference than RNN-Transducers. TDT models for Speech Translation achieve an absolute gain of over 1 BLEU on the MUST-C test compared with conventional Transducers, and its inference is 2.27X faster. In Speech Intent Classification and Slot Filling tasks, TDT models improve the intent accuracy up to over 1% (absolute) over conventional Transducers, while running up to 1.28X faster.

【7】 The future of hearing aid technology标题：助听器技术的未来链接：http://arxiv.org/pdf/2304.06786v1

作者：Volker Hohmann机构：Department of Medical Physics and Acoustics, University of Oldenburg, Oldenburg, Hörzentrum Oldenburg gGmbH, Oldenburg, Germany, Cluster of Excellence Hearing,all Oldenburg, Germany摘要：背景。助听器技术已被证明在听力损失的康复方面是成功的，但其性能在以噪声和混响为特征的困难的日常条件下仍然受到限制。目标。介绍了助听器技术的现状，并介绍了目前的研究状况和未来的发展。方法。目前的文献进行了分析，并提出了几个具体的新的发展。结果。来自实证研究的客观和主观数据都显示了当前技术的局限性。当前研究的实例显示了基于机器学习的算法和多模态信号处理用于改善语音处理和感知的潜力，使用虚拟现实用于改善听力设备适配的潜力，以及移动健康技术用于改善听力健康服务的潜力。结论。听力设备技术仍将是听力障碍康复的关键因素。机器学习、多模态信号处理、虚拟现实和移动医疗技术等新技术将改善语音增强、个人适应和沟通培训。摘要：Background. Hearing aid technology has proven successful in the rehabilitation of hearing loss, but its performance is still limited in difficult everyday conditions characterized by noise and reverberation. Objectives. Introduction to the current state of hearing aid technology and presentation of the current state of research and future development. Methods. Current literature is analyzed and several specific new developments are presented. Results. Both objective and subjective data from empirical studies show the limitation of current technology. Examples of current research show the potential of machine-learning based algorithms and multi-modal signal processing for improving speech processing and perception, of using virtual reality for improving hearing device fitting and of mobile health technology for improving hearing-health services. Conclusions. Hearing device technology will remain a key factor in the rehabilitation of hearing impairment. New technology such as machine learning, and multi-modal signal processing, virtual reality and mobile health technology will improve speech enhancement, individual fitting and communication training.
eess.AS音频处理

【1】 HCAM -- Hierarchical Cross Attention Model for Multi-modal Emotion Recognition标题：HCAM--多通道情绪识别的分层交叉注意模型链接：http://arxiv.org/pdf/2304.06910v1

*与cs.SD语音【5】为同一篇

作者：Soumya Dutta,Sriram Ganapathy机构：Indian Instituteof Science备注：11 pages, 6 figures, submitted to IEEE TASLP摘要：由于情感表达的多模态性质，会话中的情感识别具有挑战性。我们提出了一个层次交叉注意模型（HCAM）的方法，多模态情感识别使用的组合经常性和共同注意神经网络模型。模型的输入由两种模态组成，i）通过可学习的wav 2 vec方法处理的音频数据，以及ii）使用来自Transformers的双向编码器表示（BERT）模型表示的文本数据。音频和文本表示使用一组具有自我注意力的双向递归神经网络层来处理，该神经网络层将给定对话中的每个话语转换为固定维度的嵌入。为了将上下文知识和跨两种模态的信息结合起来，使用共同关注层来组合音频和文本嵌入，该共同关注层尝试对与情感识别的任务相关的话语级嵌入进行加权。音频层、文本层以及多模态共同注意层中的神经网络参数被分层地训练用于情感分类任务。我们在三个已建立的数据集上进行实验，即IEMOCAP，MELD和CMU-MOSI，在那里我们说明了所提出的模型比其他基准有显着改进，并有助于在所有这些数据集上实现最先进的结果。摘要：Emotion recognition in conversations is challenging due to the multi-modal nature of the emotion expression. We propose a hierarchical cross-attention model (HCAM) approach to multi-modal emotion recognition using a combination of recurrent and co-attention neural network models. The input to the model consists of two modalities, i) audio data, processed through a learnable wav2vec approach and, ii) text data represented using a bidirectional encoder representations from transformers (BERT) model. The audio and text representations are processed using a set of bi-directional recurrent neural network layers with self-attention that converts each utterance in a given conversation to a fixed dimensional embedding. In order to incorporate contextual knowledge and the information across the two modalities, the audio and text embeddings are combined using a co-attention layer that attempts to weigh the utterance level embeddings relevant to the task of emotion recognition. The neural network parameters in the audio layers, text layers as well as the multi-modal co-attention layers, are hierarchically trained for the emotion classification task. We perform experiments on three established datasets namely, IEMOCAP, MELD and CMU-MOSI, where we illustrate that the proposed model improves significantly over other benchmarks and helps achieve state-of-art results on all these datasets.

【2】 Efficient Sequence Transduction by Jointly Predicting Tokens and Durations标题：联合预测标记和持续时间的高效序列转导链接：http://arxiv.org/pdf/2304.06795v1

*与cs.SD语音【6】为同一篇

作者：Hainan Xu,Fei Jia,Somshubra Majumdar,He Huang,Shinji Watanabe,Boris Ginsburg机构：USA 2Carnegie Mellon University摘要：本文介绍了一种新的令牌和持续时间转换器（TDT）结构的序列到序列的任务。TDT通过联合预测令牌及其持续时间（即由发射的令牌覆盖的输入帧的数量）来扩展传统的RNN换能器架构。这是通过使用具有两个输出的联合网络来实现的，这两个输出被独立地归一化以生成令牌和持续时间上的分布。在推断期间，TDT模型可以跳过由预测持续时间输出引导的输入帧，这使得它们比逐帧处理编码器输出的常规换能器显著更快。TDT模型在不同的序列转导任务上实现了比常规转导器更好的准确性和显著更快的推断。用于语音识别的TDT模型实现了更高的准确性，推理速度比RNN传感器快2.82倍。用于语音翻译的TDT模型在MUST-C测试中与传统换能器相比实现了超过1 BLEU的绝对增益，其推理速度提高了2.27倍。在语音意图分类和槽填充任务中，TDT模型将意图准确度提高到传统换能器的1%（绝对值）以上，同时运行速度提高了1.28倍。摘要：This paper introduces a novel Token-and-Duration Transducer (TDT) architecture for sequence-to-sequence tasks. TDT extends conventional RNN-Transducer architectures by jointly predicting both a token and its duration, i.e. the number of input frames covered by the emitted token. This is achieved by using a joint network with two outputs which are independently normalized to generate distributions over tokens and durations. During inference, TDT models can skip input frames guided by the predicted duration output, which makes them significantly faster than conventional Transducers which process the encoder output frame by frame. TDT models achieve both better accuracy and significantly faster inference than conventional Transducers on different sequence transduction tasks. TDT models for Speech Recognition achieve better accuracy and up to 2.82X faster inference than RNN-Transducers. TDT models for Speech Translation achieve an absolute gain of over 1 BLEU on the MUST-C test compared with conventional Transducers, and its inference is 2.27X faster. In Speech Intent Classification and Slot Filling tasks, TDT models improve the intent accuracy up to over 1% (absolute) over conventional Transducers, while running up to 1.28X faster.

【3】 The future of hearing aid technology标题：助听器技术的未来链接：http://arxiv.org/pdf/2304.06786v1

*与cs.SD语音【7】为同一篇

作者：Volker Hohmann机构：Department of Medical Physics and Acoustics, University of Oldenburg, Oldenburg, Hörzentrum Oldenburg gGmbH, Oldenburg, Germany, Cluster of Excellence Hearing,all Oldenburg, Germany摘要：背景。助听器技术已被证明在听力损失的康复方面是成功的，但其性能在以噪声和混响为特征的困难的日常条件下仍然受到限制。目标。介绍了助听器技术的现状，并介绍了目前的研究状况和未来的发展。方法。目前的文献进行了分析，并提出了几个具体的新的发展。结果。来自实证研究的客观和主观数据都显示了当前技术的局限性。当前研究的实例显示了基于机器学习的算法和多模态信号处理用于改善语音处理和感知的潜力，使用虚拟现实用于改善听力设备适配的潜力，以及移动健康技术用于改善听力健康服务的潜力。结论。听力设备技术仍将是听力障碍康复的关键因素。机器学习、多模态信号处理、虚拟现实和移动医疗技术等新技术将改善语音增强、个人适应和沟通培训。摘要：Background. Hearing aid technology has proven successful in the rehabilitation of hearing loss, but its performance is still limited in difficult everyday conditions characterized by noise and reverberation. Objectives. Introduction to the current state of hearing aid technology and presentation of the current state of research and future development. Methods. Current literature is analyzed and several specific new developments are presented. Results. Both objective and subjective data from empirical studies show the limitation of current technology. Examples of current research show the potential of machine-learning based algorithms and multi-modal signal processing for improving speech processing and perception, of using virtual reality for improving hearing device fitting and of mobile health technology for improving hearing-health services. Conclusions. Hearing device technology will remain a key factor in the rehabilitation of hearing impairment. New technology such as machine learning, and multi-modal signal processing, virtual reality and mobile health technology will improve speech enhancement, individual fitting and communication training.

【4】 Adapting Meter Tracking Models to Latin American Music

标题：调整节拍跟踪模式以适应拉美音乐链接：http://arxiv.org/pdf/2304.07186v1

*与cs.SD语音【1】为同一篇

作者：Lucas S. Maia,Martín Rocamora,Luiz W. P. Biscainho,Magdalena Fuentes机构：PEECOPPE, Federal University of Rio de Janeiro, Brazil, FING, Universidad de la República, Uruguay, MARL-IDM, New York University, United States备注：Accepted at ISMIR 2022. This version was made after a bug fix in the code, which lead to minor modifications in the results (updated in Figure 1 and Table 1). The paper's conclusions remain unchanged摘要：近年来，随着深度学习方法的引入，节拍和下拍跟踪模型得到了显着改进。然而，尽管有这些改进，仍然存在一些挑战。特别是，现有的模型，以代表性不足的音乐传统的MIR的适应通常是同义词收集和注释大量的数据，这是不切实际的和耗时的。迁移学习、数据增强和微调技术已经在相关任务中得到了非常成功的应用，并且被认为可以缓解这一瓶颈。此外，在研究这些音乐传统时，模型不需要推广到多种主流音乐流派，而是在更受约束的同质条件下表现良好。在这项工作中，我们调查简单而有效的策略，以适应节拍和强拍跟踪模型，以两种不同的拉丁美洲音乐传统，并分析这些适应在现实世界中的应用程序的数据和计算要求的可行性。与普遍的看法相反，我们的研究结果表明，只需花几分钟时间注释一部分数据并在标准CPU机器中训练模型，就可以实现良好的性能，所需的精确资源量取决于任务和数据集的复杂性。摘要：Beat and downbeat tracking models have improved significantly in recent years with the introduction of deep learning methods. However, despite these improvements, several challenges remain. Particularly, the adaptation of available models to underrepresented music traditions in MIR is usually synonymous with collecting and annotating large amounts of data, which is impractical and time-consuming. Transfer learning, data augmentation, and fine-tuning techniques have been used quite successfully in related tasks and are known to alleviate this bottleneck. Furthermore, when studying these music traditions, models are not required to generalize to multiple mainstream music genres but to perform well in more constrained, homogeneous conditions. In this work, we investigate simple yet effective strategies to adapt beat and downbeat tracking models to two different Latin American music traditions and analyze the feasibility of these adaptations in real-world applications concerning the data and computational requirements. Contrary to common belief, our findings show it is possible to achieve good performance by spending just a few minutes annotating a portion of the data and training a model in a standard CPU machine, with the precise amount of resources needed depending on the task and the complexity of the dataset.

【5】 On Data Sampling Strategies for Training Neural Network Speech Separation Models标题：训练神经网络语音分离模型的数据采样策略研究链接：http://arxiv.org/pdf/2304.07142v1

*与cs.SD语音【2】为同一篇

作者：William Ravenscroft,Stefan Goetze,Thomas Hain机构：Department of Computer Science, The University of Sheffield, Sheffield, United Kingdom备注：Submitted to EUSIPCO 2023摘要：语音分离是多说话人信号处理的一个重要领域。深度神经网络（DNN）模型在许多语音分离基准测试中表现最佳。其中一些模型可能需要花费大量时间来训练，并且具有较高的内存要求。以前的工作已经提出了缩短训练示例来解决这些问题，但这对模型性能的影响尚未得到很好的理解。在这项工作中，应用这些训练信号长度（TSL）限制的影响被分析用于两个语音分离模型：SepFormer，一个Transformer模型，Conv-TasNet，一个卷积模型。WJS 0 - 2 Mix、WHAMR和Libri 2 Mix数据集在信号长度分布及其对训练效率的影响方面进行了分析。结果表明，对于特定的分布，应用特定的TSL限值会产生更好的性能。这被示出主要是由于随机采样的波形的开始索引，从而导致更独特的训练的例子。使用4.42s的TSL限制和动态混合（DM）训练的SepFormer模型被示出为与使用DM和无限信号长度训练的性能最佳的SepFormer模型相匹配。此外，4.42s的TSL限制导致WHAMR的训练时间减少44%。摘要：Speech separation remains an important area of multi-speaker signal processing. Deep neural network (DNN) models have attained the best performance on many speech separation benchmarks. Some of these models can take significant time to train and have high memory requirements. Previous work has proposed shortening training examples to address these issues but the impact of this on model performance is not yet well understood. In this work, the impact of applying these training signal length (TSL) limits is analysed for two speech separation models: SepFormer, a transformer model, and Conv-TasNet, a convolutional model. The WJS0-2Mix, WHAMR and Libri2Mix datasets are analysed in terms of signal length distribution and its impact on training efficiency. It is demonstrated that, for specific distributions, applying specific TSL limits results in better performance. This is shown to be mainly due to randomly sampling the start index of the waveforms resulting in more unique examples for training. A SepFormer model trained using a TSL limit of 4.42s and dynamic mixing (DM) is shown to match the best-performing SepFormer model trained with DM and unlimited signal lengths. Furthermore, the 4.42s TSL limit results in a 44% reduction in training time with WHAMR.

【6】 Using a one-dimensional finite-element approximation of Webster's horn equation to estimate individual ear canal acoustic transfer from input impedances标题：用韦氏喇叭方程的一维有限元近似从输入阻抗估计单个耳道的声学传递链接：http://arxiv.org/pdf/2304.07131v1

作者：Nick Wulbusch,Reinhild Roden,Alexey Chernov,Matthias Blau机构：Institut f¨ur Mathematik, Carl-von-Ossietzky Universit¨at, Oldenburg, Germany, Institut f¨ur Audiologie und H¨ortechnik, Jade-Hochschule, Oldenburg, Germany, Cluster of Excellence ”Hearing,All”备注：16 pages, 15 figures摘要：在许多应用中，声压传递到鼓膜的知识是重要的。传递受到耳道的形状及其声学特性（诸如鼓膜处的声阻抗）的高度影响。测量鼓膜处的声压的侵入性过程通常是复杂的或昂贵的。在这项工作中，我们提出了一种数值方法来估计传递阻抗在鼓膜只输入阻抗测量在耳道入口处，通过使用一维一阶有限元和Nelder-Mead优化算法。实现了对耳道的面积函数和鼓膜处的声阻抗的估计。使用从三维有限元模拟合成生成的数据，通过对十种不同耳道几何形状和鼓膜处三种不同声阻抗的数值模拟来验证结果。摘要：In many applications, knowledge of the sound pressure transfer to the eardrum is important. The transfer is highly influenced by the shape of the ear canal and its acoustic properties, such as the acoustic impedance at the eardrum. Invasive procedures to measure the sound pressure at the eardrum are usually elaborate or costly. In this work, we propose a numerical method to estimate the transfer impedance at the eardrum given only input impedance measurements at the ear canal entrance by using one-dimensional first-order finite elements and Nelder-Mead optimization algorithm. Estimations on the area function of the ear canal and the acoustic impedance at the eardrum are achieved. Results are validated through numerical simulations on ten different ear canal geometries and three different acoustic impedances at the eardrum using synthetically generated data from three-dimensional finite element simulations.

【7】 Tempo vs. Pitch: understanding self-supervised tempo estimation标题：节奏与音高：理解自我监督的节奏估计链接：http://arxiv.org/pdf/2304.06868v1

*与cs.SD语音【3】为同一篇

作者：Giovana Morais,Matthew E. P. Davies,Marcelo Queiroz,Magdalena Fuentes机构：Computer Science Dept., University of S˜ao Paulo, S˜ao Paulo, Brazil, SiriusXM, USA, MARL-IDM, New York University, New York, USA备注：5 pages, 3 figures, published on 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing摘要：自我监督方法通过解决不需要人工生成标签的借口任务来学习表示，从而减轻了对耗时注释的需求。这些方法已经应用于计算机视觉、自然语言处理、环境声音分析，并且最近应用于音乐信息检索，例如用于音调估计。特别是在音乐的背景下，关于这些模型在不同数据分布方面的脆弱性以及如何减轻这些脆弱性的见解很少。在本文中，我们探讨这些问题，通过解剖一个自我监督模型的音高估计适应节奏估计，通过严格的实验与合成数据。具体来说，我们研究了自监督节奏估计的输入表示和数据分布之间的关系。摘要：Self-supervision methods learn representations by solving pretext tasks that do not require human-generated labels, alleviating the need for time-consuming annotations. These methods have been applied in computer vision, natural language processing, environmental sound analysis, and recently in music information retrieval, e.g. for pitch estimation. Particularly in the context of music, there are few insights about the fragility of these models regarding different distributions of data, and how they could be mitigated. In this paper, we explore these questions by dissecting a self-supervised model for pitch estimation adapted for tempo estimation via rigorous experimentation with synthetic data. Specifically, we study the relationship between the input representation and data distribution for self-supervised tempo estimation.

【8】 Level generation for rhythm VR games标题：节奏VR游戏的关卡生成链接：http://arxiv.org/pdf/2304.06809v1

*与cs.SD语音【4】为同一篇

作者：Mariia Rizhko机构：Supervised by Andrii Boichuk, Lviv, Spring备注：Bachelor's thesis, 35 pages, 25 figures摘要：Ragnarock是一款虚拟现实（VR）节奏游戏，在游戏中你扮演一名维京船长参加长船比赛。有了两个锤子，任务是粉碎传入的符文与史诗维京音乐同步。符文由玩家可以手动创建的节拍图定义。创建节拍图需要几个小时。这项工作旨在自动化节拍图创建的过程，也称为学习编排的任务。作业分为三部分：确定节拍的定时（动作放置）、确定与所选节拍连接的符文应该放置在空间中的何处（动作选择）以及web应用程序创建。对于动作放置的第一任务，使用从音乐记录中提取主导局部脉冲（PLP）信息。这种方法允许学习应该放置在何处以及放置多少节拍。对于动作选择的第二个任务，使用递归神经网络（RNN），特别是门控递归单元（GRU）来学习节拍序列及其模式，以便能够重新创建这些规则并接收全新的水平。最后一项任务是为非技术人员构建一个解决方案，任务是将第一部分和第二部分的结果组合到一个Web应用程序中，以便于使用。对于这项任务，前端使用JavaScript和React构建，后端使用python和FastAPI构建。摘要：Ragnarock is a virtual reality (VR) rhythm game in which you play a Viking captain competing in a longship race. With two hammers, the task is to crush the incoming runes in sync with epic Viking music. The runes are defined by a beat map which the player can manually create. The creation of beat maps takes hours. This work aims to automate the process of beat map creation, also known as the task of learning to choreograph. The assignment is broken down into three parts: determining the timing of the beats (action placement), determining where in space the runes connected with the chosen beats should be placed (action selection) and web-application creation. For the first task of action placement, extraction of predominant local pulse (PLP) information from music recordings is used. This approach allows to learn where and how many beats are supposed to be placed. For the second task of action selection, Recurrent Neural Networks (RNN) are used, specifically Gated recurrent unit (GRU) to learn sequences of beats, and their patterns to be able to recreate those rules and receive completely new levels. Then the last task was to build a solution for non-technical players, the task was to combine the results of the first and the second parts into a web application for easy use. For this task the frontend was built using JavaScript and React and the backend - python and FastAPI.

机器翻译，仅供参考