今日论文合集：cs.SD语音2篇，eess.AS音频处理4篇。
cs.SD语音

【1】 TACOformer:Token-channel compounded Cross Attention for Multimodal  Emotion Recognition标题：TACOform：用于多通道情绪识别的令牌-通道复合交叉注意链接：https://arxiv.org/abs/2306.13592

作者：Xinda Li机构：Beihang University备注：Accepted by IJCAI 2023- AI4TS workshop摘要：近年来，基于生理信号的情感识别已成为一个研究热点。多模态、多通道生理信号的利用显著提高了情感识别系统的性能，由于它们的互补性。然而，有效地整合来自不同模态的情感相关语义信息和捕获模态间的依赖关系仍然是一个具有挑战性的问题。许多现有的多模态融合方法忽略了来自不同模态的多通道信号的令牌到令牌或通道到通道的相关性，这在一定程度上限制了模型的分类能力。在本文中，我们提出了一个综合的视角，多模态融合，集成了通道级和令牌级的跨模态交互。具体而言，我们引入了一个统一的交叉注意模块，称为令牌信道信道COmpound（TACO）交叉注意执行多模态融合，同时模型的通道级和令牌级的模式之间的依赖关系。此外，我们提出了一个2D位置编码方法，以保存有关的脑电信号通道的空间分布的信息，然后我们使用两个Transformer编码器前面的融合模块捕获长期的时间依赖性从脑电信号和外周生理信号，分别。情感数据集DEAP和Dreamer上的受试者独立实验表明，所提出的模型达到了最先进的性能。摘要：Recently, emotion recognition based on physiological signals has emerged as a field with intensive research. The utilization of multi-modal, multi-channel physiological signals has significantly improved the performance of emotion recognition systems, due to their complementarity. However, effectively integrating emotion-related semantic information from different modalities and capturing inter-modal dependencies remains a challenging issue. Many existing multimodal fusion methods ignore either token-to-token or channel-to-channel correlations of multichannel signals from different modalities, which limits the classification capability of the models to some extent. In this paper, we propose a comprehensive perspective of multimodal fusion that integrates channel-level and token-level cross-modal interactions. Specifically, we introduce a unified cross attention module called Token-chAnnel COmpound (TACO) Cross Attention to perform multimodal fusion, which simultaneously models channel-level and token-level dependencies between modalities. Additionally, we propose a 2D position encoding method to preserve information about the spatial distribution of EEG signal channels, then we use two transformer encoders ahead of the fusion module to capture long-term temporal dependencies from the EEG signal and the peripheral physiological signal, respectively. Subject-independent experiments on emotional dataset DEAP and Dreamer demonstrate that the proposed model achieves state-of-the-art performance.

【2】 DISCO-10M: A Large-Scale Music Dataset标题：DISCO-10M：一个大规模的音乐数据集链接：https://arxiv.org/abs/2306.13512
作者：Luca A. Lanzendörfer,Florian Grötschla,Emil Funke,Roger Wattenhofer机构：ETH Zurich, Zurich, Switzerland摘要：音乐数据集在推动音乐机器学习研究方面发挥着至关重要的作用。然而，现有的音乐数据集遭受有限的大小、可访问性和缺乏音频资源。为了解决这些缺点，我们提出了DISCO-10 M，一个新颖的和广泛的音乐数据集，超过了以前最大的音乐数据集的数量级。为了确保高质量的数据，我们实施了多级过滤过程。该过程结合基于文本描述和音频嵌入的相似性。此外，我们提供了预计算CLAP嵌入与DISCO-10 M，促进了各种下游任务的直接应用。这些嵌入使得能够在所提供的数据上有效地探索机器学习应用。通过DISCO-10 M，我们的目标是推动新研究的民主化和促进，以帮助推动新的音乐机器学习模型的开发。摘要：Music datasets play a crucial role in advancing research in machine learning for music. However, existing music datasets suffer from limited size, accessibility, and lack of audio resources. To address these shortcomings, we present DISCO-10M, a novel and extensive music dataset that surpasses the largest previously available music dataset by an order of magnitude. To ensure high-quality data, we implement a multi-stage filtering process. This process incorporates similarities based on textual descriptions and audio embeddings. Moreover, we provide precomputed CLAP embeddings alongside DISCO-10M, facilitating direct application on various downstream tasks. These embeddings enable efficient exploration of machine learning applications on the provided data. With DISCO-10M, we aim to democratize and facilitate new research to help advance the development of novel machine learning models for music.
eess.AS音频处理
【1】 Towards Effective and Compact Contextual Representation for Conformer  Transducer Speech Recognition Systems标题：面向换能器语音识别系统的高效紧凑上下文表示链接：https://arxiv.org/abs/2306.13307
作者：Mingyu Cui,Jiawen Kang,Jiajun Deng,Xi Yin,Yutao Xie,Xie Chen,Xunying Liu机构：The Chinese University of Hong Kong, Hong Kong SAR, China,  MoE Key Lab of Artificial Intelligence, AI Institute, X-LANCE Lab, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China,  Peng Cheng Laboratory, Shenzhen, China备注：Accepted by INTERSPEECH 2023摘要：目前的ASR系统主要在话语水平上进行训练和评估。可以并入长距离交叉话语上下文。一个关键的任务是得到一个合适的紧凑表示最相关的历史背景。与以前的研究相比，基于LSTM-RNN编码的历史，衰减来自较长范围上下文的信息，或Transformer上下文嵌入的帧级级联，在本文中，紧凑的低维交叉话语上下文特征在Conformer-Transducer编码器中使用专门设计的注意力池层，该注意力池层应用于高效缓存的先前话语历史向量。1000小时Gigaspeech语料库上的实验表明，所提出的上下文化流Conformer-Transducers优于基线使用话语内部上下文只有统计显着的WER减少0.7%至0.5%绝对（4.3%至3.1%相对）的开发和测试数据。摘要：Current ASR systems are mainly trained and evaluated at the utterance level. Long range cross utterance context can be incorporated. A key task is to derive a suitable compact representation of the most relevant history contexts. In contrast to previous researches based on either LSTM-RNN encoded histories that attenuate the information from longer range contexts, or frame level concatenation of transformer context embeddings, in this paper compact low-dimensional cross utterance contextual features are learned in the Conformer-Transducer Encoder using specially designed attention pooling layers that are applied over efficiently cached preceding utterances history vectors. Experiments on the 1000-hr Gigaspeech corpus demonstrate that the proposed contextualized streaming Conformer-Transducers outperform the baseline using utterance internal context only with statistically significant WER reductions of 0.7% to 0.5% absolute (4.3% to 3.1% relative) on the dev and test data.
【2】 TACOformer:Token-channel compounded Cross Attention for Multimodal  Emotion Recognition标题：TACOform：用于多通道情绪识别的令牌-通道复合交叉注意链接：https://arxiv.org/abs/2306.13592
*与cs.SD语音【1】为同一篇
作者：Xinda Li机构：Beihang University备注：Accepted by IJCAI 2023- AI4TS workshop摘要：近年来，基于生理信号的情感识别已成为一个研究热点。多模态、多通道生理信号的利用显著提高了情感识别系统的性能，由于它们的互补性。然而，有效地整合来自不同模态的情感相关语义信息和捕获模态间的依赖关系仍然是一个具有挑战性的问题。许多现有的多模态融合方法忽略了来自不同模态的多通道信号的令牌到令牌或通道到通道的相关性，这在一定程度上限制了模型的分类能力。在本文中，我们提出了一个综合的视角，多模态融合，集成了通道级和令牌级的跨模态交互。具体而言，我们引入了一个统一的交叉注意模块，称为令牌信道信道COmpound（TACO）交叉注意执行多模态融合，同时模型的通道级和令牌级的模式之间的依赖关系。此外，我们提出了一个2D位置编码方法，以保存有关的脑电信号通道的空间分布的信息，然后我们使用两个Transformer编码器前面的融合模块捕获长期的时间依赖性从脑电信号和外周生理信号，分别。情感数据集DEAP和Dreamer上的受试者独立实验表明，所提出的模型达到了最先进的性能。摘要：Recently, emotion recognition based on physiological signals has emerged as a field with intensive research. The utilization of multi-modal, multi-channel physiological signals has significantly improved the performance of emotion recognition systems, due to their complementarity. However, effectively integrating emotion-related semantic information from different modalities and capturing inter-modal dependencies remains a challenging issue. Many existing multimodal fusion methods ignore either token-to-token or channel-to-channel correlations of multichannel signals from different modalities, which limits the classification capability of the models to some extent. In this paper, we propose a comprehensive perspective of multimodal fusion that integrates channel-level and token-level cross-modal interactions. Specifically, we introduce a unified cross attention module called Token-chAnnel COmpound (TACO) Cross Attention to perform multimodal fusion, which simultaneously models channel-level and token-level dependencies between modalities. Additionally, we propose a 2D position encoding method to preserve information about the spatial distribution of EEG signal channels, then we use two transformer encoders ahead of the fusion module to capture long-term temporal dependencies from the EEG signal and the peripheral physiological signal, respectively. Subject-independent experiments on emotional dataset DEAP and Dreamer demonstrate that the proposed model achieves state-of-the-art performance.
【3】 DISCO-10M: A Large-Scale Music Dataset标题：DISCO-10M：一个大规模的音乐数据集链接：https://arxiv.org/abs/2306.13512
*与cs.SD语音【2】为同一篇
作者：Luca A. Lanzendörfer,Florian Grötschla,Emil Funke,Roger Wattenhofer机构：ETH Zurich, Zurich, Switzerland摘要：音乐数据集在推动音乐机器学习研究方面发挥着至关重要的作用。然而，现有的音乐数据集遭受有限的大小、可访问性和缺乏音频资源。为了解决这些缺点，我们提出了DISCO-10 M，一个新颖的和广泛的音乐数据集，超过了以前最大的音乐数据集的数量级。为了确保高质量的数据，我们实施了多级过滤过程。该过程结合基于文本描述和音频嵌入的相似性。此外，我们提供了预计算CLAP嵌入与DISCO-10 M，促进了各种下游任务的直接应用。这些嵌入使得能够在所提供的数据上有效地探索机器学习应用。通过DISCO-10 M，我们的目标是推动新研究的民主化和促进，以帮助推动新的音乐机器学习模型的开发。摘要：Music datasets play a crucial role in advancing research in machine learning for music. However, existing music datasets suffer from limited size, accessibility, and lack of audio resources. To address these shortcomings, we present DISCO-10M, a novel and extensive music dataset that surpasses the largest previously available music dataset by an order of magnitude. To ensure high-quality data, we implement a multi-stage filtering process. This process incorporates similarities based on textual descriptions and audio embeddings. Moreover, we provide precomputed CLAP embeddings alongside DISCO-10M, facilitating direct application on various downstream tasks. These embeddings enable efficient exploration of machine learning applications on the provided data. With DISCO-10M, we aim to democratize and facilitate new research to help advance the development of novel machine learning models for music.
【4】 A Reference-less Quality Metric for Automatic Speech Recognition via  Contrastive-Learning of a Multi-Language Model with Self-Supervision标题：一种基于多语言模型对比学习的无参考质量自动语音识别方法链接：https://arxiv.org/abs/2306.13114
作者：Kamer Ali Yuksel,Thiago Ferreira,Ahmet Gunduz,Mohamed Al-Badrashiny,Golara Javadi机构：aiXplain Inc., Los Gatos, CA, USA备注：arXiv admin note: substantial text overlap with arXiv:2306.12577摘要：用于自动语音识别（ASR）系统的质量评估的通用标准是基于参考的度量，诸如词错误率（WER），其使用手动地面实况转录来计算，其耗时且获得昂贵。这项工作提出了一个多语言的referenceless质量度量，它允许比较不同的ASR模型的性能在没有地面实况转录的语音数据集。为了估计ASR假设的质量，预训练的语言模型（LM）微调与对比学习在自我监督学习的方式。在几个看不见的测试数据集组成的输出从顶级商业ASR引擎在各种语言，建议的参考度量获得了更高的相关性与WER分数和他们的排名比复杂度度量从国家的最先进的多语言LM在所有实验，也减少了WER超过7%$时，用于集成假设。微调模型和实验可用于再现性：https://github.com/aixplain/NoRefER摘要：The common standard for quality evaluation of automatic speech recognition (ASR) systems is reference-based metrics such as the Word Error Rate (WER), computed using manual ground-truth transcriptions that are time-consuming and expensive to obtain. This work proposes a multi-language referenceless quality metric, which allows comparing the performance of different ASR models on a speech dataset without ground truth transcriptions. To estimate the quality of ASR hypotheses, a pre-trained language model (LM) is fine-tuned with contrastive learning in a self-supervised learning manner. In experiments conducted on several unseen test datasets consisting of outputs from top commercial ASR engines in various languages, the proposed referenceless metric obtains a much higher correlation with WER scores and their ranks than the perplexity metric from the state-of-art multi-lingual LM in all experiments, and also reduces WER by more than $7\%$ when used for ensembling hypotheses. The fine-tuned model and experiments are made available for the reproducibility: https://github.com/aixplain/NoRefER
机器翻译，仅供参考