今日论文合集：cs.SD语音8篇，eess.AS音频处理9篇。【1】 ASR Bundestag: A Large-Scale political debate dataset in German
标题：ASR Bundestag：德国大规模政治辩论数据集
链接：https://arxiv.org/abs/2302.06008
作者：Johannes Wirth,René Peinl机构：University of Applied Sciences Hof, Alfons-Goppel-Platz , Hof, Germany备注：13 pages, 2 tables, 4 figures摘要：我们介绍了ASR Bundestag，这是一个用于德语自动语音识别的数据集，包括用于监督训练的610小时对齐的音频-转录本对，以及用于自我监督学习的1，038小时未标记的音频片段，这些数据基于德国议会全体会议和委员会会议的原始音频数据和转录本。此外，我们还讨论了自动创建语音数据集的方法，并根据预先训练的最新模型的评估和微调来评估所得数据集的质量。我们公开数据集，包括所有子集。摘要：We present ASR Bundestag, a dataset for automatic speech recognition in German, consisting of 610 hours of aligned audio-transcript pairs for supervised training as well as 1,038 hours of unlabeled audio snippets for self-supervised learning, based on raw audio data and transcriptions from plenary sessions and committee meetings of the German parliament. In addition, we discuss utilized approaches for the automated creation of speech datasets and assess the quality of the resulting dataset based on evaluations and finetuning of a pre-trained state of the art model. We make the dataset publicly available, including all subsets. 【2】 SemanticAC: Semantics-Assisted Framework for Audio Classification
标题：语义辅助音频分类框架--语义AC
链接：https://arxiv.org/abs/2302.05940
作者：Yicheng Xiao,Yue Ma,Shuyan Li,Hantao Zhou,Ran Liao,Xiu Li机构：Tsinghua Shenzhen International Graduate School, Tsinghua University, China摘要：为了更好地利用音频分类中的语义信息，本文提出了一个语义辅助的音频分类框架SemanticAC。不同于传统的音频分类方法将类别标签视为离散向量，我们使用语言模型从标签中提取丰富的语义，并优化音频信号与其标签之间的语义一致性。我们验证了来自标签的简单文本信息和高级预训练模型能够实现更丰富的语义监督以获得更好的性能。具体地说，我们设计了一个文本编码器，从标签的文本扩展中获取语义信息。然后通过音频编码器和相似度计算模块将音频信号映射到对应类别标签的语义上，以增强语义一致性。在ESC-50和US 8 K两个音频数据集上的实验结果表明，本文提出的方法在分类性能上优于同类方法.摘要：In this paper, we propose SemanticAC, a semantics-assisted framework for Audio Classification to better leverage the semantic information. Unlike conventional audio classification methods that treat class labels as discrete vectors, we employ a language model to extract abundant semantics from labels and optimize the semantic consistency between audio signals and their labels. We verify that simple textual information from labels and advanced pretraining models enable more abundant semantic supervision for better performance. Specifically, we design a text encoder to capture the semantic information from the text extension of labels. Then we map the audio signals to align with the semantics of corresponding class labels via an audio encoder and a similarity calculation module so as to enforce the semantic consistency. Extensive experiments on two audio datasets, ESC-50 and US8K demonstrate that our proposed method consistently outperforms the compared audio classification methods.【3】 Parameterizable Acoustical Modeling and Auralization of Cultural Heritage Sites based on Photogrammetry
标题：基于摄影测量的文化遗址可参数声学建模与可听化
链接：https://arxiv.org/abs/2302.05725
机构：Research Group DIGITAL ORGANOLOGY, Leipzig University, Leipzig (Germany)备注：6 pages, 3 figures, 27th Conference on Cultural Heritage and New Technologies (Vienna, 2022)摘要：文化遗址的摄影测量和重建建模主要集中在视觉感知方面，但如果其预期目的是以声音为重点的文化行为的表现，则重要的是考虑保护其声学行为，使其以真实的方式可听。这尤其适用于作为摄影测量模型的流行对象的祭祀和音乐会环境，其包含可用于定位和分类声学相关表面特性的几何和纹理信息。随着历史声学空间转换或破坏的不断深入，保留其独特的声学特征变得越来越重要，而三维可听化也得到了广泛的应用。拟议的研究报告介绍了利用摄影测量数据进行声学建模的新方法的现状，并介绍了一个可参数化的管道，该管道将作为一个具有图形用户界面的开放源码软件提供。摘要：The photogrammetric and reconstructive modeling of cultural heritage sites is mostly focused on visually perceivable aspects, but if their intended purpose is the performance of cultural acts with a sonic emphasis, it is important to consider the preservation of their acoustical behaviour to make them audible in an authentic way. This applies in particular to sacral and concert environments as popular objects for photogrammetric models, which contain geometrical and textural information that can be used to locate and classify acoustically relevant surface properties. With the advancing conversion or destruction of historical acoustical spaces, it becomes even more important to preserve their unique sonic characters, while three-dimensional auralizations become widely applicable. The proposed study presents the current state of a new methodological approach to acoustical modeling using photogrammetric data and introduces a parameterizable pipeline that will be accessible as an open-source software with a graphical user interface.【4】 Local spectral attention for full-band speech enhancement
标题：全频带语音增强中的局部谱注意
链接：https://arxiv.org/abs/2302.05693
作者：Zhongshu Hou,Qinwen Hu,Kai Chen,Jing Lu机构：Key Laboratory of Modern Acoustics, Nanjing University, Nanjing , China, NJU-Horizon Intelligent Audio Lab, Horizon Robotics, Beijing , China, Nanjing Institute of Advanced Artificial Intelligence, Nanjing , China摘要：注意力机制在语音增强中得到了广泛的应用，因为它在理论上可以有效地在时域和谱域模拟信号的内在联系。通常情况下，注意力的广度在时域上是有限的，而在频域上的注意力则跨越整个频率范围。在本文中，我们注意到对整个频率范围的关注阻碍了对全带SE的推断，并可能导致过量的残留噪声。为了解决这一问题，通过限制注意广度，将局部谱注意（LSA）引入到全频带SE模型中。在最新的SOTA全频带SE模型上进行的烧蚀测试表明，局部频率关注可以有效地提高整体性能。改进后的模型在全频带VoiceBank+DEMAND集合上取得了最好的客观评分。摘要：Attention mechanism has been widely utilized in speech enhancement (SE) because theoretically it can effectively model the inherent connection of signal both in time domain and spectrum domain. Usually, the span of attention is limited in time domain while the attention in frequency domain spans the whole frequency range. In this paper, we notice that the attention over the whole frequency range hampers the inference for full-band SE and possibly leads to excessive residual noise. To alleviate this problem, we introduce local spectral attention (LSA) into full-band SE model by limiting the span of attention. The ablation test on the state-of-the-art (SOTA) full-band SE model reveals that the local frequency attention can effectively improve overall performance. The improved model achieves the best objective score on the full-band VoiceBank+DEMAND set.【5】 Attention does not guarantee best performance in speech enhancement
标题：注意力不能保证语音增强的最佳性能
链接：https://arxiv.org/abs/2302.05690
作者：Zhongshu Hou,Qinwen Hu,Kai Chen,Jing Lu机构：Key Laboratory of Modern Acoustics, Nanjing University, Nanjing , China, NJU-Horizon Intelligent Audio Lab, Horizon Robotics, Beijing , China, Nanjing Institute of Advanced Artificial Intelligence, Nanjing , China摘要：注意力机制在语音增强中得到了广泛的应用，因为它在理论上可以有效地在时域和谱域模拟信号的长期内在联系。然而，通常使用的全局注意机制可能不是最好的选择，因为在语音增强中，相邻信息自然比相距较远的信息施加更大的影响。本文通过将注意力替换为RNN，在两个典型的SOTA模型--带轴向注意力的多尺度时频卷积网络（MTFAA）和基于一致性的度量GAN网络（CMGAN）中验证了这一猜想.摘要：Attention mechanism has been widely utilized in speech enhancement (SE) because theoretically it can effectively model the long-term inherent connection of signal both in time domain and spectrum domain. However, the generally used global attention mechanism might not be the best choice since the adjacent information naturally imposes more influence than the far-apart information in speech enhancement. In this paper, we validate this conjecture by replacing attention with RNN in two typical state-of-the-art (SOTA) models, multi-scale temporal frequency convolutional network (MTFAA) with axial attention and conformer-based metric-GAN network (CMGAN).【6】 Fast and small footprint Hybrid HMM-HiFiGAN based system for speech synthesis in Indian languages
标题：基于混合HMM-HiFiGAN的印度语语音合成系统
链接：https://arxiv.org/abs/2302.06227
作者：Sudhanshu Srivastava,Ishika Gupta,Anusha Prakash,Jom Kuriakose,Hema A. Murthy机构：Department of Computer Science and Engineering, Indian Institute of Technology, Madras, Department of Electrical Engineering, Indian Institute of Technology Madras摘要：基于隐马尔可夫模型（HMM）的文本到语音（HTS）提供说话风格的灵活性以及快速训练和合成，同时计算强度较低。HTS即使在资源不足的情况下也表现良好。主要缺点是语音质量与E2 E系统的语音质量相比较差。提出了一种基于HMM的特征生成和基于神经网络的HiFi-GAN声码器相结合的混合方法来提高HTS合成质量。HTS是在高分辨率梅尔谱图上而不是在常规梅尔广义系数（MGC）上训练的，并且对应于输入文本的输出梅尔谱图被用在在印度语上训练的HiFi-GAN声码器中，以产生与E2 E系统的自然度等效的自然度，如从DMOS和PC测试所证明的。摘要：Hidden-Markov-model (HMM) based text-to-speech (HTS) offers flexibility in speaking styles along with fast training and synthesis while being computationally less intense. HTS performs well even in low-resource scenarios. The primary drawback is that the voice quality is poor compared to that of E2E systems. A hybrid approach combining HMM-based feature generation and neural-network-based HiFi-GAN vocoder to improve HTS synthesis quality is proposed. HTS is trained on high-resolution mel-spectrograms instead of conventional mel generalized coefficients (MGC), and the output mel-spectrogram corresponding to the input text is used in a HiFi-GAN vocoder trained on Indic languages, to produce naturalness that is equivalent to that of E2E systems, as evidenced from the DMOS and PC tests. 【7】 Improved Decoding of Attentional Selection in Multi-Talker Environments with Self-Supervised Learned Speech Representation
标题：基于自监督学习语音表征的多人环境下注意选择译码改进
链接：https://arxiv.org/abs/2302.05756
作者：Cong Han,Vishal Choudhari,Yinghao Aaron Li,Nima Mesgarani机构：1Department of Electrical Engineering, Columbia University摘要：听觉注意解码（AAD）是一种用于在嘈杂环境中识别和放大听者所关注的说话者的技术。这是通过将听者的脑电波与所有声源的代表进行比较，以找到最接近的匹配来完成的。该表示通常是声音的波形或声谱图。这些陈述对AAD的有效性尚不确定。在本研究中，我们检验了使用自监督学习语音表征来提高AAD的准确性和速度。我们使用侵入性皮层电图记录了三个受试者在听两段对话并专注于一段对话时的大脑活动。我们使用WavLM来提取每个说话者的潜在表征，并训练时空滤波器来将大脑活动映射到语音的中间表征。在评估期间，将重构的表示与每个说话者的表示进行比较以确定目标说话者。实验结果表明，小波线性模型的语音表示比语音包络和语音谱图具有更好的解码精度和速度。我们的研究结果证明了自监督学习语音表征在听觉注意解码中的优势，为开发脑控听觉技术奠定了基础。摘要：Auditory attention decoding (AAD) is a technique used to identify and amplify the talker that a listener is focused on in a noisy environment. This is done by comparing the listener's brainwaves to a representation of all the sound sources to find the closest match. The representation is typically the waveform or spectrogram of the sounds. The effectiveness of these representations for AAD is uncertain. In this study, we examined the use of self-supervised learned speech representation in improving the accuracy and speed of AAD. We recorded the brain activity of three subjects using invasive electrocorticography (ECoG) as they listened to two conversations and focused on one. We used WavLM to extract a latent representation of each talker and trained a spatiotemporal filter to map brain activity to intermediate representations of speech. During the evaluation, the reconstructed representation is compared to each speaker's representation to determine the target speaker. Our results indicate that speech representation from WavLM provides better decoding accuracy and speed than the speech envelope and spectrogram. Our findings demonstrate the advantages of self-supervised learned speech representation for auditory attention decoding and pave the way for developing brain-controlled hearable technologies.【8】 ASDF: A Differential Testing Framework for Automatic Speech Recognition Systems
标题：ASDF：一种用于自动语音识别系统的差分测试框架
链接：https://arxiv.org/abs/2302.05582
作者：Daniel Hao Xian Yuen,Andrew Yong Chen Pang,Zhou Yang,Chun Yong Chong,Mei Kuan Lim,David Lo机构：∗School of Information Technology, Monash University Malaysia, †School of Computing and Information Systems, Singapore Management University备注：Accpeted by ICST 2023 Tool Demo Track摘要：近年来，自动语音识别（ASR）技术在各个领域得到了广泛的应用。因此，评估和提高ASR系统的质量具有重要意义。本文提出了一个用于测试ASR系统的自动语音识别差分测试框架ASDF。ASDF扩展了现有的ASR测试工具CrossASR ++，它从文本语料库中合成测试用例。然而，CrossASR ++未能有效地利用文本语料库，并且提供了关于失败的测试用例如何改进ASR系统的有限信息。为了解决这些局限性，我们的工具整合了两个新功能：（1）文本转换模块，用于增加所生成的测试实例的数量并揭示ASR系统中的更多错误，以及（2）语音分析模块，用于识别ASR系统倾向于在哪些音素上产生错误。ASDF通过应用各种文本转换方法（例如，改变时态）到失败测试用例中的文本。通过这样做，ASDF可以利用一个小的文本语料库来生成大量的音频测试用例，这是CrossASR ++所不能做到的。此外，ASDF实现了更多的度量标准，从多个角度评估ASR系统的性能。ASDF对识别出的失败测试用例执行语音分析，以识别ASR系统倾向于错误转录的音素，为开发人员改进ASR系统提供有用的信息。我们的工具的演示视频在线制作，网址为www.example.comhttps://www.youtube.com/watch? v = DzVwfc3h9As。该实现可在www.example.com上获得。https://github.com/danielyuenhx/asdf-differential-testing.摘要：Recent years have witnessed wider adoption of Automated Speech Recognition (ASR) techniques in various domains. Consequently, evaluating and enhancing the quality of ASR systems is of great importance. This paper proposes ASDF, an Automated Speech Recognition Differential Testing Framework for testing ASR systems. ASDF extends an existing ASR testing tool, the CrossASR++, which synthesizes test cases from a text corpus. However, CrossASR++ fails to make use of the text corpus efficiently and provides limited information on how the failed test cases can improve ASR systems. To address these limitations, our tool incorporates two novel features: (1) a text transformation module to boost the number of generated test cases and uncover more errors in ASR systems and (2) a phonetic analysis module to identify on which phonemes the ASR system tend to produce errors. ASDF generates more high-quality test cases by applying various text transformation methods (e.g., change tense) to the texts in failed test cases. By doing so, ASDF can utilize a small text corpus to generate a large number of audio test cases, something which CrossASR++ is not capable of. In addition, ASDF implements more metrics to evaluate the performance of ASR systems from multiple perspectives. ASDF performs phonetic analysis on the identified failed test cases to identify the phonemes that ASR systems tend to transcribe incorrectly, providing useful information for developers to improve ASR systems. The demonstration video of our tool is made online at https://www.youtube.com/watch?v=DzVwfc3h9As. The implementation is available at https://github.com/danielyuenhx/asdf-differential-testing.【1】 AV-data2vec: Self-supervised Learning of Audio-Visual Speech Representations with Contextualized Target Representations
标题：AV-data2vec：基于语境化目标表征的视听语音表征的自监督学习
链接：https://arxiv.org/abs/2302.06419
作者：Jiachen Lian,Alexei Baevski,Wei-Ning Hsu,Michael Auli机构：The visual information helps to improve the robustness andEqual advising 1University of California摘要：自我监督通过极大地减少构建良好系统所需的标记数据量，在视听语音识别方面显示出巨大的潜力。然而，现有方法不是完全端到端的，或者没有训练两种模态的联合表示。本文介绍了AV-data 2 vec算法，该算法解决了这些问题，并在预测情境化表示的基础上构建了视听表示，该算法在单模态情况下取得了成功。该模型对音频和视频使用共享的Transformer编码器，并且可以结合两种模态来提高语音识别。LRS 3上的结果表明，AV-data 2 vec在大多数设置下始终优于现有方法。摘要：Self-supervision has shown great potential for audio-visual speech recognition by vastly reducing the amount of labeled data required to build good systems. However, existing methods are either not entirely end-to-end or do not train joint representations of both modalities. In this paper, we introduce AV-data2vec which addresses these challenges and builds audio-visual representations based on predicting contextualized representations which has been successful in the uni-modal case. The model uses a shared transformer encoder for both audio and video and can combine both modalities to improve speech recognition. Results on LRS3 show that AV-data2vec consistently outperforms existing methods under most settings.【2】 Fast and small footprint Hybrid HMM-HiFiGAN based system for speech synthesis in Indian languages
标题：基于混合HMM-HiFiGAN的印度语语音合成系统
链接：https://arxiv.org/abs/2302.06227
*与cs.SD语音【6】为同一篇
作者：Sudhanshu Srivastava,Ishika Gupta,Anusha Prakash,Jom Kuriakose,Hema A. Murthy机构：Department of Computer Science and Engineering, Indian Institute of Technology, Madras, Department of Electrical Engineering, Indian Institute of Technology Madras摘要：基于隐马尔可夫模型（HMM）的文本到语音（HTS）提供说话风格的灵活性以及快速训练和合成，同时计算强度较低。HTS即使在资源不足的情况下也表现良好。主要缺点是语音质量与E2 E系统的语音质量相比较差。提出了一种基于HMM的特征生成和基于神经网络的HiFi-GAN声码器相结合的混合方法来提高HTS合成质量。HTS是在高分辨率梅尔谱图上而不是在常规梅尔广义系数（MGC）上训练的，并且对应于输入文本的输出梅尔谱图被用在在印度语上训练的HiFi-GAN声码器中，以产生与E2 E系统的自然度等效的自然度，如从DMOS和PC测试所证明的。摘要：Hidden-Markov-model (HMM) based text-to-speech (HTS) offers flexibility in speaking styles along with fast training and synthesis while being computationally less intense. HTS performs well even in low-resource scenarios. The primary drawback is that the voice quality is poor compared to that of E2E systems. A hybrid approach combining HMM-based feature generation and neural-network-based HiFi-GAN vocoder to improve HTS synthesis quality is proposed. HTS is trained on high-resolution mel-spectrograms instead of conventional mel generalized coefficients (MGC), and the output mel-spectrogram corresponding to the input text is used in a HiFi-GAN vocoder trained on Indic languages, to produce naturalness that is equivalent to that of E2E systems, as evidenced from the DMOS and PC tests.【3】 Improved Decoding of Attentional Selection in Multi-Talker Environments with Self-Supervised Learned Speech Representation
标题：基于自监督学习语音表征的多人环境下注意选择译码改进
链接：https://arxiv.org/abs/2302.05756
*与cs.SD语音【7】为同一篇
作者：Cong Han,Vishal Choudhari,Yinghao Aaron Li,Nima Mesgarani机构：1Department of Electrical Engineering, Columbia University摘要：听觉注意解码（AAD）是一种用于在嘈杂环境中识别和放大听者所关注的说话者的技术。这是通过将听者的脑电波与所有声源的代表进行比较，以找到最接近的匹配来完成的。该表示通常是声音的波形或声谱图。这些陈述对AAD的有效性尚不确定。在本研究中，我们检验了使用自监督学习语音表征来提高AAD的准确性和速度。我们使用侵入性皮层电图记录了三个受试者在听两段对话并专注于一段对话时的大脑活动。我们使用WavLM来提取每个说话者的潜在表征，并训练时空滤波器来将大脑活动映射到语音的中间表征。在评估期间，将重构的表示与每个说话者的表示进行比较以确定目标说话者。实验结果表明，小波线性模型的语音表示比语音包络和语音谱图具有更好的解码精度和速度。我们的研究结果证明了自监督学习语音表征在听觉注意解码中的优势，为开发脑控听觉技术奠定了基础。摘要：Auditory attention decoding (AAD) is a technique used to identify and amplify the talker that a listener is focused on in a noisy environment. This is done by comparing the listener's brainwaves to a representation of all the sound sources to find the closest match. The representation is typically the waveform or spectrogram of the sounds. The effectiveness of these representations for AAD is uncertain. In this study, we examined the use of self-supervised learned speech representation in improving the accuracy and speed of AAD. We recorded the brain activity of three subjects using invasive electrocorticography (ECoG) as they listened to two conversations and focused on one. We used WavLM to extract a latent representation of each talker and trained a spatiotemporal filter to map brain activity to intermediate representations of speech. During the evaluation, the reconstructed representation is compared to each speaker's representation to determine the target speaker. Our results indicate that speech representation from WavLM provides better decoding accuracy and speed than the speech envelope and spectrogram. Our findings demonstrate the advantages of self-supervised learned speech representation for auditory attention decoding and pave the way for developing brain-controlled hearable technologies.【4】 ASDF: A Differential Testing Framework for Automatic Speech Recognition Systems
标题：ASDF：一种用于自动语音识别系统的差分测试框架
链接：https://arxiv.org/abs/2302.05582
*与cs.SD语音【8】为同一篇
作者：Daniel Hao Xian Yuen,Andrew Yong Chen Pang,Zhou Yang,Chun Yong Chong,Mei Kuan Lim,David Lo机构：∗School of Information Technology, Monash University Malaysia, †School of Computing and Information Systems, Singapore Management University备注：Accpeted by ICST 2023 Tool Demo Track摘要：近年来，自动语音识别（ASR）技术在各个领域得到了广泛的应用。因此，评估和提高ASR系统的质量具有重要意义。本文提出了一个用于测试ASR系统的自动语音识别差分测试框架ASDF。ASDF扩展了现有的ASR测试工具CrossASR ++，它从文本语料库中合成测试用例。然而，CrossASR ++未能有效地利用文本语料库，并且提供了关于失败的测试用例如何改进ASR系统的有限信息。为了解决这些局限性，我们的工具整合了两个新功能：（1）文本转换模块，用于增加所生成的测试实例的数量并揭示ASR系统中的更多错误，以及（2）语音分析模块，用于识别ASR系统倾向于在哪些音素上产生错误。ASDF通过应用各种文本转换方法（例如，改变时态）到失败测试用例中的文本。通过这样做，ASDF可以利用一个小的文本语料库来生成大量的音频测试用例，这是CrossASR ++所不能做到的。此外，ASDF实现了更多的度量标准，从多个角度评估ASR系统的性能。ASDF对识别出的失败测试用例执行语音分析，以识别ASR系统倾向于错误转录的音素，为开发人员改进ASR系统提供有用的信息。我们的工具的演示视频在线制作，网址为www.example.comhttps://www.youtube.com/watch? v = DzVwfc3h9As。该实现可在www.example.com上获得。https://github.com/danielyuenhx/asdf-differential-testing.摘要：Recent years have witnessed wider adoption of Automated Speech Recognition (ASR) techniques in various domains. Consequently, evaluating and enhancing the quality of ASR systems is of great importance. This paper proposes ASDF, an Automated Speech Recognition Differential Testing Framework for testing ASR systems. ASDF extends an existing ASR testing tool, the CrossASR++, which synthesizes test cases from a text corpus. However, CrossASR++ fails to make use of the text corpus efficiently and provides limited information on how the failed test cases can improve ASR systems. To address these limitations, our tool incorporates two novel features: (1) a text transformation module to boost the number of generated test cases and uncover more errors in ASR systems and (2) a phonetic analysis module to identify on which phonemes the ASR system tend to produce errors. ASDF generates more high-quality test cases by applying various text transformation methods (e.g., change tense) to the texts in failed test cases. By doing so, ASDF can utilize a small text corpus to generate a large number of audio test cases, something which CrossASR++ is not capable of. In addition, ASDF implements more metrics to evaluate the performance of ASR systems from multiple perspectives. ASDF performs phonetic analysis on the identified failed test cases to identify the phonemes that ASR systems tend to transcribe incorrectly, providing useful information for developers to improve ASR systems. The demonstration video of our tool is made online at https://www.youtube.com/watch?v=DzVwfc3h9As. The implementation is available at https://github.com/danielyuenhx/asdf-differential-testing.【5】 ASR Bundestag: A Large-Scale political debate dataset in German
标题：ASR Bundestag：德国大规模政治辩论数据集
链接：https://arxiv.org/abs/2302.06008
*与cs.SD语音【1】为同一篇
作者：Johannes Wirth,René Peinl机构：University of Applied Sciences Hof, Alfons-Goppel-Platz , Hof, Germany备注：13 pages, 2 tables, 4 figures摘要：我们介绍了ASR Bundestag，这是一个用于德语自动语音识别的数据集，包括用于监督训练的610小时对齐的音频-转录本对，以及用于自我监督学习的1，038小时未标记的音频片段，这些数据基于德国议会全体会议和委员会会议的原始音频数据和转录本。此外，我们还讨论了自动创建语音数据集的方法，并根据预先训练的最新模型的评估和微调来评估所得数据集的质量。我们公开数据集，包括所有子集。摘要：We present ASR Bundestag, a dataset for automatic speech recognition in German, consisting of 610 hours of aligned audio-transcript pairs for supervised training as well as 1,038 hours of unlabeled audio snippets for self-supervised learning, based on raw audio data and transcriptions from plenary sessions and committee meetings of the German parliament. In addition, we discuss utilized approaches for the automated creation of speech datasets and assess the quality of the resulting dataset based on evaluations and finetuning of a pre-trained state of the art model. We make the dataset publicly available, including all subsets.【6】 SemanticAC: Semantics-Assisted Framework for Audio Classification
标题：语义辅助音频分类框架--语义AC
链接：https://arxiv.org/abs/2302.05940
*与cs.SD语音【2】为同一篇
作者：Yicheng Xiao,Yue Ma,Shuyan Li,Hantao Zhou,Ran Liao,Xiu Li机构：Tsinghua Shenzhen International Graduate School, Tsinghua University, China摘要：为了更好地利用音频分类中的语义信息，本文提出了一个语义辅助的音频分类框架SemanticAC。不同于传统的音频分类方法将类别标签视为离散向量，我们使用语言模型从标签中提取丰富的语义，并优化音频信号与其标签之间的语义一致性。我们验证了来自标签的简单文本信息和高级预训练模型能够实现更丰富的语义监督以获得更好的性能。具体地说，我们设计了一个文本编码器，从标签的文本扩展中获取语义信息。然后通过音频编码器和相似度计算模块将音频信号映射到对应类别标签的语义上，以增强语义一致性。在ESC-50和US 8 K两个音频数据集上的实验结果表明，本文提出的方法在分类性能上优于同类方法.摘要：In this paper, we propose SemanticAC, a semantics-assisted framework for Audio Classification to better leverage the semantic information. Unlike conventional audio classification methods that treat class labels as discrete vectors, we employ a language model to extract abundant semantics from labels and optimize the semantic consistency between audio signals and their labels. We verify that simple textual information from labels and advanced pretraining models enable more abundant semantic supervision for better performance. Specifically, we design a text encoder to capture the semantic information from the text extension of labels. Then we map the audio signals to align with the semantics of corresponding class labels via an audio encoder and a similarity calculation module so as to enforce the semantic consistency. Extensive experiments on two audio datasets, ESC-50 and US8K demonstrate that our proposed method consistently outperforms the compared audio classification methods.【7】 Parameterizable Acoustical Modeling and Auralization of Cultural Heritage Sites based on Photogrammetry
标题：基于摄影测量的文化遗址可参数声学建模与可听化
链接：https://arxiv.org/abs/2302.05725
*与cs.SD语音【3】为同一篇
机构：Research Group DIGITAL ORGANOLOGY, Leipzig University, Leipzig (Germany)备注：6 pages, 3 figures, 27th Conference on Cultural Heritage and New Technologies (Vienna, 2022)摘要：文化遗址的摄影测量和重建建模主要集中在视觉感知方面，但如果其预期目的是以声音为重点的文化行为的表现，则重要的是考虑保护其声学行为，使其以真实的方式可听。这尤其适用于作为摄影测量模型的流行对象的祭祀和音乐会环境，其包含可用于定位和分类声学相关表面特性的几何和纹理信息。随着历史声学空间转换或破坏的不断深入，保留其独特的声学特征变得越来越重要，而三维可听化也得到了广泛的应用。拟议的研究报告介绍了利用摄影测量数据进行声学建模的新方法的现状，并介绍了一个可参数化的管道，该管道将作为一个具有图形用户界面的开放源码软件提供。摘要：The photogrammetric and reconstructive modeling of cultural heritage sites is mostly focused on visually perceivable aspects, but if their intended purpose is the performance of cultural acts with a sonic emphasis, it is important to consider the preservation of their acoustical behaviour to make them audible in an authentic way. This applies in particular to sacral and concert environments as popular objects for photogrammetric models, which contain geometrical and textural information that can be used to locate and classify acoustically relevant surface properties. With the advancing conversion or destruction of historical acoustical spaces, it becomes even more important to preserve their unique sonic characters, while three-dimensional auralizations become widely applicable. The proposed study presents the current state of a new methodological approach to acoustical modeling using photogrammetric data and introduces a parameterizable pipeline that will be accessible as an open-source software with a graphical user interface.【8】 Local spectral attention for full-band speech enhancement
标题：全频带语音增强中的局部谱注意
链接：https://arxiv.org/abs/2302.05693
*与cs.SD语音【4】为同一篇
作者：Zhongshu Hou,Qinwen Hu,Kai Chen,Jing Lu机构：Key Laboratory of Modern Acoustics, Nanjing University, Nanjing , China, NJU-Horizon Intelligent Audio Lab, Horizon Robotics, Beijing , China, Nanjing Institute of Advanced Artificial Intelligence, Nanjing , China摘要：注意力机制在语音增强中得到了广泛的应用，因为它在理论上可以有效地在时域和谱域模拟信号的内在联系。通常情况下，注意力的广度在时域上是有限的，而在频域上的注意力则跨越整个频率范围。在本文中，我们注意到对整个频率范围的关注阻碍了对全带SE的推断，并可能导致过量的残留噪声。为了解决这一问题，通过限制注意广度，将局部谱注意（LSA）引入到全频带SE模型中。在最新的SOTA全频带SE模型上进行的烧蚀测试表明，局部频率关注可以有效地提高整体性能。改进后的模型在全频带VoiceBank+DEMAND集合上取得了最好的客观评分。摘要：Attention mechanism has been widely utilized in speech enhancement (SE) because theoretically it can effectively model the inherent connection of signal both in time domain and spectrum domain. Usually, the span of attention is limited in time domain while the attention in frequency domain spans the whole frequency range. In this paper, we notice that the attention over the whole frequency range hampers the inference for full-band SE and possibly leads to excessive residual noise. To alleviate this problem, we introduce local spectral attention (LSA) into full-band SE model by limiting the span of attention. The ablation test on the state-of-the-art (SOTA) full-band SE model reveals that the local frequency attention can effectively improve overall performance. The improved model achieves the best objective score on the full-band VoiceBank+DEMAND set.【9】 Attention does not guarantee best performance in speech enhancement
标题：注意力不能保证语音增强的最佳性能
链接：https://arxiv.org/abs/2302.05690
*与cs.SD语音【5】为同一篇
作者：Zhongshu Hou,Qinwen Hu,Kai Chen,Jing Lu机构：Key Laboratory of Modern Acoustics, Nanjing University, Nanjing , China, NJU-Horizon Intelligent Audio Lab, Horizon Robotics, Beijing , China, Nanjing Institute of Advanced Artificial Intelligence, Nanjing , China摘要：注意力机制在语音增强中得到了广泛的应用，因为它在理论上可以有效地在时域和谱域模拟信号的长期内在联系。然而，通常使用的全局注意机制可能不是最好的选择，因为在语音增强中，相邻信息自然比相距较远的信息施加更大的影响。本文通过将注意力替换为RNN，在两个典型的SOTA模型--带轴向注意力的多尺度时频卷积网络（MTFAA）和基于一致性的度量GAN网络（CMGAN）中验证了这一猜想.摘要：Attention mechanism has been widely utilized in speech enhancement (SE) because theoretically it can effectively model the long-term inherent connection of signal both in time domain and spectrum domain. However, the generally used global attention mechanism might not be the best choice since the adjacent information naturally imposes more influence than the far-apart information in speech enhancement. In this paper, we validate this conjecture by replacing attention with RNN in two typical state-of-the-art (SOTA) models, multi-scale temporal frequency convolutional network (MTFAA) with axial attention and conformer-based metric-GAN network (CMGAN).机器翻译，仅供参考
每日语音速递