今日论文合集：cs.SD语音15篇，eess.AS音频处理12篇。
cs.SD语音
【1】 Systematic Analysis of Music Representations from BERT标题：BERT音乐表现手法的系统分析链接：https://arxiv.org/abs/2306.04628
作者：Sangjun Han,Hyeongrae Ihm,Woohyung Lim机构：LG AI Research摘要：已经有许多尝试将原始数据表示为有效地捕获语义和上下文信息的数字向量。然而，在符号音乐领域，以前的作品试图通过观察各种微调任务的性能改善来验证他们的音乐嵌入。在这项工作中，我们直接分析嵌入从BERT和BERT与对比学习培训的酒吧级MIDI，检查他们的音乐信息，可以从MIDI事件。我们观察到的嵌入表现出不同的信息特征，这取决于对比的目标和层的选择。我们的代码可在www.example.com上获得https://github.com/sjhan91/MusicBERT。摘要：There have been numerous attempts to represent raw data as numerical vectors that effectively capture semantic and contextual information. However, in the field of symbolic music, previous works have attempted to validate their music embeddings by observing the performance improvement of various fine-tuning tasks. In this work, we directly analyze embeddings from BERT and BERT with contrastive learning trained on bar-level MIDI, inspecting their musical information that can be obtained from MIDI events. We observe that the embeddings exhibit distinct characteristics of information depending on the contrastive objectives and the choice of layers. Our code is available at https://github.com/sjhan91/MusicBERT.
【2】 Zambezi Voice: A Multilingual Speech Corpus for Zambian Languages标题：赞比西亚语：赞比亚语言的多语种语音语料库链接：https://arxiv.org/abs/2306.04428
作者：Claytone Sikasote,Kalinda Siaminwe,Stanly Mwape,Bangiwe Zulu,Mofya Phiri,Martin Phiri,David Zulu,Mayumbo Nyirenda,Antonios Anastasopoulos机构：University of Zambia, Zambia, George Mason University, U.S.A备注：Accepted at INTERSPEECH 2023摘要：这项工作介绍了赞比西之声，赞比亚语言的开源多语言语音资源。它包含两个数据集集合：无线电新闻和脱口秀节目的未标记的音频记录（160小时）和标记的数据（超过80小时），所述标记的数据包括从来源于公开可用的文学书籍的文本记录的阅读语音。该数据集是为语音识别创建的，但可以扩展到多语言语音处理研究，用于监督和无监督学习方法。据我们所知，这是为赞比亚语言创建的第一个多语言语音数据集。我们通过微调Wav2Vec2.0大规模多语言预训练模型来利用预训练和跨语言迁移学习，为我们的基线模型构建端到端（E2 E）语音识别模型。该数据集在Creative Commons BY-NC-ND 4.0许可下公开发布，可以通过项目存储库访问。参见https://github.com/unza-speech-lab/zambezi-voice摘要：This work introduces Zambezi Voice, an open-source multilingual speech resource for Zambian languages. It contains two collections of datasets: unlabelled audio recordings of radio news and talk shows programs (160 hours) and labelled data (over 80 hours) consisting of read speech recorded from text sourced from publicly available literature books. The dataset is created for speech recognition but can be extended to multilingual speech processing research for both supervised and unsupervised learning approaches. To our knowledge, this is the first multilingual speech dataset created for Zambian languages. We exploit pretraining and cross-lingual transfer learning by finetuning the Wav2Vec2.0 large-scale multilingual pre-trained model to build end-to-end (E2E) speech recognition models for our baseline models. The dataset is released publicly under a Creative Commons BY-NC-ND 4.0 license and can be accessed through the project repository. See https://github.com/unza-speech-lab/zambezi-voice
【3】 Label Aware Speech Representation Learning For Language Identification标题：标签感知语音表征学习在语言识别中的应用链接：https://arxiv.org/abs/2306.04374
作者：Shikhar Vashishth,Shikhar Bharadwaj,Sriram Ganapathy,Ankur Bapna,Min Ma,Wei Han,Vera Axelrod,Partha Talukdar机构：Google Research备注：Accepted at Interspeech 2023摘要：用于诸如语言识别的非语义任务的语音表示学习方法已经探索了使用分类器模型的监督嵌入提取方法或使用原始数据的自监督表示学习方法。在本文中，我们提出了一个新的框架相结合的自监督表示学习的语言标签信息的预训练任务。这个框架，被称为标签感知语音表示（LASR）学习，使用基于三元组的目标函数，将语言标签与自监督损失函数。针对下游任务进一步微调语音表示。语言识别实验在两个公共数据集上进行- FLEURS和Dhwani。在这些实验中，我们说明了所提出的LASR框架提高了语言识别的最先进的系统。我们还报告了一个分析的鲁棒性LASR方法的噪声/丢失的标签，以及其应用程序的多语言语音识别任务。摘要：Speech representation learning approaches for non-semantic tasks such as language recognition have either explored supervised embedding extraction methods using a classifier model or self-supervised representation learning approaches using raw data. In this paper, we propose a novel framework of combining self-supervised representation learning with the language label information for the pre-training task. This framework, termed as Label Aware Speech Representation (LASR) learning, uses a triplet based objective function to incorporate language labels along with the self-supervised loss function. The speech representations are further fine-tuned for the downstream task. The language recognition experiments are performed on two public datasets - FLEURS and Dhwani. In these experiments, we illustrate that the proposed LASR framework improves over the state-of-the-art systems on language identification. We also report an analysis of the robustness of LASR approach to noisy/missing labels as well as its application to multi-lingual speech recognition tasks.
【4】 Arabic Dysarthric Speech Recognition Using Adversarial and Signal-Based  Augmentation标题：基于对抗性和基于信号增强的阿拉伯文双韵律语音识别链接：https://arxiv.org/abs/2306.04368
作者：Massa Baali,Ibrahim Almakky,Shady Shehata,Fakhri Karray机构：Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE备注：Accepted to Interspeech 2023摘要：尽管自动语音识别（ASR）取得了重大进展，但最先进的ASR系统仍难以处理受损的语音，即使是高资源语言。在阿拉伯语中，这一挑战被放大，从构音障碍的说话者那里收集数据的复杂性增加了。在本文中，我们的目标是提高阿拉伯语构音障碍自动语音识别的性能，通过多阶段增强的方法。为此，我们首先提出了一种基于信号的方法来生成构音障碍的阿拉伯语语音从健康的阿拉伯语语音通过修改其速度和节奏。我们还提出了一个第二阶段的并行波生成（PWG）对抗模型，该模型在英语构音障碍数据集上进行训练，以捕获语言无关的构音障碍语音模式，并进一步增强信号调整后的语音样本。此外，我们提出了一个微调和文本校正策略的阿拉伯语顺应在不同的构音障碍的讲话严重程度。我们的微调构象实现了18%的单词错误率（WER）和17.2%的字符错误率（CER）的合成生成的构音障碍的语音从阿拉伯语的普通语音数据集。这表明与仅在健康数据上训练的基线模型相比，WER显著改善了81.8%。我们对真正的英语构音障碍语音进行进一步验证，与仅在健康英语LJSpeech数据集上训练的基线相比，WER改善了124%。摘要：Despite major advancements in Automatic Speech Recognition (ASR), the state-of-the-art ASR systems struggle to deal with impaired speech even with high-resource languages. In Arabic, this challenge gets amplified, with added complexities in collecting data from dysarthric speakers. In this paper, we aim to improve the performance of Arabic dysarthric automatic speech recognition through a multi-stage augmentation approach. To this effect, we first propose a signal-based approach to generate dysarthric Arabic speech from healthy Arabic speech by modifying its speed and tempo. We also propose a second stage Parallel Wave Generative (PWG) adversarial model that is trained on an English dysarthric dataset to capture language-independant dysarthric speech patterns and further augment the signal-adjusted speech samples. Furthermore, we propose a fine-tuning and text-correction strategies for Arabic Conformer at different dysarthric speech severity levels. Our fine-tuned Conformer achieved 18% Word Error Rate (WER) and 17.2% Character Error Rate (CER) on synthetically generated dysarthric speech from the Arabic commonvoice speech dataset. This shows significant WER improvement of 81.8% compared to the baseline model trained solely on healthy data. We perform further validation on real English dysarthric speech showing a WER improvement of 124% compared to the baseline trained only on healthy English LJSpeech dataset.
【5】 Allophant: Cross-lingual Phoneme Recognition with Articulatory  Attributes标题：Allophant：具有节律属性的跨语言音素识别链接：https://arxiv.org/abs/2306.04306
作者：Kevin Glocker,Aaricia Herygers,Munir Georges机构：AImotion Bavaria, Technische Hochschule Ingolstadt, Germany, Intel Labs, Germany备注：5 pages, 2 figures, 2 tables, accepted to INTERSPEECH 2023摘要：本文提出了一种多语种音素识别器Allophant。它只需要一个音素库存跨语言转移到目标语言，允许低资源识别。该架构结合了一个组成的电话嵌入方法与单独监督的语音属性分类器在一个多任务架构。我们还介绍了Allophoible，PHOIBLE数据库的扩展。当与基于距离的映射方法结合用于字素到音素输出时，它允许我们直接在PHOIBLE库存上训练。通过对34种语言的训练和评估，我们发现，多任务学习的加入提高了模型应用于看不见的音素和音素库存的能力。在有监督的语言上，我们实现了11个百分点的音素错误率改进（pp.）。与没有多任务学习的基线相比。对84种语言的zero-shot迁移进行评估，PER降低了2.63 pp。超过基线。摘要：This paper proposes Allophant, a multilingual phoneme recognizer. It requires only a phoneme inventory for cross-lingual transfer to a target language, allowing for low-resource recognition. The architecture combines a compositional phone embedding approach with individually supervised phonetic attribute classifiers in a multi-task architecture. We also introduce Allophoible, an extension of the PHOIBLE database. When combined with a distance based mapping approach for grapheme-to-phoneme outputs, it allows us to train on PHOIBLE inventories directly. By training and evaluating on 34 languages, we found that the addition of multi-task learning improves the model's capability of being applied to unseen phonemes and phoneme inventories. On supervised languages we achieve phoneme error rate improvements of 11 percentage points (pp.) compared to a baseline without multi-task learning. Evaluation of zero-shot transfer on 84 languages yielded a decrease in PER of 2.63 pp. over the baseline.
【6】 Interpretable Style Transfer for Text-to-Speech with ControlVAE and  Diffusion Bridge标题：利用ControlVAE和扩散桥实现文语转换的可解释风格转换链接：https://arxiv.org/abs/2306.04301
作者：Wenhao Guan,Tao Li,Yishuang Li,Hukai Huang,Qingyang Hong,Lin Li机构：School of Informatics, Xiamen University, China, Institute of Artificial Intelligence, Xiamen University, China, School of Electronic Science and Engineering, Xiamen University, China备注：Accepted at Interspeech2023摘要：随着语音自动控制和个性化生成的需求，文本到语音（TTS）中的风格控制和转换变得越来越重要。在本文中，我们提出了一个新的TTS系统，可以执行风格转换的解释性和高保真度。首先，我们设计了一个结合变分自动编码器（VAE）和扩散细化器的TTS系统，以获得细化的梅尔谱图。具体地，分别设计了两级和一级系统，以提高音频质量和风格转换的性能。其次，设计了量化的VAE扩散桥，有效地学习复杂的离散风格表示，提高风格迁移的性能。为了更好地传递风格，我们引入ControlVAE来提高重建质量，同时具有良好的可解释性。在LibriTTS数据集上的实验表明，该方法比基线模型更有效。摘要：With the demand for autonomous control and personalized speech generation, the style control and transfer in Text-to-Speech (TTS) is becoming more and more important. In this paper, we propose a new TTS system that can perform style transfer with interpretability and high fidelity. Firstly, we design a TTS system that combines variational autoencoder (VAE) and diffusion refiner to get refined mel-spectrograms. Specifically, a two-stage and a one-stage system are designed respectively, to improve the audio quality and the performance of style transfer. Secondly, a diffusion bridge of quantized VAE is designed to efficiently learn complex discrete style representations and improve the performance of style transfer. To have a better ability of style transfer, we introduce ControlVAE to improve the reconstruction quality and have good interpretability simultaneously. Experiments on LibriTTS dataset demonstrate that our method is more effective than baseline models.
【7】 A Mask Free Neural Network for Monaural Speech Enhancement标题：一种用于单声道语音增强的无掩码神经网络链接：https://arxiv.org/abs/2306.04286
作者：Liang Liu,Haixin Guan,Jinlong Ma,Wei Dai,Guangyong Wang,Shaowei Ding机构：Unisound AI Technology Co. Ltd, China, University of Science and Technology of China, China摘要：在语音增强中，目标语音阶段缺乏清晰的结构特征，需要使用保守和繁琐的网络框架。使用直接方法和简单的网络架构似乎很难实现有竞争力的性能。然而，我们提出了MFNet，这是一个直接而简单的网络，不仅可以映射语音，还可以映射反向噪声。该网络通过堆叠全局局部前体块（GLFB）来构建，其结合了Mobileblock用于全局处理和Metaformer架构用于局部交互的优点。我们的实验结果表明，我们的网络使用映射方法优于掩蔽方法，和反向噪声的直接映射是在强噪声环境中的最佳解决方案。在2020年深度噪声抑制（DNS）挑战测试集的水平比较中，据我们所知，MFNet是当前最先进的（SOTA）映射模型。摘要：In speech enhancement, the lack of clear structural characteristics in the target speech phase requires the use of conservative and cumbersome network frameworks. It seems difficult to achieve competitive performance using direct methods and simple network architectures. However, we propose the MFNet, a direct and simple network that can not only map speech but also map reverse noise. This network is constructed by stacking global local former blocks (GLFBs), which combine the advantages of Mobileblock for global processing and Metaformer architecture for local interaction. Our experimental results demonstrate that our network using mapping method outperforms masking methods, and direct mapping of reverse noise is the optimal solution in strong noise environments. In a horizontal comparison on the 2020 Deep Noise Suppression (DNS) challenge test set without reverberation, to the best of our knowledge, MFNet is the current state-of-the-art (SOTA) mapping model.
【8】 Multi-microphone Automatic Speech Segmentation in Meetings Based on  Circular Harmonics Features标题：基于环形谐波特征的会议多麦克风自动语音分割链接：https://arxiv.org/abs/2306.04268
作者：Théo Mariotte,Anthony Larcher,Silvio Montrésor,Jean-Hugh Thomas机构：LAUM UMR CNRS , IAGS, Le Mans Universit´e, France, LIUM, Le Mans Universit´e, France备注：Interspeech 2023, international Speech Communication Association (ISCA), Aug 2023, Dublin, Ireland摘要：发言人日记是回答谁发言和何时发言的任务。在音频流中。流水线系统依赖于语音分割来提取说话人的片段并实现鲁棒的说话人日记化。本文提出了一个通用框架来解决远程语音场景中的三个分割任务：语音活动检测（VAD）、重叠语音检测（OSD）和说话者改变检测（SCD）。在文献中，一些研究调查了多麦克风远距离语音场景。在这项工作中，我们提出了一组新的空间特征的基础上到达方向估计的圆谐域（CH-DOA）。这些空间特征是从多麦克风音频数据中提取的，并与标准声学特征相结合。AMI会议语料库上的实验表明，CH-DOA可以提高分割，同时在停用麦克风的情况下是鲁棒的。摘要：Speaker diarization is the task of answering Who spoke and when? in an audio stream. Pipeline systems rely on speech segmentation to extract speakers' segments and achieve robust speaker diarization. This paper proposes a common framework to solve three segmentation tasks in the distant speech scenario: Voice Activity Detection (VAD), Overlapped Speech Detection (OSD), and Speaker Change Detection (SCD). In the literature, a few studies investigate the multi-microphone distant speech scenario. In this work, we propose a new set of spatial features based on direction-of-arrival estimations in the circular harmonic domain (CH-DOA). These spatial features are extracted from multi-microphone audio data and combined with standard acoustic features. Experiments on the AMI meeting corpus show that CH-DOA can improve the segmentation while being robust in the case of deactivated microphones.
【9】 Transfer Learning from Pre-trained Language Models Improves End-to-End  Speech Summarization标题：来自预先训练的语言模型的迁移学习改善了端到端语音摘要链接：https://arxiv.org/abs/2306.04233
作者：Kohei Matsuura,Takanori Ashihara,Takafumi Moriya,Tomohiro Tanaka,Takatomo Kano,Atsunori Ogawa,Marc Delcroix机构：NTT Corporation, Japan备注：Accepted by Interspeech 2023摘要：端到端语音摘要（E2E SSum）通过单一模型直接将输入语音摘要为易于阅读的短句。这种方法是有前途的，因为它，在传统的级联方法相比，可以利用完整的声学信息，并减轻转录错误的传播。然而，由于收集语音摘要对的高成本，E2E SSum模型倾向于遭受训练数据稀缺并输出不自然的句子。为了克服这一缺点，我们首次提出通过迁移学习将预先训练的语言模型（LM）集成到E2E SSum解码器中，该模型非常能够生成自然句子。此外，为了减少独立预训练的编码器和解码器之间的差距，我们还建议转移基线E2E SSum编码器，而不是常用的自动语音识别编码器。实验结果表明，该模型优于基线和数据增强模型。摘要：End-to-end speech summarization (E2E SSum) directly summarizes input speech into easy-to-read short sentences with a single model. This approach is promising because it, in contrast to the conventional cascade approach, can utilize full acoustical information and mitigate to the propagation of transcription errors. However, due to the high cost of collecting speech-summary pairs, an E2E SSum model tends to suffer from training data scarcity and output unnatural sentences. To overcome this drawback, we propose for the first time to integrate a pre-trained language model (LM), which is highly capable of generating natural sentences, into the E2E SSum decoder via transfer learning. In addition, to reduce the gap between the independently pre-trained encoder and decoder, we also propose to transfer the baseline E2E SSum encoder instead of the commonly used automatic speech recognition encoder. Experimental results show that the proposed model outperforms baseline and data augmented models.
【10】 An ASR-Based Tutor for Learning to Read: How to Optimize Feedback to  First Graders标题：基于ASR的阅读学习辅导：如何优化对一年级学生的反馈链接：https://arxiv.org/abs/2306.04190
作者：Yu Bai,Cristian Tejedor-Garcia,Ferdy Hubers,Catia Cucchiarini,Helmer Strik机构：Centre for Language and Speech Technology (CLST), Radboud University,  Centre for Language Studies (CLS), Radboud University Nijmegen,  Donders Institute for Brain, Cognition and Behaviour, Radboud University,  NovoLearning B.V., Nijmegen备注：None摘要：近年来，在阅读实践的应用中采用自动语音识别（ASR）的兴趣一直在增长。在以前的研究中，我们提出了一个基于ASR的荷兰阅读导师的应用程序，开发提供即时反馈，一年级学生学习阅读。我们看到ASR在阅读过程的这个阶段具有潜力，因为结果表明，学生通过使用该软件在阅读准确性和流畅性方面取得了进步。在目前的研究中，我们使用儿童的语音从现有的语料库（JASMIN）开发两个新的ASR系统，并比较的结果，以前的研究。我们分析的ASR系统的正确/不正确的分类使用人类转录本在词水平，通过评价措施，如科恩的Kappa，马修斯相关系数（MCC），精度，召回和F-措施的手段。我们观察到新开发的ASR系统的改进，与基于人为的判断和正确的拒绝（CR）的协议。ASR系统的准确性因不同的阅读任务和单词类型而异。我们的研究结果表明，在目前的配置，这是很难分类孤立的词。我们讨论了这些结果，可能的方法来改进我们的系统和未来的研究途径。摘要：The interest in employing automatic speech recognition (ASR) in applications for reading practice has been growing in recent years. In a previous study, we presented an ASR-based Dutch reading tutor application that was developed to provide instantaneous feedback to first-graders learning to read. We saw that ASR has potential at this stage of the reading process, as the results suggested that pupils made progress in reading accuracy and fluency by using the software. In the current study, we used children's speech from an existing corpus (JASMIN) to develop two new ASR systems, and compared the results to those of the previous study. We analyze correct/incorrect classification of the ASR systems using human transcripts at word level, by means of evaluation measures such as Cohen's Kappa, Matthews Correlation Coefficient (MCC), precision, recall and F-measures. We observe improvements for the newly developed ASR systems regarding the agreement with human-based judgment and correct rejection (CR). The accuracy of the ASR systems varies for different reading tasks and word types. Our results suggest that, in the current configuration, it is difficult to classify isolated words. We discuss these results, possible ways to improve our systems and avenues for future research.
【11】 SANGEET: A XML based Open Dataset for Research in Hindustani Sangeet标题：Sangeet：一个用于印度斯坦Sangeet研究的基于XML的开放数据集链接：https://arxiv.org/abs/2306.04148
作者：Chandan Misra,Swarup Chattopadhyay机构：School of Computer Science & Engineering, XIM University摘要：访问丰富的音乐数据集非常重要，它在各种应用程序中都很有用。目前，现有的数据集大多集中在存储声乐或器乐录音数据，而忽略了其可视化表示和检索的要求。本文试图建立一个基于XML的公共数据集，称为SANGEET，存储由著名音乐学家Pt写的印度斯坦Sangeet（北印度古典音乐）作品的全面信息。毗湿奴·纳拉扬·巴特坎德SANGEET以标准化的方式保存任何给定作品的所有所需信息，包括元数据，结构，标记，节奏和旋律信息，以便轻松有效地存储和提取音乐信息。该数据集旨在为音乐信息研究任务提供基础事实信息，从而从机器学习的角度支持几种数据驱动的分析。我们目前的有用性的数据集，通过展示其应用程序的音乐信息检索使用XQuery，通过Omenad渲染系统的可视化。最后，我们提出了一些方法来转换数据集，以执行统计和机器学习任务，从而更好地了解印度斯坦语Sangeet。该数据集可以在www.example.com上找到https://github.com/cmisra/Sangeet。摘要：It is very important to access a rich music dataset that is useful in a wide variety of applications. Currently, available datasets are mostly focused on storing vocal or instrumental recording data and ignoring the requirement of its visual representation and retrieval. This paper attempts to build an XML-based public dataset, called SANGEET, that stores comprehensive information of Hindustani Sangeet (North Indian Classical Music) compositions written by famous musicologist Pt. Vishnu Narayan Bhatkhande. SANGEET preserves all the required information of any given composition including metadata, structural, notational, rhythmic, and melodic information in a standardized way for easy and efficient storage and extraction of musical information. The dataset is intended to provide the ground truth information for music information research tasks, thereby supporting several data-driven analysis from a machine learning perspective. We present the usefulness of the dataset by demonstrating its application on music information retrieval using XQuery, visualization through Omenad rendering system. Finally, we propose approaches to transform the dataset for performing statistical and machine learning tasks for a better understanding of Hindustani Sangeet. The dataset can be found at https://github.com/cmisra/Sangeet.
【12】 RISC: A Corpus for Shout Type Classification and Shout Intensity  Prediction标题：RISC：用于呼喊类型分类和呼喊强度预测的语料库链接：https://arxiv.org/abs/2306.04143
作者：Takahiro Fukumori,Taito Ishida,Yoichi Yamashita机构：†College of Information Science and Engineering, Ritsumeikan University, Japan., ††Graduate School of Information Science and Engineering, Ritsumeikan University, Japan.摘要：在音频监视和监听中，对被呼叫语音的检测是至关重要的。尽管希望安全系统能够识别紧急情况，但是现有的语料库仅提供二进制标签（即，喊出或正常），使得难以预测喊出强度。此外，大多数语料库仅包括典型的危险情况的话语，这意味着分类器不能学习区分这种话语与典型的不太危险的情况的喊叫，例如欢呼。因此，本文提出了一种新的研究来源，Ritsumeikan喊语料库（RISC），其中包含各种类型的录音实验中收集的喊语音样本。RISC中的每个被喊的语音样本具有喊类型，并且还经由众包服务被分配喊强度等级。我们还对语音类型分类任务和呼叫强度预测任务的深度学习方法进行了全面的性能比较。结果表明，基于谱域和倒谱域的特征学习，实现了高性能，无论使用哪种网络架构。研究结果还表明，呼喊类型分类和强度预测仍然是具有挑战性的任务，RISC有望有助于进一步发展在这一研究领域。摘要：The detection of shouted speech is crucial in audio surveillance and monitoring. Although it is desirable for a security system to be able to identify emergencies, existing corpora provide only a binary label (i.e., shouted or normal) for each speech sample, making it difficult to predict the shout intensity. Furthermore, most corpora comprise only utterances typical of hazardous situations, meaning that classifiers cannot learn to discriminate such utterances from shouts typical of less hazardous situations, such as cheers. Thus, this paper presents a novel research source, the RItsumeikan Shout Corpus (RISC), which contains wide variety types of shouted speech samples collected in recording experiments. Each shouted speech sample in RISC has a shout type and is also assigned shout intensity ratings via a crowdsourcing service. We also present a comprehensive performance comparison among deep learning approaches for speech type classification tasks and a shout intensity prediction task. The results show that feature learning based on the spectral and cepstral domains achieves high performance, no matter which network architecture is used. The results also demonstrate that shout type classification and intensity prediction are still challenging tasks, and RISC is expected to contribute to further development in this research area.
【13】 Text-only Domain Adaptation using Unified Speech-Text Representation in  Transducer标题：在Transducer中使用统一语音-文本表示的纯文本域自适应链接：https://arxiv.org/abs/2306.04076
作者：Lu Huang,Boyu Li,Jun Zhang,Lu Lu,Zejun Ma机构：ByteDance备注：Submitted to Interspeech 2023摘要：在端到端（E2 E）语音识别中，使用纯文本语料库的域自适应是具有挑战性的。通过TTS从文本合成音频的适应是消耗资源的。我们提出了一种方法来学习统一的语音-文本表示在构象转换器（USTR-CT），使快速域适应使用纯文本语料库。与之前的文本图方法不同，我们在工作中引入了一个额外的文本编码器来学习文本表示，并在推理过程中删除，因此无需对在线部署进行修改。为了提高自适应的效率，单步和多步的自适应也进行了探索。将LibriSpeech应用于SPGISpeech的实验表明，该方法在目标域上的误词率（WER）降低了44%，优于TTS方法和文本图方法。此外，它示出所提出的方法可以与内部语言模型估计（ILME），以进一步提高性能。摘要：Domain adaptation using text-only corpus is challenging in end-to-end(E2E) speech recognition. Adaptation by synthesizing audio from text through TTS is resource-consuming. We present a method to learn Unified Speech-Text Representation in Conformer Transducer(USTR-CT) to enable fast domain adaptation using the text-only corpus. Different from the previous textogram method, an extra text encoder is introduced in our work to learn text representation and is removed during inference, so there is no modification for online deployment. To improve the efficiency of adaptation, single-step and multi-step adaptations are also explored. The experiments on adapting LibriSpeech to SPGISpeech show the proposed method reduces the word error rate(WER) by relatively 44% on the target domain, which is better than those of TTS method and textogram method. Also, it is shown the proposed method can be combined with internal language model estimation(ILME) to further improve the performance.
【14】 Test experiments with distributed acoustic sensing and hydrophone arrays  for locating underwater sound sources标题：分布式声学传感和水听器阵列定位水下声源的试验研究链接：https://arxiv.org/abs/2306.04276
作者：Jörg Rychen,Patrick Paitz,Pascal Edme,Krystyna Smolinski,Joeri Brackenhoff,Andreas Fichtner机构：Fichtner,,  Department of Information Technology and Electrical Engineering, ETH Zurich, Zurich, Switzerland,  Department of Earth Sciences, ETH Zurich, Zurich, Switzerland, Dataset:备注：Data description摘要：鲸鱼和海豚依靠声音进行导航和交流，这使它们成为研究语言进化的有趣主题。传统的水听器阵列已被用来记录他们的声学行为，但光纤已成为一个有前途的替代品。本研究探讨了使用分布式声学传感（DAS），一种技术，检测光纤中的局部应力，水下声音记录。在苏黎世湖进行了一项实验，在那里部署了光纤电缆和自制的水听器阵列。测试信号在不同的位置广播，得到的数据被同步并合并到文件中。分析揭示了不同的频率响应在DAS渠道，并提供了在湖中的声音传播的见解。确定了与电缆灵敏度、采样率和广播保真度相关的挑战。该数据集为推进水下环境中的声学传感技术，特别是研究海洋哺乳动物的发声行为提供了宝贵的资源。摘要：Whales and dolphins rely on sound for navigation and communication, making them an intriguing subject for studying language evolution. Traditional hydrophone arrays have been used to record their acoustic behavior, but optical fibers have emerged as a promising alternative. This study explores the use of distributed acoustic sensing (DAS), a technique that detects local stress in optical fibers, for underwater sound recording. An experiment was conducted in Lake Zurich, where a fiber-optic cable and a self-made hydrophone array were deployed. A test signal was broadcasted at various locations, and the resulting data was synchronized and consolidated into files. Analysis revealed distinct frequency responses in the DAS channels and provided insights into sound propagation in the lake. Challenges related to cable sensitivity, sample rate, and broadcast fidelity were identified. This dataset serves as a valuable resource for advancing acoustic sensing techniques in underwater environments, especially for studying marine mammal vocal behavior.
【15】 RescueSpeech: A German Corpus for Speech Recognition in Search and  Rescue Domain标题：RescueSpeech：用于搜救领域语音识别的德语语料库链接：https://arxiv.org/abs/2306.04054
作者：Sangeet Sagar,Mirco Ravanelli,Bernd Kiefer,Ivana Kruijff Korbayova,Josef van Genabith机构：Genabith, Saarland University, Germany, Concordia University, Mila-Quebec AI Institute, Canada, German Research Center for Artificial Intelligence (DFKI), Germany摘要：尽管最近在语音识别方面取得了进步，但在嘈杂和混响的声学环境中准确地转录会话和情感语音仍然存在困难。这在搜索和救援（SAR）领域提出了一个特殊的挑战，其中转录救援队成员之间的对话对于支持实时决策至关重要。在SAR场景中，语音数据和相关背景噪声的稀缺使得难以部署鲁棒的语音识别系统。  为了解决这个问题，我们创建并公开了一个名为RescueSpeech的德语语音数据集。该数据集包括来自模拟救援演习的真实语音记录。此外，我们还发布了有竞争力的训练配方和预训练模型。我们的研究表明，目前的性能水平所取得的国家的最先进的方法仍然是远远不能接受的。摘要：Despite recent advancements in speech recognition, there are still difficulties in accurately transcribing conversational and emotional speech in noisy and reverberant acoustic environments. This poses a particular challenge in the search and rescue (SAR) domain, where transcribing conversations among rescue team members is crucial to support real-time decision-making. The scarcity of speech data and associated background noise in SAR scenarios make it difficult to deploy robust speech recognition systems.  To address this issue, we have created and made publicly available a German speech dataset called RescueSpeech. This dataset includes real speech recordings from simulated rescue exercises. Additionally, we have released competitive training recipes and pre-trained models. Our study indicates that the current level of performance achieved by state-of-the-art methods is still far from being acceptable.
eess.AS音频处理
【1】 Self-supervised Audio Teacher-Student Transformer for Both Clip-level  and Frame-level Tasks标题：用于片段级和帧级任务的自我监督音频教师-学生转换器链接：https://arxiv.org/abs/2306.04186
作者：Xian Li,Nian Shao,Xiaofei Li机构：cn 1Westlake Institute for Advanced Study & 2Westlake University备注：Submitted to IEEE TASLP. arXiv admin note: text overlap with arXiv:2204.12076摘要：近年来，自监督学习（SSL）已经成为学习音频表示的流行方法。音频自监督预训练的最终目标是将知识转移到下游音频任务，通常包括剪辑级和帧级任务。剪辑级任务对整个音频剪辑的场景或声音进行分类，例如：音频标记、仪器识别等。虽然帧级任务从音频剪辑检测事件级时间戳，例如，声音事件检测、说话者日志等。先前的研究主要评估剪辑级下游任务。帧级任务对于细粒度声学场景/事件理解是重要的，并且通常比剪辑级任务更具挑战性。为了解决剪辑级和帧级任务，本文提出了两种自监督音频表示学习方法：ATST-Clip和ATST-Frame，分别负责学习剪辑级和帧级表示。ATST是Audio Teacher-Student Transformer的缩写，这意味着这两种方法都使用了变压器编码器和师生训练方案。实验结果表明，我们的ATST-Frame模型在大多数剪辑级和帧级下游任务上都获得了最先进的（SOTA）性能。特别是，它优于其他模型的帧级声音事件检测任务的大幅度。此外，通过知识蒸馏将两种模型结合起来，可以进一步提高性能。摘要：In recent years, self-supervised learning (SSL) has emerged as a popular approach for learning audio representations. The ultimate goal of audio self-supervised pre-training is to transfer knowledge to downstream audio tasks, generally including clip-level and frame-level tasks. Clip-level tasks classify the scene or sound of an entire audio clip, e.g. audio tagging, instrument recognition, etc. While frame-level tasks detect event-level timestamps from an audio clip, e.g. sound event detection, speaker diarization, etc. Prior studies primarily evaluate on clip-level downstream tasks. Frame-level tasks are important for fine-grained acoustic scene/event understanding, and are generally more challenging than clip-level tasks. In order to tackle both clip-level and frame-level tasks, this paper proposes two self-supervised audio representation learning methods: ATST-Clip and ATST-Frame, responsible for learning clip-level and frame-level representations, respectively. ATST stands for Audio Teacher-Student Transformer, which means both methods use a transformer encoder and a teacher-student training scheme.Experimental results show that our ATST-Frame model obtains state-of-the-art (SOTA) performance on most of the clip-level and frame-level downstream tasks. Especially, it outperforms other models by a large margin on the frame-level sound event detection task. In addition, the performance can be further improved by combining the two models through knowledge distillation.
【2】 RescueSpeech: A German Corpus for Speech Recognition in Search and  Rescue Domain标题：RescueSpeech：用于搜救领域语音识别的德语语料库链接：https://arxiv.org/abs/2306.04054
*与cs.SD语音【15】为同一篇
作者：Sangeet Sagar,Mirco Ravanelli,Bernd Kiefer,Ivana Kruijff Korbayova,Josef van Genabith机构：Genabith, Saarland University, Germany, Concordia University, Mila-Quebec AI Institute, Canada, German Research Center for Artificial Intelligence (DFKI), Germany摘要：尽管最近在语音识别方面取得了进步，但在嘈杂和混响的声学环境中准确地转录会话和情感语音仍然存在困难。这在搜索和救援（SAR）领域提出了一个特殊的挑战，其中转录救援队成员之间的对话对于支持实时决策至关重要。在SAR场景中，语音数据和相关背景噪声的稀缺使得难以部署鲁棒的语音识别系统。  为了解决这个问题，我们创建并公开了一个名为RescueSpeech的德语语音数据集。该数据集包括来自模拟救援演习的真实语音记录。此外，我们还发布了有竞争力的训练配方和预训练模型。我们的研究表明，目前的性能水平所取得的国家的最先进的方法仍然是远远不能接受的。摘要：Despite recent advancements in speech recognition, there are still difficulties in accurately transcribing conversational and emotional speech in noisy and reverberant acoustic environments. This poses a particular challenge in the search and rescue (SAR) domain, where transcribing conversations among rescue team members is crucial to support real-time decision-making. The scarcity of speech data and associated background noise in SAR scenarios make it difficult to deploy robust speech recognition systems.  To address this issue, we have created and made publicly available a German speech dataset called RescueSpeech. This dataset includes real speech recordings from simulated rescue exercises. Additionally, we have released competitive training recipes and pre-trained models. Our study indicates that the current level of performance achieved by state-of-the-art methods is still far from being acceptable.
【3】 Systematic Analysis of Music Representations from BERT标题：BERT音乐表现手法的系统分析链接：https://arxiv.org/abs/2306.04628
*与cs.SD语音【1】为同一篇
作者：Sangjun Han,Hyeongrae Ihm,Woohyung Lim机构：LG AI Research摘要：已经有许多尝试将原始数据表示为有效地捕获语义和上下文信息的数字向量。然而，在符号音乐领域，以前的作品试图通过观察各种微调任务的性能改善来验证他们的音乐嵌入。在这项工作中，我们直接分析嵌入从BERT和BERT与对比学习培训的酒吧级MIDI，检查他们的音乐信息，可以从MIDI事件。我们观察到的嵌入表现出不同的信息特征，这取决于对比的目标和层的选择。我们的代码可在www.example.com上获得https://github.com/sjhan91/MusicBERT。摘要：There have been numerous attempts to represent raw data as numerical vectors that effectively capture semantic and contextual information. However, in the field of symbolic music, previous works have attempted to validate their music embeddings by observing the performance improvement of various fine-tuning tasks. In this work, we directly analyze embeddings from BERT and BERT with contrastive learning trained on bar-level MIDI, inspecting their musical information that can be obtained from MIDI events. We observe that the embeddings exhibit distinct characteristics of information depending on the contrastive objectives and the choice of layers. Our code is available at https://github.com/sjhan91/MusicBERT.
【4】 Zambezi Voice: A Multilingual Speech Corpus for Zambian Languages标题：赞比西亚语：赞比亚语言的多语种语音语料库链接：https://arxiv.org/abs/2306.04428
*与cs.SD语音【2】为同一篇
作者：Claytone Sikasote,Kalinda Siaminwe,Stanly Mwape,Bangiwe Zulu,Mofya Phiri,Martin Phiri,David Zulu,Mayumbo Nyirenda,Antonios Anastasopoulos机构：University of Zambia, Zambia, George Mason University, U.S.A备注：Accepted at INTERSPEECH 2023摘要：这项工作介绍了赞比西之声，赞比亚语言的开源多语言语音资源。它包含两个数据集集合：无线电新闻和脱口秀节目的未标记的音频记录（160小时）和标记的数据（超过80小时），所述标记的数据包括从来源于公开可用的文学书籍的文本记录的阅读语音。该数据集是为语音识别创建的，但可以扩展到多语言语音处理研究，用于监督和无监督学习方法。据我们所知，这是为赞比亚语言创建的第一个多语言语音数据集。我们通过微调Wav2Vec2.0大规模多语言预训练模型来利用预训练和跨语言迁移学习，为我们的基线模型构建端到端（E2 E）语音识别模型。该数据集在Creative Commons BY-NC-ND 4.0许可下公开发布，可以通过项目存储库访问。参见https://github.com/unza-speech-lab/zambezi-voice摘要：This work introduces Zambezi Voice, an open-source multilingual speech resource for Zambian languages. It contains two collections of datasets: unlabelled audio recordings of radio news and talk shows programs (160 hours) and labelled data (over 80 hours) consisting of read speech recorded from text sourced from publicly available literature books. The dataset is created for speech recognition but can be extended to multilingual speech processing research for both supervised and unsupervised learning approaches. To our knowledge, this is the first multilingual speech dataset created for Zambian languages. We exploit pretraining and cross-lingual transfer learning by finetuning the Wav2Vec2.0 large-scale multilingual pre-trained model to build end-to-end (E2E) speech recognition models for our baseline models. The dataset is released publicly under a Creative Commons BY-NC-ND 4.0 license and can be accessed through the project repository. See https://github.com/unza-speech-lab/zambezi-voice
【5】 Label Aware Speech Representation Learning For Language Identification标题：标签感知语音表征学习在语言识别中的应用链接：https://arxiv.org/abs/2306.04374
*与cs.SD语音【3】为同一篇
作者：Shikhar Vashishth,Shikhar Bharadwaj,Sriram Ganapathy,Ankur Bapna,Min Ma,Wei Han,Vera Axelrod,Partha Talukdar机构：Google Research备注：Accepted at Interspeech 2023摘要：用于诸如语言识别的非语义任务的语音表示学习方法已经探索了使用分类器模型的监督嵌入提取方法或使用原始数据的自监督表示学习方法。在本文中，我们提出了一个新的框架相结合的自监督表示学习的语言标签信息的预训练任务。这个框架，被称为标签感知语音表示（LASR）学习，使用基于三元组的目标函数，将语言标签与自监督损失函数。针对下游任务进一步微调语音表示。语言识别实验在两个公共数据集上进行- FLEURS和Dhwani。在这些实验中，我们说明了所提出的LASR框架提高了语言识别的最先进的系统。我们还报告了一个分析的鲁棒性LASR方法的噪声/丢失的标签，以及其应用程序的多语言语音识别任务。摘要：Speech representation learning approaches for non-semantic tasks such as language recognition have either explored supervised embedding extraction methods using a classifier model or self-supervised representation learning approaches using raw data. In this paper, we propose a novel framework of combining self-supervised representation learning with the language label information for the pre-training task. This framework, termed as Label Aware Speech Representation (LASR) learning, uses a triplet based objective function to incorporate language labels along with the self-supervised loss function. The speech representations are further fine-tuned for the downstream task. The language recognition experiments are performed on two public datasets - FLEURS and Dhwani. In these experiments, we illustrate that the proposed LASR framework improves over the state-of-the-art systems on language identification. We also report an analysis of the robustness of LASR approach to noisy/missing labels as well as its application to multi-lingual speech recognition tasks.
【6】 Allophant: Cross-lingual Phoneme Recognition with Articulatory  Attributes标题：Allophant：具有节律属性的跨语言音素识别链接：https://arxiv.org/abs/2306.04306
*与cs.SD语音【5】为同一篇
作者：Kevin Glocker,Aaricia Herygers,Munir Georges机构：AImotion Bavaria, Technische Hochschule Ingolstadt, Germany, Intel Labs, Germany备注：5 pages, 2 figures, 2 tables, accepted to INTERSPEECH 2023摘要：本文提出了一种多语种音素识别器Allophant。它只需要一个音素库存跨语言转移到目标语言，允许低资源识别。该架构结合了一个组成的电话嵌入方法与单独监督的语音属性分类器在一个多任务架构。我们还介绍了Allophoible，PHOIBLE数据库的扩展。当与基于距离的映射方法结合用于字素到音素输出时，它允许我们直接在PHOIBLE库存上训练。通过对34种语言的训练和评估，我们发现，多任务学习的加入提高了模型应用于看不见的音素和音素库存的能力。在有监督的语言上，我们实现了11个百分点的音素错误率改进（pp.）。与没有多任务学习的基线相比。对84种语言的zero-shot迁移进行评估，PER降低了2.63 pp。超过基线。摘要：This paper proposes Allophant, a multilingual phoneme recognizer. It requires only a phoneme inventory for cross-lingual transfer to a target language, allowing for low-resource recognition. The architecture combines a compositional phone embedding approach with individually supervised phonetic attribute classifiers in a multi-task architecture. We also introduce Allophoible, an extension of the PHOIBLE database. When combined with a distance based mapping approach for grapheme-to-phoneme outputs, it allows us to train on PHOIBLE inventories directly. By training and evaluating on 34 languages, we found that the addition of multi-task learning improves the model's capability of being applied to unseen phonemes and phoneme inventories. On supervised languages we achieve phoneme error rate improvements of 11 percentage points (pp.) compared to a baseline without multi-task learning. Evaluation of zero-shot transfer on 84 languages yielded a decrease in PER of 2.63 pp. over the baseline.
【7】 Interpretable Style Transfer for Text-to-Speech with ControlVAE and  Diffusion Bridge标题：利用ControlVAE和扩散桥实现文语转换的可解释风格转换链接：https://arxiv.org/abs/2306.04301
*与cs.SD语音【6】为同一篇
作者：Wenhao Guan,Tao Li,Yishuang Li,Hukai Huang,Qingyang Hong,Lin Li机构：School of Informatics, Xiamen University, China, Institute of Artificial Intelligence, Xiamen University, China, School of Electronic Science and Engineering, Xiamen University, China备注：Accepted at Interspeech2023摘要：随着语音自动控制和个性化生成的需求，文本到语音（TTS）中的风格控制和转换变得越来越重要。在本文中，我们提出了一个新的TTS系统，可以执行风格转换的解释性和高保真度。首先，我们设计了一个结合变分自动编码器（VAE）和扩散细化器的TTS系统，以获得细化的梅尔谱图。具体地，分别设计了两级和一级系统，以提高音频质量和风格转换的性能。其次，设计了量化的VAE扩散桥，有效地学习复杂的离散风格表示，提高风格迁移的性能。为了更好地传递风格，我们引入ControlVAE来提高重建质量，同时具有良好的可解释性。在LibriTTS数据集上的实验表明，该方法比基线模型更有效。摘要：With the demand for autonomous control and personalized speech generation, the style control and transfer in Text-to-Speech (TTS) is becoming more and more important. In this paper, we propose a new TTS system that can perform style transfer with interpretability and high fidelity. Firstly, we design a TTS system that combines variational autoencoder (VAE) and diffusion refiner to get refined mel-spectrograms. Specifically, a two-stage and a one-stage system are designed respectively, to improve the audio quality and the performance of style transfer. Secondly, a diffusion bridge of quantized VAE is designed to efficiently learn complex discrete style representations and improve the performance of style transfer. To have a better ability of style transfer, we introduce ControlVAE to improve the reconstruction quality and have good interpretability simultaneously. Experiments on LibriTTS dataset demonstrate that our method is more effective than baseline models.
【8】 Multi-microphone Automatic Speech Segmentation in Meetings Based on  Circular Harmonics Features标题：基于环形谐波特征的会议多麦克风自动语音分割链接：https://arxiv.org/abs/2306.04268
*与cs.SD语音【8】为同一篇
作者：Théo Mariotte,Anthony Larcher,Silvio Montrésor,Jean-Hugh Thomas机构：LAUM UMR CNRS , IAGS, Le Mans Universit´e, France, LIUM, Le Mans Universit´e, France备注：Interspeech 2023, international Speech Communication Association (ISCA), Aug 2023, Dublin, Ireland摘要：发言人日记是回答谁发言和何时发言的任务。在音频流中。流水线系统依赖于语音分割来提取说话人的片段并实现鲁棒的说话人日记化。本文提出了一个通用框架来解决远程语音场景中的三个分割任务：语音活动检测（VAD）、重叠语音检测（OSD）和说话者改变检测（SCD）。在文献中，一些研究调查了多麦克风远距离语音场景。在这项工作中，我们提出了一组新的空间特征的基础上到达方向估计的圆谐域（CH-DOA）。这些空间特征是从多麦克风音频数据中提取的，并与标准声学特征相结合。AMI会议语料库上的实验表明，CH-DOA可以提高分割，同时在停用麦克风的情况下是鲁棒的。摘要：Speaker diarization is the task of answering Who spoke and when? in an audio stream. Pipeline systems rely on speech segmentation to extract speakers' segments and achieve robust speaker diarization. This paper proposes a common framework to solve three segmentation tasks in the distant speech scenario: Voice Activity Detection (VAD), Overlapped Speech Detection (OSD), and Speaker Change Detection (SCD). In the literature, a few studies investigate the multi-microphone distant speech scenario. In this work, we propose a new set of spatial features based on direction-of-arrival estimations in the circular harmonic domain (CH-DOA). These spatial features are extracted from multi-microphone audio data and combined with standard acoustic features. Experiments on the AMI meeting corpus show that CH-DOA can improve the segmentation while being robust in the case of deactivated microphones.
【9】 Transfer Learning from Pre-trained Language Models Improves End-to-End  Speech Summarization标题：来自预先训练的语言模型的迁移学习改善了端到端语音摘要链接：https://arxiv.org/abs/2306.04233
*与cs.SD语音【9】为同一篇
作者：Kohei Matsuura,Takanori Ashihara,Takafumi Moriya,Tomohiro Tanaka,Takatomo Kano,Atsunori Ogawa,Marc Delcroix机构：NTT Corporation, Japan备注：Accepted by Interspeech 2023摘要：端到端语音摘要（E2E SSum）通过单一模型直接将输入语音摘要为易于阅读的短句。这种方法是有前途的，因为它，在传统的级联方法相比，可以利用完整的声学信息，并减轻转录错误的传播。然而，由于收集语音摘要对的高成本，E2E SSum模型倾向于遭受训练数据稀缺并输出不自然的句子。为了克服这一缺点，我们首次提出通过迁移学习将预先训练的语言模型（LM）集成到E2E SSum解码器中，该模型非常能够生成自然句子。此外，为了减少独立预训练的编码器和解码器之间的差距，我们还建议转移基线E2E SSum编码器，而不是常用的自动语音识别编码器。实验结果表明，该模型优于基线和数据增强模型。摘要：End-to-end speech summarization (E2E SSum) directly summarizes input speech into easy-to-read short sentences with a single model. This approach is promising because it, in contrast to the conventional cascade approach, can utilize full acoustical information and mitigate to the propagation of transcription errors. However, due to the high cost of collecting speech-summary pairs, an E2E SSum model tends to suffer from training data scarcity and output unnatural sentences. To overcome this drawback, we propose for the first time to integrate a pre-trained language model (LM), which is highly capable of generating natural sentences, into the E2E SSum decoder via transfer learning. In addition, to reduce the gap between the independently pre-trained encoder and decoder, we also propose to transfer the baseline E2E SSum encoder instead of the commonly used automatic speech recognition encoder. Experimental results show that the proposed model outperforms baseline and data augmented models.
【10】 An ASR-Based Tutor for Learning to Read: How to Optimize Feedback to  First Graders标题：基于ASR的阅读学习辅导：如何优化对一年级学生的反馈链接：https://arxiv.org/abs/2306.04190
*与cs.SD语音【10】为同一篇
作者：Yu Bai,Cristian Tejedor-Garcia,Ferdy Hubers,Catia Cucchiarini,Helmer Strik机构：Centre for Language and Speech Technology (CLST), Radboud University,  Centre for Language Studies (CLS), Radboud University Nijmegen,  Donders Institute for Brain, Cognition and Behaviour, Radboud University,  NovoLearning B.V., Nijmegen备注：None摘要：近年来，在阅读实践的应用中采用自动语音识别（ASR）的兴趣一直在增长。在以前的研究中，我们提出了一个基于ASR的荷兰阅读导师的应用程序，开发提供即时反馈，一年级学生学习阅读。我们看到ASR在阅读过程的这个阶段具有潜力，因为结果表明，学生通过使用该软件在阅读准确性和流畅性方面取得了进步。在目前的研究中，我们使用儿童的语音从现有的语料库（JASMIN）开发两个新的ASR系统，并比较的结果，以前的研究。我们分析的ASR系统的正确/不正确的分类使用人类转录本在词水平，通过评价措施，如科恩的Kappa，马修斯相关系数（MCC），精度，召回和F-措施的手段。我们观察到新开发的ASR系统的改进，与基于人为的判断和正确的拒绝（CR）的协议。ASR系统的准确性因不同的阅读任务和单词类型而异。我们的研究结果表明，在目前的配置，这是很难分类孤立的词。我们讨论了这些结果，可能的方法来改进我们的系统和未来的研究途径。摘要：The interest in employing automatic speech recognition (ASR) in applications for reading practice has been growing in recent years. In a previous study, we presented an ASR-based Dutch reading tutor application that was developed to provide instantaneous feedback to first-graders learning to read. We saw that ASR has potential at this stage of the reading process, as the results suggested that pupils made progress in reading accuracy and fluency by using the software. In the current study, we used children's speech from an existing corpus (JASMIN) to develop two new ASR systems, and compared the results to those of the previous study. We analyze correct/incorrect classification of the ASR systems using human transcripts at word level, by means of evaluation measures such as Cohen's Kappa, Matthews Correlation Coefficient (MCC), precision, recall and F-measures. We observe improvements for the newly developed ASR systems regarding the agreement with human-based judgment and correct rejection (CR). The accuracy of the ASR systems varies for different reading tasks and word types. Our results suggest that, in the current configuration, it is difficult to classify isolated words. We discuss these results, possible ways to improve our systems and avenues for future research.
【11】 RISC: A Corpus for Shout Type Classification and Shout Intensity  Prediction标题：RISC：用于呼喊类型分类和呼喊强度预测的语料库链接：https://arxiv.org/abs/2306.04143
*与cs.SD语音【12】为同一篇
作者：Takahiro Fukumori,Taito Ishida,Yoichi Yamashita机构：†College of Information Science and Engineering, Ritsumeikan University, Japan., ††Graduate School of Information Science and Engineering, Ritsumeikan University, Japan.摘要：在音频监视和监听中，对被呼叫语音的检测是至关重要的。尽管希望安全系统能够识别紧急情况，但是现有的语料库仅提供二进制标签（即，喊出或正常），使得难以预测喊出强度。此外，大多数语料库仅包括典型的危险情况的话语，这意味着分类器不能学习区分这种话语与典型的不太危险的情况的喊叫，例如欢呼。因此，本文提出了一种新的研究来源，Ritsumeikan喊语料库（RISC），其中包含各种类型的录音实验中收集的喊语音样本。RISC中的每个被喊的语音样本具有喊类型，并且还经由众包服务被分配喊强度等级。我们还对语音类型分类任务和呼叫强度预测任务的深度学习方法进行了全面的性能比较。结果表明，基于谱域和倒谱域的特征学习，实现了高性能，无论使用哪种网络架构。研究结果还表明，呼喊类型分类和强度预测仍然是具有挑战性的任务，RISC有望有助于进一步发展在这一研究领域。摘要：The detection of shouted speech is crucial in audio surveillance and monitoring. Although it is desirable for a security system to be able to identify emergencies, existing corpora provide only a binary label (i.e., shouted or normal) for each speech sample, making it difficult to predict the shout intensity. Furthermore, most corpora comprise only utterances typical of hazardous situations, meaning that classifiers cannot learn to discriminate such utterances from shouts typical of less hazardous situations, such as cheers. Thus, this paper presents a novel research source, the RItsumeikan Shout Corpus (RISC), which contains wide variety types of shouted speech samples collected in recording experiments. Each shouted speech sample in RISC has a shout type and is also assigned shout intensity ratings via a crowdsourcing service. We also present a comprehensive performance comparison among deep learning approaches for speech type classification tasks and a shout intensity prediction task. The results show that feature learning based on the spectral and cepstral domains achieves high performance, no matter which network architecture is used. The results also demonstrate that shout type classification and intensity prediction are still challenging tasks, and RISC is expected to contribute to further development in this research area.
【12】 Text-only Domain Adaptation using Unified Speech-Text Representation in  Transducer标题：在Transducer中使用统一语音-文本表示的纯文本域自适应链接：https://arxiv.org/abs/2306.04076
*与cs.SD语音【13】为同一篇
作者：Lu Huang,Boyu Li,Jun Zhang,Lu Lu,Zejun Ma机构：ByteDance备注：Submitted to Interspeech 2023摘要：在端到端（E2 E）语音识别中，使用纯文本语料库的域自适应是具有挑战性的。通过TTS从文本合成音频的适应是消耗资源的。我们提出了一种方法来学习统一的语音-文本表示在构象转换器（USTR-CT），使快速域适应使用纯文本语料库。与之前的文本图方法不同，我们在工作中引入了一个额外的文本编码器来学习文本表示，并在推理过程中删除，因此无需对在线部署进行修改。为了提高自适应的效率，单步和多步的自适应也进行了探索。将LibriSpeech应用于SPGISpeech的实验表明，该方法在目标域上的误词率（WER）降低了44%，优于TTS方法和文本图方法。此外，它示出所提出的方法可以与内部语言模型估计（ILME），以进一步提高性能。摘要：Domain adaptation using text-only corpus is challenging in end-to-end(E2E) speech recognition. Adaptation by synthesizing audio from text through TTS is resource-consuming. We present a method to learn Unified Speech-Text Representation in Conformer Transducer(USTR-CT) to enable fast domain adaptation using the text-only corpus. Different from the previous textogram method, an extra text encoder is introduced in our work to learn text representation and is removed during inference, so there is no modification for online deployment. To improve the efficiency of adaptation, single-step and multi-step adaptations are also explored. The experiments on adapting LibriSpeech to SPGISpeech show the proposed method reduces the word error rate(WER) by relatively 44% on the target domain, which is better than those of TTS method and textogram method. Also, it is shown the proposed method can be combined with internal language model estimation(ILME) to further improve the performance.
机器翻译，仅供参考