今日论文合集：cs.SD语音21篇，eess.AS音频处理25篇。cs.SD语音【1】Amphion: An Open-Source Audio, Music and Speech Generation Toolkit标题：Amphion：一个开源的音频、音乐和语音生成工具包链接：https://arxiv.org/abs/2312.09911作者：Xueyao Zhang,Liumeng Xue,Yuancheng Wang,Yicheng Gu,Xi Chen,Zihao Fang,Haopeng Chen,Lexiao Zou,Chaoren Wang,Jun Han,Kai Chen,Haizhou Li,Zhizheng Wu备注：GitHub: this https URL摘要：Amphion是一个用于音频，音乐和语音生成的工具包。其目的是支持可重复的研究，并帮助初级研究人员和工程师开始在音频，音乐和语音生成领域的研究和开发。Amphion提供了一个独特的功能：经典模型或架构的可视化。我们相信，这些可视化对于希望更好地理解模型的初级研究人员和工程师来说是有益的。Amphion的北极星目标是提供一个平台，用于研究将任何输入转换为通用音频。Amphion旨在支持个人生成任务。除了特定的生成任务，Amphion还包括几个声码器和评估指标。声码器是产生高质量音频信号的重要模块，而评估指标对于确保生成任务中的一致指标至关重要。在本文中，我们提供了一个高层次的概述Amphion。摘要：Amphion is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development. Amphion offers a unique feature: visualizations of classic models or architectures. We believe that these visualizations are beneficial for junior researchers and engineers who wish to gain a better understanding of the model. The North-Star objective of Amphion is to offer a platform for studying the conversion of any inputs into general audio. Amphion is designed to support individual generation tasks. In addition to the specific generation tasks, Amphion also includes several vocoders and evaluation metrics. A vocoder is an important module for producing high-quality audio signals, while evaluation metrics are critical for ensuring consistent metrics in generation tasks. In this paper, we provide a high-level overview of Amphion.【2】 Generative Context-aware Fine-tuning of Self-supervised Speech Models标题：自监督语音模型的产生式上下文感知微调链接：https://arxiv.org/abs/2312.09895作者：Suwon Shon,Kwangyoun Kim,Prashant Sridhar,Yi-Te Hsu,Shinji Watanabe,Karen Livescu摘要：当执行诸如自动语音识别或针对给定话语的口语理解之类的任务时，访问先前的文本或音频提供上下文信息可以提高性能。考虑到生成式大型语言模型（LLM）的最新进展，我们假设LLM可以使用前面的文本生成有用的上下文信息。通过适当的提示，LLM可以生成下一个句子或抽象文本（如标题或主题）的预测。在本文中，我们研究了使用LLM生成的上下文信息，并提出了一种方法来提取生成的信息在微调的自监督语音模型，我们称之为生成上下文感知微调。这种方法允许微调模型在推理时无需访问真实的周围片段或LLM即可进行改进的预测，同时仅需要非常小的附加上下文模块。我们使用SLUE和Libri-Light基准测试评估了所提出的方法，用于几个下游任务：自动语音识别，命名实体识别和情感分析。结果表明，生成上下文感知微调优于上下文注入微调方法，访问地面真相以前的文本，并与生成上下文注入微调方法，需要在推理时间的LLM竞争。摘要：When performing tasks like automatic speech recognition or spoken language understanding for a given utterance, access to preceding text or audio provides contextual information can improve performance. Considering the recent advances in generative large language models (LLM), we hypothesize that an LLM could generate useful context information using the preceding text. With appropriate prompts, LLM could generate a prediction of the next sentence or abstractive text like titles or topics. In this paper, we study the use of LLM-generated context information and propose an approach to distill the generated information during fine-tuning of self-supervised speech models, which we refer to as generative context-aware fine-tuning. This approach allows the fine-tuned model to make improved predictions without access to the true surrounding segments or to the LLM at inference time, while requiring only a very small additional context module. We evaluate the proposed approach using the SLUE and Libri-light benchmarks for several downstream tasks: automatic speech recognition, named entity recognition, and sentiment analysis. The results show that generative context-aware fine-tuning outperforms a context injection fine-tuning approach that accesses the ground-truth previous text, and is competitive with a generative context injection fine-tuning approach that requires the LLM at inference time.【3】 On the compression of shallow non-causal ASR models using knowledge distillation and tied-and-reduced decoder for low-latency on-device speech recognition标题：用于低延迟设备语音识别的浅层非因果ASR模型的知识提取和捆绑简化译码压缩研究链接：https://arxiv.org/abs/2312.09842作者：Nagaraj Adiga,Jinhwan Park,Chintigari Shiva Kumar,Shatrughan Singh,Kyungmin Lee,Chanwoo Kim,Dhananjaya Gowda摘要：最近，级联两遍架构已成为设备上自动语音识别（ASR）的有力竞争者。因果和浅非因果编码器的级联与共享解码器耦合，使得能够在流模式和前瞻模式下操作。在本文中，我们提出了浅级联模型，结合各种模型压缩技术，如知识蒸馏，共享解码器，捆绑和减少传感器网络，以减少模型的足迹。共享解码器变成一个捆绑和减少的网络。级联两通模型进一步压缩使用知识蒸馏使用模型后验的Kullback-Leibler发散损失。我们证明了一个41 M参数级联教师模型的大小减少了50%，ASR准确性没有明显下降，延迟减少了30%摘要：Recently, the cascaded two-pass architecture has emerged as a strong contender for on-device automatic speech recognition (ASR). A cascade of causal and shallow non-causal encoders coupled with a shared decoder enables operation in both streaming and look-ahead modes. In this paper, we propose shallow cascaded model by combining various model compression techniques such as knowledge distillation, shared decoder, and tied-and-reduced transducer network in order to reduce the model footprint. The shared decoder is changed into a tied-and-reduced network. The cascaded two-pass model is further compressed using knowledge distillation using a Kullback-Leibler divergence loss on the model posteriors. We demonstrate a 50% reduction in the size of a 41 M parameter cascaded teacher model with no noticeable degradation in ASR accuracy and a 30% reduction in latency【4】 Automatic channel selection and spatial feature integration for multi-channel speech recognition across various array topologies标题：用于跨各种阵列拓扑的多通道语音识别的自动通道选择和空间特征集成链接：https://arxiv.org/abs/2312.09746作者：Bingshen Mu,Pengcheng Guo,Dake Guo,Pan Zhou,Wei Chen,Lei Xie备注：Accepted by ICASSP 2024摘要：自动语音识别（ASR）已经取得了显着的进展，但它仍然面临着挑战，在现实世界中的远程场景中，在各种阵列拓扑结构中，每个阵列拓扑结构都有多个记录设备。CHiME-7 Distant ASR任务的重点是设计一个统一的系统，该系统能够推广具有多个记录设备的各种阵列拓扑，并在现实环境中提供可靠的识别性能。为了解决这一任务，我们引入了一个ASR系统，该系统在各种阵列拓扑中表现出卓越的性能。首先，我们提出了两个基于注意力的自动通道选择模块，从多个记录设备中为每个话语选择最有利的多通道信号子集。此外，我们引入通道间空间特征来增强多帧跨通道注意的有效性，帮助其提高空间信息感知能力。最后，我们提出了一个多层卷积融合模块借鉴U-Net架构的灵感，将多通道输出集成到一个单通道输出。CHiME-7语料库与Oracle分割的实验结果表明，在我们提出的ASR系统中引入的改进导致相对减少了40.1%的宏观日记属性字错误率（DA-WER）相比，基线ASR系统的Eval集。摘要：Automatic Speech Recognition (ASR) has shown remarkable progress, yet it still faces challenges in real-world distant scenarios across various array topologies each with multiple recording devices. The focal point of the CHiME-7 Distant ASR task is to devise a unified system capable of generalizing various array topologies that have multiple recording devices and offering reliable recognition performance in real-world environments. Addressing this task, we introduce an ASR system that demonstrates exceptional performance across various array topologies. First of all, we propose two attention-based automatic channel selection modules to select the most advantageous subset of multi-channel signals from multiple recording devices for each utterance. Furthermore, we introduce inter-channel spatial features to augment the effectiveness of multi-frame cross-channel attention, aiding it in improving the capability of spatial information awareness. Finally, we propose a multi-layer convolution fusion module drawing inspiration from the U-Net architecture to integrate the multi-channel output into a single-channel output. Experimental results on the CHiME-7 corpus with oracle segmentation demonstrate that the improvements introduced in our proposed ASR system lead to a relative reduction of 40.1% in the Macro Diarization Attributed Word Error Rates (DA-WER) when compared to the baseline ASR system on the Eval sets.【5】 HEAR: Hearing Enhanced Audio Response for Video-grounded Dialogue标题：Hear：针对基于视频的对话的听觉增强音频响应链接：https://arxiv.org/abs/2312.09736作者：Sunjae Yoon,Dahyun Kim,Eunseop Yoon,Hee Suk Yoon,Junyeong Kim,Chnag D. Yoo备注：EMNLP 2023, 14 pages, 13 figures摘要：基于视频的对话（VGD）旨在回答关于给定的多模态输入（包括视频、音频和对话历史）的问题。尽管已经在开发VGD系统以提高其响应的质量方面做出了许多努力，但是现有系统仅能够将信息合并到视频和文本中，并且当生成对问题的适当响应时，在从音频中提取必要的信息方面趋于困难。VGD系统似乎是聋的，因此，我们将当前系统忽略音频数据的这种症状称为聋反应。为了克服聋人响应问题，听力增强音频响应（Hearing Enhanced Audio Response，HEAR）框架被提出来通过在问题需要时选择性地关注音频来执行明智的听力。HEAR框架以模型不可知的方式增强VGD系统的准确性和可听度。HEAR在VGD数据集上得到验证（即，AVSD@DSTC 7和AVSD@DSTC 8），并显示了各种VGD系统的有效性。摘要：Video-grounded Dialogue (VGD) aims to answer questions regarding a given multi-modal input comprising video, audio, and dialogue history. Although there have been numerous efforts in developing VGD systems to improve the quality of their responses, existing systems are competent only to incorporate the information in the video and text and tend to struggle in extracting the necessary information from the audio when generating appropriate responses to the question. The VGD system seems to be deaf, and thus, we coin this symptom of current systems' ignoring audio data as a deaf response. To overcome the deaf response problem, Hearing Enhanced Audio Response (HEAR) framework is proposed to perform sensible listening by selectively attending to audio whenever the question requires it. The HEAR framework enhances the accuracy and audibility of VGD systems in a model-agnostic manner. HEAR is validated on VGD datasets (i.e., AVSD@DSTC7 and AVSD@DSTC8) and shows effectiveness with various VGD systems.【6】 LiteVSR: Efficient Visual Speech Recognition by Learning from Speech Representations of Unlabeled Data标题：LiteVSR：从未标记数据的语音表示中学习的高效视觉语音识别链接：https://arxiv.org/abs/2312.09727作者：Hendrik Laux,Emil Mededovic,Ahmed Hallawa,Lukas Martin,Arne Peine,Anke Schmeink备注：Accepted for publication at ICASSP 2024摘要：本文提出了一种新的，资源有效的方法，视觉语音识别（VSR）利用语音表示产生的任何训练的自动语音识别（ASR）模型。远离最近文献中普遍存在的资源密集型趋势，我们的方法从经过训练的基于Conformer的ASR模型中提取知识，在标准VSR基准测试中实现具有竞争力的性能，并且资源利用率显著降低。仅使用未标记的视听数据，我们的基线模型在LRS2和LRS3测试基准上分别实现了47.4%和54.7%的单词错误率（WER）。在有限的标记数据微调模型后，单词错误率降低到35%（LRS2）和45.7%（LRS3）。我们的模型可以在几天内在单个消费级GPU上进行训练，并且能够在过时的硬件上执行实时端到端VSR，这表明了一条通往更易于访问和资源高效的VSR方法的道路。摘要：This paper proposes a novel, resource-efficient approach to Visual Speech Recognition (VSR) leveraging speech representations produced by any trained Automatic Speech Recognition (ASR) model. Moving away from the resource-intensive trends prevalent in recent literature, our method distills knowledge from a trained Conformer-based ASR model, achieving competitive performance on standard VSR benchmarks with significantly less resource utilization. Using unlabeled audio-visual data only, our baseline model achieves a word error rate (WER) of 47.4% and 54.7% on the LRS2 and LRS3 test benchmarks, respectively. After fine-tuning the model with limited labeled data, the word error rate reduces to 35% (LRS2) and 45.7% (LRS3). Our model can be trained on a single consumer-grade GPU within a few days and is capable of performing real-time end-to-end VSR on dated hardware, suggesting a path towards more accessible and resource-efficient VSR methodologies.【7】 What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection标题：记住什么：用于音频深伪检测的自适应持续学习链接：https://arxiv.org/abs/2312.09651作者：Xiaohui Zhang,Jiangyan Yi,Chenglong Wang,Chuyuan Zhang,Siding Zeng,Jianhua Tao备注：Accepted by the main track The 38th Annual AAAI Conference on Artificial Intelligence (AAAI 2024)摘要：语音合成和语音转换的快速发展引起了人们的极大关注，因为这种技术可能会被滥用，这促使人们迫切需要有效的音频深度伪造检测机制。现有的检测模型在识别已知的deepfake音频方面取得了显着的成功，但在遇到新的攻击类型时会遇到困难。为了应对这一挑战，一个紧急有效的方法是持续学习。在本文中，我们提出了一种称为弧度权重修改（RWM）的持续学习方法，用于音频deepfake检测。RWM的基本概念涉及将所有类分为两组：那些在任务中具有紧凑特征分布的类，如真实音频，以及那些具有更广泛分布的类，如各种类型的假音频。这些区别通过类内余弦距离量化，随后作为RWM的基础，为不同的数据类型引入可训练的梯度修改方向。针对主流持续学习方法的实验评估揭示了RWM在知识获取和减轻音频deepfake检测中的遗忘方面的优势。此外，RWM的适用性超出了音频深度伪造检测，证明了其在图像识别等不同机器学习领域的潜在意义。摘要：The rapid evolution of speech synthesis and voice conversion has raised substantial concerns due to the potential misuse of such technology, prompting a pressing need for effective audio deepfake detection mechanisms. Existing detection models have shown remarkable success in discriminating known deepfake audio, but struggle when encountering new attack types. To address this challenge, one of the emergent effective approaches is continual learning. In this paper, we propose a continual learning approach called Radian Weight Modification (RWM) for audio deepfake detection. The fundamental concept underlying RWM involves categorizing all classes into two groups: those with compact feature distributions across tasks, such as genuine audio, and those with more spread-out distributions, like various types of fake audio. These distinctions are quantified by means of the in-class cosine distance, which subsequently serves as the basis for RWM to introduce a trainable gradient modification direction for distinct data types. Experimental evaluations against mainstream continual learning methods reveal the superiority of RWM in terms of knowledge acquisition and mitigating forgetting in audio deepfake detection. Furthermore, RWM's applicability extends beyond audio deepfake detection, demonstrating its potential significance in diverse machine learning domains such as image recognition.【8】 Stethoscope-guided Supervised Contrastive Learning for Cross-domain Adaptation on Respiratory Sound Classification标题：听诊器引导的呼吸音分类跨域适应监督对比学习链接：https://arxiv.org/abs/2312.09603作者：June-Woo Kim,Sangmin Bae,Won-Yang Cho,Byungjo Lee,Ho-Young Jung备注：accepted to ICASSP 2024摘要：尽管深度学习技术取得了显着进步，但由于可用数据的稀缺性，在肺音分类中实现令人满意的性能仍然是一个挑战。此外，呼吸声样本是从各种电子听诊器收集的，这可能会在训练模型中引入偏差。当测试数据集内或实际场景中发生显著的分布偏移时，它会大大降低性能。为了解决这个问题，我们引入跨域适应技术，将知识从源域转移到不同的目标域。特别是，通过考虑不同的听诊器类型作为单独的域，我们提出了一种新的听诊器引导的监督对比学习方法。该方法可以减轻任何域相关的差异，并且因此使得模型能够区分听诊器的记录变化的呼吸声音。在ICBHI数据集上的实验结果表明，所提出的方法有效地降低了领域依赖性，ICBHI得分为61.71%，比基线显著提高了2.16%。摘要：Despite the remarkable advances in deep learning technology, achieving satisfactory performance in lung sound classification remains a challenge due to the scarcity of available data. Moreover, the respiratory sound samples are collected from a variety of electronic stethoscopes, which could potentially introduce biases into the trained models. When a significant distribution shift occurs within the test dataset or in a practical scenario, it can substantially decrease the performance. To tackle this issue, we introduce cross-domain adaptation techniques, which transfer the knowledge from a source domain to a distinct target domain. In particular, by considering different stethoscope types as individual domains, we propose a novel stethoscope-guided supervised contrastive learning approach. This method can mitigate any domain-related disparities and thus enables the model to distinguish respiratory sounds of the recording variation of the stethoscope. The experimental results on the ICBHI dataset demonstrate that the proposed methods are effective in reducing the domain dependency and achieving the ICBHI Score of 61.71%, which is a significant improvement of 2.16% over the baseline.【9】 Leveraging Language ID to Calculate Intermediate CTC Loss for Enhanced Code-Switching Speech Recognition标题：利用语言ID计算增强码切换语音识别的中间CTC损失链接：https://arxiv.org/abs/2312.09583作者：Tzu-Ting Yang,Hsin-Wei Wang,Berlin Chen备注：Accepted to The 28th International Conference on Technologies and Applications of Artificial Intelligence (TAAI), in Chinese language摘要：近年来，端到端语音识别已经成为一种技术，它集成了传统自动语音识别模型的声学，发音字典和语言模型组件。不需要事先建立发音词典，就可以实现类似人类的识别。然而，由于语码转换的训练数据相对稀缺，当遇到这种现象时，ASR模型的性能往往会急剧下降。以往的研究大多将语码转换任务分解为处理单一语言的多个任务，然后分别学习每种语言的特定领域知识，从而简化了该模型的学习复杂性。因此，在本文中，我们试图将语言识别信息引入到ASR模型的编码器的中间层。我们的目标是生成声学特征，以更隐式的方式暗示语言区别，减少模型在处理语言切换时的混乱。摘要：In recent years, end-to-end speech recognition has emerged as a technology that integrates the acoustic, pronunciation dictionary, and language model components of the traditional Automatic Speech Recognition model. It is possible to achieve human-like recognition without the need to build a pronunciation dictionary in advance. However, due to the relative scarcity of training data on code-switching, the performance of ASR models tends to degrade drastically when encountering this phenomenon. Most past studies have simplified the learning complexity of the model by splitting the code-switching task into multiple tasks dealing with a single language and then learning the domain-specific knowledge of each language separately. Therefore, in this paper, we attempt to introduce language identification information into the middle layer of the ASR model's encoder. We aim to generate acoustic features that imply language distinctions in a more implicit way, reducing the model's confusion when dealing with language switching.【10】 Phoneme-aware Encoding for Prefix-tree-based Contextual ASR标题：基于前缀树的上下文ASR音素感知编码链接：https://arxiv.org/abs/2312.09582作者：Hayato Futami,Emiru Tsunoo,Yosuke Kashiwagi,Hiroaki Ogawa,Siddhant Arora,Shinji Watanabe备注：Accepted to ICASSP2024摘要：在语音识别应用中，识别上下文特定的生僻词（如专有名词）是很重要的。树约束的指针生成器（TCPGen）已经显示出用于此目的的承诺，它有效地偏置这样的词与前缀树。虽然最初的TCPGen依赖于基于字素的编码，但我们建议使用音素感知编码对其进行扩展，以更好地识别不寻常发音的单词。由于TCPGen处理偏置字作为子字单元，我们建议通过使用音素和子字之间的对齐来获得子字级音素感知编码。此外，我们建议将CTC的音素级预测注入到TCPGen的查询中，以便该模型更好地解释音素感知编码。我们用TCPGen对RNN传感器进行了ASR实验。我们观察到，在英语LibriSpeech和日语CSJ数据集上，提出的音素感知编码优于普通的基于图素的编码，证明了我们的方法在不同语言中的鲁棒性。摘要：In speech recognition applications, it is important to recognize context-specific rare words, such as proper nouns. Tree-constrained Pointer Generator (TCPGen) has shown promise for this purpose, which efficiently biases such words with a prefix tree. While the original TCPGen relies on grapheme-based encoding, we propose extending it with phoneme-aware encoding to better recognize words of unusual pronunciations. As TCPGen handles biasing words as subword units, we propose obtaining subword-level phoneme-aware encoding by using alignment between phonemes and subwords. Furthermore, we propose injecting phoneme-level predictions from CTC into queries of TCPGen so that the model better interprets the phoneme-aware encodings. We conducted ASR experiments with TCPGen for RNN transducer. We observed that proposed phoneme-aware encoding outperformed ordinary grapheme-based encoding on both the English LibriSpeech and Japanese CSJ datasets, demonstrating the robustness of our approach across linguistically diverse languages.【11】 A 1.6-mW Sparse Deep Learning Accelerator for Speech Separation标题：一种1.6 mW的稀疏深度学习语音分离加速器链接：https://arxiv.org/abs/2312.09580作者：Chih-Chyau Yang,Tian-Sheuan Chang备注：None摘要：语音处理上的低功耗深度学习加速器可以在边缘设备上实现实时应用。然而，大多数现有的加速器遭受高功耗和专注于图像应用。本文提出了一种低功耗的语音分离加速器，通过算法和硬件优化。在算法级，该模型被压缩与结构化的敏感性以及非结构化的修剪，并进一步量化为移位的8位浮点格式，而不是32位浮点格式。通过分解扩张卷积和转置卷积，跳过具有零内核和零激活值的计算。在硬件级别，压缩模型然后由具有八个独立乘法器和MAC（MAC）的架构支持，该架构具有简单的零跳过硬件以利用激活稀疏性和低功率处理。该方法使模型规模减少了95.44%，计算复杂度降低了93.88%.最终实现与台积电40纳米$工艺可以实现实时语音分离和功耗1.6 mW时，工作在150 MHz。归一化能量效率和面积效率分别为2.344 TOPS/W和14.42 GOPS/mm 2。摘要：Low power deep learning accelerators on the speech processing enable real-time applications on edge devices. However, most of the existing accelerators suffer from high power consumption and focus on image applications only. This paper presents a low power accelerator for speech separation through algorithm and hardware optimizations. At the algorithm level, the model is compressed with structured sensitivity as well as unstructured pruning, and further quantized to the shifted 8-bit floating-point format instead of the 32-bit floating-point format. The computations with the zero kernel and zero activation values are skipped by decomposition of the dilated and transposed convolutions. At the hardware level, the compressed model is then supported by an architecture with eight independent multipliers and accumulators (MACs) with a simple zero-skipping hardware to take advantage of the activation sparsity and low power processing. The proposed approach reduces the model size by 95.44\% and computation complexity by 93.88\%. The final implementation with the TSMC 40 $nm$ process can achieve real-time speech separation and consumes 1.6 mW power when operated at 150 MHz. The normalized energy efficiency and area efficiency are 2.344 TOPS/W and 14.42 GOPS/mm$^2$, respectively.【12】 Audio-visual fine-tuning of audio-only ASR models标题：纯音频ASR模型的视听微调链接：https://arxiv.org/abs/2312.09369作者：Avner May,Dmitriy Serdyuk,Ankit Parag Shah,Otavio Braga,Olivier Siohan摘要：视听自动语音识别（AV-ASR）模型在降低噪声语音的单词错误率方面非常有效，但需要大量转录的AV训练数据。最近，已经开发了视听自监督学习（SSL）方法来减少对转录AV数据的依赖，但这些方法非常复杂且计算昂贵。在这项工作中，我们建议用一个简单快速的SSL方法来代替这些昂贵的AV-SSL方法，然后进行AV监督微调。我们表明，这种方法在LRS 3-TED基准任务上与最先进的（SOTA）AV-SSL方法竞争（绝对WER在0.5%以内），同时更简单，更高效（预训练速度快12- 30倍）。此外，我们可以扩展这种方法，将SOTA仅音频ASR模型转换为AV模型。通过这样做，我们匹配SOTA AV-SSL结果，即使在预训练期间没有使用AV数据。摘要：Audio-visual automatic speech recognition (AV-ASR) models are very effective at reducing word error rates on noisy speech, but require large amounts of transcribed AV training data. Recently, audio-visual self-supervised learning (SSL) approaches have been developed to reduce this dependence on transcribed AV data, but these methods are quite complex and computationally expensive. In this work, we propose replacing these expensive AV-SSL methods with a simple and fast \textit{audio-only} SSL method, and then performing AV supervised fine-tuning. We show that this approach is competitive with state-of-the-art (SOTA) AV-SSL methods on the LRS3-TED benchmark task (within 0.5% absolute WER), while being dramatically simpler and more efficient (12-30x faster to pre-train). Furthermore, we show we can extend this approach to convert a SOTA audio-only ASR model into an AV model. By doing so, we match SOTA AV-SSL results, even though no AV data was used during pre-training.【13】 Efficient speech detection in environmental audio using acoustic recognition and knowledge distillation标题：基于声学识别和知识提取的环境音频语音检测链接：https://arxiv.org/abs/2312.09269作者：Drew Priebe,Burooj Ghani,Dan Stowell摘要：由于土地使用变化和全球变暖等因素，目前的生物多样性危机突出表明需要有效的生态监测方法。生物多样性声学监测已成为一种重要的监测工具。在声景监测项目中检测人类声音对于分析人类干扰和隐私过滤都是有用的。尽管近年来深度学习取得了重大进展，但由于内存和延迟限制，在紧凑设备上部署大型神经网络带来了挑战。我们的方法侧重于利用知识蒸馏技术来设计高效，轻量级的学生模型，用于生物声学中的语音检测。特别是，我们采用MobileNetV 3-Small-Pi模型来创建紧凑而有效的学生架构，以与更大的EcoVAD教师模型进行比较，EcoVAD教师模型是生态声学监测中备受好评的语音检测架构。比较分析包括检查MobileNetV 3-Small-Pi衍生学生模型的各种配置，以确定最佳性能。此外，对不同的蒸馏技术进行了全面的评价，以确定最有效的模型选择方法。我们的研究结果表明，蒸馏模型表现出相当的性能EcoVAD教师模型，表明一个有前途的方法来克服实时生态监测的计算障碍。摘要：The ongoing biodiversity crisis, driven by factors such as land-use change and global warming, emphasizes the need for effective ecological monitoring methods. Acoustic monitoring of biodiversity has emerged as an important monitoring tool. Detecting human voices in soundscape monitoring projects is useful both for analysing human disturbance and for privacy filtering. Despite significant strides in deep learning in recent years, the deployment of large neural networks on compact devices poses challenges due to memory and latency constraints. Our approach focuses on leveraging knowledge distillation techniques to design efficient, lightweight student models for speech detection in bioacoustics. In particular, we employed the MobileNetV3-Small-Pi model to create compact yet effective student architectures to compare against the larger EcoVAD teacher model, a well-regarded voice detection architecture in eco-acoustic monitoring. The comparative analysis included examining various configurations of the MobileNetV3-Small-Pi derived student models to identify optimal performance. Additionally, a thorough evaluation of different distillation techniques was conducted to ascertain the most effective method for model selection. Our findings revealed that the distilled models exhibited comparable performance to the EcoVAD teacher model, indicating a promising approach to overcoming computational barriers for real-time ecological monitoring.【14】 Acoustic models of Brazilian Portuguese Speech based on Neural Transformers标题：基于神经转换器的巴西葡萄牙语语音声学模型链接：https://arxiv.org/abs/2312.09265作者：Marcelo Matheus Gauy,Marcelo Finger备注：Under review at Journal of Brazilian Computer Society摘要：在大量未标记数据上训练的声学模型由可用于解决下游任务的自监督学习语音表示组成，可能在相应下游任务中对模型进行微调之后。在这项工作中，我们通过Transformer神经网络建立了巴西葡萄牙语语音的声学模型。该模型使用预训练技术的组合，在超过800 $小时的巴西葡萄牙语语音上进行了预训练。使用为检测巴西葡萄牙语使用者的呼吸功能不全而收集的标记数据集，我们在以下任务上微调预训练的Transformer神经网络：呼吸功能不全检测，性别识别和年龄组分类。我们将预先训练的Transformers在这些任务上的表现与没有预先训练的Transformers进行了比较，注意到显着的改善。特别是，呼吸功能不全检测的性能获得了迄今为止最好的报告结果，表明这种声学模型作为一个有前途的工具，语音作为生物标志物的方法。此外，性别识别的性能与英语中的最先进模型相当。摘要：An acoustic model, trained on a significant amount of unlabeled data, consists of a self-supervised learned speech representation useful for solving downstream tasks, perhaps after a fine-tuning of the model in the respective downstream task. In this work, we build an acoustic model of Brazilian Portuguese Speech through a Transformer neural network. This model was pretrained on more than $800$ hours of Brazilian Portuguese Speech, using a combination of pretraining techniques. Using a labeled dataset collected for the detection of respiratory insufficiency in Brazilian Portuguese speakers, we fine-tune the pretrained Transformer neural network on the following tasks: respiratory insufficiency detection, gender recognition and age group classification. We compare the performance of pretrained Transformers on these tasks with that of Transformers without previous pretraining, noting a significant improvement. In particular, the performance of respiratory insufficiency detection obtains the best reported results so far, indicating this kind of acoustic model as a promising tool for speech-as-biomarker approach. Moreover, the performance of gender recognition is comparable to the state of the art models in English.【15】 Multi-level graph learning for audio event classification and human-perceived annoyance rating prediction标题：基于多层图学习的音频事件分类和人的烦扰程度预测链接：https://arxiv.org/abs/2312.09952作者：Yuanbo Hou,Qiaoqiao Ren,Siyang Song,Yuxin Song,Wenwu Wang,Dick Botteldooren备注：Accepted by ICASSP 2024摘要：世界卫生组织关于环境噪声的报告估计，2200万人患有与各种来源的音频事件（AE）引起的噪声相关的慢性烦恼。烦恼可能导致健康问题和对代谢和认知系统的不良影响。在城市中，监测噪声水平并不能提供对明显AE的见解，更不用说它们与烦恼的关系了。为了创建与烦恼相关的监测，本文提出了一种基于图的模型来识别声景中的AE，并探讨不同AE与人类感知烦恼等级（AR）之间的关系。具体而言，本文提出了一种基于局部和全局语义图的轻量级多级图学习（MLGL），以同时执行音频事件分类（AEC）和人类烦恼等级预测（ARP）。实验表明：1）具有4.1M参数的MLGL通过使用局部和全局上下文感知图中的语义节点信息来改进AEC和ARP结果; 2）MLGL很好地捕获粗粒度和细粒度AE与AR之间的关系; 3）MLGL结果的统计分析表明，来自不同来源的一些AE与AR显著相关，这与先前关于人类对这些声源的感知的研究一致。摘要：WHO's report on environmental noise estimates that 22 M people suffer from chronic annoyance related to noise caused by audio events (AEs) from various sources. Annoyance may lead to health issues and adverse effects on metabolic and cognitive systems. In cities, monitoring noise levels does not provide insights into noticeable AEs, let alone their relations to annoyance. To create annoyance-related monitoring, this paper proposes a graph-based model to identify AEs in a soundscape, and explore relations between diverse AEs and human-perceived annoyance rating (AR). Specifically, this paper proposes a lightweight multi-level graph learning (MLGL) based on local and global semantic graphs to simultaneously perform audio event classification (AEC) and human annoyance rating prediction (ARP). Experiments show that: 1) MLGL with 4.1 M parameters improves AEC and ARP results by using semantic node information in local and global context aware graphs; 2) MLGL captures relations between coarse and fine-grained AEs and AR well; 3) Statistical analysis of MLGL results shows that some AEs from different sources significantly correlate with AR, which is consistent with previous research on human perception of these sound sources.【16】 Decoding Envelope and Frequency-Following EEG Responses to Continuous Speech Using Deep Neural Networks标题：用深度神经网络对连续语音的包络和频率跟随脑电响应进行解码链接：https://arxiv.org/abs/2312.09768作者：Mike Thornton,Danilo Mandic,Tobias Reichenbach摘要：脑电图（EEG）提供了一种非侵入性的手段，通过它可以在连续语音感知期间监测听者的听觉系统。可靠的脑电图解码器可以促进听力障碍的客观诊断，或在认知导向助听器中找到应用。在此之前，我们为ICASSP听觉EEG信号处理大挑战赛（SPGC）开发了解码器。这些解码器旨在解决匹配-不匹配任务：给定EEG记录的短时间段和两个候选语音段，任务是识别两个语音段中的哪一个与EEG段在时间上对齐或匹配。解码器利用皮层对语音包络的反应，以及与语音相关的频率跟随反应，将EEG记录与语音刺激相关联。在这里，我们全面记录解码器的开发方法。我们通过探索说话者特征（音高和性别）与分类准确性之间的关联来扩展我们之前的分析，并提供对数据集的一部分进行评估的解码器的最终性能的全面统计分析。最后，解码器的泛化能力的特点，通过使用一个完全不同的数据集，其中包含在各种语音收听条件下记录的EEG进行评估。结果表明，匹配-失配解码器实现了准确和鲁棒的分类精度，它们甚至可以作为听觉注意解码器，而无需额外的训练。摘要：The electroencephalogram (EEG) offers a non-invasive means by which a listener's auditory system may be monitored during continuous speech perception. Reliable auditory-EEG decoders could facilitate the objective diagnosis of hearing disorders, or find applications in cognitively-steered hearing aids. Previously, we developed decoders for the ICASSP Auditory EEG Signal Processing Grand Challenge (SPGC). These decoders aimed to solve the match-mismatch task: given a short temporal segment of EEG recordings, and two candidate speech segments, the task is to identify which of the two speech segments is temporally aligned, or matched, with the EEG segment. The decoders made use of cortical responses to the speech envelope, as well as speech-related frequency-following responses, to relate the EEG recordings to the speech stimuli. Here we comprehensively document the methods by which the decoders were developed. We extend our previous analysis by exploring the association between speaker characteristics (pitch and sex) and classification accuracy, and provide a full statistical analysis of the final performance of the decoders as evaluated on a heldout portion of the dataset. Finally, the generalisation capabilities of the decoders are characterised, by evaluating them using an entirely different dataset which contains EEG recorded under a variety of speech-listening conditions. The results show that the match-mismatch decoders achieve accurate and robust classification accuracies, and they can even serve as auditory attention decoders without additional training.【17】 U2-KWS: Unified Two-pass Open-vocabulary Keyword Spotting with Keyword Bias标题：U2-KWS：带有关键词偏向的统一两遍开放词汇表关键词发现链接：https://arxiv.org/abs/2312.09760作者：Ao Zhang,Pan Zhou,Kaixun Huang,Yong Zou,Ming Liu,Lei Xie备注：Accepted by ASRU2023摘要：开放式词汇表关键词发现（KWS），它允许用户自定义关键词，吸引了越来越多的兴趣。然而，基于声学模型和后处理的现有方法利用ASR训练标准来训练声学模型以对所有音素进行建模，使得声学模型对于KWS任务未被优化。为了解决这个问题，我们提出了一个新的统一的两遍开放词汇表KWS（U2-KWS）框架的启发两遍ASR模型U2。具体来说，我们采用CTC分支作为第一阶段模型来检测潜在的关键字候选人和解码器分支作为第二阶段模型来验证候选人。为了增强任何定制的关键字，我们重新设计了U2-KWS的U2训练过程，并通过音频和文本交叉注意将关键字信息添加到两个分支中。我们在我们的内部数据集和Aishell-1上进行实验。实验结果表明，当虚警率固定为0.5次/h时，U2-KWS系统相对于传统定制KWS系统的唤醒率有41%的显著提高。摘要：Open-vocabulary keyword spotting (KWS), which allows users to customize keywords, has attracted increasingly more interest. However, existing methods based on acoustic models and post-processing train the acoustic model with ASR training criteria to model all phonemes, making the acoustic model under-optimized for the KWS task. To solve this problem, we propose a novel unified two-pass open-vocabulary KWS (U2-KWS) framework inspired by the two-pass ASR model U2. Specifically, we employ the CTC branch as the first stage model to detect potential keyword candidates and the decoder branch as the second stage model to validate candidates. In order to enhance any customized keywords, we redesign the U2 training procedure for U2-KWS and add keyword information by audio and text cross-attention into both branches. We perform experiments on our internal dataset and Aishell-1. The results show that U2-KWS can achieve a significant relative wake-up rate improvement of 41% compared to the traditional customized KWS systems when the false alarm rate is fixed to 0.5 times per hour.【18】 Toward Deep Drum Source Separation标题：走向深鼓声源分选链接：https://arxiv.org/abs/2312.09663作者：Alessandro Ilic Mezza,Riccardo Giampiccolo,Alberto Bernardini,Augusto Sarti备注：9 pages, 2 figures. Submitted to Pattern Recognition Letters摘要：过去，由于数据可用性有限，鼓源分离领域面临着重大挑战，阻碍了在其他相关音频应用中取得成功的尖端深度学习方法的采用。在这篇手稿中，我们介绍了StemGMD，一个大规模的孤立的单乐器鼓干音频数据集。每一个音频剪辑是合成从表现鼓表演的录音，使用十个真正的冠冕堂皇的声学鼓包。StemGMD是迄今为止最大的鼓音频数据集，也是第一个包含标准九件套鼓套件中每种乐器的独立音频片段的数据集。我们利用StemGMD开发LarsNet，一种新的深度鼓源分离模型。通过一组专用的U-Net，LarsNet可以比实时更快地从立体声鼓混合中分离出五个茎，并且表现出显着优于最先进的非负谱时间因子分解方法。摘要：In the past, the field of drum source separation faced significant challenges due to limited data availability, hindering the adoption of cutting-edge deep learning methods that have found success in other related audio applications. In this manuscript, we introduce StemGMD, a large-scale audio dataset of isolated single-instrument drum stems. Each audio clip is synthesized from MIDI recordings of expressive drums performances using ten real-sounding acoustic drum kits. Totaling 1224 hours, StemGMD is the largest audio dataset of drums to date and the first to comprise isolated audio clips for every instrument in a canonical nine-piece drum kit. We leverage StemGMD to develop LarsNet, a novel deep drum source separation model. Through a bank of dedicated U-Nets, LarsNet can separate five stems from a stereo drum mixture faster than real-time and is shown to significantly outperform state-of-the-art nonnegative spectro-temporal factorization methods.【19】 Fine-Tuned Self-Supervised Speech Representations for Language Diarization in Multilingual Code-Switched Speech标题：用于多语码转换语音中语言对象化的微调自监督语音表示链接：https://arxiv.org/abs/2312.09645作者：Geoffrey Frost,Emily Morris,Joshua Jansen van Vüren,Thomas Niesler备注：Presented at SACAIR 2022摘要：注释多语言代码转换语料库是一个艰苦的过程，需要专业的语言学知识。这部分是由于大量的语言组合，可能会出现在内和跨话语，这可能需要几个注释不同的语言专业知识，以考虑发言顺序。这既费时又费钱。如果在注释开始之前已知话语中的口语语言及其边界，则允许将片段并行地分配给相关语言专家，这将是有用的。为了解决这个问题，我们研究了一个连续的多语言diarizer使用微调的语音表示从一个大的预训练的自监督架构（WavLM）提取的发展。我们的实验与代码切换语料库组成的五个南非语言（isiZulu，isiXhosa，Seturan，Sesuran和英语），并显示大量的日记错误率改善的语言家庭，语言群体，和个别语言的基线系统。摘要：Annotating a multilingual code-switched corpus is a painstaking process requiring specialist linguistic expertise. This is partly due to the large number of language combinations that may appear within and across utterances, which might require several annotators with different linguistic expertise to consider an utterance sequentially. This is time-consuming and costly. It would be useful if the spoken languages in an utterance and the boundaries thereof were known before annotation commences, to allow segments to be assigned to the relevant language experts in parallel. To address this, we investigate the development of a continuous multilingual language diarizer using fine-tuned speech representations extracted from a large pre-trained self-supervised architecture (WavLM). We experiment with a code-switched corpus consisting of five South African languages (isiZulu, isiXhosa, Setswana, Sesotho and English) and show substantial diarization error rate improvements for language families, language groups, and individual languages over baseline systems.【20】 Self-Supervised Learning for Anomalous Sound Detection标题：用于异常声音检测的自监督学习链接：https://arxiv.org/abs/2312.09578作者：Kevin Wilkinghoff备注：Accepted for presentation at IEEE ICASSP 2024摘要：最先进的异常声音检测（ASD）系统通常通过使用辅助分类任务来学习嵌入空间来训练。这样做使系统能够学习对噪声鲁棒并且忽略非目标声音事件但需要手动注释的Meta信息用作类别标签的嵌入。然而，分类任务变得越不困难，嵌入的信息就越少，并且所得到的ASD性能就越差。解决这个问题的一个方法是使用自监督学习（SSL）。在这项工作中，功能交换（EASTEX），一个简单而有效的SSL方法ASD，提出。此外，还将CNOEx与现有的SSL方法进行了比较和结合。作为主要结果，获得了DCASE2023 ASD数据集的新的最先进性能，该性能大大优于该数据集上所有其他已发表的结果。摘要：State-of-the-art anomalous sound detection (ASD) systems are often trained by using an auxiliary classification task to learn an embedding space. Doing so enables the system to learn embeddings that are robust to noise and are ignoring non-target sound events but requires manually annotated meta information to be used as class labels. However, the less difficult the classification task becomes, the less informative are the embeddings and the worse is the resulting ASD performance. A solution to this problem is to utilize self-supervised learning (SSL). In this work, feature exchange (FeatEx), a simple yet effective SSL approach for ASD, is proposed. In addition, FeatEx is compared to and combined with existing SSL approaches. As the main result, a new state-of-the-art performance for the DCASE2023 ASD dataset is obtained that outperforms all other published results on this dataset by a large margin.【21】 IR-UWB Radar-Based Contactless Silent Speech Recognition of Vowels, Consonants, Words, and Phrases标题：基于IR-UWB雷达的元音、辅音、单词和短语的非接触式无声语音识别链接：https://arxiv.org/abs/2312.09572作者：Sunghwa Lee,Younghoon Shin,Myungjong Kim,Jiwon Seo备注：Submitted to IEEE Access摘要：已经提出了几种用于无声语音识别（SSR）的感测技术;然而，这些方法中的许多方法需要侵入性过程或使用胶带或胶水将传感器附接到皮肤，使得它们不适合在日常生活中频繁使用。相比之下，脉冲无线电超宽带（IR-UWB）雷达可以在不与用户的关节和相关身体部位发生物理接触的情况下工作，为SSR提供了几个优势。这些优点包括高距离分辨率、高穿透性、低功耗、对外部光或声音干扰的鲁棒性以及嵌入空间受限的手持设备中的能力。这项研究表明，IR-UWB雷达为基础的非接触SSR使用四种类型的语音刺激（元音，辅音，单词和短语）。为了实现这一目标，提出了一种新的语音特征提取算法专门设计的IR-UWB雷达基SSR。通过对提取的语音特征应用分类算法来识别每个语音刺激。两种不同的算法，多维动态时间规整（MD-DTW）和深度神经网络-隐马尔可夫模型（DNN-HMM），进行了比较的分类任务。此外，有利的雷达天线位置，无论是在用户的嘴唇前面或用户的下巴下面，被确定为实现更高的识别精度。实验结果表明，所提出的语音特征提取算法结合DNN-HMM分类元音，辅音，单词和短语的有效性。值得注意的是，这项研究代表了第一次使用非接触式雷达的音素级SSR演示。摘要：Several sensing techniques have been proposed for silent speech recognition (SSR); however, many of these methods require invasive processes or sensor attachment to the skin using adhesive tape or glue, rendering them unsuitable for frequent use in daily life. By contrast, impulse radio ultra-wideband (IR-UWB) radar can operate without physical contact with users' articulators and related body parts, offering several advantages for SSR. These advantages include high range resolution, high penetrability, low power consumption, robustness to external light or sound interference, and the ability to be embedded in space-constrained handheld devices. This study demonstrated IR-UWB radar-based contactless SSR using four types of speech stimuli (vowels, consonants, words, and phrases). To achieve this, a novel speech feature extraction algorithm specifically designed for IR-UWB radar-based SSR is proposed. Each speech stimulus is recognized by applying a classification algorithm to the extracted speech features. Two different algorithms, multidimensional dynamic time warping (MD-DTW) and deep neural network-hidden Markov model (DNN-HMM), were compared for the classification task. Additionally, a favorable radar antenna position, either in front of the user's lips or below the user's chin, was determined to achieve higher recognition accuracy. Experimental results demonstrated the efficacy of the proposed speech feature extraction algorithm combined with DNN-HMM for classifying vowels, consonants, words, and phrases. Notably, this study represents the first demonstration of phoneme-level SSR using contactless radar.eess.AS音频处理【1】 Multi-level graph learning for audio event classification and human-perceived annoyance rating prediction标题：基于多层图学习的音频事件分类和人的烦扰程度预测链接：https://arxiv.org/abs/2312.09952作者：Yuanbo Hou,Qiaoqiao Ren,Siyang Song,Yuxin Song,Wenwu Wang,Dick Botteldooren备注：Accepted by ICASSP 2024摘要：世界卫生组织关于环境噪声的报告估计，2200万人患有与各种来源的音频事件（AE）引起的噪声相关的慢性烦恼。烦恼可能导致健康问题和对代谢和认知系统的不良影响。在城市中，监测噪声水平并不能提供对明显AE的见解，更不用说它们与烦恼的关系了。为了创建与烦恼相关的监测，本文提出了一种基于图的模型来识别声景中的AE，并探讨不同AE与人类感知烦恼等级（AR）之间的关系。具体而言，本文提出了一种基于局部和全局语义图的轻量级多级图学习（MLGL），以同时执行音频事件分类（AEC）和人类烦恼等级预测（ARP）。实验表明：1）具有4.1M参数的MLGL通过使用局部和全局上下文感知图中的语义节点信息来改进AEC和ARP结果; 2）MLGL很好地捕获粗粒度和细粒度AE与AR之间的关系; 3）MLGL结果的统计分析表明，来自不同来源的一些AE与AR显著相关，这与先前关于人类对这些声源的感知的研究一致。摘要：WHO's report on environmental noise estimates that 22 M people suffer from chronic annoyance related to noise caused by audio events (AEs) from various sources. Annoyance may lead to health issues and adverse effects on metabolic and cognitive systems. In cities, monitoring noise levels does not provide insights into noticeable AEs, let alone their relations to annoyance. To create annoyance-related monitoring, this paper proposes a graph-based model to identify AEs in a soundscape, and explore relations between diverse AEs and human-perceived annoyance rating (AR). Specifically, this paper proposes a lightweight multi-level graph learning (MLGL) based on local and global semantic graphs to simultaneously perform audio event classification (AEC) and human annoyance rating prediction (ARP). Experiments show that: 1) MLGL with 4.1 M parameters improves AEC and ARP results by using semantic node information in local and global context aware graphs; 2) MLGL captures relations between coarse and fine-grained AEs and AR well; 3) Statistical analysis of MLGL results shows that some AEs from different sources significantly correlate with AR, which is consistent with previous research on human perception of these sound sources.【2】 Decoding Envelope and Frequency-Following EEG Responses to Continuous Speech Using Deep Neural Networks标题：使用深度神经网络解码连续语音的包络和频率跟随EEG响应链接：https://arxiv.org/abs/2312.09768作者：Mike Thornton,Danilo Mandic,Tobias Reichenbach摘要：脑电图（EEG）提供了一种非侵入性的手段，通过它可以在连续语音感知期间监测听者的听觉系统。可靠的脑电图解码器可以促进听力障碍的客观诊断，或在认知导向助听器中找到应用。在此之前，我们为ICASSP听觉EEG信号处理大挑战赛（SPGC）开发了解码器。这些解码器旨在解决匹配-不匹配任务：给定EEG记录的短时间段和两个候选语音段，任务是识别两个语音段中的哪一个与EEG段在时间上对齐或匹配。解码器利用皮层对语音包络的反应，以及与语音相关的频率跟随反应，将EEG记录与语音刺激相关联。在这里，我们全面记录解码器的开发方法。我们通过探索说话者特征（音高和性别）与分类准确性之间的关联来扩展我们之前的分析，并提供对数据集的一部分进行评估的解码器的最终性能的全面统计分析。最后，解码器的泛化能力的特点，通过使用一个完全不同的数据集，其中包含在各种语音收听条件下记录的EEG进行评估。结果表明，匹配-失配解码器实现了准确和鲁棒的分类精度，它们甚至可以作为听觉注意解码器，而无需额外的训练。摘要：The electroencephalogram (EEG) offers a non-invasive means by which a listener's auditory system may be monitored during continuous speech perception. Reliable auditory-EEG decoders could facilitate the objective diagnosis of hearing disorders, or find applications in cognitively-steered hearing aids. Previously, we developed decoders for the ICASSP Auditory EEG Signal Processing Grand Challenge (SPGC). These decoders aimed to solve the match-mismatch task: given a short temporal segment of EEG recordings, and two candidate speech segments, the task is to identify which of the two speech segments is temporally aligned, or matched, with the EEG segment. The decoders made use of cortical responses to the speech envelope, as well as speech-related frequency-following responses, to relate the EEG recordings to the speech stimuli. Here we comprehensively document the methods by which the decoders were developed. We extend our previous analysis by exploring the association between speaker characteristics (pitch and sex) and classification accuracy, and provide a full statistical analysis of the final performance of the decoders as evaluated on a heldout portion of the dataset. Finally, the generalisation capabilities of the decoders are characterised, by evaluating them using an entirely different dataset which contains EEG recorded under a variety of speech-listening conditions. The results show that the match-mismatch decoders achieve accurate and robust classification accuracies, and they can even serve as auditory attention decoders without additional training.【3】 U2-KWS: Unified Two-pass Open-vocabulary Keyword Spotting with Keyword Bias标题：U2-KWS：带有关键词偏向的统一两遍开放词汇表关键词发现链接：https://arxiv.org/abs/2312.09760作者：Ao Zhang,Pan Zhou,Kaixun Huang,Yong Zou,Ming Liu,Lei Xie备注：Accepted by ASRU2023摘要：开放式词汇表关键词发现（KWS），它允许用户自定义关键词，吸引了越来越多的兴趣。然而，基于声学模型和后处理的现有方法利用ASR训练标准来训练声学模型以对所有音素进行建模，使得声学模型对于KWS任务未被优化。为了解决这个问题，我们提出了一个新的统一的两遍开放词汇表KWS（U2-KWS）框架的启发两遍ASR模型U2。具体来说，我们采用CTC分支作为第一阶段模型来检测潜在的关键字候选人和解码器分支作为第二阶段模型来验证候选人。为了增强任何定制的关键字，我们重新设计了U2-KWS的U2训练过程，并通过音频和文本交叉注意将关键字信息添加到两个分支中。我们在我们的内部数据集和Aishell-1上进行实验。实验结果表明，当虚警率固定为0.5次/h时，U2-KWS系统相对于传统定制KWS系统的唤醒率有41%的显著提高。摘要：Open-vocabulary keyword spotting (KWS), which allows users to customize keywords, has attracted increasingly more interest. However, existing methods based on acoustic models and post-processing train the acoustic model with ASR training criteria to model all phonemes, making the acoustic model under-optimized for the KWS task. To solve this problem, we propose a novel unified two-pass open-vocabulary KWS (U2-KWS) framework inspired by the two-pass ASR model U2. Specifically, we employ the CTC branch as the first stage model to detect potential keyword candidates and the decoder branch as the second stage model to validate candidates. In order to enhance any customized keywords, we redesign the U2 training procedure for U2-KWS and add keyword information by audio and text cross-attention into both branches. We perform experiments on our internal dataset and Aishell-1. The results show that U2-KWS can achieve a significant relative wake-up rate improvement of 41% compared to the traditional customized KWS systems when the false alarm rate is fixed to 0.5 times per hour.【4】 SELM: Speech Enhancement Using Discrete Tokens and Language Models标题：SELM：使用离散标记和语言模型的语音增强链接：https://arxiv.org/abs/2312.09747作者：Ziqian Wang,Xinfa Zhu,Zihan Zhang,YuanJun Lv,Ning Jiang,Guoqing Zhao,Lei Xie备注：5 pages, 2 figures. ACCEPTED BY ICASSP 2024摘要：近年来，语言模型在各种语音生成任务中表现出了优异的性能，证明了其强大的语义上下文建模能力。鉴于语音生成和语音增强之间的内在相似性，利用语义信息对语音增强任务具有潜在的优势。鉴于此，我们提出了SELM，一种新的语音增强范例，它集成了离散的令牌，并利用语言模型。SELM包括三个阶段：编码、建模和解码。我们使用预训练的自监督学习（SSL）模型和k-means标记器将连续波形信号转换为离散标记。然后，语言模型在这些标记中捕获全面的上下文信息。最后，去标记器和HiFi-GAN将它们恢复为增强的语音。实验结果表明，SELM实现了相当的客观指标的性能，同时在主观感知的优越结果。我们的演示可以在https://honee-w.github.io/SELM/上找到。摘要：Language models (LMs) have shown superior performances in various speech generation tasks recently, demonstrating their powerful ability for semantic context modeling. Given the intrinsic similarity between speech generation and speech enhancement, harnessing semantic information holds potential advantages for speech enhancement tasks. In light of this, we propose SELM, a novel paradigm for speech enhancement, which integrates discrete tokens and leverages language models. SELM comprises three stages: encoding, modeling, and decoding. We transform continuous waveform signals into discrete tokens using pre-trained self-supervised learning (SSL) models and a k-means tokenizer. Language models then capture comprehensive contextual information within these tokens. Finally, a detokenizer and HiFi-GAN restore them into enhanced speech. Experimental results demonstrate that SELM achieves comparable performance in objective metrics alongside superior results in subjective perception. Our demos are available https://honee-w.github.io/SELM/.【5】 Toward Deep Drum Source Separation标题：走向深鼓声源分选链接：https://arxiv.org/abs/2312.09663作者：Alessandro Ilic Mezza,Riccardo Giampiccolo,Alberto Bernardini,Augusto Sarti备注：9 pages, 2 figures. Submitted to Pattern Recognition Letters摘要：过去，由于数据可用性有限，鼓源分离领域面临着重大挑战，阻碍了在其他相关音频应用中取得成功的尖端深度学习方法的采用。在这篇手稿中，我们介绍了StemGMD，一个大规模的孤立的单乐器鼓干音频数据集。每一个音频剪辑是合成从表现鼓表演的录音，使用十个真正的冠冕堂皇的声学鼓包。StemGMD是迄今为止最大的鼓音频数据集，也是第一个包含标准九件套鼓套件中每种乐器的独立音频片段的数据集。我们利用StemGMD开发LarsNet，一种新的深度鼓源分离模型。通过一组专用的U-Net，LarsNet可以比实时更快地从立体声鼓混合中分离出五个茎，并且表现出显着优于最先进的非负谱时间因子分解方法。摘要：In the past, the field of drum source separation faced significant challenges due to limited data availability, hindering the adoption of cutting-edge deep learning methods that have found success in other related audio applications. In this manuscript, we introduce StemGMD, a large-scale audio dataset of isolated single-instrument drum stems. Each audio clip is synthesized from MIDI recordings of expressive drums performances using ten real-sounding acoustic drum kits. Totaling 1224 hours, StemGMD is the largest audio dataset of drums to date and the first to comprise isolated audio clips for every instrument in a canonical nine-piece drum kit. We leverage StemGMD to develop LarsNet, a novel deep drum source separation model. Through a bank of dedicated U-Nets, LarsNet can separate five stems from a stereo drum mixture faster than real-time and is shown to significantly outperform state-of-the-art nonnegative spectro-temporal factorization methods.【6】 Fine-Tuned Self-Supervised Speech Representations for Language Diarization in Multilingual Code-Switched Speech标题：用于多语码转换语音中语言对象化的微调自监督语音表示链接：https://arxiv.org/abs/2312.09645作者：Geoffrey Frost,Emily Morris,Joshua Jansen van Vüren,Thomas Niesler备注：Presented at SACAIR 2022摘要：注释多语言代码转换语料库是一个艰苦的过程，需要专业的语言学知识。这部分是由于大量的语言组合，可能会出现在内和跨话语，这可能需要几个注释不同的语言专业知识，以考虑发言顺序。这既费时又费钱。如果在注释开始之前已知话语中的口语语言及其边界，则允许将片段并行地分配给相关语言专家，这将是有用的。为了解决这个问题，我们研究了一个连续的多语言diarizer使用微调的语音表示从一个大的预训练的自监督架构（WavLM）提取的发展。我们的实验与代码切换语料库组成的五个南非语言（isiZulu，isiXhosa，Seturan，Sesuran和英语），并显示大量的日记错误率改善的语言家庭，语言群体，和个别语言的基线系统。摘要：Annotating a multilingual code-switched corpus is a painstaking process requiring specialist linguistic expertise. This is partly due to the large number of language combinations that may appear within and across utterances, which might require several annotators with different linguistic expertise to consider an utterance sequentially. This is time-consuming and costly. It would be useful if the spoken languages in an utterance and the boundaries thereof were known before annotation commences, to allow segments to be assigned to the relevant language experts in parallel. To address this, we investigate the development of a continuous multilingual language diarizer using fine-tuned speech representations extracted from a large pre-trained self-supervised architecture (WavLM). We experiment with a code-switched corpus consisting of five South African languages (isiZulu, isiXhosa, Setswana, Sesotho and English) and show substantial diarization error rate improvements for language families, language groups, and individual languages over baseline systems.【7】 A Deep Representation Learning-based Speech Enhancement Method Using Complex Convolution Recurrent Variational Autoencoder标题：基于深度表示学习的复卷积递归变分自动编码器语音增强方法链接：https://arxiv.org/abs/2312.09620作者：Yang Xiang,Jingguang Tian,Xinhui Hu,Xinkang Xu,ZhaoHui Yin备注：Accepted by ICASSP 2024摘要：通常，深度神经网络（DNN）的性能在很大程度上取决于数据表示学习的质量。我们的初步工作强调了深度表示学习（DRL）在语音增强（SE）应用中的重要性。具体来说，我们的初始SE算法采用了门控的经常性单位变分自编码器（VAE）与高斯分布，以提高某些现有的SE系统的性能。基于我们的初步框架，本文介绍了一种使用具有VAE的深度复杂卷积递归网络（DCCRN-VAE）进行SE的新方法。DCCRN-VAE假设信号的潜变量遵循DCCRN建模的复高斯分布，因为这些分布可以更好地捕获复杂信号的行为。此外，我们提出了DCCRN-VAE中的残余损失的应用，以进一步提高增强语音的质量。{与我们的初步工作相比，DCCRN-VAE为DRL引入了更复杂的DCCRN结构和概率分布。此外，与DCCRN相比，DCCRN-VAE采用更先进的DRL策略。实验结果表明，所提出的SE算法优于我们的初步SE框架和国家的最先进的DCCRN SE方法的尺度不变的信号失真比，语音质量和语音清晰度。摘要：Generally, the performance of deep neural networks (DNNs) heavily depends on the quality of data representation learning. Our preliminary work has emphasized the significance of deep representation learning (DRL) in the context of speech enhancement (SE) applications. Specifically, our initial SE algorithm employed a gated recurrent unit variational autoencoder (VAE) with a Gaussian distribution to enhance the performance of certain existing SE systems. Building upon our preliminary framework, this paper introduces a novel approach for SE using deep complex convolutional recurrent networks with a VAE (DCCRN-VAE). DCCRN-VAE assumes that the latent variables of signals follow complex Gaussian distributions that are modeled by DCCRN, as these distributions can better capture the behaviors of complex signals. Additionally, we propose the application of a residual loss in DCCRN-VAE to further improve the quality of the enhanced speech. {Compared to our preliminary work, DCCRN-VAE introduces a more sophisticated DCCRN structure and probability distribution for DRL. Furthermore, in comparison to DCCRN, DCCRN-VAE employs a more advanced DRL strategy. The experimental results demonstrate that the proposed SE algorithm outperforms both our preliminary SE framework and the state-of-the-art DCCRN SE method in terms of scale-invariant signal-to-distortion ratio, speech quality, and speech intelligibility.【8】 Self-Supervised Learning for Anomalous Sound Detection标题：用于异常声音检测的自监督学习链接：https://arxiv.org/abs/2312.09578作者：Kevin Wilkinghoff备注：Accepted for presentation at IEEE ICASSP 2024摘要：最先进的异常声音检测（ASD）系统通常通过使用辅助分类任务来学习嵌入空间来训练。这样做使系统能够学习对噪声鲁棒并且忽略非目标声音事件但需要手动注释的Meta信息用作类别标签的嵌入。然而，分类任务变得越不困难，嵌入的信息就越少，并且所得到的ASD性能就越差。解决这个问题的一个方法是使用自监督学习（SSL）。在这项工作中，功能交换（EASTEX），一个简单而有效的SSL方法ASD，提出。此外，还将CNOEx与现有的SSL方法进行了比较和结合。作为主要结果，获得了DCASE2023 ASD数据集的新的最先进性能，该性能大大优于该数据集上所有其他已发表的结果。摘要：State-of-the-art anomalous sound detection (ASD) systems are often trained by using an auxiliary classification task to learn an embedding space. Doing so enables the system to learn embeddings that are robust to noise and are ignoring non-target sound events but requires manually annotated meta information to be used as class labels. However, the less difficult the classification task becomes, the less informative are the embeddings and the worse is the resulting ASD performance. A solution to this problem is to utilize self-supervised learning (SSL). In this work, feature exchange (FeatEx), a simple yet effective SSL approach for ASD, is proposed. In addition, FeatEx is compared to and combined with existing SSL approaches. As the main result, a new state-of-the-art performance for the DCASE2023 ASD dataset is obtained that outperforms all other published results on this dataset by a large margin.【9】 IR-UWB Radar-Based Contactless Silent Speech Recognition of Vowels, Consonants, Words, and Phrases标题：基于IR-UWB雷达的元音、辅音、单词和短语的非接触式无声语音识别链接：https://arxiv.org/abs/2312.09572作者：Sunghwa Lee,Younghoon Shin,Myungjong Kim,Jiwon Seo备注：Submitted to IEEE Access摘要：已经提出了几种用于无声语音识别（SSR）的感测技术;然而，这些方法中的许多方法需要侵入性过程或使用胶带或胶水将传感器附接到皮肤，使得它们不适合在日常生活中频繁使用。相比之下，脉冲无线电超宽带（IR-UWB）雷达可以在不与用户的关节和相关身体部位发生物理接触的情况下工作，为SSR提供了几个优势。这些优点包括高距离分辨率、高穿透性、低功耗、对外部光或声音干扰的鲁棒性以及嵌入空间受限的手持设备中的能力。这项研究表明，IR-UWB雷达为基础的非接触SSR使用四种类型的语音刺激（元音，辅音，单词和短语）。为了实现这一目标，提出了一种新的语音特征提取算法专门设计的IR-UWB雷达基SSR。通过对提取的语音特征应用分类算法来识别每个语音刺激。两种不同的算法，多维动态时间规整（MD-DTW）和深度神经网络-隐马尔可夫模型（DNN-HMM），进行了比较的分类任务。此外，有利的雷达天线位置，无论是在用户的嘴唇前面或用户的下巴下面，被确定为实现更高的识别精度。实验结果表明，所提出的语音特征提取算法结合DNN-HMM分类元音，辅音，单词和短语的有效性。值得注意的是，这项研究代表了第一次使用非接触式雷达的音素级SSR演示。摘要：Several sensing techniques have been proposed for silent speech recognition (SSR); however, many of these methods require invasive processes or sensor attachment to the skin using adhesive tape or glue, rendering them unsuitable for frequent use in daily life. By contrast, impulse radio ultra-wideband (IR-UWB) radar can operate without physical contact with users' articulators and related body parts, offering several advantages for SSR. These advantages include high range resolution, high penetrability, low power consumption, robustness to external light or sound interference, and the ability to be embedded in space-constrained handheld devices. This study demonstrated IR-UWB radar-based contactless SSR using four types of speech stimuli (vowels, consonants, words, and phrases). To achieve this, a novel speech feature extraction algorithm specifically designed for IR-UWB radar-based SSR is proposed. Each speech stimulus is recognized by applying a classification algorithm to the extracted speech features. Two different algorithms, multidimensional dynamic time warping (MD-DTW) and deep neural network-hidden Markov model (DNN-HMM), were compared for the classification task. Additionally, a favorable radar antenna position, either in front of the user's lips or below the user's chin, was determined to achieve higher recognition accuracy. Experimental results demonstrated the efficacy of the proposed speech feature extraction algorithm combined with DNN-HMM for classifying vowels, consonants, words, and phrases. Notably, this study represents the first demonstration of phoneme-level SSR using contactless radar.【10】 Livestock feeding behavior: A tutorial review on automated techniques for ruminant monitoring标题：家畜饲养行为：反刍动物监测自动化技术的教程综述链接：https://arxiv.org/abs/2312.09259作者：José Chelotti,Luciano Martinez-Rau,Mariano Ferrero,Leandro Vignolo,Julio Galli,Alejandra Planisich,H. Leonardo Rufiner,Leonardo Giovanini备注：Preprint submitted to biosystems engineering journal摘要：家畜饲养行为是畜牧业和农业领域的一个有影响力的研究领域。近年来，人们对用于监测反刍动物行为的自动化系统越来越感兴趣。尽管在过去的十年中取得了进展，但仍然有很多工作要做，并了解测量和分析牲畜饲养行为的方法。自动监测系统主要使用运动、声学和图像传感器来收集动物行为数据。现有方法的性能评价是一项复杂的任务，研究之间的直接比较是困难的。从实验中使用的数据和性能指标的多样性开始，有几个因素阻止了直接比较。据我们所知，这项工作代表了第一个辅导式的反刍动物的摄食行为的分析，强调传感方法，信号处理和计算智能方法之间的关系。它评估了主要的传感方法（即基于运动，声音，图像/视频和压力）和主要技术来测量和分析与喂食行为相关的信号，评估它们在不同环境和情况下的使用。它还强调了自动监测系统的潜力，以提供有价值的信息，提高我们对牲畜饲养行为的理解。由于这些系统对生产系统和研究的影响，它们的相关性越来越重要。最后，本文结束了讨论未来的挑战和机遇，在牲畜饲养行为监测。摘要：Livestock feeding behavior is an influential research area for those involved in animal husbandry and agriculture. In recent years, there has been a growing interest in automated systems for monitoring the behavior of ruminants. Despite the developments accomplished in the last decade, there is still much to do and learn about the methods for measuring and analyzing livestock feeding behavior. Automated monitoring systems mainly use motion, acoustic, and image sensors to collect animal behavioral data. The performance evaluation of existing methods is a complex task and direct comparisons between studies are difficult. Several factors prevent a direct comparison, starting from the diversity of data and performance metrics used in the experiments. To the best of our knowledge, this work represents the first tutorial-style review on the analysis of the feeding behavior of ruminants, emphasizing the relationship between sensing methodologies, signal processing and computational intelligence methods. It assesses the main sensing methodologies (i.e. based on movement, sound, images/videos and pressure) and the main techniques to measure and analyze the signals associated with feeding behavior, evaluating their use in different settings and situations. It also highlights the potentiality of automated monitoring systems to provide valuable information that improves our understanding of livestock feeding behavior. The relevance of these systems is increasingly important due to their impact on production systems and research. Finally, the paper closes by discussing future challenges and opportunities in livestock feeding behavior monitoring.【11】 Understanding Probe Behaviors through Variational Bounds of Mutual Information标题：通过互信息的变化界理解探测行为链接：https://arxiv.org/abs/2312.10019作者：Kwanghee Choi,Jee-weon Jung,Shinji Watanabe备注：Accepted to ICASSP 2024, implementation available at this https URL摘要：随着自我监督表征的成功，研究人员寻求更好地理解表征中封装的信息。在各种可解释性方法中，我们专注于基于分类的线性探测。我们的目标是通过构建一个利用信息论的新数学框架，促进对线性探测的坚实理解并提供指导方针。首先，我们将探测与互信息（MI）的变分界限联系起来，以放松探测设计，将线性探测等同于微调。然后，我们通过我们的数学框架来研究探索的经验行为和实践。我们分析了逐层性能曲线是凸的，这似乎违反了数据处理不等式。然而，我们表明，中间表示可以有最大的MI估计，因为更好的可分性和降低MI之间的权衡。我们进一步建议，线性可分表示的边缘可以作为衡量“表示的好”的一个标准。“我们还将准确性与MI作为衡量标准进行了比较。最后，我们通过观察保留单词和音素信息的自监督语音模型来实证验证我们的主张。摘要：With the success of self-supervised representations, researchers seek a better understanding of the information encapsulated within a representation. Among various interpretability methods, we focus on classification-based linear probing. We aim to foster a solid understanding and provide guidelines for linear probing by constructing a novel mathematical framework leveraging information theory. First, we connect probing with the variational bounds of mutual information (MI) to relax the probe design, equating linear probing with fine-tuning. Then, we investigate empirical behaviors and practices of probing through our mathematical framework. We analyze the layer-wise performance curve being convex, which seemingly violates the data processing inequality. However, we show that the intermediate representations can have the biggest MI estimate because of the tradeoff between better separability and decreasing MI. We further suggest that the margin of linearly separable representations can be a criterion for measuring the "goodness of representation." We also compare accuracy with MI as the measuring criteria. Finally, we empirically validate our claims by observing the self-supervised speech models on retaining word and phoneme information.【12】 Amphion: An Open-Source Audio, Music and Speech Generation Toolkit标题：Amphion：一个开源的音频、音乐和语音生成工具包链接：https://arxiv.org/abs/2312.09911作者：Xueyao Zhang,Liumeng Xue,Yuancheng Wang,Yicheng Gu,Xi Chen,Zihao Fang,Haopeng Chen,Lexiao Zou,Chaoren Wang,Jun Han,Kai Chen,Haizhou Li,Zhizheng Wu备注：GitHub: this https URL摘要：Amphion是一个用于音频，音乐和语音生成的工具包。其目的是支持可重复的研究，并帮助初级研究人员和工程师开始在音频，音乐和语音生成领域的研究和开发。Amphion提供了一个独特的功能：经典模型或架构的可视化。我们相信，这些可视化对于希望更好地理解模型的初级研究人员和工程师来说是有益的。Amphion的北极星目标是提供一个平台，用于研究将任何输入转换为通用音频。Amphion旨在支持个人生成任务。除了特定的生成任务，Amphion还包括几个声码器和评估指标。声码器是产生高质量音频信号的重要模块，而评估指标对于确保生成任务中的一致指标至关重要。在本文中，我们提供了一个高层次的概述Amphion。摘要：Amphion is a toolkit for Audio, Music, and Speech Generation. Its purpose is to support reproducible research and help junior researchers and engineers get started in the field of audio, music, and speech generation research and development. Amphion offers a unique feature: visualizations of classic models or architectures. We believe that these visualizations are beneficial for junior researchers and engineers who wish to gain a better understanding of the model. The North-Star objective of Amphion is to offer a platform for studying the conversion of any inputs into general audio. Amphion is designed to support individual generation tasks. In addition to the specific generation tasks, Amphion also includes several vocoders and evaluation metrics. A vocoder is an important module for producing high-quality audio signals, while evaluation metrics are critical for ensuring consistent metrics in generation tasks. In this paper, we provide a high-level overview of Amphion.【13】 Generative Context-aware Fine-tuning of Self-supervised Speech Models标题：自监督语音模型的生成式上下文感知微调链接：https://arxiv.org/abs/2312.09895作者：Suwon Shon,Kwangyoun Kim,Prashant Sridhar,Yi-Te Hsu,Shinji Watanabe,Karen Livescu摘要：当执行诸如自动语音识别或针对给定话语的口语理解之类的任务时，访问先前的文本或音频提供上下文信息可以提高性能。考虑到生成式大型语言模型（LLM）的最新进展，我们假设LLM可以使用前面的文本生成有用的上下文信息。通过适当的提示，LLM可以生成下一个句子或抽象文本（如标题或主题）的预测。在本文中，我们研究了使用LLM生成的上下文信息，并提出了一种方法来提取生成的信息在微调的自监督语音模型，我们称之为生成上下文感知微调。这种方法允许微调模型在推理时无需访问真实的周围片段或LLM即可进行改进的预测，同时仅需要非常小的附加上下文模块。我们使用SLUE和Libri-Light基准测试评估了所提出的方法，用于几个下游任务：自动语音识别，命名实体识别和情感分析。结果表明，生成上下文感知微调优于上下文注入微调方法，访问地面真相以前的文本，并与生成上下文注入微调方法，需要在推理时间的LLM竞争。摘要：When performing tasks like automatic speech recognition or spoken language understanding for a given utterance, access to preceding text or audio provides contextual information can improve performance. Considering the recent advances in generative large language models (LLM), we hypothesize that an LLM could generate useful context information using the preceding text. With appropriate prompts, LLM could generate a prediction of the next sentence or abstractive text like titles or topics. In this paper, we study the use of LLM-generated context information and propose an approach to distill the generated information during fine-tuning of self-supervised speech models, which we refer to as generative context-aware fine-tuning. This approach allows the fine-tuned model to make improved predictions without access to the true surrounding segments or to the LLM at inference time, while requiring only a very small additional context module. We evaluate the proposed approach using the SLUE and Libri-light benchmarks for several downstream tasks: automatic speech recognition, named entity recognition, and sentiment analysis. The results show that generative context-aware fine-tuning outperforms a context injection fine-tuning approach that accesses the ground-truth previous text, and is competitive with a generative context injection fine-tuning approach that requires the LLM at inference time.【14】 On the compression of shallow non-causal ASR models using knowledge distillation and tied-and-reduced decoder for low-latency on-device speech recognition标题：使用知识提取和捆绑简化解码器压缩浅层非因果ASR模型以实现低延迟设备上语音识别链接：https://arxiv.org/abs/2312.09842作者：Nagaraj Adiga,Jinhwan Park,Chintigari Shiva Kumar,Shatrughan Singh,Kyungmin Lee,Chanwoo Kim,Dhananjaya Gowda摘要：最近，级联两遍架构已成为设备上自动语音识别（ASR）的有力竞争者。因果和浅非因果编码器的级联与共享解码器耦合，使得能够在流模式和前瞻模式下操作。在本文中，我们提出了浅级联模型，结合各种模型压缩技术，如知识蒸馏，共享解码器，捆绑和减少传感器网络，以减少模型的足迹。共享解码器变成一个捆绑和减少的网络。级联两通模型进一步压缩使用知识蒸馏使用模型后验的Kullback-Leibler发散损失。我们证明了一个41 M参数级联教师模型的大小减少了50%，ASR准确性没有明显下降，延迟减少了30%摘要：Recently, the cascaded two-pass architecture has emerged as a strong contender for on-device automatic speech recognition (ASR). A cascade of causal and shallow non-causal encoders coupled with a shared decoder enables operation in both streaming and look-ahead modes. In this paper, we propose shallow cascaded model by combining various model compression techniques such as knowledge distillation, shared decoder, and tied-and-reduced transducer network in order to reduce the model footprint. The shared decoder is changed into a tied-and-reduced network. The cascaded two-pass model is further compressed using knowledge distillation using a Kullback-Leibler divergence loss on the model posteriors. We demonstrate a 50% reduction in the size of a 41 M parameter cascaded teacher model with no noticeable degradation in ASR accuracy and a 30% reduction in latency【15】 Automatic channel selection and spatial feature integration for multi-channel speech recognition across various array topologies标题：用于跨各种阵列拓扑的多通道语音识别的自动通道选择和空间特征集成链接：https://arxiv.org/abs/2312.09746作者：Bingshen Mu,Pengcheng Guo,Dake Guo,Pan Zhou,Wei Chen,Lei Xie备注：Accepted by ICASSP 2024摘要：自动语音识别（ASR）已经取得了显着的进展，但它仍然面临着挑战，在现实世界中的远程场景中，在各种阵列拓扑结构中，每个阵列拓扑结构都有多个记录设备。CHiME-7 Distant ASR任务的重点是设计一个统一的系统，该系统能够推广具有多个记录设备的各种阵列拓扑，并在现实环境中提供可靠的识别性能。为了解决这一任务，我们引入了一个ASR系统，该系统在各种阵列拓扑中表现出卓越的性能。首先，我们提出了两个基于注意力的自动通道选择模块，从多个记录设备中为每个话语选择最有利的多通道信号子集。此外，我们引入通道间空间特征来增强多帧跨通道注意的有效性，帮助其提高空间信息感知能力。最后，我们提出了一个多层卷积融合模块借鉴U-Net架构的灵感，将多通道输出集成到一个单通道输出。CHiME-7语料库与Oracle分割的实验结果表明，在我们提出的ASR系统中引入的改进导致相对减少了40.1%的宏观日记属性字错误率（DA-WER）相比，基线ASR系统的Eval集。摘要：Automatic Speech Recognition (ASR) has shown remarkable progress, yet it still faces challenges in real-world distant scenarios across various array topologies each with multiple recording devices. The focal point of the CHiME-7 Distant ASR task is to devise a unified system capable of generalizing various array topologies that have multiple recording devices and offering reliable recognition performance in real-world environments. Addressing this task, we introduce an ASR system that demonstrates exceptional performance across various array topologies. First of all, we propose two attention-based automatic channel selection modules to select the most advantageous subset of multi-channel signals from multiple recording devices for each utterance. Furthermore, we introduce inter-channel spatial features to augment the effectiveness of multi-frame cross-channel attention, aiding it in improving the capability of spatial information awareness. Finally, we propose a multi-layer convolution fusion module drawing inspiration from the U-Net architecture to integrate the multi-channel output into a single-channel output. Experimental results on the CHiME-7 corpus with oracle segmentation demonstrate that the improvements introduced in our proposed ASR system lead to a relative reduction of 40.1% in the Macro Diarization Attributed Word Error Rates (DA-WER) when compared to the baseline ASR system on the Eval sets.【16】 HEAR: Hearing Enhanced Audio Response for Video-grounded Dialogue标题：Hear：针对基于视频的对话的听觉增强音频响应链接：https://arxiv.org/abs/2312.09736作者：Sunjae Yoon,Dahyun Kim,Eunseop Yoon,Hee Suk Yoon,Junyeong Kim,Chnag D. Yoo备注：EMNLP 2023, 14 pages, 13 figures摘要：基于视频的对话（VGD）旨在回答关于给定的多模态输入（包括视频、音频和对话历史）的问题。尽管已经在开发VGD系统以提高其响应的质量方面做出了许多努力，但是现有系统仅能够将信息合并到视频和文本中，并且当生成对问题的适当响应时，在从音频中提取必要的信息方面趋于困难。VGD系统似乎是聋的，因此，我们将当前系统忽略音频数据的这种症状称为聋反应。为了克服聋人响应问题，听力增强音频响应（Hearing Enhanced Audio Response，HEAR）框架被提出来通过在问题需要时选择性地关注音频来执行明智的听力。HEAR框架以模型不可知的方式增强VGD系统的准确性和可听度。HEAR在VGD数据集上得到验证（即，AVSD@DSTC 7和AVSD@DSTC 8），并显示了各种VGD系统的有效性。摘要：Video-grounded Dialogue (VGD) aims to answer questions regarding a given multi-modal input comprising video, audio, and dialogue history. Although there have been numerous efforts in developing VGD systems to improve the quality of their responses, existing systems are competent only to incorporate the information in the video and text and tend to struggle in extracting the necessary information from the audio when generating appropriate responses to the question. The VGD system seems to be deaf, and thus, we coin this symptom of current systems' ignoring audio data as a deaf response. To overcome the deaf response problem, Hearing Enhanced Audio Response (HEAR) framework is proposed to perform sensible listening by selectively attending to audio whenever the question requires it. The HEAR framework enhances the accuracy and audibility of VGD systems in a model-agnostic manner. HEAR is validated on VGD datasets (i.e., AVSD@DSTC7 and AVSD@DSTC8) and shows effectiveness with various VGD systems.【17】 LiteVSR: Efficient Visual Speech Recognition by Learning from Speech Representations of Unlabeled Data标题：LiteVSR：从未标记数据的语音表示中学习的高效视觉语音识别链接：https://arxiv.org/abs/2312.09727作者：Hendrik Laux,Emil Mededovic,Ahmed Hallawa,Lukas Martin,Arne Peine,Anke Schmeink备注：Accepted for publication at ICASSP 2024摘要：本文提出了一种新的，资源有效的方法，视觉语音识别（VSR）利用语音表示产生的任何训练的自动语音识别（ASR）模型。远离最近文献中普遍存在的资源密集型趋势，我们的方法从经过训练的基于Conformer的ASR模型中提取知识，在标准VSR基准测试中实现具有竞争力的性能，并且资源利用率显著降低。仅使用未标记的视听数据，我们的基线模型在LRS2和LRS3测试基准上分别实现了47.4%和54.7%的单词错误率（WER）。在有限的标记数据微调模型后，单词错误率降低到35%（LRS2）和45.7%（LRS3）。我们的模型可以在几天内在单个消费级GPU上进行训练，并且能够在过时的硬件上执行实时端到端VSR，这表明了一条通往更易于访问和资源高效的VSR方法的道路。摘要：This paper proposes a novel, resource-efficient approach to Visual Speech Recognition (VSR) leveraging speech representations produced by any trained Automatic Speech Recognition (ASR) model. Moving away from the resource-intensive trends prevalent in recent literature, our method distills knowledge from a trained Conformer-based ASR model, achieving competitive performance on standard VSR benchmarks with significantly less resource utilization. Using unlabeled audio-visual data only, our baseline model achieves a word error rate (WER) of 47.4% and 54.7% on the LRS2 and LRS3 test benchmarks, respectively. After fine-tuning the model with limited labeled data, the word error rate reduces to 35% (LRS2) and 45.7% (LRS3). Our model can be trained on a single consumer-grade GPU within a few days and is capable of performing real-time end-to-end VSR on dated hardware, suggesting a path towards more accessible and resource-efficient VSR methodologies.【18】 What to Remember: Self-Adaptive Continual Learning for Audio Deepfake Detection标题：记住什么：用于音频深伪检测的自适应持续学习链接：https://arxiv.org/abs/2312.09651作者：Xiaohui Zhang,Jiangyan Yi,Chenglong Wang,Chuyuan Zhang,Siding Zeng,Jianhua Tao备注：Accepted by the main track The 38th Annual AAAI Conference on Artificial Intelligence (AAAI 2024)摘要：语音合成和语音转换的快速发展引起了人们的极大关注，因为这种技术可能会被滥用，这促使人们迫切需要有效的音频深度伪造检测机制。现有的检测模型在识别已知的deepfake音频方面取得了显着的成功，但在遇到新的攻击类型时会遇到困难。为了应对这一挑战，一个紧急有效的方法是持续学习。在本文中，我们提出了一种称为弧度权重修改（RWM）的持续学习方法，用于音频deepfake检测。RWM的基本概念涉及将所有类分为两组：那些在任务中具有紧凑特征分布的类，如真实音频，以及那些具有更广泛分布的类，如各种类型的假音频。这些区别通过类内余弦距离量化，随后作为RWM的基础，为不同的数据类型引入可训练的梯度修改方向。针对主流持续学习方法的实验评估揭示了RWM在知识获取和减轻音频deepfake检测中的遗忘方面的优势。此外，RWM的适用性超出了音频深度伪造检测，证明了其在图像识别等不同机器学习领域的潜在意义。摘要：The rapid evolution of speech synthesis and voice conversion has raised substantial concerns due to the potential misuse of such technology, prompting a pressing need for effective audio deepfake detection mechanisms. Existing detection models have shown remarkable success in discriminating known deepfake audio, but struggle when encountering new attack types. To address this challenge, one of the emergent effective approaches is continual learning. In this paper, we propose a continual learning approach called Radian Weight Modification (RWM) for audio deepfake detection. The fundamental concept underlying RWM involves categorizing all classes into two groups: those with compact feature distributions across tasks, such as genuine audio, and those with more spread-out distributions, like various types of fake audio. These distinctions are quantified by means of the in-class cosine distance, which subsequently serves as the basis for RWM to introduce a trainable gradient modification direction for distinct data types. Experimental evaluations against mainstream continual learning methods reveal the superiority of RWM in terms of knowledge acquisition and mitigating forgetting in audio deepfake detection. Furthermore, RWM's applicability extends beyond audio deepfake detection, demonstrating its potential significance in diverse machine learning domains such as image recognition.【19】 Stethoscope-guided Supervised Contrastive Learning for Cross-domain Adaptation on Respiratory Sound Classification标题：听诊器引导的呼吸音分类跨域适应监督对比学习链接：https://arxiv.org/abs/2312.09603作者：June-Woo Kim,Sangmin Bae,Won-Yang Cho,Byungjo Lee,Ho-Young Jung备注：accepted to ICASSP 2024摘要：尽管深度学习技术取得了显着进步，但由于可用数据的稀缺性，在肺音分类中实现令人满意的性能仍然是一个挑战。此外，呼吸声样本是从各种电子听诊器收集的，这可能会在训练模型中引入偏差。当测试数据集内或实际场景中发生显著的分布偏移时，它会大大降低性能。为了解决这个问题，我们引入跨域适应技术，将知识从源域转移到不同的目标域。特别是，通过考虑不同的听诊器类型作为单独的域，我们提出了一种新的听诊器引导的监督对比学习方法。该方法可以减轻任何域相关的差异，并且因此使得模型能够区分听诊器的记录变化的呼吸声音。在ICBHI数据集上的实验结果表明，所提出的方法有效地降低了领域依赖性，ICBHI得分为61.71%，比基线显著提高了2.16%。摘要：Despite the remarkable advances in deep learning technology, achieving satisfactory performance in lung sound classification remains a challenge due to the scarcity of available data. Moreover, the respiratory sound samples are collected from a variety of electronic stethoscopes, which could potentially introduce biases into the trained models. When a significant distribution shift occurs within the test dataset or in a practical scenario, it can substantially decrease the performance. To tackle this issue, we introduce cross-domain adaptation techniques, which transfer the knowledge from a source domain to a distinct target domain. In particular, by considering different stethoscope types as individual domains, we propose a novel stethoscope-guided supervised contrastive learning approach. This method can mitigate any domain-related disparities and thus enables the model to distinguish respiratory sounds of the recording variation of the stethoscope. The experimental results on the ICBHI dataset demonstrate that the proposed methods are effective in reducing the domain dependency and achieving the ICBHI Score of 61.71%, which is a significant improvement of 2.16% over the baseline.【20】 Leveraging Language ID to Calculate Intermediate CTC Loss for Enhanced Code-Switching Speech Recognition标题：利用语言ID计算增强码切换语音识别的中间CTC损失链接：https://arxiv.org/abs/2312.09583作者：Tzu-Ting Yang,Hsin-Wei Wang,Berlin Chen备注：Accepted to The 28th International Conference on Technologies and Applications of Artificial Intelligence (TAAI), in Chinese language摘要：近年来，端到端语音识别已经成为一种技术，它集成了传统自动语音识别模型的声学，发音字典和语言模型组件。不需要事先建立发音词典，就可以实现类似人类的识别。然而，由于语码转换的训练数据相对稀缺，当遇到这种现象时，ASR模型的性能往往会急剧下降。以往的研究大多将语码转换任务分解为处理单一语言的多个任务，然后分别学习每种语言的特定领域知识，从而简化了该模型的学习复杂性。因此，在本文中，我们试图将语言识别信息引入到ASR模型的编码器的中间层。我们的目标是生成声学特征，以更隐式的方式暗示语言区别，减少模型在处理语言切换时的混乱。摘要：In recent years, end-to-end speech recognition has emerged as a technology that integrates the acoustic, pronunciation dictionary, and language model components of the traditional Automatic Speech Recognition model. It is possible to achieve human-like recognition without the need to build a pronunciation dictionary in advance. However, due to the relative scarcity of training data on code-switching, the performance of ASR models tends to degrade drastically when encountering this phenomenon. Most past studies have simplified the learning complexity of the model by splitting the code-switching task into multiple tasks dealing with a single language and then learning the domain-specific knowledge of each language separately. Therefore, in this paper, we attempt to introduce language identification information into the middle layer of the ASR model's encoder. We aim to generate acoustic features that imply language distinctions in a more implicit way, reducing the model's confusion when dealing with language switching.【21】 Phoneme-aware Encoding for Prefix-tree-based Contextual ASR标题：基于前缀树的上下文ASR音素感知编码链接：https://arxiv.org/abs/2312.09582作者：Hayato Futami,Emiru Tsunoo,Yosuke Kashiwagi,Hiroaki Ogawa,Siddhant Arora,Shinji Watanabe备注：Accepted to ICASSP2024摘要：在语音识别应用中，识别上下文特定的生僻词（如专有名词）是很重要的。树约束的指针生成器（TCPGen）已经显示出用于此目的的承诺，它有效地偏置这样的词与前缀树。虽然最初的TCPGen依赖于基于字素的编码，但我们建议使用音素感知编码对其进行扩展，以更好地识别不寻常发音的单词。由于TCPGen处理偏置字作为子字单元，我们建议通过使用音素和子字之间的对齐来获得子字级音素感知编码。此外，我们建议将CTC的音素级预测注入到TCPGen的查询中，以便该模型更好地解释音素感知编码。我们用TCPGen对RNN传感器进行了ASR实验。我们观察到，在英语LibriSpeech和日语CSJ数据集上，提出的音素感知编码优于普通的基于图素的编码，证明了我们的方法在不同语言中的鲁棒性。摘要：In speech recognition applications, it is important to recognize context-specific rare words, such as proper nouns. Tree-constrained Pointer Generator (TCPGen) has shown promise for this purpose, which efficiently biases such words with a prefix tree. While the original TCPGen relies on grapheme-based encoding, we propose extending it with phoneme-aware encoding to better recognize words of unusual pronunciations. As TCPGen handles biasing words as subword units, we propose obtaining subword-level phoneme-aware encoding by using alignment between phonemes and subwords. Furthermore, we propose injecting phoneme-level predictions from CTC into queries of TCPGen so that the model better interprets the phoneme-aware encodings. We conducted ASR experiments with TCPGen for RNN transducer. We observed that proposed phoneme-aware encoding outperformed ordinary grapheme-based encoding on both the English LibriSpeech and Japanese CSJ datasets, demonstrating the robustness of our approach across linguistically diverse languages.【22】 A 1.6-mW Sparse Deep Learning Accelerator for Speech Separation标题：一种1.6 mW的稀疏深度学习语音分离加速器链接：https://arxiv.org/abs/2312.09580作者：Chih-Chyau Yang,Tian-Sheuan Chang备注：None摘要：语音处理上的低功耗深度学习加速器可以在边缘设备上实现实时应用。然而，大多数现有的加速器遭受高功耗和专注于图像应用。本文提出了一种低功耗的语音分离加速器，通过算法和硬件优化。在算法级，该模型被压缩与结构化的敏感性以及非结构化的修剪，并进一步量化为移位的8位浮点格式，而不是32位浮点格式。通过分解扩张卷积和转置卷积，跳过具有零内核和零激活值的计算。在硬件级别，压缩模型然后由具有八个独立乘法器和MAC（MAC）的架构支持，该架构具有简单的零跳过硬件以利用激活稀疏性和低功率处理。该方法使模型规模减少了95.44%，计算复杂度降低了93.88%.最终实现与台积电40纳米$工艺可以实现实时语音分离和功耗1.6 mW时，工作在150 MHz。归一化能量效率和面积效率分别为2.344 TOPS/W和14.42 GOPS/mm 2。摘要：Low power deep learning accelerators on the speech processing enable real-time applications on edge devices. However, most of the existing accelerators suffer from high power consumption and focus on image applications only. This paper presents a low power accelerator for speech separation through algorithm and hardware optimizations. At the algorithm level, the model is compressed with structured sensitivity as well as unstructured pruning, and further quantized to the shifted 8-bit floating-point format instead of the 32-bit floating-point format. The computations with the zero kernel and zero activation values are skipped by decomposition of the dilated and transposed convolutions. At the hardware level, the compressed model is then supported by an architecture with eight independent multipliers and accumulators (MACs) with a simple zero-skipping hardware to take advantage of the activation sparsity and low power processing. The proposed approach reduces the model size by 95.44\% and computation complexity by 93.88\%. The final implementation with the TSMC 40 $nm$ process can achieve real-time speech separation and consumes 1.6 mW power when operated at 150 MHz. The normalized energy efficiency and area efficiency are 2.344 TOPS/W and 14.42 GOPS/mm$^2$, respectively.【23】 Audio-visual fine-tuning of audio-only ASR models标题：纯音频ASR模型的视听微调链接：https://arxiv.org/abs/2312.09369作者：Avner May,Dmitriy Serdyuk,Ankit Parag Shah,Otavio Braga,Olivier Siohan摘要：视听自动语音识别（AV-ASR）模型在降低噪声语音的单词错误率方面非常有效，但需要大量转录的AV训练数据。最近，已经开发了视听自监督学习（SSL）方法来减少对转录AV数据的依赖，但这些方法非常复杂且计算昂贵。在这项工作中，我们建议用一个简单快速的SSL方法来代替这些昂贵的AV-SSL方法，然后进行AV监督微调。我们表明，这种方法在LRS 3-TED基准任务上与最先进的（SOTA）AV-SSL方法竞争（绝对WER在0.5%以内），同时更简单，更高效（预训练速度快12- 30倍）。此外，我们可以扩展这种方法，将SOTA仅音频ASR模型转换为AV模型。通过这样做，我们匹配SOTA AV-SSL结果，即使在预训练期间没有使用AV数据。摘要：Audio-visual automatic speech recognition (AV-ASR) models are very effective at reducing word error rates on noisy speech, but require large amounts of transcribed AV training data. Recently, audio-visual self-supervised learning (SSL) approaches have been developed to reduce this dependence on transcribed AV data, but these methods are quite complex and computationally expensive. In this work, we propose replacing these expensive AV-SSL methods with a simple and fast \textit{audio-only} SSL method, and then performing AV supervised fine-tuning. We show that this approach is competitive with state-of-the-art (SOTA) AV-SSL methods on the LRS3-TED benchmark task (within 0.5% absolute WER), while being dramatically simpler and more efficient (12-30x faster to pre-train). Furthermore, we show we can extend this approach to convert a SOTA audio-only ASR model into an AV model. By doing so, we match SOTA AV-SSL results, even though no AV data was used during pre-training.【24】 Efficient speech detection in environmental audio using acoustic recognition and knowledge distillation标题：基于声学识别和知识提取的环境音频语音检测链接：https://arxiv.org/abs/2312.09269作者：Drew Priebe,Burooj Ghani,Dan Stowell摘要：由于土地使用变化和全球变暖等因素，目前的生物多样性危机突出表明需要有效的生态监测方法。生物多样性声学监测已成为一种重要的监测工具。在声景监测项目中检测人类声音对于分析人类干扰和隐私过滤都是有用的。尽管近年来深度学习取得了重大进展，但由于内存和延迟限制，在紧凑设备上部署大型神经网络带来了挑战。我们的方法侧重于利用知识蒸馏技术来设计高效，轻量级的学生模型，用于生物声学中的语音检测。特别是，我们采用MobileNetV 3-Small-Pi模型来创建紧凑而有效的学生架构，以与更大的EcoVAD教师模型进行比较，EcoVAD教师模型是生态声学监测中备受好评的语音检测架构。比较分析包括检查MobileNetV 3-Small-Pi衍生学生模型的各种配置，以确定最佳性能。此外，对不同的蒸馏技术进行了全面的评价，以确定最有效的模型选择方法。我们的研究结果表明，蒸馏模型表现出相当的性能EcoVAD教师模型，表明一个有前途的方法来克服实时生态监测的计算障碍。摘要：The ongoing biodiversity crisis, driven by factors such as land-use change and global warming, emphasizes the need for effective ecological monitoring methods. Acoustic monitoring of biodiversity has emerged as an important monitoring tool. Detecting human voices in soundscape monitoring projects is useful both for analysing human disturbance and for privacy filtering. Despite significant strides in deep learning in recent years, the deployment of large neural networks on compact devices poses challenges due to memory and latency constraints. Our approach focuses on leveraging knowledge distillation techniques to design efficient, lightweight student models for speech detection in bioacoustics. In particular, we employed the MobileNetV3-Small-Pi model to create compact yet effective student architectures to compare against the larger EcoVAD teacher model, a well-regarded voice detection architecture in eco-acoustic monitoring. The comparative analysis included examining various configurations of the MobileNetV3-Small-Pi derived student models to identify optimal performance. Additionally, a thorough evaluation of different distillation techniques was conducted to ascertain the most effective method for model selection. Our findings revealed that the distilled models exhibited comparable performance to the EcoVAD teacher model, indicating a promising approach to overcoming computational barriers for real-time ecological monitoring.【25】 Acoustic models of Brazilian Portuguese Speech based on Neural Transformers标题：基于神经转换器的巴西葡萄牙语语音声学模型链接：https://arxiv.org/abs/2312.09265作者：Marcelo Matheus Gauy,Marcelo Finger备注：Under review at Journal of Brazilian Computer Society摘要：在大量未标记数据上训练的声学模型由可用于解决下游任务的自监督学习语音表示组成，可能在相应下游任务中对模型进行微调之后。在这项工作中，我们通过Transformer神经网络建立了巴西葡萄牙语语音的声学模型。该模型使用预训练技术的组合，在超过800 $小时的巴西葡萄牙语语音上进行了预训练。使用为检测巴西葡萄牙语使用者的呼吸功能不全而收集的标记数据集，我们在以下任务上微调预训练的Transformer神经网络：呼吸功能不全检测，性别识别和年龄组分类。我们将预先训练的Transformers在这些任务上的表现与没有预先训练的Transformers进行了比较，注意到显着的改善。特别是，呼吸功能不全检测的性能获得了迄今为止最好的报告结果，表明这种声学模型作为一个有前途的工具，语音作为生物标志物的方法。此外，性别识别的性能与英语中的最先进模型相当。摘要：An acoustic model, trained on a significant amount of unlabeled data, consists of a self-supervised learned speech representation useful for solving downstream tasks, perhaps after a fine-tuning of the model in the respective downstream task. In this work, we build an acoustic model of Brazilian Portuguese Speech through a Transformer neural network. This model was pretrained on more than $800$ hours of Brazilian Portuguese Speech, using a combination of pretraining techniques. Using a labeled dataset collected for the detection of respiratory insufficiency in Brazilian Portuguese speakers, we fine-tune the pretrained Transformer neural network on the following tasks: respiratory insufficiency detection, gender recognition and age group classification. We compare the performance of pretrained Transformers on these tasks with that of Transformers without previous pretraining, noting a significant improvement. In particular, the performance of respiratory insufficiency detection obtains the best reported results so far, indicating this kind of acoustic model as a promising tool for speech-as-biomarker approach. Moreover, the performance of gender recognition is comparable to the state of the art models in English.机器翻译由腾讯交互翻译提供，仅供参考
每日语音速递
