今日论文合集：cs.SD语音4篇，eess.AS音频处理6篇。
cs.SD语音

【1】CB-Conformer: Contextual biasing Conformer for biased word recognition标题：CB-Conform：用于偏倚词识别的上下文偏向整形器链接：http://arxiv.org/pdf/2304.09607v1

作者：Yaoxun Xu,Baiji Liu,Qiaochu Huang and,Xingchen Song,Zhiyong Wu,Shiyin Kang,Helen Meng机构：Shenzhen International Graduate School, Tsinghua University, Shenzhen, China, XVerse Inc., Shenzhen, China, Peng Cheng Lab, Shenzhen, China, The Chinese University of Hong Kong, Hong Kong SAR, China摘要：由于源域和目标域之间的不匹配，如何更好地利用有偏词信息来提高目标域自动语音识别模型的性能成为研究的热点。以往的方法要么使用固定的外部语言模型进行解码，要么引入一个相当大的偏置模块，这导致了适应性差和推理速度慢。在这项工作中，我们提出了CB-Conformer，通过引入上下文偏置模块和自适应语言模型香草Conformer，以提高有偏见的单词识别。上下文偏置模块将音频片段和上下文信息相结合，仅具有原始Conformer的0.2%的模型参数。自适应语言模型根据其召回率和准确率修改有偏见的单词的内部权重，从而比标准固定语言模型更关注有偏见的单词并且更成功地与自动语音识别模型集成。此外，我们构建并发布了一个基于WenetSpeech的开源汉语偏误词数据集。实验结果表明，我们提出的方法带来了15.34%的字符错误率降低，14.13%的偏见字召回增加，和6.80%的偏见字F1分数增加相比，基础的构象。摘要：Due to the mismatch between the source and target domains, how to better utilize the biased word information to improve the performance of the automatic speech recognition model in the target domain becomes a hot research topic. Previous approaches either decode with a fixed external language model or introduce a sizeable biasing module, which leads to poor adaptability and slow inference. In this work, we propose CB-Conformer to improve biased word recognition by introducing the Contextual Biasing Module and the Self-Adaptive Language Model to vanilla Conformer. The Contextual Biasing Module combines audio fragments and contextual information, with only 0.2% model parameters of the original Conformer. The Self-Adaptive Language Model modifies the internal weights of biased words based on their recall and precision, resulting in a greater focus on biased words and more successful integration with the automatic speech recognition model than the standard fixed language model. In addition, we construct and release an open-source Mandarin biased-word dataset based on WenetSpeech. Experiments indicate that our proposed method brings a 15.34% character error rate reduction, a 14.13% biased word recall increase, and a 6.80% biased word F1-score increase compared with the base Conformer.

【2】Security and Privacy Problems in Voice Assistant Applications: A Survey标题：语音助手应用中的安全和隐私问题：综述链接：http://arxiv.org/pdf/2304.09486v1

作者：Jingjin Li,Chao chen,Lei Pan,Mostafa Rahimi Azghadi,Hossein Ghodosi,Jun Zhang机构：James Cook University, RMIT University, Deakin University, Swinburne University of Technology备注：5 figures摘要：语音助理应用程序现在已经变得无所不知。两个模型为现实生活中的应用提供了两个最重要的功能（即，Google Home、Amazon Alexa、Siri等）是自动语音识别（ASR）模型和说话人识别（SI）模型。根据最近的研究，随着物联网（IoT）的快速发展，安全和隐私威胁也出现了。研究的安全问题包括针对机器学习模型和其他广泛用于语音助理应用的硬件组件的攻击技术。隐私问题包括技术上的信息窃取和策略上的隐私泄露。语音助手应用每年都占据着稳步增长的市场份额，但其隐私和安全问题从未停止过，造成了巨大的经济损失，并危及用户的个人敏感信息。因此，重要的是要有一个全面的调查，以概述目前的研究分类有关的安全和隐私问题的语音助理应用程序。本文总结和评估了网络安全和语音领域顶级会议上发表的论文中的五种安全攻击和三种隐私威胁。摘要：Voice assistant applications have become omniscient nowadays. Two models that provide the two most important functions for real-life applications (i.e., Google Home, Amazon Alexa, Siri, etc.) are Automatic Speech Recognition (ASR) models and Speaker Identification (SI) models. According to recent studies, security and privacy threats have also emerged with the rapid development of the Internet of Things (IoT). The security issues researched include attack techniques toward machine learning models and other hardware components widely used in voice assistant applications. The privacy issues include technical-wise information stealing and policy-wise privacy breaches. The voice assistant application takes a steadily growing market share every year, but their privacy and security issues never stopped causing huge economic losses and endangering users' personal sensitive information. Thus, it is important to have a comprehensive survey to outline the categorization of the current research regarding the security and privacy problems of voice assistant applications. This paper concludes and assesses five kinds of security attacks and three types of privacy threats in the papers published in the top-tier conferences of cyber security and voice domain.

【3】Dynamic Chuck Convolution For Unified Streaming And Non-streaming Conformer ASR标题：用于统一流和非流整形器ASR的动态Chuck卷积链接：http://arxiv.org/pdf/2304.09325v1

作者：Xilai Li,Goeric Huybrechts,Srikanth Ronanki,Jeff Farris,Sravan Bodapati机构：AWS AI Labs备注：5 pages, 3 figures, 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023)摘要：最近，人们越来越关注统一流式和非流式语音识别模型以降低开发、训练和部署成本。最著名的方法依赖于基于窗口或基于动态块的注意力策略和因果卷积，以最大限度地减少由于流的退化。然而，在非流和独立训练的全上下文模型之间的性能差距仍然相对较大。为了解决这个问题，我们提出了一个动态的基于块的卷积代替因果卷积的混合连接主义时间分类（CTC）-注意一致性架构。此外，我们展示了进一步的改进，通过初始化的权重从一个完整的上下文模型和并行化的卷积和自我注意模块。我们在开源的Voxpopuli，LibriSpeech和内部会话数据集上评估我们的模型。总体而言，我们提出的模型将非流全上下文模型的流模式的退化从41.7%和45.7%分别降低到LibriSpeech测试清洁和测试其他数据集的16.7%和26.2%，同时比以前的最先进的统一模型提高了相对15.5%的WER。摘要：Recently, there has been an increasing interest in unifying streaming and non-streaming speech recognition models to reduce development, training and deployment cost. The best-known approaches rely on either window-based or dynamic chunk-based attention strategy and causal convolutions to minimize the degradation due to streaming. However, the performance gap still remains relatively large between non-streaming and a full-contextual model trained independently. To address this, we propose a dynamic chunk-based convolution replacing the causal convolution in a hybrid Connectionist Temporal Classification (CTC)-Attention Conformer architecture. Additionally, we demonstrate further improvements through initialization of weights from a full-contextual model and parallelization of the convolution and self-attention modules. We evaluate our models on the open-source Voxpopuli, LibriSpeech and in-house conversational datasets. Overall, our proposed model reduces the degradation of the streaming mode over the non-streaming full-contextual model from 41.7% and 45.7% to 16.7% and 26.2% on the LibriSpeech test-clean and test-other datasets respectively, while improving by a relative 15.5% WER over the previous state-of-the-art unified model.

【4】Coded Speech Quality Measurement by a Non-Intrusive PESQ-DNN标题：基于非侵入式PESQ-DNN的编码语音质量测量链接：http://arxiv.org/pdf/2304.09226v1

作者：Ziyi Xu,Ziyue Zhao,Tim Fingscheidt机构：Ziyi Xu and Tim Fingscheidt are with the Institute for Communi-cations Technology, Technische Universit¨at Braunschweig摘要：诸如AMR-WB或EVS之类的宽带编解码器广泛用于（移动）语音通信中。编码语音质量的评估通常由绝对类别评级（ACR）收听测试主观地执行。然而，ACR测试对于语音通信网络的在线监测是不切实际的。语音质量感知评价（PESQ）是广泛使用的用于预测ACR测试结果的度量之一。然而，PESQ算法需要原始参考信号，这在网络监测中通常是不可用的，从而限制了其适用性。NISQA是一种新的基于神经网络的非侵入式语音质量测量方法，主要针对超宽带语音信号。然而，在这项工作中，我们的目标是使用非侵入式PESQ-DNN模型预测众所周知的PESQ度量。我们通过预测从AMR-WB或EVS编解码器在噪声、串联和易出错的传输条件下以不同比特率操作获得的宽带编码语音的PESQ分数来说明该模型的潜力。我们将我们的方法与QualityNet，WaweNet和DNSMOS的最先进的网络拓扑结构进行比较-所有这些都适用于PESQ预测-通过测量平均绝对误差（MAE）和线性相关系数（LCC）。所提出的PESQ-DNN在没有帧丢失的条件下分别提供0.11和0.92的最佳总MAE和LCC，并且在包括帧丢失时仍然是最佳的。注意，我们的模型可以类似地用于非侵入性地预测POLQA或其他（侵入性）指标。文章接受后，代码将在GitHub上提供。摘要：Wideband codecs such as AMR-WB or EVS are widely used in (mobile) speech communication. Evaluation of coded speech quality is often performed subjectively by an absolute category rating (ACR) listening test. However, the ACR test is impractical for online monitoring of speech communication networks. Perceptual evaluation of speech quality (PESQ) is one of the widely used metrics instrumentally predicting the results of an ACR test. However, the PESQ algorithm requires an original reference signal, which is usually unavailable in network monitoring, thus limiting its applicability. NISQA is a new non-intrusive neural-network-based speech quality measure, focusing on super-wideband speech signals. In this work, however, we aim at predicting the well-known PESQ metric using a non-intrusive PESQ-DNN model. We illustrate the potential of this model by predicting the PESQ scores of wideband-coded speech obtained from AMR-WB or EVS codecs operating at different bitrates in noisy, tandeming, and error-prone transmission conditions. We compare our methods with the state-of-the-art network topologies of QualityNet, WaweNet, and DNSMOS -- all applied to PESQ prediction -- by measuring the mean absolute error (MAE) and the linear correlation coefficient (LCC). The proposed PESQ-DNN offers the best total MAE and LCC of 0.11 and 0.92, respectively, in conditions without frame loss, and still is best when including frame loss. Note that our model could be similarly used to non-intrusively predict POLQA or other (intrusive) metrics. Upon article acceptance, code will be provided at GitHub.
eess.AS音频处理

【1】Multilingual Query-by-Example Keyword Spotting with Metric Learning and Phoneme-to-Embedding Mapping标题：基于度量学习和音素到嵌入映射的多语种示例关键词识别链接：http://arxiv.org/pdf/2304.09585v1

作者：Paul M. Reuter,Christian Rollwage,Bernd T. Meyer机构：Carl von Ossietzky University, Oldenburg, Germany, Communication Acoustics and Cluster of Excellence Hearing,all, Fraunhofer Institute for Digital Media Technology IDMT, Division Hearing, Speech and Audio Technology, Oldenburg, Germany备注：Accepted to ICASSP 2023摘要：在本文中，我们提出了一个多语言查询的例子关键字定位（KWS）系统的基础上，剩余神经网络。该模型被训练为一个分类器上的多语言关键字数据集提取的常见的声音句子和微调使用循环损失。我们证明了该模型对新语言的泛化能力，并报告了与竞争基线相比，先前看到的EER平均降低了59.2%，看不见的语言平均降低了47.9%。我们表明，通过KWS模型学习的单词嵌入可以使用简单的LSTM模型从音素序列中准确预测。我们的系统实现了一个有前途的准确性流关键字发现和关键字搜索通用语音音频，每个关键字只使用5个例子。在Hey-Snips数据集上的实验显示出良好的性能，在每小时仅0.1个假警报的情况下，假阴性率为5.4%。摘要：In this paper, we propose a multilingual query-by-example keyword spotting (KWS) system based on a residual neural network. The model is trained as a classifier on a multilingual keyword dataset extracted from Common Voice sentences and fine-tuned using circle loss. We demonstrate the generalization ability of the model to new languages and report a mean reduction in EER of 59.2 % for previously seen and 47.9 % for unseen languages compared to a competitive baseline. We show that the word embeddings learned by the KWS model can be accurately predicted from the phoneme sequences using a simple LSTM model. Our system achieves a promising accuracy for streaming keyword spotting and keyword search on Common Voice audio using just 5 examples per keyword. Experiments on the Hey-Snips dataset show a good performance with a false negative rate of 5.4 % at only 0.1 false alarms per hour.

【2】Dynamic Chuck Convolution For Unified Streaming And Non-streaming Conformer ASR标题：用于统一流和非流整形器ASR的动态Chuck卷积链接：http://arxiv.org/pdf/2304.09325v1

*与cs.SD语音【3】为同一篇

作者：Xilai Li,Goeric Huybrechts,Srikanth Ronanki,Jeff Farris,Sravan Bodapati机构：AWS AI Labs备注：5 pages, 3 figures, 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023)摘要：最近，人们越来越关注统一流式和非流式语音识别模型以降低开发、训练和部署成本。最著名的方法依赖于基于窗口或基于动态块的注意力策略和因果卷积，以最大限度地减少由于流的退化。然而，在非流和独立训练的全上下文模型之间的性能差距仍然相对较大。为了解决这个问题，我们提出了一个动态的基于块的卷积代替因果卷积的混合连接主义时间分类（CTC）-注意一致性架构。此外，我们展示了进一步的改进，通过初始化的权重从一个完整的上下文模型和并行化的卷积和自我注意模块。我们在开源的Voxpopuli，LibriSpeech和内部会话数据集上评估我们的模型。总体而言，我们提出的模型将非流全上下文模型的流模式的退化从41.7%和45.7%分别降低到LibriSpeech测试清洁和测试其他数据集的16.7%和26.2%，同时比以前的最先进的统一模型提高了相对15.5%的WER。摘要：Recently, there has been an increasing interest in unifying streaming and non-streaming speech recognition models to reduce development, training and deployment cost. The best-known approaches rely on either window-based or dynamic chunk-based attention strategy and causal convolutions to minimize the degradation due to streaming. However, the performance gap still remains relatively large between non-streaming and a full-contextual model trained independently. To address this, we propose a dynamic chunk-based convolution replacing the causal convolution in a hybrid Connectionist Temporal Classification (CTC)-Attention Conformer architecture. Additionally, we demonstrate further improvements through initialization of weights from a full-contextual model and parallelization of the convolution and self-attention modules. We evaluate our models on the open-source Voxpopuli, LibriSpeech and in-house conversational datasets. Overall, our proposed model reduces the degradation of the streaming mode over the non-streaming full-contextual model from 41.7% and 45.7% to 16.7% and 26.2% on the LibriSpeech test-clean and test-other datasets respectively, while improving by a relative 15.5% WER over the previous state-of-the-art unified model.

【3】AIRCADE: an Anechoic and IR Convolution-based Auralization Data-compilation Ensemble标题：AIRCADE：一种基于消声和红外卷积的AIRCADE数据汇编系统链接：http://arxiv.org/pdf/2304.09318v1

作者：Túlio Chiodi,Arthur dos Santos,Pedro Martins,Bruno Masiero机构：School of Acoustical Engineering, Federal University of Santa Maria, Brazil, Arthur Nicholas dos Santos, School of Electrical and Computer Engineering, University of Campinas, Brazil, Bruno Sanches Masiero备注：5 pages, 2 figures摘要：在本文中，我们介绍了一个数据汇编合奏，主要是为了作为一个资源，研究人员在该领域的去混响，特别是数据驱动的方法。它包括语音和歌曲样本，以及原声吉他声音，具有与情感识别和音乐信息检索（MIR）相关的原始注释。此外，它还包括一系列具有不同混响时间（RT）值的脉冲响应（IR）样本，为评估提供了广泛的条件。该数据编译可以与提供的Python脚本一起使用，用于生成不同大小的可听化数据集合：小号中号大号另外，所提供的元数据注释还允许对不同条件下的去混响算法的性能进行进一步分析和调查。所有数据均采用知识共享署名4.0国际许可协议进行许可。摘要：In this paper, we introduce a data-compilation ensemble, primarily intended to serve as a resource for researchers in the field of dereverberation, particularly for data-driven approaches. It comprises speech and song samples, together with acoustic guitar sounds, with original annotations pertinent to emotion recognition and Music Information Retrieval (MIR). Moreover, it includes a selection of impulse response (IR) samples with varying Reverberation Time (RT) values, providing a wide range of conditions for evaluation. This data-compilation can be used together with provided Python scripts, for generating auralized data ensembles in different sizes: tiny, small, medium and large. Additionally, the provided metadata annotations also allow for further analysis and investigation of the performance of dereverberation algorithms under different conditions. All data is licensed under Creative Commons Attribution 4.0 International License.

【4】Coded Speech Quality Measurement by a Non-Intrusive PESQ-DNN标题：基于非侵入式PESQ-DNN的编码语音质量测量链接：http://arxiv.org/pdf/2304.09226v1

*与cs.SD语音【4】为同一篇

作者：Ziyi Xu,Ziyue Zhao,Tim Fingscheidt机构：Ziyi Xu and Tim Fingscheidt are with the Institute for Communi-cations Technology, Technische Universit¨at Braunschweig摘要：诸如AMR-WB或EVS之类的宽带编解码器广泛用于（移动）语音通信中。编码语音质量的评估通常由绝对类别评级（ACR）收听测试主观地执行。然而，ACR测试对于语音通信网络的在线监测是不切实际的。语音质量感知评价（PESQ）是广泛使用的用于预测ACR测试结果的度量之一。然而，PESQ算法需要原始参考信号，这在网络监测中通常是不可用的，从而限制了其适用性。NISQA是一种新的基于神经网络的非侵入式语音质量测量方法，主要针对超宽带语音信号。然而，在这项工作中，我们的目标是使用非侵入式PESQ-DNN模型预测众所周知的PESQ度量。我们通过预测从AMR-WB或EVS编解码器在噪声、串联和易出错的传输条件下以不同比特率操作获得的宽带编码语音的PESQ分数来说明该模型的潜力。我们将我们的方法与QualityNet，WaweNet和DNSMOS的最先进的网络拓扑结构进行比较-所有这些都适用于PESQ预测-通过测量平均绝对误差（MAE）和线性相关系数（LCC）。所提出的PESQ-DNN在没有帧丢失的条件下分别提供0.11和0.92的最佳总MAE和LCC，并且在包括帧丢失时仍然是最佳的。注意，我们的模型可以类似地用于非侵入性地预测POLQA或其他（侵入性）指标。文章接受后，代码将在GitHub上提供。摘要：Wideband codecs such as AMR-WB or EVS are widely used in (mobile) speech communication. Evaluation of coded speech quality is often performed subjectively by an absolute category rating (ACR) listening test. However, the ACR test is impractical for online monitoring of speech communication networks. Perceptual evaluation of speech quality (PESQ) is one of the widely used metrics instrumentally predicting the results of an ACR test. However, the PESQ algorithm requires an original reference signal, which is usually unavailable in network monitoring, thus limiting its applicability. NISQA is a new non-intrusive neural-network-based speech quality measure, focusing on super-wideband speech signals. In this work, however, we aim at predicting the well-known PESQ metric using a non-intrusive PESQ-DNN model. We illustrate the potential of this model by predicting the PESQ scores of wideband-coded speech obtained from AMR-WB or EVS codecs operating at different bitrates in noisy, tandeming, and error-prone transmission conditions. We compare our methods with the state-of-the-art network topologies of QualityNet, WaweNet, and DNSMOS -- all applied to PESQ prediction -- by measuring the mean absolute error (MAE) and the linear correlation coefficient (LCC). The proposed PESQ-DNN offers the best total MAE and LCC of 0.11 and 0.92, respectively, in conditions without frame loss, and still is best when including frame loss. Note that our model could be similarly used to non-intrusively predict POLQA or other (intrusive) metrics. Upon article acceptance, code will be provided at GitHub.

【5】CB-Conformer: Contextual biasing Conformer for biased word recognition标题：CB-Conform：用于偏倚词识别的上下文偏向整形器链接：http://arxiv.org/pdf/2304.09607v1

*与cs.SD语音【1】为同一篇

作者：Yaoxun Xu,Baiji Liu,Qiaochu Huang and,Xingchen Song,Zhiyong Wu,Shiyin Kang,Helen Meng机构：Shenzhen International Graduate School, Tsinghua University, Shenzhen, China, XVerse Inc., Shenzhen, China, Peng Cheng Lab, Shenzhen, China, The Chinese University of Hong Kong, Hong Kong SAR, China摘要：由于源域和目标域之间的不匹配，如何更好地利用有偏词信息来提高目标域自动语音识别模型的性能成为研究的热点。以往的方法要么使用固定的外部语言模型进行解码，要么引入一个相当大的偏置模块，这导致了适应性差和推理速度慢。在这项工作中，我们提出了CB-Conformer，通过引入上下文偏置模块和自适应语言模型香草Conformer，以提高有偏见的单词识别。上下文偏置模块将音频片段和上下文信息相结合，仅具有原始Conformer的0.2%的模型参数。自适应语言模型根据其召回率和准确率修改有偏见的单词的内部权重，从而比标准固定语言模型更关注有偏见的单词并且更成功地与自动语音识别模型集成。此外，我们构建并发布了一个基于WenetSpeech的开源汉语偏误词数据集。实验结果表明，我们提出的方法带来了15.34%的字符错误率降低，14.13%的偏见字召回增加，和6.80%的偏见字F1分数增加相比，基础的构象。摘要：Due to the mismatch between the source and target domains, how to better utilize the biased word information to improve the performance of the automatic speech recognition model in the target domain becomes a hot research topic. Previous approaches either decode with a fixed external language model or introduce a sizeable biasing module, which leads to poor adaptability and slow inference. In this work, we propose CB-Conformer to improve biased word recognition by introducing the Contextual Biasing Module and the Self-Adaptive Language Model to vanilla Conformer. The Contextual Biasing Module combines audio fragments and contextual information, with only 0.2% model parameters of the original Conformer. The Self-Adaptive Language Model modifies the internal weights of biased words based on their recall and precision, resulting in a greater focus on biased words and more successful integration with the automatic speech recognition model than the standard fixed language model. In addition, we construct and release an open-source Mandarin biased-word dataset based on WenetSpeech. Experiments indicate that our proposed method brings a 15.34% character error rate reduction, a 14.13% biased word recall increase, and a 6.80% biased word F1-score increase compared with the base Conformer.

【6】Security and Privacy Problems in Voice Assistant Applications: A Survey标题：语音助手应用中的安全和隐私问题：综述链接：http://arxiv.org/pdf/2304.09486v1

*与cs.SD语音【2】为同一篇

作者：Jingjin Li,Chao chen,Lei Pan,Mostafa Rahimi Azghadi,Hossein Ghodosi,Jun Zhang机构：James Cook University, RMIT University, Deakin University, Swinburne University of Technology备注：5 figures摘要：语音助理应用程序现在已经变得无所不知。两个模型为现实生活中的应用提供了两个最重要的功能（即，Google Home、Amazon Alexa、Siri等）是自动语音识别（ASR）模型和说话人识别（SI）模型。根据最近的研究，随着物联网（IoT）的快速发展，安全和隐私威胁也出现了。研究的安全问题包括针对机器学习模型和其他广泛用于语音助理应用的硬件组件的攻击技术。隐私问题包括技术上的信息窃取和策略上的隐私泄露。语音助手应用每年都占据着稳步增长的市场份额，但其隐私和安全问题从未停止过，造成了巨大的经济损失，并危及用户的个人敏感信息。因此，重要的是要有一个全面的调查，以概述目前的研究分类有关的安全和隐私问题的语音助理应用程序。本文总结和评估了网络安全和语音领域顶级会议上发表的论文中的五种安全攻击和三种隐私威胁。摘要：Voice assistant applications have become omniscient nowadays. Two models that provide the two most important functions for real-life applications (i.e., Google Home, Amazon Alexa, Siri, etc.) are Automatic Speech Recognition (ASR) models and Speaker Identification (SI) models. According to recent studies, security and privacy threats have also emerged with the rapid development of the Internet of Things (IoT). The security issues researched include attack techniques toward machine learning models and other hardware components widely used in voice assistant applications. The privacy issues include technical-wise information stealing and policy-wise privacy breaches. The voice assistant application takes a steadily growing market share every year, but their privacy and security issues never stopped causing huge economic losses and endangering users' personal sensitive information. Thus, it is important to have a comprehensive survey to outline the categorization of the current research regarding the security and privacy problems of voice assistant applications. This paper concludes and assesses five kinds of security attacks and three types of privacy threats in the papers published in the top-tier conferences of cyber security and voice domain.

机器翻译，仅供参考