【1】 Source Tracing: Detecting Voice Spoofing
标题：来源追踪：检测语音欺骗
链接：https://arxiv.org/abs/2212.08601
作者：Tinglong Zhu,Xingming Wang,Xiaoyi Qin,Ming Li机构：∗ Data Science Research Center, Duke Kunshan University, Jiangsu, China, † School of Computer Science, Wuhan University, Wuhan, China备注：Accepted by APSIPA ASC摘要：近来的反欺骗系统集中于欺骗检测，其中的任务仅仅是确定测试音频是否是伪造的。然而，很少有研究关注虚假语音产生的方法。逻辑访问（LA）场景中常见的欺骗攻击算法，例如语音转换和语音合成，可以分为几个阶段：输入处理、转换、波形生成等。在这项工作中，我们提出了一个系统，用于分类不同的电子欺骗属性，表示整个流水线中不同模块的特征。当在不同阶段遇到不同模块的复杂组合时，对欺骗攻击的属性进行分类而不是确定整个欺骗流水线可以使系统更加鲁棒。此外，我们的系统还可以作为一个辅助系统，用于反欺骗对看不见的欺骗方法。在ASVspoof2019LA数据集上进行了实验，与传统的二进制欺骗检测方法相比，该方法取得了20%的相对改进.摘要：Recent anti-spoofing systems focus on spoofing detection, where the task is only to determine whether the test audio is fake. However, there are few studies putting attention to identifying the methods of generating fake speech. Common spoofing attack algorithms in the logical access (LA) scenario, such as voice conversion and speech synthesis, can be divided into several stages: input processing, conversion, waveform generation, etc. In this work, we propose a system for classifying different spoofing attributes, representing characteristics of different modules in the whole pipeline. Classifying attributes for the spoofing attack other than determining the whole spoofing pipeline can make the system more robust when encountering complex combinations of different modules at different stages. In addition, our system can also be used as an auxiliary system for anti-spoofing against unseen spoofing methods. The experiments are conducted on ASVspoof 2019 LA data set and the proposed method achieved a 20\% relative improvement against conventional binary spoof detection methods.
【2】 Statistical Design and Analysis for Robust Machine Learning: A Case Study from COVID-19
标题：稳健机器学习的统计设计与分析--以新冠肺炎为例
链接：https://arxiv.org/abs/2212.08571
作者：Davide Pigoli,Kieran Baker,Jobie Budd,Lorraine Butler,Harry Coppock,Sabrina Egglestone,Steven G. Gilmour,Chris Holmes,David Hurley,Radka Jersakova,Ivan Kiskin,Vasiliki Koutra,Jonathon Mellor,George Nicholson,Joe Packham,Selina Patel,Richard Payne,Stephen J. Roberts,Björn W. Schuller,Ana Tendero-Cañadas,Tracey Thornley,Alexander Titcomb机构：King’s College London, UK and The Alan Turing Institute, London, UK., UK Health Security Agency, London, UK., University of Oxford, UK and The Alan Turing Institute, London, UK., University of Surrey, UK., University of Oxford, UK.摘要：自2019年冠状病毒病（COVID-19）大流行早期以来，人们一直对使用人工智能方法基于声音音频信号（例如咳嗽录音）预测COVID-19感染状态感兴趣。然而，现有的研究在数据收集和评估所提出的预测模型的性能方面存在局限性。这篇论文使用英国卫生安全局收集的数据集，严格评估了用于基于语音音频信号预测COVID-19感染状态的最先进的机器学习技术。该数据集包括声学记录和广泛的研究参与者元数据。我们提供了基于声学特征对COVID-19感染状态进行分类的方法性能测试指南，并讨论了如何将这些方法更广泛地扩展到基于公共卫生数据集的预测方法的开发和评估。摘要：Since early in the coronavirus disease 2019 (COVID-19) pandemic, there has been interest in using artificial intelligence methods to predict COVID-19 infection status based on vocal audio signals, for example cough recordings. However, existing studies have limitations in terms of data collection and of the assessment of the performances of the proposed predictive models. This paper rigorously assesses state-of-the-art machine learning techniques used to predict COVID-19 infection status based on vocal audio signals, using a dataset collected by the UK Health Security Agency. This dataset includes acoustic recordings and extensive study participant meta-data. We provide guidelines on testing the performance of methods to classify COVID-19 infection status based on acoustic features and we discuss how these can be extended more generally to the development and assessment of predictive methods based on public health datasets.
【3】 Audio-based AI classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers
标题：基于音频的AI分类器没有证据表明新冠肺炎筛查比简单的症状检查器更好
链接：https://arxiv.org/abs/2212.08570
作者：Harry Coppock,George Nicholson,Ivan Kiskin,Vasiliki Koutra,Kieran Baker,Jobie Budd,Richard Payne,Emma Karoune,David Hurley,Alexander Titcomb,Sabrina Egglestone,Ana Tendero Cañadas,Lorraine Butler,Radka Jersakova,Jonathon Mellor,Selina Patel,Tracey Thornley,Peter Diggle,Sylvia Richardson,Josef Packham,Björn W. Schuller,Davide Pigoli,Steven Gilmour,Stephen Roberts,Chris Holmes机构：The Alan Turing Institute, UK, Imperial College London, UK, University of Oxford, UK, Surrey Institute for People-Centred AI, University of Surrey, UK, King’s College London, UK, Division of Medicine, University College London, UK摘要：最近的工作报告说，在录音上训练的AI分类器可以准确预测严重急性呼吸综合征冠状病毒2（SARSCoV 2）感染状态。在这里，我们对基于音频的深度学习分类器进行了大规模研究，作为英国政府应对流感大流行的一部分。我们收集并分析了67，842名个人的音频记录数据集，并链接了元数据，包括逆转录聚合酶链反应（PCR）检测结果，其中23，514人SARS CoV 2检测呈阳性。受试者通过英国政府国家卫生服务检测和追踪项目和社区传播实时评估（REACT）随机监测调查招募。在我们的数据集的未调整分析中，AI分类器预测SARS-CoV-2感染状态的准确性很高（受试者操作特征曲线下面积（ROCAUC）0.846 [0.838，0.854]），与先前研究的结果一致。然而，在匹配测量的混杂因素（如年龄、性别和自我报告的症状）后，我们的分类器性能要弱得多（ROC-AUC 0.619 [0.594，0.644]）。在量化基于音频的分类器在实际设置中的效用时，我们发现基于用户报告的症状的简单预测评分优于它们。摘要：Recent work has reported that AI classifiers trained on audio recordings can accurately predict severe acute respiratory syndrome coronavirus 2 (SARSCoV2) infection status. Here, we undertake a large scale study of audio-based deep learning classifiers, as part of the UK governments pandemic response. We collect and analyse a dataset of audio recordings from 67,842 individuals with linked metadata, including reverse transcription polymerase chain reaction (PCR) test outcomes, of whom 23,514 tested positive for SARS CoV 2. Subjects were recruited via the UK governments National Health Service Test-and-Trace programme and the REal-time Assessment of Community Transmission (REACT) randomised surveillance survey. In an unadjusted analysis of our dataset AI classifiers predict SARS-CoV-2 infection status with high accuracy (Receiver Operating Characteristic Area Under the Curve (ROCAUC) 0.846 [0.838, 0.854]) consistent with the findings of previous studies. However, after matching on measured confounders, such as age, gender, and self reported symptoms, our classifiers performance is much weaker (ROC-AUC 0.619 [0.594, 0.644]). Upon quantifying the utility of audio based classifiers in practical settings, we find them to be outperformed by simple predictive scores based on user reported symptoms.
【4】 Effectiveness of Text, Acoustic, and Lattice-based representations in Spoken Language Understanding tasks
标题：基于文本、声音和晶格的表征在口语理解任务中的有效性
链接：https://arxiv.org/abs/2212.08489
作者：Esaú Villatoro-Tello,Srikanth Madikeri,Juan Zuluaga-Gomez,Bidisha Sharma,Seyyed Saeed Sarfjoo,Iuliia Nigmatulina,Petr Motlicek,Alexei V. Ivanov,Aravind Ganapathiraju机构：Idiap Research Institute, Martigny, Switzerland., Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland., Institute of Computational Linguistics, University of Z¨urich, Switzerland., Uniphore Software Systems Inc., Palo Alto, CA, USA.备注：Submitted to ICASSP 2023 (Under review)摘要：本文针对口语理解（SLU）系统中的意图分类问题，对不同的表征方式进行了详尽的评估。我们对三种类型的系统进行基准测试以执行SLU意图检测任务：1）基于文本的，2）基于格的，以及新颖的3）多模态方法。我们的工作提供了一个全面的分析，什么是可以实现的性能，在不同的情况下，不同的国家的最先进的SLU系统，例如，自动与手动生成的转录本。我们在公开的SLURP口语语料库上对系统进行了测试。我们的结果表明，与1-最佳设置相比，使用更丰富形式的自动语音识别（ASR）输出允许SLU系统改进（4%相对改进）。然而，跨模态方法，从声学和文本嵌入中学习，获得与Oracle设置类似的性能，以及比1-best配置相对提高18%。因此，跨模式架构代表了一种很好的替代方案，可以克服纯自动生成的文本数据的局限性。摘要：In this paper, we perform an exhaustive evaluation of different representations to address the intent classification problem in a Spoken Language Understanding (SLU) setup. We benchmark three types of systems to perform the SLU intent detection task: 1) text-based, 2) lattice-based, and a novel 3) multimodal approach. Our work provides a comprehensive analysis of what could be the achievable performance of different state-of-the-art SLU systems under different circumstances, e.g., automatically- vs. manually-generated transcripts. We evaluate the systems on the publicly available SLURP spoken language resource corpus. Our results indicate that using richer forms of Automatic Speech Recognition (ASR) outputs allows SLU systems to improve in comparison to the 1-best setup (4% relative improvement). However, crossmodal approaches, i.e., learning from acoustic and text embeddings, obtains performance similar to the oracle setup, and a relative improvement of 18% over the 1-best configuration. Thus, crossmodal architectures represent a good alternative to overcome the limitations of working purely automatically generated textual data.
【5】 Feature Dropout: Revisiting the Role of Augmentations in Contrastive Learning
标题：特征缺失：重温扩充在对比学习中的作用
链接：https://arxiv.org/abs/2212.08378
作者：Alex Tamkin,Margalit Glasgow,Xiluo He,Noah Goodman摘要：强化在对比学习中扮演什么角色？最近的研究表明，良好的扩增是标签保留相对于一个特定的下游任务。我们通过展示标签破坏增强在基础模型设置中可能是有用的来使这幅图景复杂化，在基础模型设置中，目标是学习多个下游任务的多样化、通用的表征。我们在一系列图像和音频数据集上进行了对比学习实验，并执行了多个下游任务（例如，对于叠加在照片上的数字，预测一个与另一个的类别）。我们发现，Viewmaker Networks（一种最近提出的用于对比学习的学习增强模型）产生随机破坏不同下游任务所需特征的标签破坏增强。这些增强是可解释的（例如，改变添加到图像的形状、数字或字母），并且令人惊讶地，与专家设计的增强相比，通常产生更好的性能，尽管不保留标签信息。为了支持我们的实证结果，我们用线性模型对一个简单的对比学习环境进行了理论分析。在这种情况下，标签破坏增强对于防止一组特征抑制对另一下游任务有用的特征的学习是至关重要的。我们的结果强调了在试图解释基础模型的成功时，分析多个下游任务之间的交互作用的必要性。摘要：What role do augmentations play in contrastive learning? Recent work suggests that good augmentations are label-preserving with respect to a specific downstream task. We complicate this picture by showing that label-destroying augmentations can be useful in the foundation model setting, where the goal is to learn diverse, general-purpose representations for multiple downstream tasks. We perform contrastive learning experiments on a range of image and audio datasets with multiple downstream tasks (e.g. for digits superimposed on photographs, predicting the class of one vs. the other). We find that Viewmaker Networks, a recently proposed model for learning augmentations for contrastive learning, produce label-destroying augmentations that stochastically destroy features needed for different downstream tasks. These augmentations are interpretable (e.g. altering shapes, digits, or letters added to images) and surprisingly often result in better performance compared to expert-designed augmentations, despite not preserving label information. To support our empirical results, we theoretically analyze a simple contrastive learning setting with a linear model. In this setting, label-destroying augmentations are crucial for preventing one set of features from suppressing the learning of features useful for another downstream task. Our results highlight the need for analyzing the interaction between multiple downstream tasks when trying to explain the success of foundation models.
【6】 Towards Unified All-Neural Beamforming for Time and Frequency Domain Speech Separation
标题：时频域语音分离的统一全神经波束形成
链接：https://arxiv.org/abs/2212.08348
作者：Rongzhi Gu,Shi-Xiong Zhang,Yuexian Zou,Dong Yu摘要：近年来，频域全神经元波束形成方法在多通道语音分离中取得了显著的进展。同时，时域网络结构与波束形成的结合也得到了广泛的关注。本文提出了一种新的时域全神经元波束形成方法，尝试统一时域和频域多通道语音分离的全神经元波束形成流水线。拟议的模式包括两个模块：分离和波束形成。这两个模块都执行时间-频谱-空间建模，并使用联合损失函数进行端到端训练。本研究的新颖之处在于两方面。首先，提出了一种以目标说话人方向为条件的时域方向特征，该特征可以在时域结构内联合优化，以增强目标信号估计。其次，设计了时域全神经元波束形成网络对预分离结果进行细化。该模块的特点是参数化时变波束形成系数估计，而不显式地遵循最优滤波器的推导可能导致上界。在AISHELL-1语料库中模拟的混响重叠语音数据上对该方法进行了评估。实验结果表明，与频域最新技术、理想幅度模板和现有时域神经波束形成方法相比，该方法性能有显著提高。摘要：Recently, frequency domain all-neural beamforming methods have achieved remarkable progress for multichannel speech separation. In parallel, the integration of time domain network structure and beamforming also gains significant attention. This study proposes a novel all-neural beamforming method in time domain and makes an attempt to unify the all-neural beamforming pipelines for time domain and frequency domain multichannel speech separation. The proposed model consists of two modules: separation and beamforming. Both modules perform temporal-spectral-spatial modeling and are trained from end-to-end using a joint loss function. The novelty of this study lies in two folds. Firstly, a time domain directional feature conditioned on the direction of the target speaker is proposed, which can be jointly optimized within the time domain architecture to enhance target signal estimation. Secondly, an all-neural beamforming network in time domain is designed to refine the pre-separated results. This module features with parametric time-variant beamforming coefficient estimation, without explicitly following the derivation of optimal filters that may lead to an upper bound. The proposed method is evaluated on simulated reverberant overlapped speech data derived from the AISHELL-1 corpus. Experimental results demonstrate significant performance improvements over frequency domain state-of-the-arts, ideal magnitude masks and existing time domain neural beamforming methods.
【1】 Context-aware Fine-tuning of Self-supervised Speech Models
标题：自监督语音模型的上下文感知微调
链接：https://arxiv.org/abs/2212.08542
作者：Suwon Shon,Felix Wu,Kwangyoun Kim,Prashant Sridhar,Karen Livescu,Shinji Watanabe机构：ASAPP, Toyota Technological Institute at Chicago, Carnegie Mellon University摘要：自监督预训练Transformers改进了各种语音任务的技术状态。由于自我注意力的二次时空复杂性，它们通常在相对较短（例如，话语）片段。本文研究了语境的使用，即，在微调期间围绕片段，并且提出了一种称为上下文感知微调的新方法。我们在预训练模型的最后一层附加上下文模块，将整个片段编码为上下文嵌入向量，然后将其用作最终预测的附加特征。在微调阶段，我们引入了一个辅助损失，鼓励这个上下文嵌入向量与周围片段的上下文向量相似。这允许模型在推理时无需访问这些周围的片段就可以进行预测，并且与标准的微调模型相比，只需要很小的开销。我们使用SLUE和Librilight基准评估了几项下游任务的建议方法：自动语音识别（ASR）、命名实体识别（NER）和情感分析（SA）。实验结果表明，上下文感知微调不仅优于标准微调基线，而且可以与使用相邻语音片段进行推理的强上下文注入基线相媲美。摘要：Self-supervised pre-trained transformers have improved the state of the art on a variety of speech tasks. Due to the quadratic time and space complexity of self-attention, they usually operate at the level of relatively short (e.g., utterance) segments. In this paper, we study the use of context, i.e., surrounding segments, during fine-tuning and propose a new approach called context-aware fine-tuning. We attach a context module on top of the last layer of a pre-trained model to encode the whole segment into a context embedding vector which is then used as an additional feature for the final prediction. During the fine-tuning stage, we introduce an auxiliary loss that encourages this context embedding vector to be similar to context vectors of surrounding segments. This allows the model to make predictions without access to these surrounding segments at inference time and requires only a tiny overhead compared to standard fine-tuned models. We evaluate the proposed approach using the SLUE and Librilight benchmarks for several downstream tasks: Automatic speech recognition (ASR), named entity recognition (NER), and sentiment analysis (SA). The results show that context-aware fine-tuning not only outperforms a standard fine-tuning baseline but also rivals a strong context injection baseline that uses neighboring speech segments during inference.
【2】 Text-to-speech synthesis based on latent variable conversion using diffusion probabilistic model and variational autoencoder
标题：基于扩散概率模型潜变量转换和变分自动编码器的文语合成
链接：https://arxiv.org/abs/2212.08329
作者：Yusuke Yasuda,Tomoki Toda备注：Submitted to ICASSP 2023摘要：文本语音合成（TTS）是一种将文本转换为语音的技术。推动TTS的两个因素是概率模型和潜在表示学习的进步。提出了一种基于扩散概率模型和变分自动编码器（VAE）的隐变量转换的TTS方法。在我们的TTS方法中，我们使用基于VAE的波形模型、预测来自文本的波形模型中的潜在变量的分布的扩散模型、以及学习文本和语音潜在序列之间的对准的对准模型。我们的方法通过用扩散建模均值和方差参数来集成扩散和VAE，其中目标分布由VAE近似确定。这种潜在变量转换框架潜在地使我们能够灵活地结合各种潜在特征提取器。实验结果表明，该方法对拼写错误和对齐错误较少的语言标签具有较好的鲁棒性。摘要：Text-to-speech synthesis (TTS) is a task to convert texts into speech. Two of the factors that have been driving TTS are the advancements of probabilistic models and latent representation learning. We propose a TTS method based on latent variable conversion using a diffusion probabilistic model and the variational autoencoder (VAE). In our TTS method, we use a waveform model based on VAE, a diffusion model that predicts the distribution of latent variables in the waveform model from texts, and an alignment model that learns alignments between the text and speech latent sequences. Our method integrates diffusion with VAE by modeling both mean and variance parameters with diffusion, where the target distribution is determined by approximation from VAE. This latent variable conversion framework potentially enables us to flexibly incorporate various latent feature extractors. Our experiments show that our method is robust to linguistic labels with poor orthography and alignment errors.
【3】 Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language
标题：音调文语合成中日语PNG BERT语言模型的研究
链接：https://arxiv.org/abs/2212.08321
作者：Yusuke Yasuda,Tomoki Toda摘要：端到端文本语音合成（TTS）技术可以从原始文本中生成自然的合成语音。然而，正确的音高重音呈现对于端到端TTS来说仍然是一个具有挑战性的问题。为了解决日语端到端语音合成系统中音高重音的正确表达问题，本文采用了一种基于字符和音素域的自监督预训练模型PnG~BERT。通过修改微调条件，确定有助于音高重音推断的条件，研究了PnG~BERT捕获的特征对日语TTS的影响。通过改变TTS过程中微调的层数，将PnG~BERT特征的内容从面向文本转换为面向语音。此外，我们借由音调预测微调来教授PnG~BERT音高重音资讯作为额外的下游工作。实验结果表明，PnG~BERT的特征包含了有助于音高重音推断的信息，在听力测试中PnG~BERT的重音正确率优于Tacotron。摘要：End-to-end text-to-speech synthesis (TTS) can generate highly natural synthetic speech from raw text. However, rendering the correct pitch accents is still a challenging problem for end-to-end TTS. To tackle the challenge of rendering correct pitch accent in Japanese end-to-end TTS, we adopt PnG~BERT, a self-supervised pretrained model in the character and phoneme domain for TTS. We investigate the effects of features captured by PnG~BERT on Japanese TTS by modifying the fine-tuning condition to determine the conditions helpful inferring pitch accents. We manipulate content of PnG~BERT features from being text-oriented to speech-oriented by changing the number of fine-tuned layers during TTS. In addition, we teach PnG~BERT pitch accent information by fine-tuning with tone prediction as an additional downstream task. Our experimental results show that the features of PnG~BERT captured by pretraining contain information helpful inferring pitch accent, and PnG~BERT outperforms baseline Tacotron on accent correctness in a listening test.
【4】 Source Tracing: Detecting Voice Spoofing
标题：来源追踪：检测语音欺骗
链接：https://arxiv.org/abs/2212.08601
*与cs.SD语音【1】为同一篇
作者：Tinglong Zhu,Xingming Wang,Xiaoyi Qin,Ming Li机构：∗ Data Science Research Center, Duke Kunshan University, Jiangsu, China, † School of Computer Science, Wuhan University, Wuhan, China备注：Accepted by APSIPA ASC摘要：近来的反欺骗系统集中于欺骗检测，其中的任务仅仅是确定测试音频是否是伪造的。然而，很少有研究关注虚假语音产生的方法。逻辑访问（LA）场景中常见的欺骗攻击算法，例如语音转换和语音合成，可以分为几个阶段：输入处理、转换、波形生成等。在这项工作中，我们提出了一个系统，用于分类不同的电子欺骗属性，表示整个流水线中不同模块的特征。当在不同阶段遇到不同模块的复杂组合时，对欺骗攻击的属性进行分类而不是确定整个欺骗流水线可以使系统更加鲁棒。此外，我们的系统还可以作为一个辅助系统，用于反欺骗对看不见的欺骗方法。在ASVspoof2019LA数据集上进行了实验，与传统的二进制欺骗检测方法相比，该方法取得了20%的相对改进.摘要：Recent anti-spoofing systems focus on spoofing detection, where the task is only to determine whether the test audio is fake. However, there are few studies putting attention to identifying the methods of generating fake speech. Common spoofing attack algorithms in the logical access (LA) scenario, such as voice conversion and speech synthesis, can be divided into several stages: input processing, conversion, waveform generation, etc. In this work, we propose a system for classifying different spoofing attributes, representing characteristics of different modules in the whole pipeline. Classifying attributes for the spoofing attack other than determining the whole spoofing pipeline can make the system more robust when encountering complex combinations of different modules at different stages. In addition, our system can also be used as an auxiliary system for anti-spoofing against unseen spoofing methods. The experiments are conducted on ASVspoof 2019 LA data set and the proposed method achieved a 20\% relative improvement against conventional binary spoof detection methods.
【5】 Statistical Design and Analysis for Robust Machine Learning: A Case Study from COVID-19
标题：稳健机器学习的统计设计与分析--以新冠肺炎为例
链接：https://arxiv.org/abs/2212.08571
*与cs.SD语音【2】为同一篇
作者：Davide Pigoli,Kieran Baker,Jobie Budd,Lorraine Butler,Harry Coppock,Sabrina Egglestone,Steven G. Gilmour,Chris Holmes,David Hurley,Radka Jersakova,Ivan Kiskin,Vasiliki Koutra,Jonathon Mellor,George Nicholson,Joe Packham,Selina Patel,Richard Payne,Stephen J. Roberts,Björn W. Schuller,Ana Tendero-Cañadas,Tracey Thornley,Alexander Titcomb机构：King’s College London, UK and The Alan Turing Institute, London, UK., UK Health Security Agency, London, UK., University of Oxford, UK and The Alan Turing Institute, London, UK., University of Surrey, UK., University of Oxford, UK.摘要：自2019年冠状病毒病（COVID-19）大流行早期以来，人们一直对使用人工智能方法基于声音音频信号（例如咳嗽录音）预测COVID-19感染状态感兴趣。然而，现有的研究在数据收集和评估所提出的预测模型的性能方面存在局限性。这篇论文使用英国卫生安全局收集的数据集，严格评估了用于基于语音音频信号预测COVID-19感染状态的最先进的机器学习技术。该数据集包括声学记录和广泛的研究参与者元数据。我们提供了基于声学特征对COVID-19感染状态进行分类的方法性能测试指南，并讨论了如何将这些方法更广泛地扩展到基于公共卫生数据集的预测方法的开发和评估。摘要：Since early in the coronavirus disease 2019 (COVID-19) pandemic, there has been interest in using artificial intelligence methods to predict COVID-19 infection status based on vocal audio signals, for example cough recordings. However, existing studies have limitations in terms of data collection and of the assessment of the performances of the proposed predictive models. This paper rigorously assesses state-of-the-art machine learning techniques used to predict COVID-19 infection status based on vocal audio signals, using a dataset collected by the UK Health Security Agency. This dataset includes acoustic recordings and extensive study participant meta-data. We provide guidelines on testing the performance of methods to classify COVID-19 infection status based on acoustic features and we discuss how these can be extended more generally to the development and assessment of predictive methods based on public health datasets.
【6】 Audio-based AI classifiers show no evidence of improved COVID-19 screening over simple symptoms checkers
标题：基于音频的AI分类器没有证据表明新冠肺炎筛查比简单的症状检查器更好
链接：https://arxiv.org/abs/2212.08570
*与cs.SD语音【3】为同一篇
作者：Harry Coppock,George Nicholson,Ivan Kiskin,Vasiliki Koutra,Kieran Baker,Jobie Budd,Richard Payne,Emma Karoune,David Hurley,Alexander Titcomb,Sabrina Egglestone,Ana Tendero Cañadas,Lorraine Butler,Radka Jersakova,Jonathon Mellor,Selina Patel,Tracey Thornley,Peter Diggle,Sylvia Richardson,Josef Packham,Björn W. Schuller,Davide Pigoli,Steven Gilmour,Stephen Roberts,Chris Holmes机构：The Alan Turing Institute, UK, Imperial College London, UK, University of Oxford, UK, Surrey Institute for People-Centred AI, University of Surrey, UK, King’s College London, UK, Division of Medicine, University College London, UK摘要：Recent work has reported that AI classifiers trained on audio recordings can accurately predict severe acute respiratory syndrome coronavirus 2 (SARSCoV2) infection status. Here, we undertake a large scale study of audio-based deep learning classifiers, as part of the UK governments pandemic response. We collect and analyse a dataset of audio recordings from 67,842 individuals with linked metadata, including reverse transcription polymerase chain reaction (PCR) test outcomes, of whom 23,514 tested positive for SARS CoV 2. Subjects were recruited via the UK governments National Health Service Test-and-Trace programme and the REal-time Assessment of Community Transmission (REACT) randomised surveillance survey. In an unadjusted analysis of our dataset AI classifiers predict SARS-CoV-2 infection status with high accuracy (Receiver Operating Characteristic Area Under the Curve (ROCAUC) 0.846 [0.838, 0.854]) consistent with the findings of previous studies. However, after matching on measured confounders, such as age, gender, and self reported symptoms, our classifiers performance is much weaker (ROC-AUC 0.619 [0.594, 0.644]). Upon quantifying the utility of audio based classifiers in practical settings, we find them to be outperformed by simple predictive scores based on user reported symptoms.
【7】 Effectiveness of Text, Acoustic, and Lattice-based representations in Spoken Language Understanding tasks
标题：基于文本、声音和晶格的表征在口语理解任务中的有效性
链接：https://arxiv.org/abs/2212.08489
*与cs.SD语音【4】为同一篇
作者：Esaú Villatoro-Tello,Srikanth Madikeri,Juan Zuluaga-Gomez,Bidisha Sharma,Seyyed Saeed Sarfjoo,Iuliia Nigmatulina,Petr Motlicek,Alexei V. Ivanov,Aravind Ganapathiraju机构：Idiap Research Institute, Martigny, Switzerland., Ecole Polytechnique Federale de Lausanne (EPFL), Switzerland., Institute of Computational Linguistics, University of Z¨urich, Switzerland., Uniphore Software Systems Inc., Palo Alto, CA, USA.备注：Submitted to ICASSP 2023 (Under review)摘要：本文针对口语理解（SLU）系统中的意图分类问题，对不同的表征方式进行了详尽的评估。我们对三种类型的系统进行基准测试以执行SLU意图检测任务：1）基于文本的，2）基于格的，以及新颖的3）多模态方法。我们的工作提供了一个全面的分析，什么是可以实现的性能，在不同的情况下，不同的国家的最先进的SLU系统，例如，自动与手动生成的转录本。我们在公开的SLURP口语语料库上对系统进行了测试。我们的结果表明，与1-最佳设置相比，使用更丰富形式的自动语音识别（ASR）输出允许SLU系统改进（4%相对改进）。然而，跨模态方法，从声学和文本嵌入中学习，获得与Oracle设置类似的性能，以及比1-best配置相对提高18%。因此，跨模式架构代表了一种很好的替代方案，可以克服纯自动生成的文本数据的局限性。摘要：In this paper, we perform an exhaustive evaluation of different representations to address the intent classification problem in a Spoken Language Understanding (SLU) setup. We benchmark three types of systems to perform the SLU intent detection task: 1) text-based, 2) lattice-based, and a novel 3) multimodal approach. Our work provides a comprehensive analysis of what could be the achievable performance of different state-of-the-art SLU systems under different circumstances, e.g., automatically- vs. manually-generated transcripts. We evaluate the systems on the publicly available SLURP spoken language resource corpus. Our results indicate that using richer forms of Automatic Speech Recognition (ASR) outputs allows SLU systems to improve in comparison to the 1-best setup (4% relative improvement). However, crossmodal approaches, i.e., learning from acoustic and text embeddings, obtains performance similar to the oracle setup, and a relative improvement of 18% over the 1-best configuration. Thus, crossmodal architectures represent a good alternative to overcome the limitations of working purely automatically generated textual data.
【8】 Feature Dropout: Revisiting the Role of Augmentations in Contrastive Learning
标题：特征缺失：重温扩充在对比学习中的作用
链接：https://arxiv.org/abs/2212.08378
*与cs.SD语音【5】为同一篇
作者：Alex Tamkin,Margalit Glasgow,Xiluo He,Noah Goodman摘要：强化在对比学习中扮演什么角色？最近的研究表明，良好的扩增是标签保留相对于一个特定的下游任务。我们通过展示标签破坏增强在基础模型设置中可能是有用的来使这幅图景复杂化，在基础模型设置中，目标是学习多个下游任务的多样化、通用的表征。我们在一系列图像和音频数据集上进行了对比学习实验，并执行了多个下游任务（例如，对于叠加在照片上的数字，预测一个与另一个的类别）。我们发现，Viewmaker Networks（一种最近提出的用于对比学习的学习增强模型）产生随机破坏不同下游任务所需特征的标签破坏增强。这些增强是可解释的（例如，改变添加到图像的形状、数字或字母），并且令人惊讶地，与专家设计的增强相比，通常产生更好的性能，尽管不保留标签信息。为了支持我们的实证结果，我们用线性模型对一个简单的对比学习环境进行了理论分析。在这种情况下，标签破坏增强对于防止一组特征抑制对另一下游任务有用的特征的学习是至关重要的。我们的结果强调了在试图解释基础模型的成功时，分析多个下游任务之间的交互作用的必要性。摘要：What role do augmentations play in contrastive learning? Recent work suggests that good augmentations are label-preserving with respect to a specific downstream task. We complicate this picture by showing that label-destroying augmentations can be useful in the foundation model setting, where the goal is to learn diverse, general-purpose representations for multiple downstream tasks. We perform contrastive learning experiments on a range of image and audio datasets with multiple downstream tasks (e.g. for digits superimposed on photographs, predicting the class of one vs. the other). We find that Viewmaker Networks, a recently proposed model for learning augmentations for contrastive learning, produce label-destroying augmentations that stochastically destroy features needed for different downstream tasks. These augmentations are interpretable (e.g. altering shapes, digits, or letters added to images) and surprisingly often result in better performance compared to expert-designed augmentations, despite not preserving label information. To support our empirical results, we theoretically analyze a simple contrastive learning setting with a linear model. In this setting, label-destroying augmentations are crucial for preventing one set of features from suppressing the learning of features useful for another downstream task. Our results highlight the need for analyzing the interaction between multiple downstream tasks when trying to explain the success of foundation models.
【9】 Towards Unified All-Neural Beamforming for Time and Frequency Domain Speech Separation
标题：时频域语音分离的统一全神经波束形成
链接：https://arxiv.org/abs/2212.08348
*与cs.SD语音【6】为同一篇
作者：Rongzhi Gu,Shi-Xiong Zhang,Yuexian Zou,Dong Yu摘要：近年来，频域全神经元波束形成方法在多通道语音分离中取得了显著的进展。同时，时域网络结构与波束形成的结合也得到了广泛的关注。本文提出了一种新的时域全神经元波束形成方法，尝试统一时域和频域多通道语音分离的全神经元波束形成流水线。拟议的模式包括两个模块：分离和波束形成。这两个模块都执行时间-频谱-空间建模，并使用联合损失函数进行端到端训练。本研究的新颖之处在于两方面。首先，提出了一种以目标说话人方向为条件的时域方向特征，该特征可以在时域结构内联合优化，以增强目标信号估计。其次，设计了时域全神经元波束形成网络对预分离结果进行细化。该模块的特点是参数化时变波束形成系数估计，而不显式地遵循最优滤波器的推导可能导致上界。在AISHELL-1语料库中模拟的混响重叠语音数据上对该方法进行了评估。实验结果表明，与频域最新技术、理想幅度模板和现有时域神经波束形成方法相比，该方法性能有显著提高。摘要：Recently, frequency domain all-neural beamforming methods have achieved remarkable progress for multichannel speech separation. In parallel, the integration of time domain network structure and beamforming also gains significant attention. This study proposes a novel all-neural beamforming method in time domain and makes an attempt to unify the all-neural beamforming pipelines for time domain and frequency domain multichannel speech separation. The proposed model consists of two modules: separation and beamforming. Both modules perform temporal-spectral-spatial modeling and are trained from end-to-end using a joint loss function. The novelty of this study lies in two folds. Firstly, a time domain directional feature conditioned on the direction of the target speaker is proposed, which can be jointly optimized within the time domain architecture to enhance target signal estimation. Secondly, an all-neural beamforming network in time domain is designed to refine the pre-separated results. This module features with parametric time-variant beamforming coefficient estimation, without explicitly following the derivation of optimal filters that may lead to an upper bound. The proposed method is evaluated on simulated reverberant overlapped speech data derived from the AISHELL-1 corpus. Experimental results demonstrate significant performance improvements over frequency domain state-of-the-arts, ideal magnitude masks and existing time domain neural beamforming methods.机器翻译，仅供参考
每日语音速递