今日论文合集：cs.SD语音11篇，eess.AS音频处理12篇。
【1】 TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separation
标题：TF-GridNet：集成全带和子带建模的语音分离方法
链接：https://arxiv.org/abs/2211.12433
作者：Zhong-Qiu Wang,Samuele Cornell,Shukjae Choi,Younglo Lee,Byeong-Yeol Kim,Shinji Watanabe机构：UniversitaPolitecnica delle Marche备注：In submission. A sound demo is available at this https URL摘要：我们提出了TF-GridNet用于语音分离。该模型是一种新颖的多路径深度神经网络（DNN），在时频域（T-F）集成了全带和子带建模。该算法将多个多径块堆叠起来，每个多径块由帧内全带模块、子带时域模块和跨帧自注意模块组成。它被训练来执行复谱映射，其中输入信号的实部和虚部（RI）分量被堆叠为预测目标RI分量的特征。我们首先在单耳无回声说话人分离上评估它。在不使用数据增强和动态混合的情况下，它在WSJ 0 - 2 mix（用于两个说话人分离的标准数据集）上获得了最先进的23.5 dB的尺度不变信号失真比（SI-SDR）改进。为了证明该方法对噪声和混响的鲁棒性，我们使用SMS-WSJ数据集对单耳混响说话人分离进行了评估，使用WHAMR！对噪声-混响说话人分离进行了评估，并在两个数据集上获得最新性能。然后，通过多麦克风复谱映射将TF-GridNet扩展到多麦克风环境，并将其集成到一个两个DNN系统中，在两个DNN之间加入一个波束形成器（在早期研究中称为MISO-BF-MISO），其中本文提出的波束形成器是一种基于第一个DNN输出计算的新型多帧Wiener滤波器。在SMS-WSJ和WHAMR！的多通道任务上获得了最先进的性能。除了说话人分离之外，我们还将所提出的算法应用于语音去混响和噪声-混响语音增强。在去混响数据集和最近的L3 DAS 22多通道语音增强挑战的数据集上获得了最先进的性能。摘要：We propose TF-GridNet for speech separation. The model is a novel multi-path deep neural network (DNN) integrating full- and sub-band modeling in the time-frequency (T-F) domain. It stacks several multi-path blocks, each consisting of an intra-frame full-band module, a sub-band temporal module, and a cross-frame self-attention module. It is trained to perform complex spectral mapping, where the real and imaginary (RI) components of input signals are stacked as features to predict target RI components. We first evaluate it on monaural anechoic speaker separation. Without using data augmentation and dynamic mixing, it obtains a state-of-the-art 23.5 dB improvement in scale-invariant signal-to-distortion ratio (SI-SDR) on WSJ0-2mix, a standard dataset for two-speaker separation. To show its robustness to noise and reverberation, we evaluate it on monaural reverberant speaker separation using the SMS-WSJ dataset and on noisy-reverberant speaker separation using WHAMR!, and obtain state-of-the-art performance on both datasets. We then extend TF-GridNet to multi-microphone conditions through multi-microphone complex spectral mapping, and integrate it into a two-DNN system with a beamformer in between (named as MISO-BF-MISO in earlier studies), where the beamformer proposed in this paper is a novel multi-frame Wiener filter computed based on the outputs of the first DNN. State-of-the-art performance is obtained on the multi-channel tasks of SMS-WSJ and WHAMR!. Besides speaker separation, we apply the proposed algorithms to speech dereverberation and noisy-reverberant speech enhancement. State-of-the-art performance is obtained on a dereverberation dataset and on the dataset of the recent L3DAS22 multi-channel speech enhancement challenge.
【2】 On Narrative Information and the Distillation of Stories
标题：论叙事信息与故事的升华
链接：https://arxiv.org/abs/2211.12423
作者：Dylan R. Ashley,Vincent Herrmann,Zachary Friggstad,Jürgen Schmidhuber机构：Dalle Molle Institute for Artificial Intelligence Research, Lugano, Switzerland, Università della Svizzera italiana, Lugano, Switzerland, Scuola universitaria professionale della Svizzera italiana, Lugano, Switzerland备注：presented in the Information-Theoretic Principles in Cognitive Systems Workshop at the 36th Conference on Neural Information Processing Systems; 4 pages in main text + 2 pages of references + 8 pages of appendices, 2 figures in main text + 3 in appendices, 1 table in main text, 2 algorithms in appendices; source code available at this https URL摘要：讲故事的行为是人的本质的一部分。本文引入了叙事信息的概念，我们将其定义为故事和构成故事的项目之间的信息空间重叠。使用对比学习方法，我们展示了如何利用现代人工神经网络来提取故事和提取叙事信息的表示。然后，我们演示了进化算法如何利用这一点来提取一组叙事模板，以及这些模板如何与我们介绍的一种新颖的曲线拟合算法一起重新排序音乐专辑，以自动诱导其中的故事。在这样做的过程中，我们给出了强有力的统计证据，证明这些叙事信息模板存在于现有的相册中。虽然我们在这里只对音乐专辑进行实验，但我们工作的前提延伸到任何形式的（很大程度上）独立媒体。摘要：The act of telling stories is a fundamental part of what it means to be human. This work introduces the concept of narrative information, which we define to be the overlap in information space between a story and the items that compose the story. Using contrastive learning methods, we show how modern artificial neural networks can be leveraged to distill stories and extract a representation of the narrative information. We then demonstrate how evolutionary algorithms can leverage this to extract a set of narrative templates and how these templates -- in tandem with a novel curve-fitting algorithm we introduce -- can reorder music albums to automatically induce stories in them. In the process of doing so, we give strong statistical evidence that these narrative information templates are present in existing albums. While we experiment only with music albums here, the premises of our work extend to any form of (largely) independent media.
【3】 AERO: Audio Super Resolution in the Spectral Domain
标题：Aero：频谱域中的音频超分辨率
链接：https://arxiv.org/abs/2211.12232
作者：Moshe Mandel,Or Tal,Yossi Adi机构：School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel摘要：我们提出了一种音频超分辨率模型AERO，它在频谱域处理语音和音乐信号。AERO基于具有U-Net类跳跃连接的编码器-解码器架构。我们使用时域和频域损失函数来优化模型。具体地说，我们考虑了一组重建损失和知觉损失，形式为对抗损失函数和特征鉴别损失函数。为了更好地处理相位信息，所提出的方法使用两个单独的通道在复值谱图上操作。与现有的主要考虑低频和高频级联以实现音频超分辨率的工作不同，所提出的方法直接预测整个频率范围。我们在语音和音乐的广泛采样速率范围内展示了高性能。考虑到对数光谱距离、ViSQOL和主观MUSHRA测试，AERO优于评价的基线。音频样本和代码可从www.example.com获得https://pages.cs.huji.ac.il/adiyoss-lab/aero摘要：We present AERO, a audio super-resolution model that processes speech and music signals in the spectral domain. AERO is based on an encoder-decoder architecture with U-Net like skip connections. We optimize the model using both time and frequency domain loss functions. Specifically, we consider a set of reconstruction losses together with perceptual ones in the form of adversarial and feature discriminator loss functions. To better handle phase information the proposed method operates over the complex-valued spectrogram using two separate channels. Unlike prior work which mainly considers low and high frequency concatenation for audio super-resolution, the proposed method directly predicts the full frequency range. We demonstrate high performance across a wide range of sample rates considering both speech and music. AERO outperforms the evaluated baselines considering Log-Spectral Distance, ViSQOL, and the subjective MUSHRA test. Audio samples and code are available at https://pages.cs.huji.ac.il/adiyoss-lab/aero
【4】 A Scope Sensitive and Result Attentive Model for Multi-Intent Spoken Language Understanding
标题：一种多意图口语理解的范围敏感和结果注意模型
链接：https://arxiv.org/abs/2211.12220
作者：Lizhi Cheng,Wenmian Yang,Weijia Jia机构：Shanghai Jiao Tong University, Shanghai, PR China, Nanyang Technological University, Singapore, Singapore, BNU-UIC Institute of Artificial Intelligence and Future Networks, Beijing Normal University (Zhuhai), Guangdong Key Lab摘要：多意图口语理解作为一种新颖而复杂的口语理解模式，正受到越来越多的关注。与传统SLU不同，此场景中的每个Intent都有其特定的范围。范围外的语义信息甚至会阻碍预测，极大地增加了意图检测的难度。更严重的是，用这些不准确的意图标签来引导时隙填充会遇到错误传播问题，从而导致整体性能不令人满意。为了解决这些问题，本文提出了一种基于Transformer的范围敏感结果注意网络（SSRAN），该网络包含范围识别器（SR）和结果注意网络（RAN）。范围识别器将范围信息分配给每个令牌，减少了范围外令牌的干扰。结果注意力网络有效地利用了空位填充结果和意图检测结果之间的双向交互，缓解了错误传播问题。在两个公共数据集上的实验表明，与现有的基线算法相比，该模型显著提高了SLU的性能（总体准确率分别为5.4%和2.1%）.摘要：Multi-Intent Spoken Language Understanding (SLU), a novel and more complex scenario of SLU, is attracting increasing attention. Unlike traditional SLU, each intent in this scenario has its specific scope. Semantic information outside the scope even hinders the prediction, which tremendously increases the difficulty of intent detection. More seriously, guiding slot filling with these inaccurate intent labels suffers error propagation problems, resulting in unsatisfied overall performance. To solve these challenges, in this paper, we propose a novel Scope-Sensitive Result Attention Network (SSRAN) based on Transformer, which contains a Scope Recognizer (SR) and a Result Attention Network (RAN). Scope Recognizer assignments scope information to each token, reducing the distraction of out-of-scope tokens. Result Attention Network effectively utilizes the bidirectional interaction between results of slot filling and intent detection, mitigating the error propagation problem. Experiments on two public datasets indicate that our model significantly improves SLU performance (5.4\% and 2.1\% on Overall accuracy) over the state-of-the-art baseline.
【5】 OR-Gate: A Noisy Label Filtering Method for Speaker Verification
标题：OR-Gate：一种用于说话人确认的噪声标签滤波方法
链接：https://arxiv.org/abs/2211.12080
作者：Zhihua Fang,Hanhan Ma,Lin Li,Liang He机构：School of Information Science and Engineering, Xinjiang University, Urumqi, China, School of Electronic Science and Engineering, Xiamen University, Xiamen, China, Department of Electronic Engineering, Tsinghua University, Beijing, China备注：Submitted to 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023)摘要：用于说话人确认的深度学习模型严重依赖于大规模数据和正确的标签。然而，噪声（错误）标签经常发生，这使系统的性能恶化。不幸的是，这方面的研究相对较少。本文提出了一种在训练阶段逐步滤除噪声标签的方法。我们将不同训练时期的网络预测与地面真实标签进行比较，并通过类似于逻辑电路中的或门机制来选择可靠（被认为正确）的标签。因此，我们提出的方法被命名为或门。实验证明，或门可以有效地滤除噪声标签，具有良好的性能。摘要：The deep learning models used for speaker verification are heavily dependent on large-scale data and correct labels. However, noisy (wrong) labels often occur, which deteriorates the system's performance. Unfortunately, there are relatively few studies in this area. In this paper, we propose a method to gradually filter noisy labels out at the training stage. We compare the network predictions at different training epochs with ground-truth labels, and select reliable (considered correct) labels by using the OR gate mechanism like that in logic circuits. Therefore, our proposed method is named as OR-Gate. We experimentally demonstrated that the OR-Gate can effectively filter noisy labels out and has excellent performance.
【6】 TaylorBeamixer: Learning Taylor-Inspired All-Neural Multi-Channel Speech Enhancement from Beam-Space Dictionary Perspective
标题：Taylor Beamixer：从波束空间词典的角度学习泰勒启发的全神经多通道语音增强
链接：https://arxiv.org/abs/2211.12024
作者：Andong Li,Guochen Yu,Wenzhe Liu,Xiaodong Li,Chengshi Zheng机构：⋆ Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy, † University of Chinese Academy of Sciences, Beijing, China, ‡ Tencent Ethereal Audio Lab, Tencent Corporation, Shenzhen, China备注：In submission to ICASSP 2023, 5 pages摘要：尽管现有的基于帧的全神经元波束形成器在语音增强领域有很好的性能，但其基本机制仍不清楚。本文从波束空间字典的角度重新审视波束形成行为，并将其公式化为不同波束空间分量的学习和混合。在此基础上，提出了一种全神经元波束形成器TaylorBM来模拟泰勒级数展开操作，其中零阶项作为空间滤波器进行波束混频，而高阶项则用于消除残余噪声以进行后处理。整个系统被设计成以端到端的方式工作。在空间化LibriSpeech语料库上的实验结果表明，该方法在评价指标方面优于现有的高级基线.摘要：Despite the promising performance of existing frame-wise all-neural beamformers in the speech enhancement field, it remains unclear what the underlying mechanism exists. In this paper, we revisit the beamforming behavior from the beam-space dictionary perspective and formulate it into the learning and mixing of different beam-space components. Based on that, we propose an all-neural beamformer called TaylorBM to simulate Taylor's series expansion operation in which the 0th-order term serves as a spatial filter to conduct the beam mixing, and several high-order terms are tasked with residual noise cancellation for post-processing. The whole system is devised to work in an end-to-end manner. Experiments are conducted on the spatialized LibriSpeech corpus and results show that the proposed approach outperforms existing advanced baselines in terms of evaluation metrics.
【7】 Disentangled Feature Learning for Real-Time Neural Speech Coding
标题：用于实时神经语音编码的解缠特征学习
链接：https://arxiv.org/abs/2211.11960
作者：Xue Jiang,Xiulian Peng,Yuan Zhang,Yan Lu机构：Communication University of China, Beijing, China, Microsoft Research Asia, Beijing, China备注：Submitted to ICASSP2023摘要：最近，端到端的神经音频/语音编码已经显示出其优于传统的基于信号分析的音频编解码器的巨大潜力。这主要是通过遵循VQ-VAE范例来实现的，其中盲特征被学习、矢量量化和编码。在这篇文章中，我们提出学习解纠缠的特征来代替盲的端到端学习，用于实时神经语音编码。具体地，通过解纠缠学习更多的类似于全局的说话人身份和局部内容特征来表示语音。这种紧凑的特征分解不仅通过利用不同特征之间的比特分配实现了更好的编码效率，而且提供了在嵌入空间中进行音频编辑的灵活性，诸如实时通信中的语音转换。主观和客观的结果都证明了它的编码效率，我们发现学习的解纠缠特征在任意到任意语音转换上表现出与现代自监督语音表示学习模型相当的性能，具有更少的参数和低延迟，显示了我们的神经编码框架的潜力。摘要：Recently end-to-end neural audio/speech coding has shown its great potential to outperform traditional signal analysis based audio codecs. This is mostly achieved by following the VQ-VAE paradigm where blind features are learned, vector-quantized and coded. In this paper, instead of blind end-to-end learning, we propose to learn disentangled features for real-time neural speech coding. Specifically, more global-like speaker identity and local content features are learned with disentanglement to represent speech. Such a compact feature decomposition not only achieves better coding efficiency by exploiting bit allocation among different features but also provides the flexibility to do audio editing in embedding space, such as voice conversion in real-time communications. Both subjective and objective results demonstrate its coding efficiency and we find that the learned disentangled features show comparable performance on any-to-any voice conversion with modern self-supervised speech representation learning models with far less parameters and low latency, showing the potential of our neural coding framework.
【8】 COVID-Net Assistant: A Deep Learning-Driven Virtual Assistant for COVID-19 Symptom Prediction and Recommendation
标题：CoVID-Net Assistant：一个深度学习驱动的新冠肺炎症状预测和推荐虚拟助手
链接：https://arxiv.org/abs/2211.11944
作者：Pengyuan Shi,Yuetong Wang,Saad Abbasi,Alexander Wong机构：David R. Cheriton School of Computer Science, Department of Systems Design Engineering, University of Waterloo摘要：随着COVID-19大流行继续给全球医疗系统带来沉重负担，人们越来越有兴趣寻找廉价的症状预筛查和推荐方法，以帮助有效利用PCR检测等可用医疗资源。在这项研究中，我们介绍了COVID-Net Assistant的设计，这是一个高效的虚拟助手，旨在通过深度卷积神经网络分析用户的咳嗽记录，为COVID-19提供症状预测和建议。我们在Covid 19-Cough基准测试数据集上探索通过机器驱动设计探索生成的各种高度定制的轻量级卷积神经网络架构（我们称之为COVID-Net Assistant神经网络）。Covid 19-咳嗽数据集包括来自COVID-19阳性队列的682个咳嗽记录和来自COVID-19阴性队列的642个咳嗽记录。在682个咳嗽记录中，382个记录通过PCR试验验证。实验结果表明，COVID-Net Assistant神经网络具有较好的预测性能，其AUC得分超过0.93，最佳得分超过0.95，同时具有快速、高效的推理能力。COVID-Net Assistant模型通过COVID-Net开放计划以开源方式提供，虽然不是一个可用于生产的解决方案，但我们希望它们的可用性能成为临床科学家、机器学习研究人员以及公民科学家开发创新解决方案的良好资源。摘要：As the COVID-19 pandemic continues to put a significant burden on healthcare systems worldwide, there has been growing interest in finding inexpensive symptom pre-screening and recommendation methods to assist in efficiently using available medical resources such as PCR tests. In this study, we introduce the design of COVID-Net Assistant, an efficient virtual assistant designed to provide symptom prediction and recommendations for COVID-19 by analyzing users' cough recordings through deep convolutional neural networks. We explore a variety of highly customized, lightweight convolutional neural network architectures generated via machine-driven design exploration (which we refer to as COVID-Net Assistant neural networks) on the Covid19-Cough benchmark dataset. The Covid19-Cough dataset comprises 682 cough recordings from a COVID-19 positive cohort and 642 from a COVID-19 negative cohort. Among the 682 cough recordings labeled positive, 382 recordings were verified by PCR test. Our experimental results show promising, with the COVID-Net Assistant neural networks demonstrating robust predictive performance, achieving AUC scores of over 0.93, with the best score over 0.95 while being fast and efficient in inference. The COVID-Net Assistant models are made available in an open source manner through the COVID-Net open initiative and, while not a production-ready solution, we hope their availability acts as a good resource for clinical scientists, machine learning researchers, as well as citizen scientists to develop innovative solutions.
【9】 Latent Iterative Refinement for Modular Source Separation
标题：模源分离的潜在迭代求精方法
链接：https://arxiv.org/abs/2211.11917
作者：Dimitrios Bralios,Efthymios Tzinis,Gordon Wichern,Paris Smaragdis,Jonathan Le Roux机构：University of Illinois at Urbana-Champaign, Urbana, IL, USA, Mitsubishi Electric Research Laboratories (MERL), Cambridge, MA, USA摘要：传统的源分离方法通过最小化整个训练集上的经验风险，使用所有可用数据一次性端到端地训练深度神经网络模型。在推理方面，在训练模型之后，用户获取静态计算图，并在一些指定的观察混合信号上运行完整模型，以获得估计的源信号。此外，这些模型中的许多模型由顺序应用的几个基本处理块组成。我们认为，通过将模型的训练和推理过程重新表述为潜在信号表示的迭代映射，可以显著提高训练和推理阶段的资源效率。首先，我们可以在其输出上不止一次地应用相同的处理块，以细化输入信号并因此提高参数效率。在训练过程中，我们可以遵循一个逐块的过程，这使得能够减少对存储器的需求。因此，与端到端训练相比，可以使用明显更少的计算来训练非常复杂的网络结构。在推理过程中，我们可以使用门控模块动态调整输入信号需要多少个处理块和特定块的迭代。摘要：Traditional source separation approaches train deep neural network models end-to-end with all the data available at once by minimizing the empirical risk on the whole training set. On the inference side, after training the model, the user fetches a static computation graph and runs the full model on some specified observed mixture signal to get the estimated source signals. Additionally, many of those models consist of several basic processing blocks which are applied sequentially. We argue that we can significantly increase resource efficiency during both training and inference stages by reformulating a model's training and inference procedures as iterative mappings of latent signal representations. First, we can apply the same processing block more than once on its output to refine the input signal and consequently improve parameter efficiency. During training, we can follow a block-wise procedure which enables a reduction on memory requirements. Thus, one can train a very complicated network structure using significantly less computation compared to end-to-end training. During inference, we can dynamically adjust how many processing blocks and iterations of a specific block an input signal needs using a gating module.
【10】 Ontology-aware Learning and Evaluation for Audio Tagging
标题：基于本体的音频标注学习与评价
链接：https://arxiv.org/abs/2211.12195
作者：Haohe Liu,Qiuqiang Kong,Xubo Liu,Xinhao Mei,Wenwu Wang,Mark D. Plumbley机构：Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK, Speech, Audio, and Music Intelligence (SAMI) Group, ByteDance, China备注：Submitted to ICASSP 2023. The code is open-sourced at this https URL摘要：针对传统的平均精度（mean average precision，mAP）评价指标将不同类型的声音作为独立的类，没有考虑它们之间的关系，提出了一种新的音频标注评价指标.此外，由于声音标注中的模糊性，不能保证训练和评估集中的标注是准确和详尽的，这对使用mAP的鲁棒评估提出了挑战。提出的本体感知的平均精度（OmAP）度量通过在评估过程中利用AudioSet本体信息来解决mAP的不足。具体地说，我们在模型预测中基于本体图到目标类的距离来重新加权假阳性事件。OmAP度量还通过对本体图中不同粗粒度级别的评估，提供了对模型性能的更多洞察。我们进行了人类评估，并证明了OmAP比mAP更符合人类感知。为了进一步验证利用本体信息的重要性，我们还提出了一种新的损失函数（OBCE），该函数基于本体距离对二进制交叉熵（BCE）损失进行重新加权。实验结果表明，OBCE算法在AudioSet标注任务中能够同时提高mAP和omAP两个指标。摘要：This study defines a new evaluation metric for audio tagging tasks to overcome the limitation of the conventional mean average precision (mAP) metric, which treats different kinds of sound as independent classes without considering their relations. Also, due to the ambiguities in sound labeling, the labels in the training and evaluation set are not guaranteed to be accurate and exhaustive, which poses challenges for robust evaluation with mAP. The proposed metric, ontology-aware mean average precision (OmAP) addresses the weaknesses of mAP by utilizing the AudioSet ontology information during the evaluation. Specifically, we reweight the false positive events in the model prediction based on the ontology graph distance to the target classes. The OmAP measure also provides more insights into model performance by evaluations with different coarse-grained levels in the ontology graph. We conduct human evaluations and demonstrate that OmAP is more consistent with human perception than mAP. To further verify the importance of utilizing the ontology information, we also propose a novel loss function (OBCE) that reweights binary cross entropy (BCE) loss based on the ontology distance. Our experiment shows that OBCE can improve both mAP and OmAP metrics on the AudioSet tagging task.
【11】 PromptTTS: Controllable Text-to-Speech with Text Descriptions
标题：PromptTTS：具有文本描述的可控文本到语音转换
链接：https://arxiv.org/abs/2211.12171
作者：Zhifang Guo,Yichong Leng,Yihan Wu,Sheng Zhao,Xu Tan机构：Univerisity of Science and Technology of China, Renmin University of China, Microsoft Azure Speech, Microsoft Research Asia备注：Submitted to ICASSP 2023摘要：使用文本描述作为提示来指导文本或图像的生成（例如，GPT-3或DALLE-2）引起了广泛的关注。除了文本和图像生成，在这项工作中，我们探讨了利用文本描述来指导语音合成的可能性。因此，我们开发了一个文本到语音（TTS）系统（称为PromptTTS），它以带有风格和内容描述的提示作为输入来合成相应的语音。具体地说，PromptTTS由用于从提示中提取相应表示的样式编码器和内容编码器以及用于根据所提取的样式和内容表示合成语音的语音解码器组成。与要求用户具有声学知识以理解诸如韵律和音高的风格因素的可控TTS中的先前工作相比，PromptTTS更用户友好，因为文本描述是表达语音风格的更自然的方式（例如，"一位女士慢慢地对她的朋友说悄悄话"）。鉴于没有包含提示的TTS数据集，为了对PromptTTS的任务进行基准测试，我们构建并发布了一个包含提示的数据集，该数据集包含样式和内容信息以及相应的语音。实验表明，PromptTTS能够生成风格控制精确、语音质量高的语音。音频样本和我们的数据集是公开的。摘要：Using a text description as prompt to guide the generation of text or images (e.g., GPT-3 or DALLE-2) has drawn wide attention recently. Beyond text and image generation, in this work, we explore the possibility of utilizing text descriptions to guide speech synthesis. Thus, we develop a text-to-speech (TTS) system (dubbed as PromptTTS) that takes a prompt with both style and content descriptions as input to synthesize the corresponding speech. Specifically, PromptTTS consists of a style encoder and a content encoder to extract the corresponding representations from the prompt, and a speech decoder to synthesize speech according to the extracted style and content representations. Compared with previous works in controllable TTS that require users to have acoustic knowledge to understand style factors such as prosody and pitch, PromptTTS is more user-friendly since text descriptions are a more natural way to express speech style (e.g., ''A lady whispers to her friend slowly''). Given that there is no TTS dataset with prompts, to benchmark the task of PromptTTS, we construct and release a dataset containing prompts with style and content information and the corresponding speech. Experiments show that PromptTTS can generate speech with precise style control and high speech quality. Audio samples and our dataset are publicly available.
【1】 Ontology-aware Learning and Evaluation for Audio Tagging
标题：基于本体的音频标注学习与评价
链接：https://arxiv.org/abs/2211.12195
* 与cs.SD语音【10】为同一篇
作者：Haohe Liu,Qiuqiang Kong,Xubo Liu,Xinhao Mei,Wenwu Wang,Mark D. Plumbley机构：Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK, Speech, Audio, and Music Intelligence (SAMI) Group, ByteDance, China备注：Submitted to ICASSP 2023. The code is open-sourced at this https URL摘要：针对传统的平均精度（mean average precision，mAP）评价指标将不同类型的声音作为独立的类，没有考虑它们之间的关系，提出了一种新的音频标注评价指标.此外，由于声音标注中的模糊性，不能保证训练和评估集中的标注是准确和详尽的，这对使用mAP的鲁棒评估提出了挑战。提出的本体感知的平均精度（OmAP）度量通过在评估过程中利用AudioSet本体信息来解决mAP的不足。具体地说，我们在模型预测中基于本体图到目标类的距离来重新加权假阳性事件。OmAP度量还通过对本体图中不同粗粒度级别的评估，提供了对模型性能的更多洞察。我们进行了人类评估，并证明了OmAP比mAP更符合人类感知。为了进一步验证利用本体信息的重要性，我们还提出了一种新的损失函数（OBCE），该函数基于本体距离对二进制交叉熵（BCE）损失进行重新加权。实验结果表明，OBCE算法在AudioSet标注任务中能够同时提高mAP和omAP两个指标。摘要：This study defines a new evaluation metric for audio tagging tasks to overcome the limitation of the conventional mean average precision (mAP) metric, which treats different kinds of sound as independent classes without considering their relations. Also, due to the ambiguities in sound labeling, the labels in the training and evaluation set are not guaranteed to be accurate and exhaustive, which poses challenges for robust evaluation with mAP. The proposed metric, ontology-aware mean average precision (OmAP) addresses the weaknesses of mAP by utilizing the AudioSet ontology information during the evaluation. Specifically, we reweight the false positive events in the model prediction based on the ontology graph distance to the target classes. The OmAP measure also provides more insights into model performance by evaluations with different coarse-grained levels in the ontology graph. We conduct human evaluations and demonstrate that OmAP is more consistent with human perception than mAP. To further verify the importance of utilizing the ontology information, we also propose a novel loss function (OBCE) that reweights binary cross entropy (BCE) loss based on the ontology distance. Our experiment shows that OBCE can improve both mAP and OmAP metrics on the AudioSet tagging task.
【2】 PromptTTS: Controllable Text-to-Speech with Text Descriptions
标题：PromptTTS：具有文本描述的可控文本到语音转换
链接：https://arxiv.org/abs/2211.12171
* 与cs.SD语音【11】为同一篇
作者：Zhifang Guo,Yichong Leng,Yihan Wu,Sheng Zhao,Xu Tan机构：Univerisity of Science and Technology of China, Renmin University of China, Microsoft Azure Speech, Microsoft Research Asia备注：Submitted to ICASSP 2023摘要：使用文本描述作为提示来指导文本或图像的生成（例如，GPT-3或DALLE-2）引起了广泛的关注。除了文本和图像生成，在这项工作中，我们探讨了利用文本描述来指导语音合成的可能性。因此，我们开发了一个文本到语音（TTS）系统（称为PromptTTS），它以带有风格和内容描述的提示作为输入来合成相应的语音。具体地说，PromptTTS由用于从提示中提取相应表示的样式编码器和内容编码器以及用于根据所提取的样式和内容表示合成语音的语音解码器组成。与要求用户具有声学知识以理解诸如韵律和音高的风格因素的可控TTS中的先前工作相比，PromptTTS更用户友好，因为文本描述是表达语音风格的更自然的方式（例如，“一位女士慢慢地对她的朋友说悄悄话”）。鉴于没有包含提示的TTS数据集，为了对PromptTTS的任务进行基准测试，我们构建并发布了一个包含提示的数据集，该数据集包含样式和内容信息以及相应的语音。实验表明，PromptTTS能够生成风格控制精确、语音质量高的语音。音频样本和我们的数据集是公开的。摘要：Using a text description as prompt to guide the generation of text or images (e.g., GPT-3 or DALLE-2) has drawn wide attention recently. Beyond text and image generation, in this work, we explore the possibility of utilizing text descriptions to guide speech synthesis. Thus, we develop a text-to-speech (TTS) system (dubbed as PromptTTS) that takes a prompt with both style and content descriptions as input to synthesize the corresponding speech. Specifically, PromptTTS consists of a style encoder and a content encoder to extract the corresponding representations from the prompt, and a speech decoder to synthesize speech according to the extracted style and content representations. Compared with previous works in controllable TTS that require users to have acoustic knowledge to understand style factors such as prosody and pitch, PromptTTS is more user-friendly since text descriptions are a more natural way to express speech style (e.g., ''A lady whispers to her friend slowly''). Given that there is no TTS dataset with prompts, to benchmark the task of PromptTTS, we construct and release a dataset containing prompts with style and content information and the corresponding speech. Experiments show that PromptTTS can generate speech with precise style control and high speech quality. Audio samples and our dataset are publicly available.
【3】 Dynamic Acoustic Compensation and Adaptive Focal Training for Personalized Speech Enhancement
标题：个性化语音增强的动态声学补偿和自适应聚焦训练
链接：https://arxiv.org/abs/2211.12097
作者：Xiaofeng Ge,Jiangyu Han,Haixin Guan,Yanhua Long机构：Key Innovation Group of Digital Humanities Resource and Research, Shanghai Normal University, Shanghai, China, Unisound AI Technology Co., Ltd., Beijing, China摘要：近年来，越来越多的个性化语音增强系统（PSE）被提出，并取得了很好的性能。然而，两个关键问题仍然限制了模型的性能和泛化能力：1）测试含噪语音与目标说话人登记语音之间的声学环境失配; 2）硬样本挖掘和学习。本文提出了一种动态声学补偿（DAC）的方法，通过截取噪声语音中的噪声或环境声学片段，并将其与干净的注册语音进行混合，来减轻环境失配。为了更好地利用训练数据中的硬样本，提出了一种自适应焦点训练（AFT）策略，在训练过程中为硬样本和非硬样本分配自适应损失权值.进一步引入时频多损失训练，改进和推广了我们以前的工作sDPCCN用于PSE。在DNS4 Challenge数据集上检验了所提方法的有效性。结果表明，DAC在多个评价指标上都有较大的改善，AFT显著降低了硬采样率，MOS评分也有明显改善。摘要：Recently, more and more personalized speech enhancement systems (PSE) with excellent performance have been proposed. However, two critical issues still limit the performance and generalization ability of the model: 1) Acoustic environment mismatch between the test noisy speech and target speaker enrollment speech; 2) Hard sample mining and learning. In this paper, dynamic acoustic compensation (DAC) is proposed to alleviate the environment mismatch, by intercepting the noise or environmental acoustic segments from noisy speech and mixing it with the clean enrollment speech. To well exploit the hard samples in training data, we propose an adaptive focal training (AFT) strategy by assigning adaptive loss weights to hard and non-hard samples during training. A time-frequency multi-loss training is further introduced to improve and generalize our previous work sDPCCN for PSE. The effectiveness of proposed methods are examined on the DNS4 Challenge dataset. Results show that, the DAC brings large improvements in terms of multiple evaluation metrics, and AFT reduces the hard sample rate significantly and produces obvious MOS score improvement.
【4】 TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separation
标题：TF-GridNet：集成全带和子带建模的语音分离方法
链接：https://arxiv.org/abs/2211.12433
* 与cs.SD语音【1】为同一篇
作者：Zhong-Qiu Wang,Samuele Cornell,Shukjae Choi,Younglo Lee,Byeong-Yeol Kim,Shinji Watanabe机构：UniversitaPolitecnica delle Marche备注：In submission. A sound demo is available at this https URL摘要：我们提出了TF-GridNet用于语音分离。该模型是一种新颖的多路径深度神经网络（DNN），在时频域（T-F）集成了全带和子带建模。该算法将多个多径块堆叠起来，每个多径块由帧内全带模块、子带时域模块和跨帧自注意模块组成。它被训练来执行复谱映射，其中输入信号的实部和虚部（RI）分量被堆叠为预测目标RI分量的特征。我们首先在单耳无回声说话人分离上评估它。在不使用数据增强和动态混合的情况下，它在WSJ 0 - 2 mix（用于两个说话人分离的标准数据集）上获得了最先进的23.5 dB的尺度不变信号失真比（SI-SDR）改进。为了证明该方法对噪声和混响的鲁棒性，我们使用SMS-WSJ数据集对单耳混响说话人分离进行了评估，使用WHAMR！对噪声-混响说话人分离进行了评估，并在两个数据集上获得最新性能。然后，通过多麦克风复谱映射将TF-GridNet扩展到多麦克风环境，并将其集成到一个两个DNN系统中，在两个DNN之间加入一个波束形成器（在早期研究中称为MISO-BF-MISO），其中本文提出的波束形成器是一种基于第一个DNN输出计算的新型多帧Wiener滤波器。在SMS-WSJ和WHAMR！的多通道任务上获得了最先进的性能。除了说话人分离之外，我们还将所提出的算法应用于语音去混响和噪声-混响语音增强。在去混响数据集和最近的L3 DAS 22多通道语音增强挑战的数据集上获得了最先进的性能。摘要：We propose TF-GridNet for speech separation. The model is a novel multi-path deep neural network (DNN) integrating full- and sub-band modeling in the time-frequency (T-F) domain. It stacks several multi-path blocks, each consisting of an intra-frame full-band module, a sub-band temporal module, and a cross-frame self-attention module. It is trained to perform complex spectral mapping, where the real and imaginary (RI) components of input signals are stacked as features to predict target RI components. We first evaluate it on monaural anechoic speaker separation. Without using data augmentation and dynamic mixing, it obtains a state-of-the-art 23.5 dB improvement in scale-invariant signal-to-distortion ratio (SI-SDR) on WSJ0-2mix, a standard dataset for two-speaker separation. To show its robustness to noise and reverberation, we evaluate it on monaural reverberant speaker separation using the SMS-WSJ dataset and on noisy-reverberant speaker separation using WHAMR!, and obtain state-of-the-art performance on both datasets. We then extend TF-GridNet to multi-microphone conditions through multi-microphone complex spectral mapping, and integrate it into a two-DNN system with a beamformer in between (named as MISO-BF-MISO in earlier studies), where the beamformer proposed in this paper is a novel multi-frame Wiener filter computed based on the outputs of the first DNN. State-of-the-art performance is obtained on the multi-channel tasks of SMS-WSJ and WHAMR!. Besides speaker separation, we apply the proposed algorithms to speech dereverberation and noisy-reverberant speech enhancement. State-of-the-art performance is obtained on a dereverberation dataset and on the dataset of the recent L3DAS22 multi-channel speech enhancement challenge.
【5】 On Narrative Information and the Distillation of Stories
标题：论叙事信息与故事的升华
链接：https://arxiv.org/abs/2211.12423
* 与cs.SD语音【2】为同一篇
作者：Dylan R. Ashley,Vincent Herrmann,Zachary Friggstad,Jürgen Schmidhuber机构：Dalle Molle Institute for Artificial Intelligence Research, Lugano, Switzerland, Università della Svizzera italiana, Lugano, Switzerland, Scuola universitaria professionale della Svizzera italiana, Lugano, Switzerland备注：presented in the Information-Theoretic Principles in Cognitive Systems Workshop at the 36th Conference on Neural Information Processing Systems; 4 pages in main text + 2 pages of references + 8 pages of appendices, 2 figures in main text + 3 in appendices, 1 table in main text, 2 algorithms in appendices; source code available at this https URL摘要：讲故事的行为是人的本质的一部分。本文引入了叙事信息的概念，我们将其定义为故事和构成故事的项目之间的信息空间重叠。使用对比学习方法，我们展示了如何利用现代人工神经网络来提取故事和提取叙事信息的表示。然后，我们演示了进化算法如何利用这一点来提取一组叙事模板，以及这些模板如何与我们介绍的一种新颖的曲线拟合算法一起重新排序音乐专辑，以自动诱导其中的故事。在这样做的过程中，我们给出了强有力的统计证据，证明这些叙事信息模板存在于现有的相册中。虽然我们在这里只对音乐专辑进行实验，但我们工作的前提延伸到任何形式的（很大程度上）独立媒体。摘要：The act of telling stories is a fundamental part of what it means to be human. This work introduces the concept of narrative information, which we define to be the overlap in information space between a story and the items that compose the story. Using contrastive learning methods, we show how modern artificial neural networks can be leveraged to distill stories and extract a representation of the narrative information. We then demonstrate how evolutionary algorithms can leverage this to extract a set of narrative templates and how these templates -- in tandem with a novel curve-fitting algorithm we introduce -- can reorder music albums to automatically induce stories in them. In the process of doing so, we give strong statistical evidence that these narrative information templates are present in existing albums. While we experiment only with music albums here, the premises of our work extend to any form of (largely) independent media.
【6】 AERO: Audio Super Resolution in the Spectral Domain
标题：Aero：频谱域中的音频超分辨率
链接：https://arxiv.org/abs/2211.12232
* 与cs.SD语音【3】为同一篇
作者：Moshe Mandel,Or Tal,Yossi Adi机构：School of Computer Science and Engineering, The Hebrew University of Jerusalem, Israel摘要：我们提出了一种音频超分辨率模型AERO，它在频谱域处理语音和音乐信号。AERO基于具有U-Net类跳跃连接的编码器-解码器架构。我们使用时域和频域损失函数来优化模型。具体地说，我们考虑了一组重建损失和知觉损失，形式为对抗损失函数和特征鉴别损失函数。为了更好地处理相位信息，所提出的方法使用两个单独的通道在复值谱图上操作。与现有的主要考虑低频和高频级联以实现音频超分辨率的工作不同，所提出的方法直接预测整个频率范围。我们在语音和音乐的广泛采样速率范围内展示了高性能。考虑到对数光谱距离、ViSQOL和主观MUSHRA测试，AERO优于评价的基线。音频样本和代码可从www.example.com获得https://pages.cs.huji.ac.il/adiyoss-lab/aero摘要：We present AERO, a audio super-resolution model that processes speech and music signals in the spectral domain. AERO is based on an encoder-decoder architecture with U-Net like skip connections. We optimize the model using both time and frequency domain loss functions. Specifically, we consider a set of reconstruction losses together with perceptual ones in the form of adversarial and feature discriminator loss functions. To better handle phase information the proposed method operates over the complex-valued spectrogram using two separate channels. Unlike prior work which mainly considers low and high frequency concatenation for audio super-resolution, the proposed method directly predicts the full frequency range. We demonstrate high performance across a wide range of sample rates considering both speech and music. AERO outperforms the evaluated baselines considering Log-Spectral Distance, ViSQOL, and the subjective MUSHRA test. Audio samples and code are available at https://pages.cs.huji.ac.il/adiyoss-lab/aero
【7】 A Scope Sensitive and Result Attentive Model for Multi-Intent Spoken Language Understanding
标题：一种多意图口语理解的范围敏感和结果注意模型
链接：https://arxiv.org/abs/2211.12220
* 与cs.SD语音【4】为同一篇
作者：Lizhi Cheng,Wenmian Yang,Weijia Jia机构：Shanghai Jiao Tong University, Shanghai, PR China, Nanyang Technological University, Singapore, Singapore, BNU-UIC Institute of Artificial Intelligence and Future Networks, Beijing Normal University (Zhuhai), Guangdong Key Lab摘要：多意图口语理解作为一种新颖而复杂的口语理解模式，正受到越来越多的关注。与传统SLU不同，此场景中的每个Intent都有其特定的范围。范围外的语义信息甚至会阻碍预测，极大地增加了意图检测的难度。更严重的是，用这些不准确的意图标签来引导时隙填充会遇到错误传播问题，从而导致整体性能不令人满意。为了解决这些问题，本文提出了一种基于Transformer的范围敏感结果注意网络（SSRAN），该网络包含范围识别器（SR）和结果注意网络（RAN）。范围识别器将范围信息分配给每个令牌，减少了范围外令牌的干扰。结果注意力网络有效地利用了空位填充结果和意图检测结果之间的双向交互，缓解了错误传播问题。在两个公共数据集上的实验表明，与现有的基线算法相比，该模型显著提高了SLU的性能（总体准确率分别为5.4%和2.1%）.摘要：Multi-Intent Spoken Language Understanding (SLU), a novel and more complex scenario of SLU, is attracting increasing attention. Unlike traditional SLU, each intent in this scenario has its specific scope. Semantic information outside the scope even hinders the prediction, which tremendously increases the difficulty of intent detection. More seriously, guiding slot filling with these inaccurate intent labels suffers error propagation problems, resulting in unsatisfied overall performance. To solve these challenges, in this paper, we propose a novel Scope-Sensitive Result Attention Network (SSRAN) based on Transformer, which contains a Scope Recognizer (SR) and a Result Attention Network (RAN). Scope Recognizer assignments scope information to each token, reducing the distraction of out-of-scope tokens. Result Attention Network effectively utilizes the bidirectional interaction between results of slot filling and intent detection, mitigating the error propagation problem. Experiments on two public datasets indicate that our model significantly improves SLU performance (5.4\% and 2.1\% on Overall accuracy) over the state-of-the-art baseline.
【8】 OR-Gate: A Noisy Label Filtering Method for Speaker Verification
标题：OR-Gate：一种用于说话人确认的噪声标签滤波方法
链接：https://arxiv.org/abs/2211.12080
* 与cs.SD语音【5】为同一篇
作者：Zhihua Fang,Hanhan Ma,Lin Li,Liang He机构：School of Information Science and Engineering, Xinjiang University, Urumqi, China, School of Electronic Science and Engineering, Xiamen University, Xiamen, China, Department of Electronic Engineering, Tsinghua University, Beijing, China备注：Submitted to 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2023)摘要：用于说话人确认的深度学习模型严重依赖于大规模数据和正确的标签。然而，噪声（错误）标签经常发生，这使系统的性能恶化。不幸的是，这方面的研究相对较少。本文提出了一种在训练阶段逐步滤除噪声标签的方法。我们将不同训练时期的网络预测与地面真实标签进行比较，并通过类似于逻辑电路中的或门机制来选择可靠（被认为正确）的标签。因此，我们提出的方法被命名为或门。实验证明，或门可以有效地滤除噪声标签，具有良好的性能。摘要：The deep learning models used for speaker verification are heavily dependent on large-scale data and correct labels. However, noisy (wrong) labels often occur, which deteriorates the system's performance. Unfortunately, there are relatively few studies in this area. In this paper, we propose a method to gradually filter noisy labels out at the training stage. We compare the network predictions at different training epochs with ground-truth labels, and select reliable (considered correct) labels by using the OR gate mechanism like that in logic circuits. Therefore, our proposed method is named as OR-Gate. We experimentally demonstrated that the OR-Gate can effectively filter noisy labels out and has excellent performance.
【9】 TaylorBeamixer: Learning Taylor-Inspired All-Neural Multi-Channel Speech Enhancement from Beam-Space Dictionary Perspective
标题：Taylor Beamixer：从波束空间词典的角度学习泰勒启发的全神经多通道语音增强
链接：https://arxiv.org/abs/2211.12024
* 与cs.SD语音【6】为同一篇
作者：Andong Li,Guochen Yu,Wenzhe Liu,Xiaodong Li,Chengshi Zheng机构：⋆ Key Laboratory of Noise and Vibration Research, Institute of Acoustics, Chinese Academy, † University of Chinese Academy of Sciences, Beijing, China, ‡ Tencent Ethereal Audio Lab, Tencent Corporation, Shenzhen, China备注：In submission to ICASSP 2023, 5 pages摘要：尽管现有的基于帧的全神经元波束形成器在语音增强领域有很好的性能，但其基本机制仍不清楚。本文从波束空间字典的角度重新审视波束形成行为，并将其公式化为不同波束空间分量的学习和混合。在此基础上，提出了一种全神经元波束形成器TaylorBM来模拟泰勒级数展开操作，其中零阶项作为空间滤波器进行波束混频，而高阶项则用于消除残余噪声以进行后处理。整个系统被设计成以端到端的方式工作。在空间化LibriSpeech语料库上的实验结果表明，该方法在评价指标方面优于现有的高级基线.摘要：Despite the promising performance of existing frame-wise all-neural beamformers in the speech enhancement field, it remains unclear what the underlying mechanism exists. In this paper, we revisit the beamforming behavior from the beam-space dictionary perspective and formulate it into the learning and mixing of different beam-space components. Based on that, we propose an all-neural beamformer called TaylorBM to simulate Taylor's series expansion operation in which the 0th-order term serves as a spatial filter to conduct the beam mixing, and several high-order terms are tasked with residual noise cancellation for post-processing. The whole system is devised to work in an end-to-end manner. Experiments are conducted on the spatialized LibriSpeech corpus and results show that the proposed approach outperforms existing advanced baselines in terms of evaluation metrics.
【10】 Disentangled Feature Learning for Real-Time Neural Speech Coding
标题：用于实时神经语音编码的解缠特征学习
链接：https://arxiv.org/abs/2211.11960
* 与cs.SD语音【7】为同一篇
作者：Xue Jiang,Xiulian Peng,Yuan Zhang,Yan Lu机构：Communication University of China, Beijing, China, Microsoft Research Asia, Beijing, China备注：Submitted to ICASSP2023摘要：最近，端到端的神经音频/语音编码已经显示出其优于传统的基于信号分析的音频编解码器的巨大潜力。这主要是通过遵循VQ-VAE范例来实现的，其中盲特征被学习、矢量量化和编码。在这篇文章中，我们提出学习解纠缠的特征来代替盲的端到端学习，用于实时神经语音编码。具体地，通过解纠缠学习更多的类似于全局的说话人身份和局部内容特征来表示语音。这种紧凑的特征分解不仅通过利用不同特征之间的比特分配实现了更好的编码效率，而且提供了在嵌入空间中进行音频编辑的灵活性，诸如实时通信中的语音转换。主观和客观的结果都证明了它的编码效率，我们发现学习的解纠缠特征在任意到任意语音转换上表现出与现代自监督语音表示学习模型相当的性能，具有更少的参数和低延迟，显示了我们的神经编码框架的潜力。摘要：Recently end-to-end neural audio/speech coding has shown its great potential to outperform traditional signal analysis based audio codecs. This is mostly achieved by following the VQ-VAE paradigm where blind features are learned, vector-quantized and coded. In this paper, instead of blind end-to-end learning, we propose to learn disentangled features for real-time neural speech coding. Specifically, more global-like speaker identity and local content features are learned with disentanglement to represent speech. Such a compact feature decomposition not only achieves better coding efficiency by exploiting bit allocation among different features but also provides the flexibility to do audio editing in embedding space, such as voice conversion in real-time communications. Both subjective and objective results demonstrate its coding efficiency and we find that the learned disentangled features show comparable performance on any-to-any voice conversion with modern self-supervised speech representation learning models with far less parameters and low latency, showing the potential of our neural coding framework.
【11】 COVID-Net Assistant: A Deep Learning-Driven Virtual Assistant for COVID-19 Symptom Prediction and Recommendation
标题：CoVID-Net Assistant：一个深度学习驱动的新冠肺炎症状预测和推荐虚拟助手
链接：https://arxiv.org/abs/2211.11944
* 与cs.SD语音【8】为同一篇
作者：Pengyuan Shi,Yuetong Wang,Saad Abbasi,Alexander Wong机构：David R. Cheriton School of Computer Science, Department of Systems Design Engineering, University of Waterloo摘要：随着COVID-19大流行继续给全球医疗系统带来沉重负担，人们越来越有兴趣寻找廉价的症状预筛查和推荐方法，以帮助有效利用PCR检测等可用医疗资源。在这项研究中，我们介绍了COVID-Net Assistant的设计，这是一个高效的虚拟助手，旨在通过深度卷积神经网络分析用户的咳嗽记录，为COVID-19提供症状预测和建议。我们在Covid19-Cough基准测试数据集上探索通过机器驱动设计探索生成的各种高度定制的轻量级卷积神经网络架构（我们称之为COVID-Net Assistant神经网络）。Covid19-咳嗽数据集包括来自COVID-19阳性队列的682个咳嗽记录和来自COVID-19阴性队列的642个咳嗽记录。在682个咳嗽记录中，382个记录通过PCR试验验证。实验结果表明，COVID-Net Assistant神经网络具有较好的预测性能，其AUC得分超过0.93，最佳得分超过0.95，同时具有快速、高效的推理能力。COVID-Net Assistant模型通过COVID-Net开放计划以开源方式提供，虽然不是一个可用于生产的解决方案，但我们希望它们的可用性能成为临床科学家、机器学习研究人员以及公民科学家开发创新解决方案的良好资源。摘要：As the COVID-19 pandemic continues to put a significant burden on healthcare systems worldwide, there has been growing interest in finding inexpensive symptom pre-screening and recommendation methods to assist in efficiently using available medical resources such as PCR tests. In this study, we introduce the design of COVID-Net Assistant, an efficient virtual assistant designed to provide symptom prediction and recommendations for COVID-19 by analyzing users' cough recordings through deep convolutional neural networks. We explore a variety of highly customized, lightweight convolutional neural network architectures generated via machine-driven design exploration (which we refer to as COVID-Net Assistant neural networks) on the Covid19-Cough benchmark dataset. The Covid19-Cough dataset comprises 682 cough recordings from a COVID-19 positive cohort and 642 from a COVID-19 negative cohort. Among the 682 cough recordings labeled positive, 382 recordings were verified by PCR test. Our experimental results show promising, with the COVID-Net Assistant neural networks demonstrating robust predictive performance, achieving AUC scores of over 0.93, with the best score over 0.95 while being fast and efficient in inference. The COVID-Net Assistant models are made available in an open source manner through the COVID-Net open initiative and, while not a production-ready solution, we hope their availability acts as a good resource for clinical scientists, machine learning researchers, as well as citizen scientists to develop innovative solutions.
【12】 Latent Iterative Refinement for Modular Source Separation
标题：模源分离的潜在迭代求精方法
链接：https://arxiv.org/abs/2211.11917
* 与cs.SD语音【9】为同一篇
作者：Dimitrios Bralios,Efthymios Tzinis,Gordon Wichern,Paris Smaragdis,Jonathan Le Roux机构：University of Illinois at Urbana-Champaign, Urbana, IL, USA, Mitsubishi Electric Research Laboratories (MERL), Cambridge, MA, USA摘要：传统的源分离方法通过最小化整个训练集上的经验风险，使用所有可用数据一次性端到端地训练深度神经网络模型。在推理方面，在训练模型之后，用户获取静态计算图，并在一些指定的观察混合信号上运行完整模型，以获得估计的源信号。此外，这些模型中的许多模型由顺序应用的几个基本处理块组成。我们认为，通过将模型的训练和推理过程重新表述为潜在信号表示的迭代映射，可以显著提高训练和推理阶段的资源效率。首先，我们可以在其输出上不止一次地应用相同的处理块，以细化输入信号并因此提高参数效率。在训练过程中，我们可以遵循一个逐块的过程，这使得能够减少对存储器的需求。因此，与端到端训练相比，可以使用明显更少的计算来训练非常复杂的网络结构。在推理过程中，我们可以使用门控模块动态调整输入信号需要多少个处理块和特定块的迭代。摘要：Traditional source separation approaches train deep neural network models end-to-end with all the data available at once by minimizing the empirical risk on the whole training set. On the inference side, after training the model, the user fetches a static computation graph and runs the full model on some specified observed mixture signal to get the estimated source signals. Additionally, many of those models consist of several basic processing blocks which are applied sequentially. We argue that we can significantly increase resource efficiency during both training and inference stages by reformulating a model's training and inference procedures as iterative mappings of latent signal representations. First, we can apply the same processing block more than once on its output to refine the input signal and consequently improve parameter efficiency. During training, we can follow a block-wise procedure which enables a reduction on memory requirements. Thus, one can train a very complicated network structure using significantly less computation compared to end-to-end training. During inference, we can dynamically adjust how many processing blocks and iterations of a specific block an input signal needs using a gating module.
机器翻译，仅供参考
每日语音速递