今日论文合集：cs.SD语音7篇，eess.AS音频处理7篇。
cs.SD语音
【1】 AudioLDM 2: Learning Holistic Audio Generation with Self-supervised  Pretraining标题：AudioLDM 2：通过自我监督的预训练学习整体音频生成链接：https://arxiv.org/abs/2308.05734
作者：Haohe Liu,Qiao Tian,Yi Yuan,Xubo Liu,Xinhao Mei,Qiuqiang Kong,Yuping Wang,Wenwu Wang,Yuxuan Wang,Mark D. Plumbley备注：AudioLDM 2 project page is this https URL摘要：虽然音频生成在不同类型的音频（例如语音、音乐和声音效果）中具有共性，但是为每种类型设计模型需要仔细考虑可能与其他类型的目标和偏差显著不同的特定目标和偏差。为了使我们更接近音频生成的统一视角，本文提出了一个框架，利用相同的学习方法的语音，音乐和音效生成。我们的框架介绍了音频的一般表示，称为音频语言（LOA）。任何音频都可以基于AudioMAE转换为LOA，AudioMAE是一种自我监督的预训练表示学习模型。在生成过程中，我们通过使用GPT-2模型将任何模态转化为LOA，并且我们使用LOA条件下的潜在扩散模型进行自我监督音频生成学习。所提出的框架自然带来的优势，如在上下文学习能力和可重用的自我监督预训练AudioMAE和潜在扩散模型。文本到音频，文本到音乐，文本到语音的主要基准的实验证明了新的国家的最先进的或竞争力的性能以前的方法。我们的演示和代码可在https://audioldm.github.io/audioldm2获得。摘要：Although audio generation shares commonalities across different types of audio, such as speech, music, and sound effects, designing models for each type requires careful consideration of specific objectives and biases that can significantly differ from those of other types. To bring us closer to a unified perspective of audio generation, this paper proposes a framework that utilizes the same learning method for speech, music, and sound effect generation. Our framework introduces a general representation of audio, called language of audio (LOA). Any audio can be translated into LOA based on AudioMAE, a self-supervised pre-trained representation learning model. In the generation process, we translate any modalities into LOA by using a GPT-2 model, and we perform self-supervised audio generation learning with a latent diffusion model conditioned on LOA. The proposed framework naturally brings advantages such as in-context learning abilities and reusable self-supervised pretrained AudioMAE and latent diffusion models. Experiments on the major benchmarks of text-to-audio, text-to-music, and text-to-speech demonstrate new state-of-the-art or competitive performance to previous approaches. Our demo and code are available at https://audioldm.github.io/audioldm2.
【2】 EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech  Resynthesis标题：Expresso：离散表达语音再合成的基准与分析链接：https://arxiv.org/abs/2308.05725
作者：Tu Anh Nguyen,Wei-Ning Hsu,Antony D'Avirro,Bowen Shi,Itai Gat,Maryam Fazel-Zarani,Tal Remez,Jade Copet,Gabriel Synnaeve,Michael Hassid,Felix Kreuk,Yossi Adi,Emmanuel Dupoux摘要：最近的工作表明，有可能重新合成高质量的语音，而不是基于文本，而是基于以自我监督的方式学习的低比特率离散单元，因此可以捕获难以转录的语音表达方面（韵律，语音风格，非言语发声）。这些方法的采用仍然受到大多数语音合成数据集被读取的事实的限制，严重限制了自发性和表达性。在这里，我们介绍Expresso，一个高质量的表达语音数据集，用于无文本语音合成，包括阅读语音和即兴对话，以26种自发的表达风格呈现。我们说明了这个数据集的挑战和潜力与表现力的再合成基准，其中的任务是在低比特率的单位编码的输入和再合成它在目标语音，同时保留内容和风格。我们评估重新合成质量与自动度量不同的自监督离散编码器，并探讨质量，比特率和不变性扬声器和风格之间的权衡。所有的数据集、评估指标和基线模型都是开源的摘要：Recent work has shown that it is possible to resynthesize high-quality speech based, not on text, but on low bitrate discrete units that have been learned in a self-supervised fashion and can therefore capture expressive aspects of speech that are hard to transcribe (prosody, voice styles, non-verbal vocalization). The adoption of these methods is still limited by the fact that most speech synthesis datasets are read, severely limiting spontaneity and expressivity. Here, we introduce Expresso, a high-quality expressive speech dataset for textless speech synthesis that includes both read speech and improvised dialogues rendered in 26 spontaneous expressive styles. We illustrate the challenges and potentials of this dataset with an expressive resynthesis benchmark where the task is to encode the input in low-bitrate units and resynthesize it in a target voice while preserving content and style. We evaluate resynthesis quality with automatic metrics for different self-supervised discrete encoders, and explore tradeoffs between quality, bitrate and invariance to speaker and style. All the dataset, evaluation metrics and baseline models are open source
【3】 A Novel Self-training Approach for Low-resource Speech Recognition标题：一种新的低资源语音识别自训练方法链接：https://arxiv.org/abs/2308.05269
作者：Satwinder Singh,Feng Hou,Ruili Wang备注：Accepted to Interspeech 2023摘要：在本文中，我们提出了一种自训练的方法，自动语音识别（ASR）的低资源设置。虽然自我训练的方法已经被广泛开发和评估的高资源语言，如英语，他们的应用程序低资源语言，如旁遮普语一直有限，尽管语言正在全球数百万人使用。注释数据的稀缺阻碍了准确ASR系统的开发，特别是对于低资源语言（例如，旁遮普语和毛利语）。为了解决这个问题，我们提出了一种有效的自我训练方法，为未标记的低资源语音生成高度准确的伪标签。我们的实验分析表明，我们的方法显着提高单词错误率，实现了14.94%的相对改善相比，在四个真实的语音数据集的基线模型。此外，我们提出的方法报告的共同的声音旁遮普语数据集的最佳结果。摘要：In this paper, we propose a self-training approach for automatic speech recognition (ASR) for low-resource settings. While self-training approaches have been extensively developed and evaluated for high-resource languages such as English, their applications to low-resource languages like Punjabi have been limited, despite the language being spoken by millions globally. The scarcity of annotated data has hindered the development of accurate ASR systems, especially for low-resource languages (e.g., Punjabi and M\=aori languages). To address this issue, we propose an effective self-training approach that generates highly accurate pseudo-labels for unlabeled low-resource speech. Our experimental analysis demonstrates that our approach significantly improves word error rate, achieving a relative improvement of 14.94% compared to a baseline model across four real speech datasets. Further, our proposed approach reports the best results on the Common Voice Punjabi dataset.
【4】 Conformer-based Target-Speaker Automatic Speech Recognition for  Single-Channel Audio标题：基于一致性的单声道音频目标说话人自动语音识别链接：https://arxiv.org/abs/2308.05218
作者：Yang Zhang,Krishna C. Puvvada,Vitaly Lavrukhin,Boris Ginsburg摘要：我们提出了CONF-TSASR，一个非自回归的端到端的时间-频率域架构的单通道目标说话人自动语音识别（TS-ASR）。该模型由基于TitaNet的说话人嵌入模块、基于Conformer的掩蔽模块以及ASR模块组成。这些模块被联合优化以转录目标说话者，同时忽略来自其他说话者的语音。对于训练，我们使用连接时间分类（CTC）损失，并引入一个规模不变的频谱图重建损失，以鼓励模型更好地分离目标说话人的频谱图从混合物。我们获得了WSJ 0 - 2 mix-extr（4.2%）的最先进的目标说话人的字错误率（TS-WER）。此外，我们首次在WSJ 0 - 3 mix-extr（12.4%），LibriSpeech 2 Mix（4.2%）和LibriSpeech 3 Mix（7.6%）数据集上报告了TS-WER，为TS-ASR建立了新的基准。该模型将通过NVIDIA NeMo工具包开源。摘要：We propose CONF-TSASR, a non-autoregressive end-to-end time-frequency domain architecture for single-channel target-speaker automatic speech recognition (TS-ASR). The model consists of a TitaNet based speaker embedding module, a Conformer based masking as well as ASR modules. These modules are jointly optimized to transcribe a target-speaker, while ignoring speech from other speakers. For training we use Connectionist Temporal Classification (CTC) loss and introduce a scale-invariant spectrogram reconstruction loss to encourage the model better separate the target-speaker's spectrogram from mixture. We obtain state-of-the-art target-speaker word error rate (TS-WER) on WSJ0-2mix-extr (4.2%). Further, we report for the first time TS-WER on WSJ0-3mix-extr (12.4%), LibriSpeech2Mix (4.2%) and LibriSpeech3Mix (7.6%) datasets, establishing new benchmarks for TS-ASR. The proposed model will be open-sourced through NVIDIA NeMo toolkit.
【5】 Sound propagation in realistic interactive 3D scenes with parameterized  sources using deep neural operators标题：基于深度神经算子的参数化源真实感交互3D场景中的声音传播链接：https://arxiv.org/abs/2308.05141
作者：Nikolas Borrel-Jensen,Somdatta Goswami,Allan P. Engsig-Karup,George Em Karniadakis,Cheol-Ho Jeong备注：25 pages, 10 figures, 4 tables摘要：我们解决的挑战，声音传播模拟在3$D虚拟房间移动源，在虚拟/增强现实，游戏音频和空间计算的应用程序。波动方程的解可以描述诸如衍射和干涉之类的波动现象。然而，使用具有数百个源和接收器位置的常规数值离散化方法来模拟它们是棘手的，使得利用移动源来刺激声场是不切实际的。为了克服这一限制，我们提出使用深度算子网络来近似线性波动方程算子。这使得能够在具有移动源的真实3D声学场景中快速预测声音传播，实现毫秒级计算。通过学习一个紧凑的代理模型，我们避免了离线计算和存储所有相关的源/收听者对的脉冲响应。我们的实验，包括各种复杂的场景几何形状，显示出良好的协议与参考解决方案，均方根误差范围从0.02帕到0.10帕。值得注意的是，我们的方法意味着范式转变，因为没有以前的机器学习方法已经实现了对现实领域内完整波场的精确预测。我们预计，我们的研究结果将推动深度神经算子方法的进一步探索，推动虚拟环境中沉浸式用户体验的研究。摘要：We address the challenge of sound propagation simulations in $3$D virtual rooms with moving sources, which have applications in virtual/augmented reality, game audio, and spatial computing. Solutions to the wave equation can describe wave phenomena such as diffraction and interference. However, simulating them using conventional numerical discretization methods with hundreds of source and receiver positions is intractable, making stimulating a sound field with moving sources impractical. To overcome this limitation, we propose using deep operator networks to approximate linear wave-equation operators. This enables the rapid prediction of sound propagation in realistic 3D acoustic scenes with moving sources, achieving millisecond-scale computations. By learning a compact surrogate model, we avoid the offline calculation and storage of impulse responses for all relevant source/listener pairs. Our experiments, including various complex scene geometries, show good agreement with reference solutions, with root mean squared errors ranging from 0.02 Pa to 0.10 Pa. Notably, our method signifies a paradigm shift as no prior machine learning approach has achieved precise predictions of complete wave fields within realistic domains. We anticipate that our findings will drive further exploration of deep neural operator methods, advancing research in immersive user experiences within virtual environments.
【6】 Stabilizing Training with Soft Dynamic Time Warping: A Case Study for  Pitch Class Estimation with Weakly Aligned Targets标题：软动态时间规整稳定训练：弱对齐目标基音类估计的实例研究链接：https://arxiv.org/abs/2308.05429
作者：Johannes Zeitler,Simon Deniffel,Michael Krause,Meinard Müller备注：Accepted for ISMIR 2023, Milano, Italy摘要：软动态时间规整（SDTW）是一种可微损失函数，允许从弱对齐数据训练神经网络。通常，SDTW用于迭代地计算和细化软比对，其补偿训练数据与其弱注释目标之间的时间偏差。一个主要问题是，在早期训练阶段中估计的软对准和参考对准之间的失配导致不正确的参数更新，使得整个训练过程不稳定。在本文中，我们调查这样的稳定性问题，通过考虑的任务，从音乐记录的音高类估计作为一个说明性的案例研究。特别是，我们介绍和讨论了三个概念上不同的策略（超参数调度，对角先验和序列展开策略）的稳定中间软对齐结果的目标。最后，我们报告的实验，证明了策略的有效性，并讨论了效率和实施问题。摘要：Soft dynamic time warping (SDTW) is a differentiable loss function that allows for training neural networks from weakly aligned data. Typically, SDTW is used to iteratively compute and refine soft alignments that compensate for temporal deviations between the training data and its weakly annotated targets. One major problem is that a mismatch between the estimated soft alignments and the reference alignments in the early training stage leads to incorrect parameter updates, making the overall training procedure unstable. In this paper, we investigate such stability issues by considering the task of pitch class estimation from music recordings as an illustrative case study. In particular, we introduce and discuss three conceptually different strategies (a hyperparameter scheduling, a diagonal prior, and a sequence unfolding strategy) with the objective of stabilizing intermediate soft alignment results. Finally, we report on experiments that demonstrate the effectiveness of the strategies and discuss efficiency and implementation issues.
【7】 Analyzing the Effect of Data Impurity on the Detection Performances of  Mental Disorders标题：数据杂质对精神障碍检测性能的影响分析链接：https://arxiv.org/abs/2308.05133
作者：Rohan Kumar Gupta,Rohit Sinha摘要：用于自动识别精神障碍的主要方法传统上涉及使用二进制分类器。这些分类器使用从面试设置获得的行为数据来训练。在这个训练过程中，来自具有所考虑的特定障碍的个体的数据被归类为积极类，而来自所有其他参与者的数据构成消极类。在实践中，人们普遍认识到，某些精神障碍共有相似的症状，使得收集的行为数据包含与多种障碍相关联的各种属性。因此，与目标精神障碍相关的属性也可能存在于负面类中。这种数据杂质可能导致分类器针对感兴趣的精神障碍的次优训练。在这项研究中，我们调查了这一假设的背景下，重性抑郁症（MDD）和创伤后应激障碍检测（PTSD）。结果表明，在去除这样的数据杂质，MDD和PTSD检测性能显着提高。摘要：The primary method for identifying mental disorders automatically has traditionally involved using binary classifiers. These classifiers are trained using behavioral data obtained from an interview setup. In this training process, data from individuals with the specific disorder under consideration are categorized as the positive class, while data from all other participants constitute the negative class. In practice, it is widely recognized that certain mental disorders share similar symptoms, causing the collected behavioral data to encompass a variety of attributes associated with multiple disorders. Consequently, attributes linked to the targeted mental disorder might also be present within the negative class. This data impurity may lead to sub-optimal training of the classifier for a mental disorder of interest. In this study, we investigate this hypothesis in the context of major depressive disorder (MDD) and post-traumatic stress disorder detection (PTSD). The results show that upon removal of such data impurity, MDD and PTSD detection performances are significantly improved.
eess.AS音频处理
【1】 Stabilizing Training with Soft Dynamic Time Warping: A Case Study for  Pitch Class Estimation with Weakly Aligned Targets标题：软动态时间规整稳定训练：弱对齐目标基音类估计的实例研究链接：https://arxiv.org/abs/2308.05429
作者：Johannes Zeitler,Simon Deniffel,Michael Krause,Meinard Müller备注：Accepted for ISMIR 2023, Milano, Italy摘要：软动态时间规整（SDTW）是一种可微损失函数，允许从弱对齐数据训练神经网络。通常，SDTW用于迭代地计算和细化软比对，其补偿训练数据与其弱注释目标之间的时间偏差。一个主要问题是，在早期训练阶段中估计的软对准和参考对准之间的失配导致不正确的参数更新，使得整个训练过程不稳定。在本文中，我们调查这样的稳定性问题，通过考虑的任务，从音乐记录的音高类估计作为一个说明性的案例研究。特别是，我们介绍和讨论了三个概念上不同的策略（超参数调度，对角先验和序列展开策略）的稳定中间软对齐结果的目标。最后，我们报告的实验，证明了策略的有效性，并讨论了效率和实施问题。摘要：Soft dynamic time warping (SDTW) is a differentiable loss function that allows for training neural networks from weakly aligned data. Typically, SDTW is used to iteratively compute and refine soft alignments that compensate for temporal deviations between the training data and its weakly annotated targets. One major problem is that a mismatch between the estimated soft alignments and the reference alignments in the early training stage leads to incorrect parameter updates, making the overall training procedure unstable. In this paper, we investigate such stability issues by considering the task of pitch class estimation from music recordings as an illustrative case study. In particular, we introduce and discuss three conceptually different strategies (a hyperparameter scheduling, a diagonal prior, and a sequence unfolding strategy) with the objective of stabilizing intermediate soft alignment results. Finally, we report on experiments that demonstrate the effectiveness of the strategies and discuss efficiency and implementation issues.
【2】 AudioLDM 2: Learning Holistic Audio Generation with Self-supervised  Pretraining标题：AudioLDM 2：通过自我监督的预训练学习整体音频生成链接：https://arxiv.org/abs/2308.05734
作者：Haohe Liu,Qiao Tian,Yi Yuan,Xubo Liu,Xinhao Mei,Qiuqiang Kong,Yuping Wang,Wenwu Wang,Yuxuan Wang,Mark D. Plumbley备注：AudioLDM 2 project page is this https URL摘要：虽然音频生成在不同类型的音频（例如语音、音乐和声音效果）中具有共性，但是为每种类型设计模型需要仔细考虑可能与其他类型的目标和偏差显著不同的特定目标和偏差。为了使我们更接近音频生成的统一视角，本文提出了一个框架，利用相同的学习方法的语音，音乐和音效生成。我们的框架介绍了音频的一般表示，称为音频语言（LOA）。任何音频都可以基于AudioMAE转换为LOA，AudioMAE是一种自我监督的预训练表示学习模型。在生成过程中，我们通过使用GPT-2模型将任何模态转化为LOA，并且我们使用LOA条件下的潜在扩散模型进行自我监督音频生成学习。所提出的框架自然带来的优势，如在上下文学习能力和可重用的自我监督预训练AudioMAE和潜在扩散模型。文本到音频，文本到音乐，文本到语音的主要基准的实验证明了新的国家的最先进的或竞争力的性能以前的方法。我们的演示和代码可在https://audioldm.github.io/audioldm2获得。摘要：Although audio generation shares commonalities across different types of audio, such as speech, music, and sound effects, designing models for each type requires careful consideration of specific objectives and biases that can significantly differ from those of other types. To bring us closer to a unified perspective of audio generation, this paper proposes a framework that utilizes the same learning method for speech, music, and sound effect generation. Our framework introduces a general representation of audio, called language of audio (LOA). Any audio can be translated into LOA based on AudioMAE, a self-supervised pre-trained representation learning model. In the generation process, we translate any modalities into LOA by using a GPT-2 model, and we perform self-supervised audio generation learning with a latent diffusion model conditioned on LOA. The proposed framework naturally brings advantages such as in-context learning abilities and reusable self-supervised pretrained AudioMAE and latent diffusion models. Experiments on the major benchmarks of text-to-audio, text-to-music, and text-to-speech demonstrate new state-of-the-art or competitive performance to previous approaches. Our demo and code are available at https://audioldm.github.io/audioldm2.
【3】 EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech  Resynthesis标题：Expresso：离散表达语音再合成的基准与分析链接：https://arxiv.org/abs/2308.05725
作者：Tu Anh Nguyen,Wei-Ning Hsu,Antony D'Avirro,Bowen Shi,Itai Gat,Maryam Fazel-Zarani,Tal Remez,Jade Copet,Gabriel Synnaeve,Michael Hassid,Felix Kreuk,Yossi Adi,Emmanuel Dupoux摘要：最近的工作表明，有可能重新合成高质量的语音，而不是基于文本，而是基于以自我监督的方式学习的低比特率离散单元，因此可以捕获难以转录的语音表达方面（韵律，语音风格，非言语发声）。这些方法的采用仍然受到大多数语音合成数据集被读取的事实的限制，严重限制了自发性和表达性。在这里，我们介绍Expresso，一个高质量的表达语音数据集，用于无文本语音合成，包括阅读语音和即兴对话，以26种自发的表达风格呈现。我们说明了这个数据集的挑战和潜力与表现力的再合成基准，其中的任务是在低比特率的单位编码的输入和再合成它在目标语音，同时保留内容和风格。我们评估重新合成质量与自动度量不同的自监督离散编码器，并探讨质量，比特率和不变性扬声器和风格之间的权衡。所有的数据集、评估指标和基线模型都是开源的摘要：Recent work has shown that it is possible to resynthesize high-quality speech based, not on text, but on low bitrate discrete units that have been learned in a self-supervised fashion and can therefore capture expressive aspects of speech that are hard to transcribe (prosody, voice styles, non-verbal vocalization). The adoption of these methods is still limited by the fact that most speech synthesis datasets are read, severely limiting spontaneity and expressivity. Here, we introduce Expresso, a high-quality expressive speech dataset for textless speech synthesis that includes both read speech and improvised dialogues rendered in 26 spontaneous expressive styles. We illustrate the challenges and potentials of this dataset with an expressive resynthesis benchmark where the task is to encode the input in low-bitrate units and resynthesize it in a target voice while preserving content and style. We evaluate resynthesis quality with automatic metrics for different self-supervised discrete encoders, and explore tradeoffs between quality, bitrate and invariance to speaker and style. All the dataset, evaluation metrics and baseline models are open source
【4】 A Novel Self-training Approach for Low-resource Speech Recognition标题：一种新的低资源语音识别自训练方法链接：https://arxiv.org/abs/2308.05269
作者：Satwinder Singh,Feng Hou,Ruili Wang备注：Accepted to Interspeech 2023摘要：在本文中，我们提出了一种自训练的方法，自动语音识别（ASR）的低资源设置。虽然自我训练的方法已经被广泛开发和评估的高资源语言，如英语，他们的应用程序低资源语言，如旁遮普语一直有限，尽管语言正在全球数百万人使用。注释数据的稀缺阻碍了准确ASR系统的开发，特别是对于低资源语言（例如，旁遮普语和毛利语）。为了解决这个问题，我们提出了一种有效的自我训练方法，为未标记的低资源语音生成高度准确的伪标签。我们的实验分析表明，我们的方法显着提高单词错误率，实现了14.94%的相对改善相比，在四个真实的语音数据集的基线模型。此外，我们提出的方法报告的共同的声音旁遮普语数据集的最佳结果。摘要：In this paper, we propose a self-training approach for automatic speech recognition (ASR) for low-resource settings. While self-training approaches have been extensively developed and evaluated for high-resource languages such as English, their applications to low-resource languages like Punjabi have been limited, despite the language being spoken by millions globally. The scarcity of annotated data has hindered the development of accurate ASR systems, especially for low-resource languages (e.g., Punjabi and M\=aori languages). To address this issue, we propose an effective self-training approach that generates highly accurate pseudo-labels for unlabeled low-resource speech. Our experimental analysis demonstrates that our approach significantly improves word error rate, achieving a relative improvement of 14.94% compared to a baseline model across four real speech datasets. Further, our proposed approach reports the best results on the Common Voice Punjabi dataset.
【5】 Conformer-based Target-Speaker Automatic Speech Recognition for  Single-Channel Audio标题：基于一致性的单声道音频目标说话人自动语音识别链接：https://arxiv.org/abs/2308.05218
作者：Yang Zhang,Krishna C. Puvvada,Vitaly Lavrukhin,Boris Ginsburg摘要：我们提出了CONF-TSASR，一个非自回归的端到端的时间-频率域架构的单通道目标说话人自动语音识别（TS-ASR）。该模型由基于TitaNet的说话人嵌入模块、基于Conformer的掩蔽模块以及ASR模块组成。这些模块被联合优化以转录目标说话者，同时忽略来自其他说话者的语音。对于训练，我们使用连接时间分类（CTC）损失，并引入一个规模不变的频谱图重建损失，以鼓励模型更好地分离目标说话人的频谱图从混合物。我们获得了WSJ 0 - 2 mix-extr（4.2%）的最先进的目标说话人的字错误率（TS-WER）。此外，我们首次在WSJ 0 - 3 mix-extr（12.4%），LibriSpeech 2 Mix（4.2%）和LibriSpeech 3 Mix（7.6%）数据集上报告了TS-WER，为TS-ASR建立了新的基准。该模型将通过NVIDIA NeMo工具包开源。摘要：We propose CONF-TSASR, a non-autoregressive end-to-end time-frequency domain architecture for single-channel target-speaker automatic speech recognition (TS-ASR). The model consists of a TitaNet based speaker embedding module, a Conformer based masking as well as ASR modules. These modules are jointly optimized to transcribe a target-speaker, while ignoring speech from other speakers. For training we use Connectionist Temporal Classification (CTC) loss and introduce a scale-invariant spectrogram reconstruction loss to encourage the model better separate the target-speaker's spectrogram from mixture. We obtain state-of-the-art target-speaker word error rate (TS-WER) on WSJ0-2mix-extr (4.2%). Further, we report for the first time TS-WER on WSJ0-3mix-extr (12.4%), LibriSpeech2Mix (4.2%) and LibriSpeech3Mix (7.6%) datasets, establishing new benchmarks for TS-ASR. The proposed model will be open-sourced through NVIDIA NeMo toolkit.
【6】 Sound propagation in realistic interactive 3D scenes with parameterized  sources using deep neural operators标题：基于深度神经算子的参数化源真实感交互3D场景中的声音传播链接：https://arxiv.org/abs/2308.05141
作者：Nikolas Borrel-Jensen,Somdatta Goswami,Allan P. Engsig-Karup,George Em Karniadakis,Cheol-Ho Jeong备注：25 pages, 10 figures, 4 tables摘要：我们解决的挑战，声音传播模拟在3$D虚拟房间移动源，在虚拟/增强现实，游戏音频和空间计算的应用程序。波动方程的解可以描述诸如衍射和干涉之类的波动现象。然而，使用具有数百个源和接收器位置的常规数值离散化方法来模拟它们是棘手的，使得利用移动源来刺激声场是不切实际的。为了克服这一限制，我们提出使用深度算子网络来近似线性波动方程算子。这使得能够在具有移动源的真实3D声学场景中快速预测声音传播，实现毫秒级计算。通过学习一个紧凑的代理模型，我们避免了离线计算和存储所有相关的源/收听者对的脉冲响应。我们的实验，包括各种复杂的场景几何形状，显示出良好的协议与参考解决方案，均方根误差范围从0.02帕到0.10帕。值得注意的是，我们的方法意味着范式转变，因为没有以前的机器学习方法已经实现了对现实领域内完整波场的精确预测。我们预计，我们的研究结果将推动深度神经算子方法的进一步探索，推动虚拟环境中沉浸式用户体验的研究。摘要：We address the challenge of sound propagation simulations in $3$D virtual rooms with moving sources, which have applications in virtual/augmented reality, game audio, and spatial computing. Solutions to the wave equation can describe wave phenomena such as diffraction and interference. However, simulating them using conventional numerical discretization methods with hundreds of source and receiver positions is intractable, making stimulating a sound field with moving sources impractical. To overcome this limitation, we propose using deep operator networks to approximate linear wave-equation operators. This enables the rapid prediction of sound propagation in realistic 3D acoustic scenes with moving sources, achieving millisecond-scale computations. By learning a compact surrogate model, we avoid the offline calculation and storage of impulse responses for all relevant source/listener pairs. Our experiments, including various complex scene geometries, show good agreement with reference solutions, with root mean squared errors ranging from 0.02 Pa to 0.10 Pa. Notably, our method signifies a paradigm shift as no prior machine learning approach has achieved precise predictions of complete wave fields within realistic domains. We anticipate that our findings will drive further exploration of deep neural operator methods, advancing research in immersive user experiences within virtual environments.
【7】 Analyzing the Effect of Data Impurity on the Detection Performances of  Mental Disorders标题：数据杂质对精神障碍检测性能的影响分析链接：https://arxiv.org/abs/2308.05133
作者：Rohan Kumar Gupta,Rohit Sinha摘要：用于自动识别精神障碍的主要方法传统上涉及使用二进制分类器。这些分类器使用从面试设置获得的行为数据来训练。在这个训练过程中，来自具有所考虑的特定障碍的个体的数据被归类为积极类，而来自所有其他参与者的数据构成消极类。在实践中，人们普遍认识到，某些精神障碍共有相似的症状，使得收集的行为数据包含与多种障碍相关联的各种属性。因此，与目标精神障碍相关的属性也可能存在于负面类中。这种数据杂质可能导致分类器针对感兴趣的精神障碍的次优训练。在这项研究中，我们调查了这一假设的背景下，重性抑郁症（MDD）和创伤后应激障碍检测（PTSD）。结果表明，在去除这样的数据杂质，MDD和PTSD检测性能显着提高。摘要：The primary method for identifying mental disorders automatically has traditionally involved using binary classifiers. These classifiers are trained using behavioral data obtained from an interview setup. In this training process, data from individuals with the specific disorder under consideration are categorized as the positive class, while data from all other participants constitute the negative class. In practice, it is widely recognized that certain mental disorders share similar symptoms, causing the collected behavioral data to encompass a variety of attributes associated with multiple disorders. Consequently, attributes linked to the targeted mental disorder might also be present within the negative class. This data impurity may lead to sub-optimal training of the classifier for a mental disorder of interest. In this study, we investigate this hypothesis in the context of major depressive disorder (MDD) and post-traumatic stress disorder detection (PTSD). The results show that upon removal of such data impurity, MDD and PTSD detection performances are significantly improved.

机器翻译，仅供参考