今日论文合集：cs.SD语音8篇，eess.AS音频处理11篇。
cs.SD语音
【1】 Towards Matching Phones and Speech Representations标题：走向匹配音素和语音表示链接：https://arxiv.org/abs/2310.17558
作者：Gene-Ping Yang,Hao Tang备注：Accepted to ASRU 2023摘要：从电话实例中学习电话类型一直是一个长期存在的问题，同时仍然是开放的。在这项工作中，我们在自监督学习的背景下重新审视这个问题，并将其视为将聚类质心与电话嵌入相匹配的问题。我们研究了两个关键属性，使匹配，即，是否集群质心的自我监督表示减少电话实例的变异性和尊重电话之间的关系。然后，我们使用匹配结果来产生伪标签，并引入一个新的损失函数来改进自监督表示。我们的实验表明，匹配结果捕捉到的电话之间的关系。训练新的损失函数与常规的自监督损失，如APC和CPC，显着提高下游电话分类。摘要：Learning phone types from phone instances has been a long-standing problem, while still being open. In this work, we revisit this problem in the context of self-supervised learning, and pose it as the problem of matching cluster centroids to phone embeddings. We study two key properties that enable matching, namely, whether cluster centroids of self-supervised representations reduce the variability of phone instances and respect the relationship among phones. We then use the matching result to produce pseudo-labels and introduce a new loss function for improving self-supervised representations. Our experiments show that the matching result captures the relationship among phones. Training the new loss function jointly with the regular self-supervised losses, such as APC and CPC, significantly improves the downstream phone classification.
【2】 Controllable Generation of Artificial Speaker Embeddings through  Discovery of Principal Directions标题：基于主方向发现的人工说话人嵌入可控生成链接：https://arxiv.org/abs/2310.17502
作者：Florian Lux,Pascal Tilli,Sarina Meyer,Ngoc Thang Vu备注：Published at ISCA Interspeech 2023 this https URL摘要：在语音合成系统中定制语音和说话风格具有直观和细粒度的控制是具有挑战性的，因为具有适当标签的数据很少。此外，编辑现有人类的声音也会带来伦理问题。在本文中，我们提出了一种方法来生成人工扬声器嵌入，不能链接到一个真正的人，同时提供直观和细粒度的控制的语音和说话风格的嵌入，而不需要任何标签的扬声器或风格。人工和可控的嵌入可以被馈送到语音合成系统，在训练期间以真实人类的嵌入为条件，而不会在推理期间牺牲隐私。摘要：Customizing voice and speaking style in a speech synthesis system with intuitive and fine-grained controls is challenging, given that little data with appropriate labels is available. Furthermore, editing an existing human's voice also comes with ethical concerns. In this paper, we propose a method to generate artificial speaker embeddings that cannot be linked to a real human while offering intuitive and fine-grained control over the voice and speaking style of the embeddings, without requiring any labels for speaker or style. The artificial and controllable embeddings can be fed to a speech synthesis system, conditioned on embeddings of real humans during training, without sacrificing privacy during inference.
【3】 Content-based Controls For Music Large Language Modeling标题：基于内容的音乐大语言建模控件链接：https://arxiv.org/abs/2310.17162
作者：Liwei Lin,Gus Xia,Junyan Jiang,Yixiao Zhang摘要：近年来，大规模语言模型在音乐音频领域迅速发展。这样的模型能够端到端生成更高质量的音乐，有些模型允许使用文本描述进行条件生成。然而，文本控件对音乐的控制能力本质上是有限的，因为它们只能通过元数据（如歌手和乐器）或高级表示（如流派和情感）间接描述音乐。我们的目标是进一步为模型配备对固有音乐语言（如音高，和弦和鼓声）的直接和基于内容的控制。为此，我们贡献了Coco-Mulla，一个基于内容的音乐大语言建模控制方法。它使用了一种参数高效微调（PEFT）方法，专为基于Transformer的音频模型量身定制。实验表明，我们的方法通过低资源半监督学习实现了高质量的音乐生成，与原始模型相比，使用不到4%的参数进行调整，并在不到300首歌曲的小数据集上进行训练。此外，我们的方法使有效的基于内容的控制，我们通过和弦和节奏，音乐音频的两个最显着的特点说明了控制能力。此外，我们表明，通过结合基于内容的控制和文本描述，我们的系统实现了灵活的音乐变化的生成和风格转移。我们的源代码和演示可以在线获得。摘要：Recent years have witnessed a rapid growth of large-scale language models in the domain of music audio. Such models enable end-to-end generation of higher-quality music, and some allow conditioned generation using text descriptions. However, the control power of text controls on music is intrinsically limited, as they can only describe music indirectly through meta-data (such as singers and instruments) or high-level representations (such as genre and emotion). We aim to further equip the models with direct and content-based controls on innate music languages such as pitch, chords and drum track. To this end, we contribute Coco-Mulla, a content-based control method for music large language modeling. It uses a parameter-efficient fine-tuning (PEFT) method tailored for Transformer-based audio models. Experiments show that our approach achieved high-quality music generation with low-resource semi-supervised learning, tuning with less than 4% parameters compared to the original model and training on a small dataset with fewer than 300 songs. Moreover, our approach enables effective content-based controls, and we illustrate the control power via chords and rhythms, two of the most salient features of music audio. Furthermore, we show that by combining content-based controls and text descriptions, our system achieves flexible music variation generation and style transfer. Our source codes and demos are available online.
【4】 Learning Repeatable Speech Embeddings Using An Intra-class Correlation  Regularizer标题：使用类内相关性规则器学习可重复语音嵌入链接：https://arxiv.org/abs/2310.17049
作者：Jianwei Zhang,Suren Jayasuriya,Visar Berisha备注：Accepted by NeurIPS 2023摘要：对于特定的机器学习任务，一个好的监督嵌入只对感兴趣的标签的变化敏感，并且对其他混淆因素不敏感。我们利用测量理论中的重复性概念来描述这种特性，并提出使用类内相关系数（ICC）来评估嵌入的重复性。然后，我们提出了一种新的正则化器，ICC正则化器，作为对比损失的补充组件，以指导深度神经网络产生具有更高可重复性的嵌入。我们使用模拟数据来解释为什么ICC正则化器在最小化类内方差方面比单独的对比损失效果更好。我们实现了ICC正则化，并将其应用于三个语音任务：说话人验证，语音风格转换，以及用于检测发音困难的语音的临床应用。实验结果表明，与仅使用对比损失相比，添加ICC正则化器可以提高学习嵌入的可重复性;此外，这些嵌入可以提高这些下游任务的性能。摘要：A good supervised embedding for a specific machine learning task is only sensitive to changes in the label of interest and is invariant to other confounding factors. We leverage the concept of repeatability from measurement theory to describe this property and propose to use the intra-class correlation coefficient (ICC) to evaluate the repeatability of embeddings. We then propose a novel regularizer, the ICC regularizer, as a complementary component for contrastive losses to guide deep neural networks to produce embeddings with higher repeatability. We use simulated data to explain why the ICC regularizer works better on minimizing the intra-class variance than the contrastive loss alone. We implement the ICC regularizer and apply it to three speech tasks: speaker verification, voice style conversion, and a clinical application for detecting dysphonic voice. The experimental results demonstrate that adding an ICC regularizer can improve the repeatability of learned embeddings compared to only using the contrastive loss; further, these embeddings lead to improved performance in these downstream tasks.
【5】 Single channel speech enhancement by colored spectrograms标题：基于彩色谱图的单通道语音增强链接：https://arxiv.org/abs/2310.17142
作者：Sania Gul,Muhammad Salman Khan,Muhammad Fazeel备注：18 pages, 6 figures, 5 tables摘要：语音增强涉及从目标语音中去除不需要的背景声音以提高其质量和可懂度所需的过程。本文提出了一种新的基于彩色谱图的单通道语音增强方法。我们提出使用一种深度神经网络（DNN）架构，该架构改编自pix 2 pix生成对抗网络（GAN），并在语音的彩色频谱图上对其进行训练以对其进行降噪。去噪后，光谱图的颜色被转换为短时傅里叶变换（STFT）的幅度使用浅回归神经网络。这些估计的STFT幅度随后与噪声相位组合以获得增强的语音。结果表明，在语音质量的感知评价（PESQ）和1%的短期客观可懂度（STOI）在未经处理的噪声数据的近0.84点的改善。未经处理的信号的质量和可懂度的增益几乎等于用于与所提出的模型进行比较的基线方法所实现的增益，但计算成本大大降低。所提出的解决方案提供了一个比较PESQ分数，其计算成本比类似的基线模型降低了近10倍，该基线模型生成了在灰度频谱图上训练的最高PESQ分数，而与另一个基于卷积神经网络GAN（CNN-GAN）的基线系统相比，它在STOI中仅提供了1%的赤字，计算成本降低了28倍，产生了最清晰的语音。摘要：Speech enhancement concerns the processes required to remove unwanted background sounds from the target speech to improve its quality and intelligibility. In this paper, a novel approach for single-channel speech enhancement is presented, using colored spectrograms. We propose the use of a deep neural network (DNN) architecture adapted from the pix2pix generative adversarial network (GAN) and train it over colored spectrograms of speech to denoise them. After denoising, the colors of spectrograms are translated to magnitudes of short-time Fourier transform (STFT) using a shallow regression neural network. These estimated STFT magnitudes are later combined with the noisy phases to obtain an enhanced speech. The results show an improvement of almost 0.84 points in the perceptual evaluation of speech quality (PESQ) and 1% in the short-term objective intelligibility (STOI) over the unprocessed noisy data. The gain in quality and intelligibility over the unprocessed signal is almost equal to the gain achieved by the baseline methods used for comparison with the proposed model, but at a much reduced computational cost. The proposed solution offers a comparative PESQ score at almost 10 times reduced computational cost than a similar baseline model that has generated the highest PESQ score trained on grayscaled spectrograms, while it provides only a 1% deficit in STOI at 28 times reduced computational cost when compared to another baseline system based on convolutional neural network-GAN (CNN-GAN) that produces the most intelligible speech.
【6】 Real-time Neonatal Chest Sound Separation using Deep Learning标题：基于深度学习的新生儿实时胸音分离链接：https://arxiv.org/abs/2310.17116
作者：Yang Yi Poh,Ethan Grooby,Kenneth Tan,Lindsay Zhou,Arrabella King,Ashwin Ramanathan,Atul Malhotra,Mehrtash Harandi,Faezeh Marzbanrad摘要：新生儿听诊是一种简单、无创的心血管和呼吸系统疾病诊断方法。这种诊断通常需要在听诊期间捕获高质量的心脏和肺音。然而，在大多数情况下，由于胸部声音包含心脏、肺和噪声声音的混合，因此获得这种高质量的声音是不平凡的。因此，需要额外的预处理来将胸部声音分离成心脏和肺声音。本文提出了一种新的深度学习方法，将这种胸部声音分离为心脏和肺音。受Conv-TasNet模型的启发，该模型具有编码器，解码器和掩码生成器。编码器由1D卷积模型组成，解码器由转置的1D卷积组成。掩码生成器使用堆叠的1D卷积和Transformers构造。所提出的模型优于以前的方法，在人工数据集的客观失真措施2.01 dB至5.06 dB，以及计算时间，至少有17倍的改善。因此，我们提出的模型可以是任何基于心音图的健康监测系统的一个合适的预处理步骤。摘要：Auscultation for neonates is a simple and non-invasive method of providing diagnosis for cardiovascular and respiratory disease. Such diagnosis often requires high-quality heart and lung sounds to be captured during auscultation. However, in most cases, obtaining such high-quality sounds is non-trivial due to the chest sounds containing a mixture of heart, lung, and noise sounds. As such, additional preprocessing is needed to separate the chest sounds into heart and lung sounds. This paper proposes a novel deep-learning approach to separate such chest sounds into heart and lung sounds. Inspired by the Conv-TasNet model, the proposed model has an encoder, decoder, and mask generator. The encoder consists of a 1D convolution model and the decoder consists of a transposed 1D convolution. The mask generator is constructed using stacked 1D convolutions and transformers. The proposed model outperforms previous methods in terms of objective distortion measures by 2.01 dB to 5.06 dB in the artificial dataset, as well as computation time, with at least a 17-time improvement. Therefore, our proposed model could be a suitable preprocessing step for any phonocardiogram-based health monitoring system.
【7】 Multi-Speaker Expressive Speech Synthesis via Semi-supervised  Contrastive Learning标题：基于半监督对比学习的多说话人表达语音合成链接：https://arxiv.org/abs/2310.17101
作者：Xinfa Zhu,Yuke Li,Yi Lei,Ning Jiang,Guoqing Zhao,Lei Xie备注：5 pages, 3 figures摘要：本文旨在建立一个具有表达力的多说话人TTS系统，合成具有多种风格和情感的目标说话人的语音。为此，我们提出了一种新的基于对比学习的TTS方法来跨扬声器传输风格和情感。具体来说，我们在话语和类别（如情感快乐或风格诗人或扬声器A）水平上构建正负样本对，并利用对比学习来更好地从语音中提取分离的风格，情感和扬声器表示。此外，我们引入了一个半监督训练策略，以有效地利用多领域的数据，包括风格标记的数据，情感标记的数据和未标记的数据。我们将学习到的表征集成到改进的VITS模型中，使其能够为目标说话者合成具有不同风格和情感的表达性语音。在多领域数据上的实验证明了该模型的良好设计。摘要：This paper aims to build an expressive TTS system for multi-speakers, synthesizing a target speaker's speech with multiple styles and emotions. To this end, we propose a novel contrastive learning-based TTS approach to transfer style and emotion across speakers. Specifically, we construct positive-negative sample pairs at both utterance and category (such as emotion-happy or style-poet or speaker A) levels and leverage contrastive learning to better extract disentangled style, emotion, and speaker representations from speech. Furthermore, we introduce a semi-supervised training strategy to the proposed approach to effectively leverage multi-domain data, including style-labeled data, emotion-labeled data, and unlabeled data. We integrate the learned representations into an improved VITS model, enabling it to synthesize expressive speech with diverse styles and emotions for a target speaker. Experiments on multi-domain data demonstrate the good design of our model.
【8】 Improved Panning on Non-Equidistant Loudspeakers with Direct Sound Level  Compensation标题：具有直接声级补偿的非等距扬声器上的改进摇摄链接：https://arxiv.org/abs/2310.17004
作者：Jan-Hendrik Hanschke,Daniel Arteaga,Giulio Cengarle,Joshua Lando,Mark R. P. Thomas,Alan Seefeldt备注：None摘要：创建幻象声源的扬声器渲染技术通常假定等距扬声器布局。典型的家庭设置可能不满足此条件，因为扬声器偏离规范位置，因此需要相应的校准。标准的方法是补偿延迟和匹配每个扬声器的响度在收听者的location.It被发现，幻影图像的移位发生时，这种校准过程被应用和一对扬声器中的一个是显着更接近听者比其他。在本文中，提出了一种新的方法来平移非等距扬声器布局，其中平移位置是由直接声音和感知响度是由全脉冲响应。主观听力测试验证的方法和量化的感知效果的补偿。在标准校准导致平均误差为10度的设置中，所提出的直接声音补偿很大程度上将幻象源返回到其预期位置。摘要：Loudspeaker rendering techniques that create phantom sound sources often assume an equidistant loudspeaker layout. Typical home setups might not fulfill this condition as loudspeakers deviate from canonical positions, thus requiring a corresponding calibration. The standard approach is to compensate for delays and to match the loudness of each loudspeaker at the listener's location.It was found that a shift of the phantom image occurs when this calibration procedure is applied and one of a pair of loudspeakers is significantly closer to the listener than the other. In this paper, a novel approach to panning on non-equidistant loudspeaker layouts is presented whereby the panning position is governed by the direct sound and the perceived loudness is governed by the full impulse response. Subjective listening tests are presented that validate the approach and quantify the perceived effect of the compensation. In a setup where the standard calibration leads to an average error of 10 degrees, the proposed direct sound compensation largely returns the phantom source to its intended position.
eess.AS音频处理
【1】 Privacy-preserving Representation Learning for Speech Understanding标题：语音理解中的隐私保护表征学习链接：https://arxiv.org/abs/2310.17194
作者：Minh Tran,Mohammad Soleymani备注：INTERSPEECH 2023摘要：现有的隐私保护语音表示学习方法针对单个应用领域。在本文中，我们提出了一种新的框架，以匿名化由预训练的编码器生成的话语级语音嵌入，并显示其有效性的一系列语音分类任务。具体来说，给定来自预先训练的编码器的表示，我们训练Transformer来估计其他说话者所说的相同话语的表示。在推理过程中，提取的表示可以转换为不同的身份，以保护隐私。我们将结果与VoicePrivacy 2022挑战的语音匿名基线进行了比较。我们评估了我们的框架，说话人识别的隐私和情感识别，抑郁症分类，和意图分类的效用。我们的方法在语言任务中的隐私和实用性方面优于基线，并在意图分类方面取得了相当的性能。摘要：Existing privacy-preserving speech representation learning methods target a single application domain. In this paper, we present a novel framework to anonymize utterance-level speech embeddings generated by pre-trained encoders and show its effectiveness for a range of speech classification tasks. Specifically, given the representations from a pre-trained encoder, we train a Transformer to estimate the representations for the same utterances spoken by other speakers. During inference, the extracted representations can be converted into different identities to preserve privacy. We compare the results with the voice anonymization baselines from the VoicePrivacy 2022 challenge. We evaluate our framework on speaker identification for privacy and emotion recognition, depression classification, and intent classification for utility. Our method outperforms the baselines on privacy and utility in paralinguistic tasks and achieves comparable performance for intent classification.
【2】 Single channel speech enhancement by colored spectrograms标题：基于彩色谱图的单通道语音增强链接：https://arxiv.org/abs/2310.17142
作者：Sania Gul,Muhammad Salman Khan,Muhammad Fazeel备注：18 pages, 6 figures, 5 tables摘要：语音增强涉及从目标语音中去除不需要的背景声音以提高其质量和可懂度所需的过程。本文提出了一种新的基于彩色谱图的单通道语音增强方法。我们提出使用一种深度神经网络（DNN）架构，该架构改编自pix 2 pix生成对抗网络（GAN），并在语音的彩色频谱图上对其进行训练以对其进行降噪。去噪后，光谱图的颜色被转换为短时傅里叶变换（STFT）的幅度使用浅回归神经网络。这些估计的STFT幅度随后与噪声相位组合以获得增强的语音。结果表明，在语音质量的感知评价（PESQ）和1%的短期客观可懂度（STOI）在未经处理的噪声数据的近0.84点的改善。未经处理的信号的质量和可懂度的增益几乎等于用于与所提出的模型进行比较的基线方法所实现的增益，但计算成本大大降低。所提出的解决方案提供了一个比较PESQ分数，其计算成本比类似的基线模型降低了近10倍，该基线模型生成了在灰度频谱图上训练的最高PESQ分数，而与另一个基于卷积神经网络GAN（CNN-GAN）的基线系统相比，它在STOI中仅提供了1%的赤字，计算成本降低了28倍，产生了最清晰的语音。摘要：Speech enhancement concerns the processes required to remove unwanted background sounds from the target speech to improve its quality and intelligibility. In this paper, a novel approach for single-channel speech enhancement is presented, using colored spectrograms. We propose the use of a deep neural network (DNN) architecture adapted from the pix2pix generative adversarial network (GAN) and train it over colored spectrograms of speech to denoise them. After denoising, the colors of spectrograms are translated to magnitudes of short-time Fourier transform (STFT) using a shallow regression neural network. These estimated STFT magnitudes are later combined with the noisy phases to obtain an enhanced speech. The results show an improvement of almost 0.84 points in the perceptual evaluation of speech quality (PESQ) and 1% in the short-term objective intelligibility (STOI) over the unprocessed noisy data. The gain in quality and intelligibility over the unprocessed signal is almost equal to the gain achieved by the baseline methods used for comparison with the proposed model, but at a much reduced computational cost. The proposed solution offers a comparative PESQ score at almost 10 times reduced computational cost than a similar baseline model that has generated the highest PESQ score trained on grayscaled spectrograms, while it provides only a 1% deficit in STOI at 28 times reduced computational cost when compared to another baseline system based on convolutional neural network-GAN (CNN-GAN) that produces the most intelligible speech.
【3】 Real-time Neonatal Chest Sound Separation using Deep Learning标题：基于深度学习的新生儿实时胸音分离链接：https://arxiv.org/abs/2310.17116
作者：Yang Yi Poh,Ethan Grooby,Kenneth Tan,Lindsay Zhou,Arrabella King,Ashwin Ramanathan,Atul Malhotra,Mehrtash Harandi,Faezeh Marzbanrad摘要：新生儿听诊是一种简单、无创的心血管和呼吸系统疾病诊断方法。这种诊断通常需要在听诊期间捕获高质量的心脏和肺音。然而，在大多数情况下，由于胸部声音包含心脏、肺和噪声声音的混合，因此获得这种高质量的声音是不平凡的。因此，需要额外的预处理来将胸部声音分离成心脏和肺声音。本文提出了一种新的深度学习方法，将这种胸部声音分离为心脏和肺音。受Conv-TasNet模型的启发，该模型具有编码器，解码器和掩码生成器。编码器由1D卷积模型组成，解码器由转置的1D卷积组成。掩码生成器使用堆叠的1D卷积和Transformers构造。所提出的模型优于以前的方法，在人工数据集的客观失真措施2.01 dB至5.06 dB，以及计算时间，至少有17倍的改善。因此，我们提出的模型可以是任何基于心音图的健康监测系统的一个合适的预处理步骤。摘要：Auscultation for neonates is a simple and non-invasive method of providing diagnosis for cardiovascular and respiratory disease. Such diagnosis often requires high-quality heart and lung sounds to be captured during auscultation. However, in most cases, obtaining such high-quality sounds is non-trivial due to the chest sounds containing a mixture of heart, lung, and noise sounds. As such, additional preprocessing is needed to separate the chest sounds into heart and lung sounds. This paper proposes a novel deep-learning approach to separate such chest sounds into heart and lung sounds. Inspired by the Conv-TasNet model, the proposed model has an encoder, decoder, and mask generator. The encoder consists of a 1D convolution model and the decoder consists of a transposed 1D convolution. The mask generator is constructed using stacked 1D convolutions and transformers. The proposed model outperforms previous methods in terms of objective distortion measures by 2.01 dB to 5.06 dB in the artificial dataset, as well as computation time, with at least a 17-time improvement. Therefore, our proposed model could be a suitable preprocessing step for any phonocardiogram-based health monitoring system.
【4】 Multi-Speaker Expressive Speech Synthesis via Semi-supervised  Contrastive Learning标题：基于半监督对比学习的多说话人表达语音合成链接：https://arxiv.org/abs/2310.17101
作者：Xinfa Zhu,Yuke Li,Yi Lei,Ning Jiang,Guoqing Zhao,Lei Xie备注：5 pages, 3 figures摘要：本文旨在建立一个具有表达力的多说话人TTS系统，合成具有多种风格和情感的目标说话人的语音。为此，我们提出了一种新的基于对比学习的TTS方法来跨扬声器传输风格和情感。具体来说，我们在话语和类别（如情感快乐或风格诗人或扬声器A）水平上构建正负样本对，并利用对比学习来更好地从语音中提取分离的风格，情感和扬声器表示。此外，我们引入了一个半监督训练策略，以有效地利用多领域的数据，包括风格标记的数据，情感标记的数据和未标记的数据。我们将学习到的表征集成到改进的VITS模型中，使其能够为目标说话者合成具有不同风格和情感的表达性语音。在多领域数据上的实验证明了该模型的良好设计。摘要：This paper aims to build an expressive TTS system for multi-speakers, synthesizing a target speaker's speech with multiple styles and emotions. To this end, we propose a novel contrastive learning-based TTS approach to transfer style and emotion across speakers. Specifically, we construct positive-negative sample pairs at both utterance and category (such as emotion-happy or style-poet or speaker A) levels and leverage contrastive learning to better extract disentangled style, emotion, and speaker representations from speech. Furthermore, we introduce a semi-supervised training strategy to the proposed approach to effectively leverage multi-domain data, including style-labeled data, emotion-labeled data, and unlabeled data. We integrate the learned representations into an improved VITS model, enabling it to synthesize expressive speech with diverse styles and emotions for a target speaker. Experiments on multi-domain data demonstrate the good design of our model.
【5】 Improved Panning on Non-Equidistant Loudspeakers with Direct Sound Level  Compensation标题：采用直接声级补偿的非等距扬声器的改进平移链接：https://arxiv.org/abs/2310.17004
作者：Jan-Hendrik Hanschke,Daniel Arteaga,Giulio Cengarle,Joshua Lando,Mark R. P. Thomas,Alan Seefeldt备注：None摘要：创建幻象声源的扬声器渲染技术通常假定等距扬声器布局。典型的家庭设置可能不满足此条件，因为扬声器偏离规范位置，因此需要相应的校准。标准的方法是补偿延迟和匹配每个扬声器的响度在收听者的location.It被发现，幻影图像的移位发生时，这种校准过程被应用和一对扬声器中的一个是显着更接近听者比其他。在本文中，提出了一种新的方法来平移非等距扬声器布局，其中平移位置是由直接声音和感知响度是由全脉冲响应。主观听力测试验证的方法和量化的感知效果的补偿。在标准校准导致平均误差为10度的设置中，所提出的直接声音补偿很大程度上将幻象源返回到其预期位置。摘要：Loudspeaker rendering techniques that create phantom sound sources often assume an equidistant loudspeaker layout. Typical home setups might not fulfill this condition as loudspeakers deviate from canonical positions, thus requiring a corresponding calibration. The standard approach is to compensate for delays and to match the loudness of each loudspeaker at the listener's location.It was found that a shift of the phantom image occurs when this calibration procedure is applied and one of a pair of loudspeakers is significantly closer to the listener than the other. In this paper, a novel approach to panning on non-equidistant loudspeaker layouts is presented whereby the panning position is governed by the direct sound and the perceived loudness is governed by the full impulse response. Subjective listening tests are presented that validate the approach and quantify the perceived effect of the compensation. In a setup where the standard calibration leads to an average error of 10 degrees, the proposed direct sound compensation largely returns the phantom source to its intended position.
【6】 Towards Matching Phones and Speech Representations标题：走向匹配音素和语音表示链接：https://arxiv.org/abs/2310.17558
作者：Gene-Ping Yang,Hao Tang备注：Accepted to ASRU 2023摘要：从电话实例中学习电话类型一直是一个长期存在的问题，同时仍然是开放的。在这项工作中，我们在自监督学习的背景下重新审视这个问题，并将其视为将聚类质心与电话嵌入相匹配的问题。我们研究了两个关键属性，使匹配，即，是否集群质心的自我监督表示减少电话实例的变异性和尊重电话之间的关系。然后，我们使用匹配结果来产生伪标签，并引入一个新的损失函数来改进自监督表示。我们的实验表明，匹配结果捕捉到的电话之间的关系。训练新的损失函数与常规的自监督损失，如APC和CPC，显着提高下游电话分类。摘要：Learning phone types from phone instances has been a long-standing problem, while still being open. In this work, we revisit this problem in the context of self-supervised learning, and pose it as the problem of matching cluster centroids to phone embeddings. We study two key properties that enable matching, namely, whether cluster centroids of self-supervised representations reduce the variability of phone instances and respect the relationship among phones. We then use the matching result to produce pseudo-labels and introduce a new loss function for improving self-supervised representations. Our experiments show that the matching result captures the relationship among phones. Training the new loss function jointly with the regular self-supervised losses, such as APC and CPC, significantly improves the downstream phone classification.
【7】 Controllable Generation of Artificial Speaker Embeddings through  Discovery of Principal Directions标题：基于主方向发现的人工说话人嵌入可控生成链接：https://arxiv.org/abs/2310.17502
作者：Florian Lux,Pascal Tilli,Sarina Meyer,Ngoc Thang Vu备注：Published at ISCA Interspeech 2023 this https URL摘要：在语音合成系统中定制语音和说话风格具有直观和细粒度的控制是具有挑战性的，因为具有适当标签的数据很少。此外，编辑现有人类的声音也会带来伦理问题。在本文中，我们提出了一种方法来生成人工扬声器嵌入，不能链接到一个真正的人，同时提供直观和细粒度的控制的语音和说话风格的嵌入，而不需要任何标签的扬声器或风格。人工和可控的嵌入可以被馈送到语音合成系统，在训练期间以真实人类的嵌入为条件，而不会在推理期间牺牲隐私。摘要：Customizing voice and speaking style in a speech synthesis system with intuitive and fine-grained controls is challenging, given that little data with appropriate labels is available. Furthermore, editing an existing human's voice also comes with ethical concerns. In this paper, we propose a method to generate artificial speaker embeddings that cannot be linked to a real human while offering intuitive and fine-grained control over the voice and speaking style of the embeddings, without requiring any labels for speaker or style. The artificial and controllable embeddings can be fed to a speech synthesis system, conditioned on embeddings of real humans during training, without sacrificing privacy during inference.
【8】 The IMS Toucan System for the Blizzard Challenge 2023标题：2023年暴雪挑战赛的IMS Toucan系统链接：https://arxiv.org/abs/2310.17499
作者：Florian Lux,Julia Koch,Sarina Meyer,Thomas Bott,Nadja Schauffler,Pavel Denisov,Antje Schweitzer,Ngoc Thang Vu备注：Published at the Blizzard Challenge Workshop 2023, colocated with the Speech Synthesis Workshop 2023, a sattelite event of the Interspeech 2023摘要：对于我们对暴雪挑战2023的贡献，我们改进了我们提交给暴雪挑战2021的系统。我们的方法需要一个基于规则的文本到音素的处理系统，其中包括基于规则的消歧在法语中的同形异义词。然后使用基于Conformer和Glow的快速高效的非自回归合成架构将音素转换为声谱图作为中间表示。基于GAN的神经声码器结合了最新的最先进的方法，将频谱图转换为最终波。我们精心设计了挑战数据的数据处理、训练和推理程序。我们的系统标识符是G。开放源代码和演示是可用的。摘要：For our contribution to the Blizzard Challenge 2023, we improved on the system we submitted to the Blizzard Challenge 2021. Our approach entails a rule-based text-to-phoneme processing system that includes rule-based disambiguation of homographs in the French language. It then transforms the phonemes to spectrograms as intermediate representations using a fast and efficient non-autoregressive synthesis architecture based on Conformer and Glow. A GAN based neural vocoder that combines recent state-of-the-art approaches converts the spectrogram to the final wave. We carefully designed the data processing, training, and inference procedures for the challenge data. Our system identifier is G. Open source code and demo are available.
【9】 Dialect Adaptation and Data Augmentation for Low-Resource ASR: TalTech  Systems for the MADASR 2023 Challenge标题：低资源ASR的方言适应和数据增强：TalTech Systems应对MADASR 2023挑战链接：https://arxiv.org/abs/2310.17448
作者：Tanel Alumäe,Jiaming Kong,Daniil Robnikov备注：None摘要：本文介绍了塔林理工大学（TalTech）为ASRU MADASR 2023挑战赛开发的系统。挑战的重点是在有限的训练音频和文本数据的情况下对方言丰富的印度语言进行自动语音识别。TalTech参加了两个轨道的挑战：轨道1，允许只使用提供的训练数据和轨道3，允许使用额外的音频数据。在这两条赛道上，我们都依赖于wav2vec2.0模型。我们的方法在两个关键点上不同于传统的预训练wav2vec2.0模型的微调过程：首先，通过实施对齐数据增强技术来增强训练数据的语言多样性，其次，通过应用深度前缀调整来适应wav2vec2.0模型的方言。在这两个方面，我们的方法都比提供的基线有了显著的改进，在所有参与团队中实现了最低的单词错误率。摘要：This paper describes Tallinn University of Technology (TalTech) systems developed for the ASRU MADASR 2023 Challenge. The challenge focuses on automatic speech recognition of dialect-rich Indian languages with limited training audio and text data. TalTech participated in two tracks of the challenge: Track 1 that allowed using only the provided training data and Track 3 which allowed using additional audio data. In both tracks, we relied on wav2vec2.0 models. Our methodology diverges from the traditional procedure of finetuning pretrained wav2vec2.0 models in two key points: firstly, through the implementation of the aligned data augmentation technique to enhance the linguistic diversity of the training data, and secondly, via the application of deep prefix tuning for dialect adaptation of wav2vec2.0 models. In both tracks, our approach yielded significant improvements over the provided baselines, achieving the lowest word error rates across all participating teams.
【10】 Content-based Controls For Music Large Language Modeling标题：基于内容的音乐大语言建模控件链接：https://arxiv.org/abs/2310.17162
作者：Liwei Lin,Gus Xia,Junyan Jiang,Yixiao Zhang摘要：近年来，大规模语言模型在音乐音频领域迅速发展。这样的模型能够端到端生成更高质量的音乐，有些模型允许使用文本描述进行条件生成。然而，文本控件对音乐的控制能力本质上是有限的，因为它们只能通过元数据（如歌手和乐器）或高级表示（如流派和情感）间接描述音乐。我们的目标是进一步为模型配备对固有音乐语言（如音高，和弦和鼓声）的直接和基于内容的控制。为此，我们贡献了Coco-Mulla，一个基于内容的音乐大语言建模控制方法。它使用了一种参数高效微调（PEFT）方法，专为基于Transformer的音频模型量身定制。实验表明，我们的方法通过低资源半监督学习实现了高质量的音乐生成，与原始模型相比，使用不到4%的参数进行调整，并在不到300首歌曲的小数据集上进行训练。此外，我们的方法使有效的基于内容的控制，我们通过和弦和节奏，音乐音频的两个最显着的特点说明了控制能力。此外，我们表明，通过结合基于内容的控制和文本描述，我们的系统实现了灵活的音乐变化的生成和风格转移。我们的源代码和演示可以在线获得。摘要：Recent years have witnessed a rapid growth of large-scale language models in the domain of music audio. Such models enable end-to-end generation of higher-quality music, and some allow conditioned generation using text descriptions. However, the control power of text controls on music is intrinsically limited, as they can only describe music indirectly through meta-data (such as singers and instruments) or high-level representations (such as genre and emotion). We aim to further equip the models with direct and content-based controls on innate music languages such as pitch, chords and drum track. To this end, we contribute Coco-Mulla, a content-based control method for music large language modeling. It uses a parameter-efficient fine-tuning (PEFT) method tailored for Transformer-based audio models. Experiments show that our approach achieved high-quality music generation with low-resource semi-supervised learning, tuning with less than 4% parameters compared to the original model and training on a small dataset with fewer than 300 songs. Moreover, our approach enables effective content-based controls, and we illustrate the control power via chords and rhythms, two of the most salient features of music audio. Furthermore, we show that by combining content-based controls and text descriptions, our system achieves flexible music variation generation and style transfer. Our source codes and demos are available online.
【11】 Learning Repeatable Speech Embeddings Using An Intra-class Correlation  Regularizer标题：使用类内相关性规则器学习可重复语音嵌入链接：https://arxiv.org/abs/2310.17049
作者：Jianwei Zhang,Suren Jayasuriya,Visar Berisha备注：Accepted by NeurIPS 2023摘要：对于特定的机器学习任务，一个好的监督嵌入只对感兴趣的标签的变化敏感，并且对其他混淆因素不敏感。我们利用测量理论中的重复性概念来描述这种特性，并提出使用类内相关系数（ICC）来评估嵌入的重复性。然后，我们提出了一种新的正则化器，ICC正则化器，作为对比损失的补充组件，以指导深度神经网络产生具有更高可重复性的嵌入。我们使用模拟数据来解释为什么ICC正则化器在最小化类内方差方面比单独的对比损失效果更好。我们实现了ICC正则化，并将其应用于三个语音任务：说话人验证，语音风格转换，以及用于检测发音困难的语音的临床应用。实验结果表明，与仅使用对比损失相比，添加ICC正则化器可以提高学习嵌入的可重复性;此外，这些嵌入可以提高这些下游任务的性能。摘要：A good supervised embedding for a specific machine learning task is only sensitive to changes in the label of interest and is invariant to other confounding factors. We leverage the concept of repeatability from measurement theory to describe this property and propose to use the intra-class correlation coefficient (ICC) to evaluate the repeatability of embeddings. We then propose a novel regularizer, the ICC regularizer, as a complementary component for contrastive losses to guide deep neural networks to produce embeddings with higher repeatability. We use simulated data to explain why the ICC regularizer works better on minimizing the intra-class variance than the contrastive loss alone. We implement the ICC regularizer and apply it to three speech tasks: speaker verification, voice style conversion, and a clinical application for detecting dysphonic voice. The experimental results demonstrate that adding an ICC regularizer can improve the repeatability of learned embeddings compared to only using the contrastive loss; further, these embeddings lead to improved performance in these downstream tasks.
机器翻译，仅供参考