今日论文合集：cs.SD语音8篇，eess.AS音频处理8篇。
cs.SD语音

【1】 Pitchclass2vec: Symbolic Music Structure Segmentation with Chord Embeddings标题：PitchClass2vec：基于和弦嵌入的符号音乐结构分割链接：http://arxiv.org/pdf/2303.15306v1

作者：Nicolas Lazzari,Andrea Poltronieri,Valentina Presutti机构：Department of Computer Science and Engineering, University of Bologna, Mura Anteo Zamboni, Bologna , Italy, LILEC, University of Bologna, Via Cartoleria, Bologna , ItalyJournal-ref：Proceedings of the 1st Workshop on Artificial Intelligence and Creativity co-located with 21th International Conference of the Italian Association for Artificial Intelligence(AIxIA 2022), Udine, Italy, November 28 - December 3, 2022摘要：结构知觉是人类音乐认知的一个基本方面。从历史上看，音乐的结构层次组织作为一种叙事手段，用于传达意义，创造期望，唤起听众的情感。因此，音乐结构在音乐创作中起着至关重要的作用，因为它们塑造了作曲家组织其思想的音乐话语。在本文中，我们提出了一种新的音乐分割方法，pitchclass2vec，基于符号和弦注释，这是嵌入到连续的矢量表示使用自然语言处理技术和定制的编码。我们的算法基于长短期记忆（LSTM）神经网络，并且优于该领域中基于符号和弦注释的最先进技术。摘要：Structure perception is a fundamental aspect of music cognition in humans. Historically, the hierarchical organization of music into structures served as a narrative device for conveying meaning, creating expectancy, and evoking emotions in the listener. Thereby, musical structures play an essential role in music composition, as they shape the musical discourse through which the composer organises his ideas. In this paper, we present a novel music segmentation method, pitchclass2vec, based on symbolic chord annotations, which are embedded into continuous vector representations using both natural language processing techniques and custom-made encodings. Our algorithm is based on long-short term memory (LSTM) neural network and outperforms the state-of-the-art techniques based on symbolic chord annotations in the field.

【2】 Data Augmentation for Environmental Sound Classification Using Diffusion Probabilistic Model with Top-k Selection Discriminator标题：基于Top-k选择判别的扩散概率模型用于环境声分类的数据增强链接：http://arxiv.org/pdf/2303.15161v1

作者：Yunhao Chen,Yunjie Zhu,Zihui Yan,Jianlu Shen,Zhen Ren,Yifan Huang机构：Jiangnan University, Wuxi, China, University of Leeds, Leeds, LS,JT, United Kingdom摘要：尽管近年来强大的深度学习技术不断进步，但模型仍然需要大量的训练数据来避免过度拟合。最近已经产生了使用生成对抗网络（GAN）的合成数据集来克服这个问题。然而，尽管取得了进步，但基于GAN的方法通常难以训练或无法生成高质量的数据样本。在本文中，我们提出了一种环境声音分类增强技术的基础上的扩散概率模型与DPM-Solver$++$快速采样。此外，为了确保生成的频谱图的质量，我们在数据集上训练了一个top-k选择鉴别器。根据实验结果，合成的频谱图具有相似的功能，以原始数据集，并可以显着提高不同的国家的最先进的模型的分类精度相比，传统的数据增强技术。公共代码可在 url{https：github.com JNAIC DPMs-for-Audio-Data-Augmentation}上获得。摘要：Despite consistent advancement in powerful deep learning techniques in recent years, large amounts of training data are still necessary for the models to avoid overfitting. Synthetic datasets using generative adversarial networks (GAN) have recently been generated to overcome this problem. Nevertheless, despite advancements, GAN-based methods are usually hard to train or fail to generate high-quality data samples. In this paper, we propose an environmental sound classification augmentation technique based on the diffusion probabilistic model with DPM-Solver$++$ for fast sampling. In addition, to ensure the quality of the generated spectrograms, we train a top-k selection discriminator on the dataset. According to the experiment results, the synthesized spectrograms have similar features to the original dataset and can significantly increase the classification accuracy of different state-of-the-art models compared with traditional data augmentation techniques. The public code is available on url{https: github.com JNAIC DPMs-for-Audio-Data-Augmentation}.

【3】 Time-domain Speech Enhancement Assisted by Multi-resolution Frequency Encoder and Decoder标题：多分辨率频率编解码器辅助的时域语音增强链接：http://arxiv.org/pdf/2303.14593v1

作者：Hao Shi,Masato Mimura,Longbiao Wang,Jianwu Dang,Tatsuya Kawahara机构：Graduate School of Informatics, Kyoto University, Kyoto, Japan, Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University, Tianjin, China摘要：时域语音增强（SE）最近得到了广泛的研究。在最近的作品中，DEMUCS引入了多分辨率STFT损失，以提高性能。然而，用于STFT的一些分辨率包含非平稳信号，并且仅用一个输出同时学习多分辨率频率损失是具有挑战性的。为了更好地利用多分辨率的频率信息，我们补充多个频谱在不同的帧长度到时域编码器。它们提取窄带和宽带中的稳定频率信息。我们还采用了多个解码器输出，每个输出计算其相应的分辨率频率损失。实验结果表明：（1）在编码器中融合平稳频率特征比融合非平稳特征更有效;（2）多个输出与频率损失一致，提高了性能。在Voice-Bank数据集上的实验结果表明，该方法的PESQ提高了0.14。摘要：Time-domain speech enhancement (SE) has recently been intensively investigated. Among recent works, DEMUCS introduces multi-resolution STFT loss to enhance performance. However, some resolutions used for STFT contain non-stationary signals, and it is challenging to learn multi-resolution frequency losses simultaneously with only one output. For better use of multi-resolution frequency information, we supplement multiple spectrograms in different frame lengths into the time-domain encoders. They extract stationary frequency information in both narrowband and wideband. We also adopt multiple decoder outputs, each of which computes its corresponding resolution frequency loss. Experimental results show that (1) it is more effective to fuse stationary frequency features than non-stationary features in the encoder, and (2) the multiple outputs consistent with the frequency loss improve performance. Experiments on the Voice-Bank dataset show that the proposed method obtained a 0.14 PESQ improvement.

【4】 Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels标题：Auto-AVSR：带自动标签的视听语音识别链接：http://arxiv.org/pdf/2303.14307v1

作者：Pingchuan Ma,Alexandros Haliassos,Adriana Fernandez-Lopez,Honglie Chen,Stavros Petridis,Maja Pantic机构：Imperial College London, UK, Meta AI, UK备注：Accepted to ICASSP 2023摘要：视听语音识别由于其对声学噪声的鲁棒性而受到了广泛的关注。最近，自动、视觉和视听语音识别（分别为ASR、VSR和AV-ASR）的性能已经得到了实质性的改善，这主要是由于使用了更大的模型和训练集。然而，数据集的准确标记是耗时且昂贵的。因此，在这项工作中，我们研究了使用自动生成的转录的未标记的数据集，以增加训练集的大小。为此，我们使用公开的预训练ASR模型来自动转录未标记的数据集，如AVSpeech和VoxCeleb 2。然后，我们在由LRS 2和LRS 3数据集以及额外的自动转录数据组成的增强训练集上训练ASR、VSR和AV-ASR模型。我们证明，增加训练集的大小，最近的趋势，在文献中，导致减少WER，尽管使用嘈杂的转录。所提出的模型在LRS 2和LRS 3上实现了新的最先进的AV-ASR性能。特别是，它在LRS 3上实现了0.9%的WER，比当前最先进的方法相对提高了30%，并且优于在非公开数据集上训练的方法，训练数据是其他方法的26倍。摘要：Audio-visual speech recognition has received a lot of attention due to its robustness against acoustic noise. Recently, the performance of automatic, visual, and audio-visual speech recognition (ASR, VSR, and AV-ASR, respectively) has been substantially improved, mainly due to the use of larger models and training sets. However, accurate labelling of datasets is time-consuming and expensive. Hence, in this work, we investigate the use of automatically-generated transcriptions of unlabelled datasets to increase the training set size. For this purpose, we use publicly-available pre-trained ASR models to automatically transcribe unlabelled datasets such as AVSpeech and VoxCeleb2. Then, we train ASR, VSR and AV-ASR models on the augmented training set, which consists of the LRS2 and LRS3 datasets as well as the additional automatically-transcribed data. We demonstrate that increasing the size of the training set, a recent trend in the literature, leads to reduced WER despite using noisy transcriptions. The proposed model achieves new state-of-the-art performance on AV-ASR on LRS2 and LRS3. In particular, it achieves a WER of 0.9% on LRS3, a relative improvement of 30% over the current state-of-the-art approach, and outperforms methods that have been trained on non-publicly available datasets with 26 times more training data.

【5】 A Deliberation-based Joint Acoustic and Text Decoder标题：一种基于商议的声文联合解码器链接：http://arxiv.org/pdf/2303.15293v1

作者：Sepand Mavandadi,Tara N. Sainath,Ke Hu,Zelin Wu机构：Google Inc., U.S.A备注：Interspeech 2021摘要：本文提出了一种新的两遍E2 E语音识别模型，该模型通过在成对数据和非成对文本数据的组合上训练来提高ASR性能。以前，联合声学和文本解码器（JATD）已经显示出有前途的结果，通过在模型训练过程中使用的文本数据和最近推出的审议架构，通过利用第一遍解码结果减少了识别错误。我们的方法称为Deliberation-JATD，它将Deliberation-JATD的拼写纠正能力与JATD对不成对文本数据的使用结合起来，以进一步提高性能。所提出的模型在多个测试集上产生了实质性的收益，特别是那些专注于罕见单词的测试集，它将单词错误率（WER）相对降低了12%至22.5%。这是在不增加模型大小或需要多阶段训练的情况下完成的，使得Deliberation-JATD成为设备上应用的有效候选者。摘要：We propose a new two-pass E2E speech recognition model that improves ASR performance by training on a combination of paired data and unpaired text data. Previously, the joint acoustic and text decoder (JATD) has shown promising results through the use of text data during model training and the recently introduced deliberation architecture has reduced recognition errors by leveraging first-pass decoding results. Our method, dubbed Deliberation-JATD, combines the spelling correcting abilities of deliberation with JATD's use of unpaired text data to further improve performance. The proposed model produces substantial gains across multiple test sets, especially those focused on rare words, where it reduces word error rate (WER) by between 12% and 22.5% relative. This is done without increasing model size or requiring multi-stage training, making Deliberation-JATD an efficient candidate for on-device applications.

【6】 Cross-utterance ASR Rescoring with Graph-based Label Propagation标题：基于图的标签传播的串音ASR研究链接：http://arxiv.org/pdf/2303.15132v1

作者：Srinath Tankasala,Long Chen,Andreas Stolcke,Anirudh Raju,Qianli Deng,Chander Chandak,Aparna Khare,Roland Maas,Venkatesh Ravichandran机构：The University of Texas at Austin, USA, Amazon Alexa AI, USA备注：To appear in IEEE ICASSP 2023摘要：我们提出了一种新的方法，利用跨话语声学相似性的ASR N-最好的假设rescoring与基于图的标签传播。与传统的基于神经语言模型（LM）的ASR重新评分 重新排名模型相比，我们的方法侧重于声学信息，并在话语之间协作进行重新评分，而不是单独进行。VCTK数据集上的实验表明，我们的方法不断提高ASR性能，以及不同口音的扬声器组之间的公平性。我们的方法提供了一种低成本的解决方案，以减轻ASR系统的多数偏见，而不需要训练新的域或口音特定的模型。摘要：We propose a novel approach for ASR N-best hypothesis rescoring with graph-based label propagation by leveraging cross-utterance acoustic similarity. In contrast to conventional neural language model (LM) based ASR rescoring reranking models, our approach focuses on acoustic information and conducts the rescoring collaboratively among utterances, instead of individually. Experiments on the VCTK dataset demonstrate that our approach consistently improves ASR performance, as well as fairness across speaker groups with different accents. Our approach provides a low-cost solution for mitigating the majoritarian bias of ASR systems, without the need to train new domain- or accent-specific models.

【7】 Partially Adaptive Multichannel Joint Reduction of Ego-noise and Environmental Noise标题：部分自适应多通道联合抑制自噪声和环境噪声链接：http://arxiv.org/pdf/2303.15042v1

作者：Huajian Fang,Niklas Wittmer,Johannes Twiefel,Stefan Wermter,Timo Gerkmann机构：Signal Processing (SP),Knowledge Technology (WTM), Universit¨at Hamburg, Germany, exXxa GmbH, Hamburg, GermanyJournal-ref：ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)摘要：人机交互依赖于噪声鲁棒音频处理模块，该模块能够从受环境噪声以及自诱导噪声（所谓的自我噪声）影响的音频记录中估计目标语音。虽然外部环境噪声源因环境而异，但自噪声主要由机器人的内部电机和关节引起。自我噪声和环境噪声减少通常是解耦的，即，在不考虑环境噪声的情况下执行自我噪声降低。最近，一个变分自动编码器（VAE）为基础的语音模型已经结合了一个完全自适应的非负矩阵分解（NMF）噪声模型，以恢复干净的语音在不同的环境噪声干扰。然而，其增强性能在涉及例如自我噪声的不利声学场景中受到限制。在本文中，我们提出了一种多通道部分自适应方案，联合模型自我噪声和环境噪声利用VAE-NMF框架，在那里我们利用空间和频谱结构特征的自我噪声预训练的自我噪声模型，同时保留适应未知的环境噪声的能力。实验结果表明，我们提出的方法优于基于一个完全固定的计划和一个完全自适应的计划，当自我噪声和环境噪声同时存在的方法。摘要：Human-robot interaction relies on a noise-robust audio processing module capable of estimating target speech from audio recordings impacted by environmental noise, as well as self-induced noise, so-called ego-noise. While external ambient noise sources vary from environment to environment, ego-noise is mainly caused by the internal motors and joints of a robot. Ego-noise and environmental noise reduction are often decoupled, i.e., ego-noise reduction is performed without considering environmental noise. Recently, a variational autoencoder (VAE)-based speech model has been combined with a fully adaptive non-negative matrix factorization (NMF) noise model to recover clean speech under different environmental noise disturbances. However, its enhancement performance is limited in adverse acoustic scenarios involving, e.g. ego-noise. In this paper, we propose a multichannel partially adaptive scheme to jointly model ego-noise and environmental noise utilizing the VAE-NMF framework, where we take advantage of spatially and spectrally structured characteristics of ego-noise by pre-training the ego-noise model, while retaining the ability to adapt to unknown environmental noise. Experimental results show that our proposed approach outperforms the methods based on a completely fixed scheme and a fully adaptive scheme when ego-noise and environmental noise are present simultaneously.

【8】 Text is All You Need: Personalizing ASR Models using Controllable Speech Synthesis标题：文本就是您需要的一切：使用可控语音合成来个性化ASR模型链接：http://arxiv.org/pdf/2303.14885v1

作者：Karren Yang,Ting-Yao Hu,Jen-Hao Rick Chang,Hema Swetha Koppula,Oncel Tuzel机构：Apple备注：ICASSP 2023摘要：由于个性化数据的稀缺性，使通用语音识别模型适应特定个体是一个具有挑战性的问题。最近的工作已经提出使用个性化的文本到语音合成来提高训练数据的量。在这里，我们要问两个关于这个策略的基本问题：合成数据何时对个性化有效，以及为什么在这些情况下有效？为了解决第一个问题，我们采用了一个国家的最先进的自动语音识别（ASR）模型的目标扬声器从四个基准数据集代表不同的扬声器类型。我们表明，合成数据的ASR个性化在所有情况下都是有效的，但特别是当（i）目标扬声器在全局数据中代表不足，以及（ii）全局模型的容量有限时。为了解决第二个问题，为什么个性化的合成数据是有效的，我们使用可控的语音合成，以产生不同的风格和内容的语音。令人惊讶的是，我们发现，文本内容的合成数据，而不是风格，是重要的发言人适应。这些结果使我们提出了一个数据选择策略的基础上语音内容的ASR个性化。摘要：Adapting generic speech recognition models to specific individuals is a challenging problem due to the scarcity of personalized data. Recent works have proposed boosting the amount of training data using personalized text-to-speech synthesis. Here, we ask two fundamental questions about this strategy: when is synthetic data effective for personalization, and why is it effective in those cases? To address the first question, we adapt a state-of-the-art automatic speech recognition (ASR) model to target speakers from four benchmark datasets representative of different speaker types. We show that ASR personalization with synthetic data is effective in all cases, but particularly when (i) the target speaker is underrepresented in the global data, and (ii) the capacity of the global model is limited. To address the second question of why personalized synthetic data is effective, we use controllable speech synthesis to generate speech with varied styles and content. Surprisingly, we find that the text content of the synthetic data, rather than style, is important for speaker adaptation.
eess.AS音频处理

【1】 A Deliberation-based Joint Acoustic and Text Decoder标题：一种基于商议的声文联合解码器链接：http://arxiv.org/pdf/2303.15293v1

*与cs.SD语音【5】为同一篇

作者：Sepand Mavandadi,Tara N. Sainath,Ke Hu,Zelin Wu机构：Google Inc., U.S.A备注：Interspeech 2021摘要：本文提出了一种新的两遍E2 E语音识别模型，该模型通过在成对数据和非成对文本数据的组合上训练来提高ASR性能。以前，联合声学和文本解码器（JATD）已经显示出有前途的结果，通过在模型训练过程中使用的文本数据和最近推出的审议架构，通过利用第一遍解码结果减少了识别错误。我们的方法称为Deliberation-JATD，它将Deliberation-JATD的拼写纠正能力与JATD对不成对文本数据的使用结合起来，以进一步提高性能。所提出的模型在多个测试集上产生了实质性的收益，特别是那些专注于罕见单词的测试集，它将单词错误率（WER）相对降低了12%至22.5%。这是在不增加模型大小或需要多阶段训练的情况下完成的，使得Deliberation-JATD成为设备上应用的有效候选者。摘要：We propose a new two-pass E2E speech recognition model that improves ASR performance by training on a combination of paired data and unpaired text data. Previously, the joint acoustic and text decoder (JATD) has shown promising results through the use of text data during model training and the recently introduced deliberation architecture has reduced recognition errors by leveraging first-pass decoding results. Our method, dubbed Deliberation-JATD, combines the spelling correcting abilities of deliberation with JATD's use of unpaired text data to further improve performance. The proposed model produces substantial gains across multiple test sets, especially those focused on rare words, where it reduces word error rate (WER) by between 12% and 22.5% relative. This is done without increasing model size or requiring multi-stage training, making Deliberation-JATD an efficient candidate for on-device applications.

【2】 Cross-utterance ASR Rescoring with Graph-based Label Propagation标题：基于图的标签传播的串音ASR研究链接：http://arxiv.org/pdf/2303.15132v1

*与cs.SD语音【6】为同一篇

作者：Srinath Tankasala,Long Chen,Andreas Stolcke,Anirudh Raju,Qianli Deng,Chander Chandak,Aparna Khare,Roland Maas,Venkatesh Ravichandran机构：The University of Texas at Austin, USA, Amazon Alexa AI, USA备注：To appear in IEEE ICASSP 2023摘要：我们提出了一种新的方法，利用跨话语声学相似性的ASR N-最好的假设rescoring与基于图的标签传播。与传统的基于神经语言模型（LM）的ASR重新评分 重新排名模型相比，我们的方法侧重于声学信息，并在话语之间协作进行重新评分，而不是单独进行。VCTK数据集上的实验表明，我们的方法不断提高ASR性能，以及不同口音的扬声器组之间的公平性。我们的方法提供了一种低成本的解决方案，以减轻ASR系统的多数偏见，而不需要训练新的域或口音特定的模型。摘要：We propose a novel approach for ASR N-best hypothesis rescoring with graph-based label propagation by leveraging cross-utterance acoustic similarity. In contrast to conventional neural language model (LM) based ASR rescoring reranking models, our approach focuses on acoustic information and conducts the rescoring collaboratively among utterances, instead of individually. Experiments on the VCTK dataset demonstrate that our approach consistently improves ASR performance, as well as fairness across speaker groups with different accents. Our approach provides a low-cost solution for mitigating the majoritarian bias of ASR systems, without the need to train new domain- or accent-specific models.

【3】 Partially Adaptive Multichannel Joint Reduction of Ego-noise and Environmental Noise标题：部分自适应多通道联合抑制自噪声和环境噪声链接：http://arxiv.org/pdf/2303.15042v1

*与cs.SD语音【7】为同一篇

作者：Huajian Fang,Niklas Wittmer,Johannes Twiefel,Stefan Wermter,Timo Gerkmann机构：Signal Processing (SP),Knowledge Technology (WTM), Universit¨at Hamburg, Germany, exXxa GmbH, Hamburg, GermanyJournal-ref：ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)摘要：人机交互依赖于噪声鲁棒音频处理模块，该模块能够从受环境噪声以及自诱导噪声（所谓的自我噪声）影响的音频记录中估计目标语音。虽然外部环境噪声源因环境而异，但自噪声主要由机器人的内部电机和关节引起。自我噪声和环境噪声减少通常是解耦的，即，在不考虑环境噪声的情况下执行自我噪声降低。最近，一个变分自动编码器（VAE）为基础的语音模型已经结合了一个完全自适应的非负矩阵分解（NMF）噪声模型，以恢复干净的语音在不同的环境噪声干扰。然而，其增强性能在涉及例如自我噪声的不利声学场景中受到限制。在本文中，我们提出了一种多通道部分自适应方案，联合模型自我噪声和环境噪声利用VAE-NMF框架，在那里我们利用空间和频谱结构特征的自我噪声预训练的自我噪声模型，同时保留适应未知的环境噪声的能力。实验结果表明，我们提出的方法优于基于一个完全固定的计划和一个完全自适应的计划，当自我噪声和环境噪声同时存在的方法。摘要：Human-robot interaction relies on a noise-robust audio processing module capable of estimating target speech from audio recordings impacted by environmental noise, as well as self-induced noise, so-called ego-noise. While external ambient noise sources vary from environment to environment, ego-noise is mainly caused by the internal motors and joints of a robot. Ego-noise and environmental noise reduction are often decoupled, i.e., ego-noise reduction is performed without considering environmental noise. Recently, a variational autoencoder (VAE)-based speech model has been combined with a fully adaptive non-negative matrix factorization (NMF) noise model to recover clean speech under different environmental noise disturbances. However, its enhancement performance is limited in adverse acoustic scenarios involving, e.g. ego-noise. In this paper, we propose a multichannel partially adaptive scheme to jointly model ego-noise and environmental noise utilizing the VAE-NMF framework, where we take advantage of spatially and spectrally structured characteristics of ego-noise by pre-training the ego-noise model, while retaining the ability to adapt to unknown environmental noise. Experimental results show that our proposed approach outperforms the methods based on a completely fixed scheme and a fully adaptive scheme when ego-noise and environmental noise are present simultaneously.

【4】 Text is All You Need: Personalizing ASR Models using Controllable Speech Synthesis标题：文本就是您需要的一切：使用可控语音合成来个性化ASR模型链接：http://arxiv.org/pdf/2303.14885v1

*与cs.SD语音【8】为同一篇

作者：Karren Yang,Ting-Yao Hu,Jen-Hao Rick Chang,Hema Swetha Koppula,Oncel Tuzel机构：Apple备注：ICASSP 2023摘要：由于个性化数据的稀缺性，使通用语音识别模型适应特定个体是一个具有挑战性的问题。最近的工作已经提出使用个性化的文本到语音合成来提高训练数据的量。在这里，我们要问两个关于这个策略的基本问题：合成数据何时对个性化有效，以及为什么在这些情况下有效？为了解决第一个问题，我们采用了一个国家的最先进的自动语音识别（ASR）模型的目标扬声器从四个基准数据集代表不同的扬声器类型。我们表明，合成数据的ASR个性化在所有情况下都是有效的，但特别是当（i）目标扬声器在全局数据中代表不足，以及（ii）全局模型的容量有限时。为了解决第二个问题，为什么个性化的合成数据是有效的，我们使用可控的语音合成，以产生不同的风格和内容的语音。令人惊讶的是，我们发现，文本内容的合成数据，而不是风格，是重要的发言人适应。这些结果使我们提出了一个数据选择策略的基础上语音内容的ASR个性化。摘要：Adapting generic speech recognition models to specific individuals is a challenging problem due to the scarcity of personalized data. Recent works have proposed boosting the amount of training data using personalized text-to-speech synthesis. Here, we ask two fundamental questions about this strategy: when is synthetic data effective for personalization, and why is it effective in those cases? To address the first question, we adapt a state-of-the-art automatic speech recognition (ASR) model to target speakers from four benchmark datasets representative of different speaker types. We show that ASR personalization with synthetic data is effective in all cases, but particularly when (i) the target speaker is underrepresented in the global data, and (ii) the capacity of the global model is limited. To address the second question of why personalized synthetic data is effective, we use controllable speech synthesis to generate speech with varied styles and content. Surprisingly, we find that the text content of the synthetic data, rather than style, is important for speaker adaptation.

【5】 Pitchclass2vec: Symbolic Music Structure Segmentation with Chord Embeddings标题：PitchClass2vec：基于和弦嵌入的符号音乐结构分割链接：http://arxiv.org/pdf/2303.15306v1

*与cs.SD语音【1】为同一篇

作者：Nicolas Lazzari,Andrea Poltronieri,Valentina Presutti机构：Department of Computer Science and Engineering, University of Bologna, Mura Anteo Zamboni, Bologna , Italy, LILEC, University of Bologna, Via Cartoleria, Bologna , ItalyJournal-ref：Proceedings of the 1st Workshop on Artificial Intelligence and Creativity co-located with 21th International Conference of the Italian Association for Artificial Intelligence(AIxIA 2022), Udine, Italy, November 28 - December 3, 2022摘要：结构知觉是人类音乐认知的一个基本方面。从历史上看，音乐的结构层次组织作为一种叙事手段，用于传达意义，创造期望，唤起听众的情感。因此，音乐结构在音乐创作中起着至关重要的作用，因为它们塑造了作曲家组织其思想的音乐话语。在本文中，我们提出了一种新的音乐分割方法，pitchclass2vec，基于符号和弦注释，这是嵌入到连续的矢量表示使用自然语言处理技术和定制的编码。我们的算法基于长短期记忆（LSTM）神经网络，并且优于该领域中基于符号和弦注释的最先进技术。摘要：Structure perception is a fundamental aspect of music cognition in humans. Historically, the hierarchical organization of music into structures served as a narrative device for conveying meaning, creating expectancy, and evoking emotions in the listener. Thereby, musical structures play an essential role in music composition, as they shape the musical discourse through which the composer organises his ideas. In this paper, we present a novel music segmentation method, pitchclass2vec, based on symbolic chord annotations, which are embedded into continuous vector representations using both natural language processing techniques and custom-made encodings. Our algorithm is based on long-short term memory (LSTM) neural network and outperforms the state-of-the-art techniques based on symbolic chord annotations in the field.

【6】 Data Augmentation for Environmental Sound Classification Using Diffusion Probabilistic Model with Top-k Selection Discriminator标题：基于Top-k选择判别的扩散概率模型用于环境声分类的数据增强链接：http://arxiv.org/pdf/2303.15161v1

*与cs.SD语音【2】为同一篇

作者：Yunhao Chen,Yunjie Zhu,Zihui Yan,Jianlu Shen,Zhen Ren,Yifan Huang机构：Jiangnan University, Wuxi, China, University of Leeds, Leeds, LS,JT, United Kingdom摘要：尽管近年来强大的深度学习技术不断进步，但模型仍然需要大量的训练数据来避免过度拟合。最近已经产生了使用生成对抗网络（GAN）的合成数据集来克服这个问题。然而，尽管取得了进步，但基于GAN的方法通常难以训练或无法生成高质量的数据样本。在本文中，我们提出了一种环境声音分类增强技术的基础上的扩散概率模型与DPM-Solver$++$快速采样。此外，为了确保生成的频谱图的质量，我们在数据集上训练了一个top-k选择鉴别器。根据实验结果，合成的频谱图具有相似的功能，以原始数据集，并可以显着提高不同的国家的最先进的模型的分类精度相比，传统的数据增强技术。公共代码可在 url{https：github.com JNAIC DPMs-for-Audio-Data-Augmentation}上获得。摘要：Despite consistent advancement in powerful deep learning techniques in recent years, large amounts of training data are still necessary for the models to avoid overfitting. Synthetic datasets using generative adversarial networks (GAN) have recently been generated to overcome this problem. Nevertheless, despite advancements, GAN-based methods are usually hard to train or fail to generate high-quality data samples. In this paper, we propose an environmental sound classification augmentation technique based on the diffusion probabilistic model with DPM-Solver$++$ for fast sampling. In addition, to ensure the quality of the generated spectrograms, we train a top-k selection discriminator on the dataset. According to the experiment results, the synthesized spectrograms have similar features to the original dataset and can significantly increase the classification accuracy of different state-of-the-art models compared with traditional data augmentation techniques. The public code is available on url{https: github.com JNAIC DPMs-for-Audio-Data-Augmentation}.

【7】 Time-domain Speech Enhancement Assisted by Multi-resolution Frequency Encoder and Decoder标题：多分辨率频率编解码器辅助的时域语音增强链接：http://arxiv.org/pdf/2303.14593v1

*与cs.SD语音【3】为同一篇

作者：Hao Shi,Masato Mimura,Longbiao Wang,Jianwu Dang,Tatsuya Kawahara机构：Graduate School of Informatics, Kyoto University, Kyoto, Japan, Tianjin Key Laboratory of Cognitive Computing and Application, College of Intelligence and Computing, Tianjin University, Tianjin, China摘要：时域语音增强（SE）最近得到了广泛的研究。在最近的作品中，DEMUCS引入了多分辨率STFT损失，以提高性能。然而，用于STFT的一些分辨率包含非平稳信号，并且仅用一个输出同时学习多分辨率频率损失是具有挑战性的。为了更好地利用多分辨率的频率信息，我们补充多个频谱在不同的帧长度到时域编码器。它们提取窄带和宽带中的稳定频率信息。我们还采用了多个解码器输出，每个输出计算其相应的分辨率频率损失。实验结果表明：（1）在编码器中融合平稳频率特征比融合非平稳特征更有效;（2）多个输出与频率损失一致，提高了性能。在Voice-Bank数据集上的实验结果表明，该方法的PESQ提高了0.14。摘要：Time-domain speech enhancement (SE) has recently been intensively investigated. Among recent works, DEMUCS introduces multi-resolution STFT loss to enhance performance. However, some resolutions used for STFT contain non-stationary signals, and it is challenging to learn multi-resolution frequency losses simultaneously with only one output. For better use of multi-resolution frequency information, we supplement multiple spectrograms in different frame lengths into the time-domain encoders. They extract stationary frequency information in both narrowband and wideband. We also adopt multiple decoder outputs, each of which computes its corresponding resolution frequency loss. Experimental results show that (1) it is more effective to fuse stationary frequency features than non-stationary features in the encoder, and (2) the multiple outputs consistent with the frequency loss improve performance. Experiments on the Voice-Bank dataset show that the proposed method obtained a 0.14 PESQ improvement.

【8】 Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels标题：Auto-AVSR：带自动标签的视听语音识别链接：http://arxiv.org/pdf/2303.14307v1

*与cs.SD语音【4】为同一篇

作者：Pingchuan Ma,Alexandros Haliassos,Adriana Fernandez-Lopez,Honglie Chen,Stavros Petridis,Maja Pantic机构：Imperial College London, UK, Meta AI, UK备注：Accepted to ICASSP 2023摘要：视听语音识别由于其对声学噪声的鲁棒性而受到了广泛的关注。最近，自动、视觉和视听语音识别（分别为ASR、VSR和AV-ASR）的性能已经得到了实质性的改善，这主要是由于使用了更大的模型和训练集。然而，数据集的准确标记是耗时且昂贵的。因此，在这项工作中，我们研究了使用自动生成的转录的未标记的数据集，以增加训练集的大小。为此，我们使用公开的预训练ASR模型来自动转录未标记的数据集，如AVSpeech和VoxCeleb 2。然后，我们在由LRS 2和LRS 3数据集以及额外的自动转录数据组成的增强训练集上训练ASR、VSR和AV-ASR模型。我们证明，增加训练集的大小，最近的趋势，在文献中，导致减少WER，尽管使用嘈杂的转录。所提出的模型在LRS 2和LRS 3上实现了新的最先进的AV-ASR性能。特别是，它在LRS 3上实现了0.9%的WER，比当前最先进的方法相对提高了30%，并且优于在非公开数据集上训练的方法，训练数据是其他方法的26倍。摘要：Audio-visual speech recognition has received a lot of attention due to its robustness against acoustic noise. Recently, the performance of automatic, visual, and audio-visual speech recognition (ASR, VSR, and AV-ASR, respectively) has been substantially improved, mainly due to the use of larger models and training sets. However, accurate labelling of datasets is time-consuming and expensive. Hence, in this work, we investigate the use of automatically-generated transcriptions of unlabelled datasets to increase the training set size. For this purpose, we use publicly-available pre-trained ASR models to automatically transcribe unlabelled datasets such as AVSpeech and VoxCeleb2. Then, we train ASR, VSR and AV-ASR models on the augmented training set, which consists of the LRS2 and LRS3 datasets as well as the additional automatically-transcribed data. We demonstrate that increasing the size of the training set, a recent trend in the literature, leads to reduced WER despite using noisy transcriptions. The proposed model achieves new state-of-the-art performance on AV-ASR on LRS2 and LRS3. In particular, it achieves a WER of 0.9% on LRS3, a relative improvement of 30% over the current state-of-the-art approach, and outperforms methods that have been trained on non-publicly available datasets with 26 times more training data.

机器翻译，仅供参考