今日论文合集：cs.SD语音9篇，eess.AS音频处理9篇。
【1】 Label-free Knowledge Distillation with Contrastive Loss for Light-weight Speaker Recognition
标题：用于轻量级说话人识别的对比度损失无标签知识提取
链接：http://arxiv.org/pdf/2212.03090v1
作者：Zhiyuan Peng,Xuanji He,Ke Ding,Tan Lee,Guanglu Wan机构：Department of Electronic Engineering, The Chinese University of Hong Kong, Meituan摘要：近年来，说话人识别的超深层模型在说话人识别中的应用取得了显著的效果。然而，对于具有受限计算资源的设备上应用部署这些模型是不切实际的。另一方面，尽管轻量模型的性能不是最佳的，但在实践中它们是非常需要的。本研究旨在通过大规模无标签知识提取（KD）来改进轻量级SR模型。现有的识别方法通常需要说话人标签来学习特定任务的知识，这是由于传统的提取损失效率低。为了解决这一问题，实现无标签知识发现，我们提出利用自监督学习的对比损失进行提取。在不同来源的公共语音数据集上进行了广泛的实验。在轻量级SR模型上的实验结果表明，所提出的无标签对比损失KD方法的性能明显优于传统蒸馏方法和自监督学习方法.摘要：Very deep models for speaker recognition (SR) have demonstrated remarkable performance improvement in recent research. However, it is impractical to deploy these models for on-device applications with constrained computational resources. On the other hand, light-weight models are highly desired in practice despite their sub-optimal performance. This research aims to improve light-weight SR models through large-scale label-free knowledge distillation (KD). Existing KD approaches for SR typically require speaker labels to learn task-specific knowledge, due to the inefficiency of conventional loss for distillation. To address the inefficiency problem and achieve label-free KD, we propose to employ the contrastive loss from self-supervised learning for distillation. Extensive experiments are conducted on a collection of public speech datasets from diverse sources. Results on light-weight SR models show that the proposed approach of label-free KD with contrastive loss consistently outperforms both conventional distillation methods and self-supervised learning methods by a significant margin.
【2】 Covariance Regularization for Probabilistic Linear Discriminant Analysis
标题：概率线性判别分析中的协方差正则化
链接：http://arxiv.org/pdf/2212.03039v1
作者：Zhiyuan Peng,Mingjie Shao,Xuanji He,Xu Li,Tan Lee,Ke Ding,Guanglu Wan机构：Department of Electronic Engineering, The Chinese University of Hong Kong, Meituan, ARC Lab, Tencent PCG摘要：概率线性判别分析（PLDA）是说话人确认系统中常用的一种说话人嵌入相似度计算方法。最近的研究通过对角化其协方差来改善PLDA在域匹配条件下的性能。我们怀疑这种残酷的修剪方法会消除其对说话人嵌入的维度相关性建模的能力，从而导致域自适应性能不足。本文提出了两种可供选择的协方差正则化方法，即插值PLDA和稀疏PLDA，以解决该问题。插值后的PLDA结合余弦评分的先验知识对PLDA的协方差进行插值。稀疏PLDA引入稀疏性惩罚来更新协方差。实验结果表明，两种方法的性能都明显优于对角正则化方法。此外，当训练稀疏PLDA用于域自适应时，可以显著减少域内数据。摘要：Probabilistic linear discriminant analysis (PLDA) is commonly used in speaker verification systems to score the similarity of speaker embeddings. Recent studies improved the performance of PLDA in domain-matched conditions by diagonalizing its covariance. We suspect such brutal pruning approach could eliminate its capacity in modeling dimension correlation of speaker embeddings, leading to inadequate performance with domain adaptation. This paper explores two alternative covariance regularization approaches, namely, interpolated PLDA and sparse PLDA, to tackle the problem. The interpolated PLDA incorporates the prior knowledge from cosine scoring to interpolate the covariance of PLDA. The sparse PLDA introduces a sparsity penalty to update the covariance. Experimental results demonstrate that both approaches outperform diagonal regularization noticeably with domain adaptation. In addition, in-domain data can be significantly reduced when training sparse PLDA for domain adaptation.
【3】 Parameter Efficient Transfer Learning for Various Speech Processing Tasks
标题：不同语音处理任务的参数高效迁移学习
链接：http://arxiv.org/pdf/2212.02780v1
作者：Shinta Otake,Rei Kawakami,Nakamasa Inoue机构：Tokyo Institute of Technology摘要：自监督模型的微调是包括语音处理在内的各种领域中的一种强有力的迁移学习方法，因为它可以利用从大量未标记数据中获得的通用特征表示。然而，微调需要为每个下游任务设置新的参数，这是参数低效的。适配器体系结构通过在冻结的预训练模型中插入轻量级的可学习模块来部分解决这一问题。然而，现有的适配器体系结构不能自适应地利用存储在不同层中的低到高级别特征，这对于解决各种语音处理任务是必要的。因此，我们提出了一种新的适配器体系结构，以针对各种语音任务更灵活地获取特征表示。在实验中，我们将该适配器应用于四个语音任务的WavLM。它的性能与原始微调相当或更好，只有11%的可学习参数。它的性能也超过了现有的适配器体系结构。摘要：Fine-tuning of self-supervised models is a powerful transfer learning method in a variety of fields, including speech processing, since it can utilize generic feature representations obtained from large amounts of unlabeled data. Fine-tuning, however, requires a new parameter set for each downstream task, which is parameter inefficient. Adapter architecture is proposed to partially solve this issue by inserting lightweight learnable modules into a frozen pre-trained model. However, existing adapter architectures fail to adaptively leverage low- to high-level features stored in different layers, which is necessary for solving various kinds of speech processing tasks. Thus, we propose a new adapter architecture to acquire feature representations more flexibly for various speech tasks. In experiments, we applied this adapter to WavLM on four speech tasks. It performed on par or better than naive fine-tuning, with only 11% of learnable parameters. It also outperformed an existing adapter architecture.
【4】 Audio Latent Space Cartography
标题：音频潜伏空间制图
链接：http://arxiv.org/pdf/2212.02610v1
作者：Nicolas Jonason,Bob L. T. Sturm机构：KTH Royal Institute of Technology备注：Late Breaking Demo, ISMIR 2022 (this https URL)摘要：我们使用音频到图像生成管道来探索音频潜在空间的可视化生成。我们相信这有助于音频潜在空间的可解释性。我们在NSynth数据集上展示了各种结果。有一个Web演示可用。摘要：We explore the generation of visualisations of audio latent spaces using an audio-to-image generation pipeline. We believe this can help with the interpretability of audio latent spaces. We demonstrate a variety of results on the NSynth dataset. A web demo is available.
【5】 MAP-Music2Vec: A Simple and Effective Baseline for Self-Supervised Music Audio Representation Learning
标题：MAP-Music2Vec：一种简单有效的自我监督音乐音频表征学习基线
链接：http://arxiv.org/pdf/2212.02508v1
作者：Yizhi Li,Ruibin Yuan,Ge Zhang,Yinghao Ma,Chenghua Lin,Xingran Chen,Anton Ragni,Hanzhi Yin,Zhijie Hu,Haoyu He,Emmanouil Benetos,Norbert Gyenge,Ruibo Liu,Jie Fu机构：Department of Computer Science, University of Sheffield, UK, Beijing Academy of Artificial Intelligence, China, Centre for Digital Music, Queen Mary University of London, UK, School of Music, Carnegie Mellon University, PA, USA摘要：深度学习社区对自监督学习（SSL）的兴趣呈指数级增长。然而，如何建立用于以自监督方式学习原始音乐波形的有用表示的框架仍然是未探索的。在本文中，我们设计了Music2Vec，这是一个探索不同SSL算法组件和音乐音频录制技巧的框架。我们的模型实现了与最先进的（SOTA）音乐SSL模型Jukebox相当的结果，尽管显著地更小，具有少于后者的2%的参数。该模型将在Huggingface上发布（请参考：https: huggingface.co m-a-p music2vec-v1)摘要：The deep learning community has witnessed an exponentially growing interest in self-supervised learning (SSL). However, it still remains unexplored how to build a framework for learning useful representations of raw music waveforms in a self-supervised manner. In this work, we design Music2Vec, a framework exploring different SSL algorithmic components and tricks for music audio recordings. Our model achieves comparable results to the state-of-the-art (SOTA) music SSL model Jukebox, despite being significantly smaller with less than 2% of parameters of the latter. The model will be released on Huggingface(Please refer to: https: huggingface.co m-a-p music2vec-v1)
【6】 FretNet: Continuous-Valued Pitch Contour Streaming for Polyphonic Guitar Tablature Transcription
标题：FretNet：用于复调吉他音型转录的连续值基音轮廓流
链接：http://arxiv.org/pdf/2212.03023v1
作者：Frank Cwitkowitz,Toni Hirvonen,Anssi Klapuri机构：University of Rochester,Yousician备注：Submitted to ICASSP 2023摘要：近年来，从音频估计音符的各种属性的自动音乐转录（AMT）的任务已经受到越来越多的关注。同时，多基音估计（MPE）的相关任务仍然是几乎所有AMT方法的一个具有挑战性但必要的组成部分，即使只是隐含的。在AMT的上下文中，音调信息通常被量化为西方音阶的标称音调。甚至在更一般的情况下，MPE系统通常产生具有某种量化程度的音调预测。在AMT的某些应用中，如吉他谱记谱法转录（GTT），估计连续值的音高轮廓更有意义。吉他指法谱有能力代表各种演奏技巧，其中一些涉及音高调制。AMT的当代方法不足以解决音调调制，并且以更高的模型复杂性为代价仅提供更少的量化。在这篇文章中，我们提出了一个GTT公式来估计连续值的音高轮廓，根据它们的弦和原始音档来分组。我们证明，对于这个任务，所提出的方法显著提高了MPE的分辨率，同时产生与基线模型竞争的谱估计结果。摘要：In recent years, the task of Automatic Music Transcription (AMT), whereby various attributes of music notes are estimated from audio, has received increasing attention. At the same time, the related task of Multi-Pitch Estimation (MPE) remains a challenging but necessary component of almost all AMT approaches, even if only implicitly. In the context of AMT, pitch information is typically quantized to the nominal pitches of the Western music scale. Even in more general contexts, MPE systems typically produce pitch predictions with some degree of quantization. In certain applications of AMT, such as Guitar Tablature Transcription (GTT), it is more meaningful to estimate continuous-valued pitch contours. Guitar tablature has the capacity to represent various playing techniques, some of which involve pitch modulation. Contemporary approaches to AMT do not adequately address pitch modulation, and offer only less quantization at the expense of more model complexity. In this paper, we present a GTT formulation that estimates continuous-valued pitch contours, grouping them according to their string and fret of origin. We demonstrate that for this task, the proposed method significantly improves the resolution of MPE and simultaneously yields tablature estimation results competitive with baseline models.
【7】 BC-VAD: A Robust Bone Conduction Voice Activity Detection
标题：BC-VAD：一种健壮的骨传导语音活动检测方法
链接：http://arxiv.org/pdf/2212.02996v1
作者：Niccolo' Polvani,Damien Ronssin,Milos Cernak机构：´Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Lausanne, Switzerland, Logitech Europe, Lausanne, Switzerland摘要：语音活动检测（VAD）是许多音频应用中的基本模块。最新的VAD系统通常基于神经网络，但它们需要的计算预算通常超过了小型电池供电设备的能力，同时保持了较大模型的性能。在这项工作中，我们依赖于来自骨导麦克风（BCM）的输入来设计一个有效的VAD（BC-VAD），它对来自环境或未佩戴BCM的说话者的残余非平稳噪声具有鲁棒性。我们首先证明了一个更大的VAD系统（58k参数）在公共可用的基准测试中获得了最先进的结果，但在骨导信号上运行时失败。然后，我们将其变体BC-VAD（5k个参数并在BC数据上训练）与专门为BCM设计的基线进行了比较，并表明所提出的方法在保持微控制器的实时处理要求的同时，在各种度量下获得了更好的性能。摘要：Voice Activity Detection (VAD) is a fundamental module in many audio applications. Recent state-of-the-art VAD systems are often based on neural networks, but they require a computational budget that usually exceeds the capabilities of a small battery-operated device when preserving the performance of larger models. In this work, we rely on the input from a bone conduction microphone (BCM) to design an efficient VAD (BC-VAD) robust against residual non-stationary noises originating from the environment or speakers not wearing the BCM.We first show that a larger VAD system (58k parameters) achieves state-of-the-art results on a publicly available benchmark but fails when running on bone conduction signals. We then compare its variant BC-VAD (5k parameters and trained on BC data) with a baseline especially designed for a BCM and show that the proposed method achieves better performances under various metrics while keeping the realtime processing requirement for a microcontroller.
【8】 Self-Supervised Audio-Visual Speech Representations Learning By Multimodal Self-Distillation
标题：基于多通道自蒸馏的自监督视听语音表征学习
链接：http://arxiv.org/pdf/2212.02782v1
作者：Jing-Xuan Zhang,Genshun Wan,Zhen-Hua Ling,Jia Pan,Jianqing Gao,Cong Liu机构：University of Science and Technology of China, China, FLYTEK Research, iFLYTEK Co. Ltd., China备注：submitted to ICASSP 2023摘要：本文提出了一种基于多模态自提取的视听语音学习方法AV 2 vec。AV 2 vec具有学生和教师模块，其中学生使用由教师在线生成的多模态目标特征执行掩蔽潜在特征回归任务。教师模型的参数是学生的动量更新。由于我们的目标特征是在线生成的，AV 2 vec不需要像AV-HuBERT那样的迭代步骤，总训练时间成本减少到不到五分之一。我们在本研究中进一步提出AV 2 vec-MLM，它使用多任务学习以掩蔽语言模型（MLM）式损失来增强AV 2 vec。我们的实验结果表明，AV 2 vec实现了与AV-HuBERT基线相当的性能。当与MLM类型的损失相结合时，AV 2 vec-MLM的表现优于基线，并在下游任务中实现了最佳性能。摘要：In this work, we present a novel method, named AV2vec, for learning audio-visual speech representations by multimodal self-distillation. AV2vec has a student and a teacher module, in which the student performs a masked latent feature regression task using the multimodal target features generated online by the teacher. The parameters of the teacher model are a momentum update of the student. Since our target features are generated online, AV2vec needs no iteration step like AV-HuBERT and the total training time cost is reduced to less than one-fifth. We further propose AV2vec-MLM in this study, which augments AV2vec with a masked language model (MLM)-style loss using multitask learning. Our experimental results show that AV2vec achieved comparable performance to the AV-HuBERT baseline. When combined with an MLM-style loss, AV2vec-MLM outperformed baselines and achieved the best performance on the downstream tasks.
【9】 Standard sound emergence as a predictor of short-term annoyance from wind turbine noise
标题：标准声音出现作为风力机噪声短期烦扰的预测指标
链接：http://arxiv.org/pdf/2212.02616v1
作者：Elise Ruaud,Guillaume Dutilleux机构：Acoustics Group, Department of Electronic Systems, NTNU, Trondheim, Norway a)备注：Submitted to the Journal or the Acoustical Society of America. 17 pages, 8 figures, 3 tables摘要：虽然一些国家使用声出现来规范风能开发，但没有公开的证据表明它是用于此目的的相关噪声描述符。在本研究中，我们进行了两次听力测试，以评估声音出现的优点。考虑了声音出现的三种定义：ISO 1996-1中的标准，可听度条件下的声涌现$e_{UAC}$和频谱涌现$e_{SP}$。我们还考虑了具体的残差比和响度度量。在每一次听力测试中，声刺激包括48个声刺激，分别为3个A计权声压级30、40、50 dB和4个比剩余比-10、-5、0、+5dB。结果得出的结论是，无论对声音出现的定义如何，总声压级比声音出现更好地预测短期烦恼度，或者比剩余噪声比更好地预测短期烦恼度。$e_{UAC}$比$e$对短期烦恼的预测稍好，而$e$比$e_{SP}$更好。总声压级和响度指标表现相似。此外，该结果提供了证据，即声音出现是风力涡轮机声音可听度的不良预测器。摘要：While sound emergence is used in several countries to regulate wind energy development, there is no published evidence that it is a relevant noise descriptor for this purpose. In the present work, we carried out two listening tests to evaluate the merits of sound emergence. Three definitions of sound emergence were considered: the one in ISO 1996-1, sound emergence under audibility condition $e_{UAC}$, and spectral emergence $e_{SP}$. We also considered the specific to residual ratio and loudness metrics. In each listening test, the sound stimuli consisted of 48 sound stimuli at 3 A-weighted sound pressure levels $ {30, 40, 50 }$~dB and 4 specific-to-residual ratios $ {-10, -5, 0, +5 }$~dB. The results lead to the conclusion that short term annoyance is better predicted by the total sound pressure level than by sound emergence, whatever the definition considered for the latter, or than by the specific to residual ratio. Short-term annoyance is slightly better predicted by $e_{UAC}$ than by $e$, while $e$ is a better predictor than $e_{SP}$. The total sound pressure level and the loudness metrics performed similarly. Furthermore, the results provide evidence that sound emergence is a poor predictor of the audibility of wind turbine sounds.
【1】 FretNet: Continuous-Valued Pitch Contour Streaming for Polyphonic Guitar Tablature Transcription
标题：FretNet：用于复调吉他音型转录的连续值基音轮廓流
链接：http://arxiv.org/pdf/2212.03023v1
*与cs.SD语音【6】为同一篇
作者：Frank Cwitkowitz,Toni Hirvonen,Anssi Klapuri机构：University of Rochester,Yousician备注：Submitted to ICASSP 2023摘要：近年来，从音频估计音符的各种属性的自动音乐转录（AMT）的任务已经受到越来越多的关注。同时，多基音估计（MPE）的相关任务仍然是几乎所有AMT方法的一个具有挑战性但必要的组成部分，即使只是隐含的。在AMT的上下文中，音调信息通常被量化为西方音阶的标称音调。甚至在更一般的情况下，MPE系统通常产生具有某种量化程度的音调预测。在AMT的某些应用中，如吉他谱记谱法转录（GTT），估计连续值的音高轮廓更有意义。吉他指法谱有能力代表各种演奏技巧，其中一些涉及音高调制。AMT的当代方法不足以解决音调调制，并且以更高的模型复杂性为代价仅提供更少的量化。在这篇文章中，我们提出了一个GTT公式来估计连续值的音高轮廓，根据它们的弦和原始音档来分组。我们证明，对于这个任务，所提出的方法显著提高了MPE的分辨率，同时产生与基线模型竞争的谱估计结果。摘要：In recent years, the task of Automatic Music Transcription (AMT), whereby various attributes of music notes are estimated from audio, has received increasing attention. At the same time, the related task of Multi-Pitch Estimation (MPE) remains a challenging but necessary component of almost all AMT approaches, even if only implicitly. In the context of AMT, pitch information is typically quantized to the nominal pitches of the Western music scale. Even in more general contexts, MPE systems typically produce pitch predictions with some degree of quantization. In certain applications of AMT, such as Guitar Tablature Transcription (GTT), it is more meaningful to estimate continuous-valued pitch contours. Guitar tablature has the capacity to represent various playing techniques, some of which involve pitch modulation. Contemporary approaches to AMT do not adequately address pitch modulation, and offer only less quantization at the expense of more model complexity. In this paper, we present a GTT formulation that estimates continuous-valued pitch contours, grouping them according to their string and fret of origin. We demonstrate that for this task, the proposed method significantly improves the resolution of MPE and simultaneously yields tablature estimation results competitive with baseline models.
【2】 BC-VAD: A Robust Bone Conduction Voice Activity Detection
标题：BC-VAD：一种健壮的骨传导语音活动检测方法
链接：http://arxiv.org/pdf/2212.02996v1
*与cs.SD语音【7】为同一篇
作者：Niccolo' Polvani,Damien Ronssin,Milos Cernak机构：´Ecole Polytechnique F´ed´erale de Lausanne (EPFL), Lausanne, Switzerland, Logitech Europe, Lausanne, Switzerland摘要：语音活动检测（VAD）是许多音频应用中的基本模块。最新的VAD系统通常基于神经网络，但它们需要的计算预算通常超过了小型电池供电设备的能力，同时保持了较大模型的性能。在这项工作中，我们依赖于来自骨导麦克风（BCM）的输入来设计一个有效的VAD（BC-VAD），它对来自环境或未佩戴BCM的说话者的残余非平稳噪声具有鲁棒性。我们首先证明了一个更大的VAD系统（58k参数）在公共可用的基准测试中获得了最先进的结果，但在骨导信号上运行时失败。然后，我们将其变体BC-VAD（5k个参数并在BC数据上训练）与专门为BCM设计的基线进行了比较，并表明所提出的方法在保持微控制器的实时处理要求的同时，在各种度量下获得了更好的性能。摘要：Voice Activity Detection (VAD) is a fundamental module in many audio applications. Recent state-of-the-art VAD systems are often based on neural networks, but they require a computational budget that usually exceeds the capabilities of a small battery-operated device when preserving the performance of larger models. In this work, we rely on the input from a bone conduction microphone (BCM) to design an efficient VAD (BC-VAD) robust against residual non-stationary noises originating from the environment or speakers not wearing the BCM.We first show that a larger VAD system (58k parameters) achieves state-of-the-art results on a publicly available benchmark but fails when running on bone conduction signals. We then compare its variant BC-VAD (5k parameters and trained on BC data) with a baseline especially designed for a BCM and show that the proposed method achieves better performances under various metrics while keeping the realtime processing requirement for a microcontroller.
【3】 Self-Supervised Audio-Visual Speech Representations Learning By Multimodal Self-Distillation
标题：基于多通道自蒸馏的自监督视听语音表征学习
链接：http://arxiv.org/pdf/2212.02782v1
*与cs.SD语音【8】为同一篇
作者：Jing-Xuan Zhang,Genshun Wan,Zhen-Hua Ling,Jia Pan,Jianqing Gao,Cong Liu机构：University of Science and Technology of China, China, FLYTEK Research, iFLYTEK Co. Ltd., China备注：submitted to ICASSP 2023摘要：本文提出了一种基于多模态自提取的视听语音学习方法AV 2 vec。AV 2 vec具有学生和教师模块，其中学生使用由教师在线生成的多模态目标特征执行掩蔽潜在特征回归任务。教师模型的参数是学生的动量更新。由于我们的目标特征是在线生成的，AV 2 vec不需要像AV-HuBERT那样的迭代步骤，总训练时间成本减少到不到五分之一。我们在本研究中进一步提出AV 2 vec-MLM，它使用多任务学习以掩蔽语言模型（MLM）式损失来增强AV 2 vec。我们的实验结果表明，AV 2 vec实现了与AV-HuBERT基线相当的性能。当与MLM类型的损失相结合时，AV 2 vec-MLM的表现优于基线，并在下游任务中实现了最佳性能。摘要：In this work, we present a novel method, named AV2vec, for learning audio-visual speech representations by multimodal self-distillation. AV2vec has a student and a teacher module, in which the student performs a masked latent feature regression task using the multimodal target features generated online by the teacher. The parameters of the teacher model are a momentum update of the student. Since our target features are generated online, AV2vec needs no iteration step like AV-HuBERT and the total training time cost is reduced to less than one-fifth. We further propose AV2vec-MLM in this study, which augments AV2vec with a masked language model (MLM)-style loss using multitask learning. Our experimental results show that AV2vec achieved comparable performance to the AV-HuBERT baseline. When combined with an MLM-style loss, AV2vec-MLM outperformed baselines and achieved the best performance on the downstream tasks.
【4】 Standard sound emergence as a predictor of short-term annoyance from wind turbine noise
标题：标准声音出现作为风力机噪声短期烦扰的预测指标
链接：http://arxiv.org/pdf/2212.02616v1
*与cs.SD语音【9】为同一篇
作者：Elise Ruaud,Guillaume Dutilleux机构：Acoustics Group, Department of Electronic Systems, NTNU, Trondheim, Norway a)备注：Submitted to the Journal or the Acoustical Society of America. 17 pages, 8 figures, 3 tables摘要：虽然一些国家使用声出现来规范风能开发，但没有公开的证据表明它是用于此目的的相关噪声描述符。在本研究中，我们进行了两次听力测试，以评估声音出现的优点。考虑了声音出现的三种定义：ISO 1996-1中的标准，可听度条件下的声涌现$e_{UAC}$和频谱涌现$e_{SP}$。我们还考虑了具体的残差比和响度度量。在每一次听力测试中，声刺激包括48个声刺激，分别为3个A计权声压级30、40、50 dB和4个比剩余比-10、-5、0、+5dB。结果得出的结论是，无论对声音出现的定义如何，总声压级比声音出现更好地预测短期烦恼度，或者比剩余噪声比更好地预测短期烦恼度。$e_{UAC}$比$e$对短期烦恼的预测稍好，而$e$比$e_{SP}$更好。总声压级和响度指标表现相似。此外，该结果提供了证据，即声音出现是风力涡轮机声音可听度的不良预测器。摘要：While sound emergence is used in several countries to regulate wind energy development, there is no published evidence that it is a relevant noise descriptor for this purpose. In the present work, we carried out two listening tests to evaluate the merits of sound emergence. Three definitions of sound emergence were considered: the one in ISO 1996-1, sound emergence under audibility condition $e_{UAC}$, and spectral emergence $e_{SP}$. We also considered the specific to residual ratio and loudness metrics. In each listening test, the sound stimuli consisted of 48 sound stimuli at 3 A-weighted sound pressure levels $ {30, 40, 50 }$~dB and 4 specific-to-residual ratios $ {-10, -5, 0, +5 }$~dB. The results lead to the conclusion that short term annoyance is better predicted by the total sound pressure level than by sound emergence, whatever the definition considered for the latter, or than by the specific to residual ratio. Short-term annoyance is slightly better predicted by $e_{UAC}$ than by $e$, while $e$ is a better predictor than $e_{SP}$. The total sound pressure level and the loudness metrics performed similarly. Furthermore, the results provide evidence that sound emergence is a poor predictor of the audibility of wind turbine sounds.
【5】 Label-free Knowledge Distillation with Contrastive Loss for Light-weight Speaker Recognition
标题：用于轻量级说话人识别的对比度损失无标签知识提取
链接：http://arxiv.org/pdf/2212.03090v1
*与cs.SD语音【1】为同一篇
作者：Zhiyuan Peng,Xuanji He,Ke Ding,Tan Lee,Guanglu Wan机构：Department of Electronic Engineering, The Chinese University of Hong Kong, Meituan摘要：近年来，说话人识别的超深层模型在说话人识别中的应用取得了显著的效果。然而，对于具有受限计算资源的设备上应用部署这些模型是不切实际的。另一方面，尽管轻量模型的性能不是最佳的，但在实践中它们是非常需要的。本研究旨在通过大规模无标签知识提取（KD）来改进轻量级SR模型。现有的识别方法通常需要说话人标签来学习特定任务的知识，这是由于传统的提取损失效率低。为了解决这一问题，实现无标签知识发现，我们提出利用自监督学习的对比损失进行提取。在不同来源的公共语音数据集上进行了广泛的实验。在轻量级SR模型上的实验结果表明，所提出的无标签对比损失KD方法的性能明显优于传统蒸馏方法和自监督学习方法.摘要：Very deep models for speaker recognition (SR) have demonstrated remarkable performance improvement in recent research. However, it is impractical to deploy these models for on-device applications with constrained computational resources. On the other hand, light-weight models are highly desired in practice despite their sub-optimal performance. This research aims to improve light-weight SR models through large-scale label-free knowledge distillation (KD). Existing KD approaches for SR typically require speaker labels to learn task-specific knowledge, due to the inefficiency of conventional loss for distillation. To address the inefficiency problem and achieve label-free KD, we propose to employ the contrastive loss from self-supervised learning for distillation. Extensive experiments are conducted on a collection of public speech datasets from diverse sources. Results on light-weight SR models show that the proposed approach of label-free KD with contrastive loss consistently outperforms both conventional distillation methods and self-supervised learning methods by a significant margin.
【6】 Covariance Regularization for Probabilistic Linear Discriminant Analysis
标题：概率线性判别分析中的协方差正则化
链接：http://arxiv.org/pdf/2212.03039v1
*与cs.SD语音【2】为同一篇
作者：Zhiyuan Peng,Mingjie Shao,Xuanji He,Xu Li,Tan Lee,Ke Ding,Guanglu Wan机构：Department of Electronic Engineering, The Chinese University of Hong Kong, Meituan, ARC Lab, Tencent PCG摘要：概率线性判别分析（PLDA）是说话人确认系统中常用的一种说话人嵌入相似度计算方法。最近的研究通过对角化其协方差来改善PLDA在域匹配条件下的性能。我们怀疑这种残酷的修剪方法会消除其对说话人嵌入的维度相关性建模的能力，从而导致域自适应性能不足。本文提出了两种可供选择的协方差正则化方法，即插值PLDA和稀疏PLDA，以解决该问题。插值后的PLDA结合余弦评分的先验知识对PLDA的协方差进行插值。稀疏PLDA引入稀疏性惩罚来更新协方差。实验结果表明，两种方法的性能都明显优于对角正则化方法。此外，当训练稀疏PLDA用于域自适应时，可以显著减少域内数据。摘要：Probabilistic linear discriminant analysis (PLDA) is commonly used in speaker verification systems to score the similarity of speaker embeddings. Recent studies improved the performance of PLDA in domain-matched conditions by diagonalizing its covariance. We suspect such brutal pruning approach could eliminate its capacity in modeling dimension correlation of speaker embeddings, leading to inadequate performance with domain adaptation. This paper explores two alternative covariance regularization approaches, namely, interpolated PLDA and sparse PLDA, to tackle the problem. The interpolated PLDA incorporates the prior knowledge from cosine scoring to interpolate the covariance of PLDA. The sparse PLDA introduces a sparsity penalty to update the covariance. Experimental results demonstrate that both approaches outperform diagonal regularization noticeably with domain adaptation. In addition, in-domain data can be significantly reduced when training sparse PLDA for domain adaptation.
【7】 Parameter Efficient Transfer Learning for Various Speech Processing Tasks
标题：不同语音处理任务的参数高效迁移学习
链接：http://arxiv.org/pdf/2212.02780v1
*与cs.SD语音【3】为同一篇
作者：Shinta Otake,Rei Kawakami,Nakamasa Inoue机构：Tokyo Institute of Technology摘要：自监督模型的微调是包括语音处理在内的各种领域中的一种强有力的迁移学习方法，因为它可以利用从大量未标记数据中获得的通用特征表示。然而，微调需要为每个下游任务设置新的参数，这是参数低效的。适配器体系结构通过在冻结的预训练模型中插入轻量级的可学习模块来部分解决这一问题。然而，现有的适配器体系结构不能自适应地利用存储在不同层中的低到高级别特征，这对于解决各种语音处理任务是必要的。因此，我们提出了一种新的适配器体系结构，以针对各种语音任务更灵活地获取特征表示。在实验中，我们将该适配器应用于四个语音任务的WavLM。它的性能与原始微调相当或更好，只有11%的可学习参数。它的性能也超过了现有的适配器体系结构。摘要：Fine-tuning of self-supervised models is a powerful transfer learning method in a variety of fields, including speech processing, since it can utilize generic feature representations obtained from large amounts of unlabeled data. Fine-tuning, however, requires a new parameter set for each downstream task, which is parameter inefficient. Adapter architecture is proposed to partially solve this issue by inserting lightweight learnable modules into a frozen pre-trained model. However, existing adapter architectures fail to adaptively leverage low- to high-level features stored in different layers, which is necessary for solving various kinds of speech processing tasks. Thus, we propose a new adapter architecture to acquire feature representations more flexibly for various speech tasks. In experiments, we applied this adapter to WavLM on four speech tasks. It performed on par or better than naive fine-tuning, with only 11% of learnable parameters. It also outperformed an existing adapter architecture.
【8】 Audio Latent Space Cartography
标题：音频潜伏空间制图
链接：http://arxiv.org/pdf/2212.02610v1
*与cs.SD语音【4】为同一篇
作者：Nicolas Jonason,Bob L. T. Sturm机构：KTH Royal Institute of Technology备注：Late Breaking Demo, ISMIR 2022 (this https URL)摘要：我们使用音频到图像生成管道来探索音频潜在空间的可视化生成。我们相信这有助于音频潜在空间的可解释性。我们在NSynth数据集上展示了各种结果。有一个Web演示可用。摘要：We explore the generation of visualisations of audio latent spaces using an audio-to-image generation pipeline. We believe this can help with the interpretability of audio latent spaces. We demonstrate a variety of results on the NSynth dataset. A web demo is available.
【9】 MAP-Music2Vec: A Simple and Effective Baseline for Self-Supervised Music Audio Representation Learning
标题：MAP-Music2Vec：一种简单有效的自我监督音乐音频表征学习基线
链接：http://arxiv.org/pdf/2212.02508v1
*与cs.SD语音【5】为同一篇
作者：Yizhi Li,Ruibin Yuan,Ge Zhang,Yinghao Ma,Chenghua Lin,Xingran Chen,Anton Ragni,Hanzhi Yin,Zhijie Hu,Haoyu He,Emmanouil Benetos,Norbert Gyenge,Ruibo Liu,Jie Fu机构：Department of Computer Science, University of Sheffield, UK, Beijing Academy of Artificial Intelligence, China, Centre for Digital Music, Queen Mary University of London, UK, School of Music, Carnegie Mellon University, PA, USA摘要：深度学习社区对自监督学习（SSL）的兴趣呈指数级增长。然而，如何建立用于以自监督方式学习原始音乐波形的有用表示的框架仍然是未探索的。在本文中，我们设计了Music2Vec，这是一个探索不同SSL算法组件和音乐音频录制技巧的框架。我们的模型实现了与最先进的（SOTA）音乐SSL模型Jukebox相当的结果，尽管显著地更小，具有少于后者的2%的参数。该模型将在Huggingface上发布（请参考：https: huggingface.co m-a-p music2vec-v1)摘要：The deep learning community has witnessed an exponentially growing interest in self-supervised learning (SSL). However, it still remains unexplored how to build a framework for learning useful representations of raw music waveforms in a self-supervised manner. In this work, we design Music2Vec, a framework exploring different SSL algorithmic components and tricks for music audio recordings. Our model achieves comparable results to the state-of-the-art (SOTA) music SSL model Jukebox, despite being significantly smaller with less than 2% of parameters of the latter. The model will be released on Huggingface(Please refer to: https: huggingface.co m-a-p music2vec-v1)
机器翻译，仅供参考
每日语音速递