今日论文合集：cs.SD语音11篇，eess.AS音频处理11篇。【1】 B-CLEAN-SC: CLEAN-SC for broadband sources标题：B-CLEAN-SC：宽带源的CLEAN-SC链接：https://arxiv.org/abs/2307.06181作者：Armin Goudarzi机构：German Aerospace Center (DLR), G¨ottingen, Germany摘要：本文介绍了B-CLEAN-SC，CLEAN-SC的宽带源的变化。与CLEAN-SC不同，B-CLEAN-SC处理频率间隔，CLEAN-SC对每个频率的波束成形图进行“去卷积”。B-CLEAN-SC不是在最大电平的位置处执行去卷积迭代，而是在过频率平均最大值的位置处执行去卷积迭代以改进位置估计。该方法进行了验证，并与标准CLEAN-SC合成的情况下，和现实世界的实验，宽和窄带源。它改善了在低频和高频的源重建并抑制噪声，同时它只增加了对存储器的需求，而不是计算工作量。摘要：This paper presents B-CLEAN-SC, a variation of CLEAN-SC for broadband sources. Opposed to CLEAN-SC, which ``deconvolves'' the beamforming map for each frequency individually, B-CLEAN-SC processes frequency intervals. Instead of performing a deconvolution iteration at the location of the maximum level, B-CLEAN-SC performs it at the location of the over-frequency-averaged maximum to improve the location estimation. The method is validated and compared to standard CLEAN-SC on synthetic cases, and real-world experiments, for broad- and narrowband sources. It improves the source reconstruction at low and high frequencies and suppresses noise, while it only increases the need for memory but not computational effort.【2】 Can Large Language Models Aid in Annotating Speech Emotional Data? Uncovering New Frontiers标题：大型语言模型能帮助注释语音情感数据吗？发现新的前沿链接：https://arxiv.org/abs/2307.06090作者：Siddique Latif,Muhammad Usama,Mohammad Ibrahim Malik,Björn W. Schuller机构：Bj¨orn W. Schuller, Fellow, IEEE备注：Under review摘要：尽管最近在语音情感识别（SER）模型方面取得了进展，但最先进的深度学习（DL）方法面临着注释数据的有限可用性的挑战。大型语言模型（LLM）彻底改变了我们对自然语言的理解，引入了扩展语言，语音和视觉理解的新兴属性。本文探讨了潜在的LLM注释丰富的语音数据，旨在提高国家的最先进的SER。我们使用公开的语音情感分类数据集在各种设置中评估这种能力。利用ChatGPT，我们实验证明了LLM在语音情感数据注释中的重要作用。我们的评估包括单镜头和几个镜头的情况下，揭示SER的性能变化。值得注意的是，我们通过数据增强实现了改进的结果，将ChatGPT注释的样本纳入现有数据集。我们的工作揭示了语音情感分类的新领域，突出了LLM在这一领域的重要性。摘要：Despite recent advancements in speech emotion recognition (SER) models, state-of-the-art deep learning (DL) approaches face the challenge of the limited availability of annotated data. Large language models (LLMs) have revolutionised our understanding of natural language, introducing emergent properties that broaden comprehension in language, speech, and vision. This paper examines the potential of LLMs to annotate abundant speech data, aiming to enhance the state-of-the-art in SER. We evaluate this capability across various settings using publicly available speech emotion classification datasets. Leveraging ChatGPT, we experimentally demonstrate the promising role of LLMs in speech emotion data annotation. Our evaluation encompasses single-shot and few-shots scenarios, revealing performance variability in SER. Notably, we achieve improved results through data augmentation, incorporating ChatGPT-annotated samples into existing datasets. Our work uncovers new frontiers in speech emotion classification, highlighting the increasing significance of LLMs in this field moving forward.【3】 Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognition标题：用于多语言和代码切换语音识别的专家的语言路由混合链接：https://arxiv.org/abs/2307.05956作者：Wenxuan Wang,Guodong Ma,Yuke Li,Binbin Du机构：NetEase Yidun AI Lab, Hangzhou, China备注：To appear in Proc. INTERSPEECH 2023, August 20-24, 2023, Dublin, Ireland摘要：单语和语码转换的多语言语音识别是一项具有挑战性的任务。近年来，基于专家混合算法（Mixture of Experts，MoE）的多语种和语码转换ASR研究取得了很大进展，但随着支持语言的增加，计算复杂度也随之增加。在这项工作中，我们提出了一个计算效率的网络名为语言路由混合专家（LR-MoE）的多语言和代码交换ASR。LR-MoE通过混合语言专家（MLE）提取特定于语言的表示，该混合语言专家（MLE）由逐帧语言路由机制引导学习。权重共享的帧级语言识别（LID）网络被联合训练为每个MoE层的共享预路由器。实验结果表明，该方法在计算效率相当的情况下，显著提高了多语种和语码切换语音识别性能。摘要：Multilingual speech recognition for both monolingual and code-switching speech is a challenging task. Recently, based on the Mixture of Experts (MoE), many works have made good progress in multilingual and code-switching ASR, but present huge computational complexity with the increase of supported languages. In this work, we propose a computation-efficient network named Language-Routing Mixture of Experts (LR-MoE) for multilingual and code-switching ASR. LR-MoE extracts language-specific representations through the Mixture of Language Experts (MLE), which is guided to learn by a frame-wise language routing mechanism. The weight-shared frame-level language identification (LID) network is jointly trained as the shared pre-router of each MoE layer. Experiments show that the proposed method significantly improves multilingual and code-switching speech recognition performances over baseline with comparable computational efficiency.【4】 SnakeSynth: New Interactions for Generative Audio Synthesis标题：SnakeSynth：生成性音频合成的新交互链接：https://arxiv.org/abs/2307.05830作者：Eric Easthope机构：University of British Columbia, Vancouver, British Columbia, Canada摘要：我介绍了“SnakeSynth”，这是一个基于Web的轻量级音频合成器，它结合了由深度生成模型生成的音频和实时连续二维（2D）输入，通过2D交互手势创建和控制可变长度的生成声音。交互手势是触摸和移动兼容的，类似于弹奏、鞠躬和弹拨乐器控制。点击和拖放手势直接控制音频播放长度，我表明，声音的长度和强度调制的互动与可编程的二维坐标网格。利用Google的TensorFlow.js中基于浏览器的音频和硬件加速的速度和普遍性，我们生成具有实时交互性的时变高保真声音。SnakeSynth在模型训练期间遇到的声音之间自适应地再现和插值，特别是没有长时间的训练，我简要讨论了深度生成模型作为音乐表达的交互式范例的可能未来。摘要：I present "SnakeSynth," a web-based lightweight audio synthesizer that combines audio generated by a deep generative model and real-time continuous two-dimensional (2D) input to create and control variable-length generative sounds through 2D interaction gestures. Interaction gestures are touch and mobile-compatible with analogies to strummed, bowed, and plucked musical instrument controls. Point-and-click and drag-and-drop gestures directly control audio playback length and I show that sound length and intensity are modulated by interactions with a programmable 2D coordinate grid. Leveraging the speed and ubiquity of browser-based audio and hardware acceleration in Google's TensorFlow.js we generate time-varying high-fidelity sounds with real-time interactivity. SnakeSynth adaptively reproduces and interpolates between sounds encountered during model training, notably without long training times, and I briefly discuss possible futures for deep generative models as an interactive paradigm for musical expression.【5】 Collaborative Song Dataset (CoSoD): An annotated dataset of multi-artist collaborations in popular music标题：协作歌曲数据集(CoSoD)：流行音乐中多个艺术家合作的带注释的数据集链接：https://arxiv.org/abs/2307.05588作者：Michèle Duguay,Kate Mancey,Johanna Devaney机构：Department of Music, Harvard University, Brooklyn College and Graduate Center, City University of New York备注：To be published in the Proceedings of the 24th International Society for Music Information Retrieval Conference (ISMIR)摘要：协作歌曲数据集（CoSoD）是2010-2019年Billboard“Hot 100”年终排行榜中331位多艺术家合作的语料库。语料库被注释有正式部分、声音产生的方面（包括混响、分层、平移和表演者的性别）和相关元数据。CoSoD通过专注于独立行为之间的音乐合作来补充其他流行音乐数据集。除了促进歌曲形式和声乐制作的研究，CoSoD允许性别的深入研究，因为它涉及到各种音色，音高和音乐合作中的正式参数。在本文中，我们详细介绍了数据集的内容，并概述了注释过程。我们还提出了一个实验，使用CoSoD，探讨如何使用混响，分层和平移与艺术家的性别。在这个实验中，我们发现，男性的声音平均处理混响较少，在立体声组合中占据的位置比女性的声音更窄。摘要：The Collaborative Song Dataset (CoSoD) is a corpus of 331 multi-artist collaborations from the 2010-2019 Billboard "Hot 100" year-end charts. The corpus is annotated with formal sections, aspects of vocal production (including reverberation, layering, panning, and gender of the performers), and relevant metadata. CoSoD complements other popular music datasets by focusing exclusively on musical collaborations between independent acts. In addition to facilitating the study of song form and vocal production, CoSoD allows for the in-depth study of gender as it relates to various timbral, pitch, and formal parameters in musical collaborations. In this paper, we detail the contents of the dataset and outline the annotation process. We also present an experiment using CoSoD that examines how the use of reverberation, layering, and panning are related to the gender of the artist. In this experiment, we find that men's voices are on average treated with less reverberation and occupy a more narrow position in the stereo mix than women's voices.【6】 The Ethical Implications of Generative Audio Models: A Systematic Literature Review标题：生成性音频模型的伦理意蕴：系统的文献综述链接：https://arxiv.org/abs/2307.05527作者：Julia Barnett机构：Northwestern University, Evanston, IL, USA备注：In proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES '23). 10 pages, 1 figure摘要：生成音频模型通常将其应用集中在音乐和语音生成中，最近的模型在其音频输出中具有类似人类的质量。本文对生成音频模型领域的884篇论文进行了系统的文献回顾，以量化该领域的研究人员考虑潜在的负面影响的程度，并确定该领域研究人员需要考虑的伦理影响类型。虽然65%的生成音频研究论文指出了他们的工作的积极潜在影响，但只有不到10%的论文讨论了任何负面影响。考虑负面影响的论文比例如此之小，尤其令人担忧，因为少数论文所揭示的问题正在引起严重的道德影响和与更广泛领域相关的担忧，例如欺诈，深度伪造和侵犯版权的可能性。通过量化生成音频研究中缺乏伦理考虑并确定潜在危害的关键领域，本文为该领域未来的工作奠定了基础，以便在关键时刻指导更认真的研究。摘要：Generative audio models typically focus their applications in music and speech generation, with recent models having human-like quality in their audio output. This paper conducts a systematic literature review of 884 papers in the area of generative audio models in order to both quantify the degree to which researchers in the field are considering potential negative impacts and identify the types of ethical implications researchers in this area need to consider. Though 65% of generative audio research papers note positive potential impacts of their work, less than 10% discuss any negative impacts. This jarringly small percentage of papers considering negative impact is particularly worrying because the issues brought to light by the few papers doing so are raising serious ethical implications and concerns relevant to the broader field such as the potential for fraud, deep-fakes, and copyright infringement. By quantifying this lack of ethical consideration in generative audio research and identifying key areas of potential harm, this paper lays the groundwork for future work in the field at a critical point in time in order to guide more conscientious research as this field progresses.【7】 Feature Embeddings from Large-Scale Acoustic Bird Classifiers Enable Few-Shot Transfer Learning标题：大规模声学Bird分类器的特征嵌入使小概率迁移学习成为可能链接：https://arxiv.org/abs/2307.06292作者：Burooj Ghani,Tom Denton,Stefan Kahl,Holger Klinck机构：Naturalis Biodiversity Center,Google LLC, K. Lisa Yang Center for Conservation Bioacoustics, Cornell Lab of Ornithology, Cornell University, Chemnitz University of Technology摘要：自动化生物声学分析有助于理解和保护海洋和陆地动物及其栖息地跨越广泛的时空尺度，并且通常涉及分析大量的声学数据集合。随着深度学习模型的出现，这些数据集中重要信号的分类得到了显着改善。这些模型为生物多样性监测、动物行为研究和自然资源管理的研究和决策提供了动力。然而，深度学习模型通常需要大量数据，并且需要大量标记的训练数据才能良好地执行。虽然足够的训练数据可用于某些分类学组（例如，常见鸟类物种）、许多类别（例如稀有和濒危物种、许多非鸟类分类群和叫声类型），缺乏足够的数据来从头训练健壮的模型。本研究调查了从大规模音频分类模型中提取的特征嵌入的实用性，以识别这些模型最初训练的生物声学类。我们在不同的数据集上评估模型，包括不同的鸟类叫声和方言类型、蝙蝠叫声、海洋哺乳动物叫声和两栖动物叫声。从在鸟类发声数据上训练的模型中提取的嵌入始终允许比在一般音频数据集上训练的嵌入更高质量的分类。这项研究的结果表明，大规模声学鸟类分类器的高质量特征嵌入可以用于Few-Shot迁移学习，从而能够从有限的训练数据中学习新的类别。我们的研究结果揭示了高效分析新的生物声学任务的潜力，即使在可用的训练数据仅限于几个样本的情况下。摘要：Automated bioacoustic analysis aids understanding and protection of both marine and terrestrial animals and their habitats across extensive spatiotemporal scales, and typically involves analyzing vast collections of acoustic data. With the advent of deep learning models, classification of important signals from these datasets has markedly improved. These models power critical data analyses for research and decision-making in biodiversity monitoring, animal behaviour studies, and natural resource management. However, deep learning models are often data-hungry and require a significant amount of labeled training data to perform well. While sufficient training data is available for certain taxonomic groups (e.g., common bird species), many classes (such as rare and endangered species, many non-bird taxa, and call-type), lack enough data to train a robust model from scratch. This study investigates the utility of feature embeddings extracted from large-scale audio classification models to identify bioacoustic classes other than the ones these models were originally trained on. We evaluate models on diverse datasets, including different bird calls and dialect types, bat calls, marine mammals calls, and amphibians calls. The embeddings extracted from the models trained on bird vocalization data consistently allowed higher quality classification than the embeddings trained on general audio datasets. The results of this study indicate that high-quality feature embeddings from large-scale acoustic bird classifiers can be harnessed for few-shot transfer learning, enabling the learning of new classes from a limited quantity of training data. Our findings reveal the potential for efficient analyses of novel bioacoustic tasks, even in scenarios where available training data is limited to a few samples.【8】 Rhythm Modeling for Voice Conversion标题：语音转换中的节奏建模链接：https://arxiv.org/abs/2307.06040作者：Benjamin van Niekerk,Marc-André Carbonneau,Herman Kamper机构：Kamper are with the Department of Electricaland Electronic Engineering, Stellenbosch University备注：5 pages, 4 figures, 4 tables, submitted to IEEE Signal Processing Letters摘要：语音转换旨在将源语音转换为不同的目标语音。然而，典型的语音转换系统不考虑节奏，节奏是感知说话者身份的重要因素。为了弥合这一差距，我们引入了Urrhythm-一种无监督的节奏转换方法，不需要并行的数据或文本转录。使用自我监督表示，我们首先将源音频划分成近似的sonorants，obstruents和沉默的段。然后，我们通过估计说话速率或每个片段类型的持续时间分布来建模节奏。最后，我们通过对语音片段进行时间拉伸来匹配目标说话速率或节奏。实验表明，Urrhythmic在质量和韵律方面优于现有的无监督方法。代码和检查点：https://github.com/bshall/urhythmic。音频演示页面：https://ubisoft-laforge.github.io/speech/urhythmic。摘要：Voice conversion aims to transform source speech into a different target voice. However, typical voice conversion systems do not account for rhythm, which is an important factor in the perception of speaker identity. To bridge this gap, we introduce Urhythmic-an unsupervised method for rhythm conversion that does not require parallel data or text transcriptions. Using self-supervised representations, we first divide source audio into segments approximating sonorants, obstruents, and silences. Then we model rhythm by estimating speaking rate or the duration distribution of each segment type. Finally, we match the target speaking rate or rhythm by time-stretching the speech segments. Experiments show that Urhythmic outperforms existing unsupervised methods in terms of quality and prosody. Code and checkpoints: https://github.com/bshall/urhythmic. Audio demo page: https://ubisoft-laforge.github.io/speech/urhythmic.【9】 Point to the Hidden: Exposing Speech Audio Splicing via Signal Pointer Nets标题：指向隐藏：通过信号指针网揭露语音音频拼接链接：https://arxiv.org/abs/2307.05641作者：Denise Moussa,Germans Hirsch,Sebastian Wankerl,Christian Riess机构：Federal Criminal Police Office (BKA), Germany, IT Security Infrastructures Lab, Friedrich-Alexander University of Erlangen-N¨urnberg, Germany, Data Science Chair, Julius-Maximilians University of W¨urzburg, Germany备注：accepted at Interspeech 2023, code will be made available after paper presentation (at the latest)摘要：为刑事调查核实录音证据的完整性是音频法医分析师工作的一个组成部分。这里，一个焦点是检测删除或插入操作，所谓的音频拼接。虽然这是一个相当容易的方法来改变口语，仔细编辑可以产生相当令人信服的结果。对于困难的情况或大量的数据，自动化工具可以支持检测潜在的编辑位置。为此，现在已经提出了几种分析和深度学习方法。尽管如此，很少解决实践中预期的不受约束的剪接情况。与SigPointer，我们提出了一个指针网络框架，连续输入，发现拼接位置自然，更有效地比现有的作品。对具有取证挑战性的数据（如强压缩和噪声信号）进行的大量实验量化了指针机制的好处，性能提高了约6至10个百分点。摘要：Verifying the integrity of voice recording evidence for criminal investigations is an integral part of an audio forensic analyst's work. Here, one focus is on detecting deletion or insertion operations, so called audio splicing. While this is a rather easy approach to alter spoken statements, careful editing can yield quite convincing results. For difficult cases or big amounts of data, automated tools can support in detecting potential editing locations. To this end, several analytical and deep learning methods have been proposed by now. Still, few address unconstrained splicing scenarios as expected in practice. With SigPointer, we propose a pointer network framework for continuous input that uncovers splice locations naturally and more efficiently than existing works. Extensive experiments on forensically challenging data like strongly compressed and noisy signals quantify the benefit of the pointer mechanism with performance increases between about 6 to 10 percentage points.【10】 Speech Diarization and ASR with GMM标题：基于GMM的语音二值化和ASR链接：https://arxiv.org/abs/2307.05637作者：Aayush Kumar Sharma,Vineet Bhavikatti,Amogh Nidawani,Dr. Siddappaji,Sanath P,Dr Geetishree Mishra机构：Dept. of Electronics and, Communication Engineering, BMS College of Engineering, Bangalore, India, Bangalore,India摘要：在这篇研究论文中，我们深入研究了语音日记化和自动语音识别（ASR）的主题。语音日志化涉及在音频流内分离各个说话者。通过采用ASR转录本，日记化过程旨在分离每个说话者的话语，基于其独特的音频特性对它们进行分组。另一方面，自动语音识别是指机器或程序识别并将口语单词和短语转换为机器可读格式的能力。在我们的语音日记化方法中，我们利用高斯混合模型（GMM）来表示语音段。基于GMM参数计算簇间距离，距离阈值作为停止准则。ASR需要将未知语音波形转换成相应的书面转录。使用同步算法分析语音信号，考虑到音调频率。我们的主要目标通常围绕着开发一个模型，最大限度地减少语音转录过程中的单词错误率（WER）度量。摘要：In this research paper, we delve into the topics of Speech Diarization and Automatic Speech Recognition (ASR). Speech diarization involves the separation of individual speakers within an audio stream. By employing the ASR transcript, the diarization process aims to segregate each speaker's utterances, grouping them based on their unique audio characteristics. On the other hand, Automatic Speech Recognition refers to the capability of a machine or program to identify and convert spoken words and phrases into a machine-readable format. In our speech diarization approach, we utilize the Gaussian Mixer Model (GMM) to represent speech segments. The inter-cluster distance is computed based on the GMM parameters, and the distance threshold serves as the stopping criterion. ASR entails the conversion of an unknown speech waveform into a corresponding written transcription. The speech signal is analyzed using synchronized algorithms, taking into account the pitch frequency. Our primary objective typically revolves around developing a model that minimizes the Word Error Rate (WER) metric during speech transcription.【11】 Anti-noise window: Subjective perception of active noise reduction and effect of informational masking标题：抗噪窗：有源降噪的主观感知和信息掩蔽的效果链接：https://arxiv.org/abs/2307.05533作者：Bhan Lam,Kelvin Chee Quan Lim,Kenneth Ooi,Zhen-Ting Ong,Dongyuan Shi,Woon-Seng Gan机构：School of Electrical and Electronic Engineering, Nanyang Technological University, Nanyang Avenue, Singapore , Singapore, A R T I C L E I N F O备注：None摘要：恢复自然通风（NV）的城市可持续性提出了室内声学舒适性的挑战。主动控制和基于干扰的降噪策略，例如使用扬声器，提供了在保持NV的同时实现声学舒适性的潜在解决方案。然而，这些方法通常没有从感知的角度进行集成或评估。本研究探讨了感性和客观方面的主动噪声控制（ANC）为基础的“抗噪声”窗口（ANW）和它的集成与信息掩蔽（IM）在一个模型的卧室。40名参与者评估了ANW的三向互动，涉及噪声类型（交通，训练和飞机），面具（鸟，水），和ANC（开，关）。评价的重点是感知烦恼（PAY; ISO/TS 15666）、感知情感质量（ISO/TS 12913-2）、响度（PLN），并包括开放式定性评估。尽管基于分贝的指标的客观降低最小，心理声学锐度略有增加，但仅ANW就表现出PAY和PLN的显著降低，以及所有噪声类型的ISO愉悦度的改善。掩蔽剂的添加通常增强了整体声学舒适度，尽管水掩蔽导致PLN增加。此外，ANC与掩蔽剂的组合显示出相互作用效应，与单独ANC相比，两种掩蔽剂均显著降低PAY。摘要：Reviving natural ventilation (NV) for urban sustainability presents challenges for indoor acoustic comfort. Active control and interference-based noise mitigation strategies, such as the use of loudspeakers, offer potential solutions to achieve acoustic comfort while maintaining NV. However, these approaches are not commonly integrated or evaluated from a perceptual standpoint. This study examines the perceptual and objective aspects of an active-noise-control (ANC)-based "anti-noise" window (ANW) and its integration with informational masking (IM) in a model bedroom. Forty participants assessed the ANW in a three-way interaction involving noise types (traffic, train, and aircraft), maskers (bird, water), and ANC (on, off). The evaluation focused on perceived annoyance (PAY; ISO/TS 15666), perceived affective quality (ISO/TS 12913-2), loudness (PLN), and included an open-ended qualitative assessment. Despite minimal objective reduction in decibel-based indicators and a slight increase in psychoacoustic sharpness, the ANW alone demonstrated significant reductions in PAY and PLN, as well as an improvement in ISO pleasantness across all noise types. The addition of maskers generally enhanced overall acoustic comfort, although water masking led to increased PLN. Furthermore, the combination of ANC with maskers showed interaction effects, with both maskers significantly reducing PAY compared to ANC alone.【1】 Feature Embeddings from Large-Scale Acoustic Bird Classifiers Enable Few-Shot Transfer Learning标题：大规模声学Bird分类器的特征嵌入使小概率迁移学习成为可能链接：https://arxiv.org/abs/2307.06292作者：Burooj Ghani,Tom Denton,Stefan Kahl,Holger Klinck机构：Naturalis Biodiversity Center,Google LLC, K. Lisa Yang Center for Conservation Bioacoustics, Cornell Lab of Ornithology, Cornell University, Chemnitz University of Technology摘要：自动化生物声学分析有助于理解和保护海洋和陆地动物及其栖息地跨越广泛的时空尺度，并且通常涉及分析大量的声学数据集合。随着深度学习模型的出现，这些数据集中重要信号的分类得到了显着改善。这些模型为生物多样性监测、动物行为研究和自然资源管理的研究和决策提供了动力。然而，深度学习模型通常需要大量数据，并且需要大量标记的训练数据才能良好地执行。虽然足够的训练数据可用于某些分类学组（例如，常见鸟类物种）、许多类别（例如稀有和濒危物种、许多非鸟类分类群和叫声类型），缺乏足够的数据来从头训练健壮的模型。本研究调查了从大规模音频分类模型中提取的特征嵌入的实用性，以识别这些模型最初训练的生物声学类。我们在不同的数据集上评估模型，包括不同的鸟类叫声和方言类型、蝙蝠叫声、海洋哺乳动物叫声和两栖动物叫声。从在鸟类发声数据上训练的模型中提取的嵌入始终允许比在一般音频数据集上训练的嵌入更高质量的分类。这项研究的结果表明，大规模声学鸟类分类器的高质量特征嵌入可以用于Few-Shot迁移学习，从而能够从有限的训练数据中学习新的类别。我们的研究结果揭示了高效分析新的生物声学任务的潜力，即使在可用的训练数据仅限于几个样本的情况下。摘要：Automated bioacoustic analysis aids understanding and protection of both marine and terrestrial animals and their habitats across extensive spatiotemporal scales, and typically involves analyzing vast collections of acoustic data. With the advent of deep learning models, classification of important signals from these datasets has markedly improved. These models power critical data analyses for research and decision-making in biodiversity monitoring, animal behaviour studies, and natural resource management. However, deep learning models are often data-hungry and require a significant amount of labeled training data to perform well. While sufficient training data is available for certain taxonomic groups (e.g., common bird species), many classes (such as rare and endangered species, many non-bird taxa, and call-type), lack enough data to train a robust model from scratch. This study investigates the utility of feature embeddings extracted from large-scale audio classification models to identify bioacoustic classes other than the ones these models were originally trained on. We evaluate models on diverse datasets, including different bird calls and dialect types, bat calls, marine mammals calls, and amphibians calls. The embeddings extracted from the models trained on bird vocalization data consistently allowed higher quality classification than the embeddings trained on general audio datasets. The results of this study indicate that high-quality feature embeddings from large-scale acoustic bird classifiers can be harnessed for few-shot transfer learning, enabling the learning of new classes from a limited quantity of training data. Our findings reveal the potential for efficient analyses of novel bioacoustic tasks, even in scenarios where available training data is limited to a few samples.【2】 Rhythm Modeling for Voice Conversion标题：语音转换中的节奏建模链接：https://arxiv.org/abs/2307.06040作者：Benjamin van Niekerk,Marc-André Carbonneau,Herman Kamper机构：Kamper are with the Department of Electricaland Electronic Engineering, Stellenbosch University备注：5 pages, 4 figures, 4 tables, submitted to IEEE Signal Processing Letters摘要：语音转换旨在将源语音转换为不同的目标语音。然而，典型的语音转换系统不考虑节奏，节奏是感知说话者身份的重要因素。为了弥合这一差距，我们引入了Urrhythm-一种无监督的节奏转换方法，不需要并行的数据或文本转录。使用自我监督表示，我们首先将源音频划分成近似的sonorants，obstruents和沉默的段。然后，我们通过估计说话速率或每个片段类型的持续时间分布来建模节奏。最后，我们通过对语音片段进行时间拉伸来匹配目标说话速率或节奏。实验表明，Urrhythmic在质量和韵律方面优于现有的无监督方法。代码和检查点：https://github.com/bshall/urhythmic。音频演示页面：https://ubisoft-laforge.github.io/speech/urhythmic。摘要：Voice conversion aims to transform source speech into a different target voice. However, typical voice conversion systems do not account for rhythm, which is an important factor in the perception of speaker identity. To bridge this gap, we introduce Urhythmic-an unsupervised method for rhythm conversion that does not require parallel data or text transcriptions. Using self-supervised representations, we first divide source audio into segments approximating sonorants, obstruents, and silences. Then we model rhythm by estimating speaking rate or the duration distribution of each segment type. Finally, we match the target speaking rate or rhythm by time-stretching the speech segments. Experiments show that Urhythmic outperforms existing unsupervised methods in terms of quality and prosody. Code and checkpoints: https://github.com/bshall/urhythmic. Audio demo page: https://ubisoft-laforge.github.io/speech/urhythmic.【3】 Point to the Hidden: Exposing Speech Audio Splicing via Signal Pointer Nets标题：指向隐藏：通过信号指针网揭露语音音频拼接链接：https://arxiv.org/abs/2307.05641作者：Denise Moussa,Germans Hirsch,Sebastian Wankerl,Christian Riess机构：Federal Criminal Police Office (BKA), Germany, IT Security Infrastructures Lab, Friedrich-Alexander University of Erlangen-N¨urnberg, Germany, Data Science Chair, Julius-Maximilians University of W¨urzburg, Germany备注：accepted at Interspeech 2023, code will be made available after paper presentation (at the latest)摘要：为刑事调查核实录音证据的完整性是音频法医分析师工作的一个组成部分。这里，一个焦点是检测删除或插入操作，所谓的音频拼接。虽然这是一个相当容易的方法来改变口语，仔细编辑可以产生相当令人信服的结果。对于困难的情况或大量的数据，自动化工具可以支持检测潜在的编辑位置。为此，现在已经提出了几种分析和深度学习方法。尽管如此，很少解决实践中预期的不受约束的剪接情况。与SigPointer，我们提出了一个指针网络框架，连续输入，发现拼接位置自然，更有效地比现有的作品。对具有取证挑战性的数据（如强压缩和噪声信号）进行的大量实验量化了指针机制的好处，性能提高了约6至10个百分点。摘要：Verifying the integrity of voice recording evidence for criminal investigations is an integral part of an audio forensic analyst's work. Here, one focus is on detecting deletion or insertion operations, so called audio splicing. While this is a rather easy approach to alter spoken statements, careful editing can yield quite convincing results. For difficult cases or big amounts of data, automated tools can support in detecting potential editing locations. To this end, several analytical and deep learning methods have been proposed by now. Still, few address unconstrained splicing scenarios as expected in practice. With SigPointer, we propose a pointer network framework for continuous input that uncovers splice locations naturally and more efficiently than existing works. Extensive experiments on forensically challenging data like strongly compressed and noisy signals quantify the benefit of the pointer mechanism with performance increases between about 6 to 10 percentage points.【4】 Speech Diarization and ASR with GMM标题：基于GMM的语音二值化和ASR链接：https://arxiv.org/abs/2307.05637作者：Aayush Kumar Sharma,Vineet Bhavikatti,Amogh Nidawani,Dr. Siddappaji,Sanath P,Dr Geetishree Mishra机构：Dept. of Electronics and, Communication Engineering, BMS College of Engineering, Bangalore, India, Bangalore,India摘要：在这篇研究论文中，我们深入研究了语音日记化和自动语音识别（ASR）的主题。语音日志化涉及在音频流内分离各个说话者。通过采用ASR转录本，日记化过程旨在分离每个说话者的话语，基于其独特的音频特性对它们进行分组。另一方面，自动语音识别是指机器或程序识别并将口语单词和短语转换为机器可读格式的能力。在我们的语音日记化方法中，我们利用高斯混合模型（GMM）来表示语音段。基于GMM参数计算簇间距离，距离阈值作为停止准则。ASR需要将未知语音波形转换成相应的书面转录。使用同步算法分析语音信号，考虑到音调频率。我们的主要目标通常围绕着开发一个模型，最大限度地减少语音转录过程中的单词错误率（WER）度量。摘要：In this research paper, we delve into the topics of Speech Diarization and Automatic Speech Recognition (ASR). Speech diarization involves the separation of individual speakers within an audio stream. By employing the ASR transcript, the diarization process aims to segregate each speaker's utterances, grouping them based on their unique audio characteristics. On the other hand, Automatic Speech Recognition refers to the capability of a machine or program to identify and convert spoken words and phrases into a machine-readable format. In our speech diarization approach, we utilize the Gaussian Mixer Model (GMM) to represent speech segments. The inter-cluster distance is computed based on the GMM parameters, and the distance threshold serves as the stopping criterion. ASR entails the conversion of an unknown speech waveform into a corresponding written transcription. The speech signal is analyzed using synchronized algorithms, taking into account the pitch frequency. Our primary objective typically revolves around developing a model that minimizes the Word Error Rate (WER) metric during speech transcription.【5】 Anti-noise window: Subjective perception of active noise reduction and effect of informational masking标题：抗噪窗：有源降噪的主观感知和信息掩蔽的效果链接：https://arxiv.org/abs/2307.05533作者：Bhan Lam,Kelvin Chee Quan Lim,Kenneth Ooi,Zhen-Ting Ong,Dongyuan Shi,Woon-Seng Gan机构：School of Electrical and Electronic Engineering, Nanyang Technological University, Nanyang Avenue, Singapore , Singapore, A R T I C L E I N F O备注：None摘要：恢复自然通风（NV）的城市可持续性提出了室内声学舒适性的挑战。主动控制和基于干扰的降噪策略，例如使用扬声器，提供了在保持NV的同时实现声学舒适性的潜在解决方案。然而，这些方法通常没有从感知的角度进行集成或评估。本研究探讨了感性和客观方面的主动噪声控制（ANC）为基础的“抗噪声”窗口（ANW）和它的集成与信息掩蔽（IM）在一个模型的卧室。40名参与者评估了ANW的三向互动，涉及噪声类型（交通，训练和飞机），面具（鸟，水），和ANC（开，关）。评价的重点是感知烦恼（PAY; ISO/TS 15666）、感知情感质量（ISO/TS 12913-2）、响度（PLN），并包括开放式定性评估。尽管基于分贝的指标的客观降低最小，心理声学锐度略有增加，但仅ANW就表现出PAY和PLN的显著降低，以及所有噪声类型的ISO愉悦度的改善。掩蔽剂的添加通常增强了整体声学舒适度，尽管水掩蔽导致PLN增加。此外，ANC与掩蔽剂的组合显示出相互作用效应，与单独ANC相比，两种掩蔽剂均显著降低PAY。摘要：Reviving natural ventilation (NV) for urban sustainability presents challenges for indoor acoustic comfort. Active control and interference-based noise mitigation strategies, such as the use of loudspeakers, offer potential solutions to achieve acoustic comfort while maintaining NV. However, these approaches are not commonly integrated or evaluated from a perceptual standpoint. This study examines the perceptual and objective aspects of an active-noise-control (ANC)-based "anti-noise" window (ANW) and its integration with informational masking (IM) in a model bedroom. Forty participants assessed the ANW in a three-way interaction involving noise types (traffic, train, and aircraft), maskers (bird, water), and ANC (on, off). The evaluation focused on perceived annoyance (PAY; ISO/TS 15666), perceived affective quality (ISO/TS 12913-2), loudness (PLN), and included an open-ended qualitative assessment. Despite minimal objective reduction in decibel-based indicators and a slight increase in psychoacoustic sharpness, the ANW alone demonstrated significant reductions in PAY and PLN, as well as an improvement in ISO pleasantness across all noise types. The addition of maskers generally enhanced overall acoustic comfort, although water masking led to increased PLN. Furthermore, the combination of ANC with maskers showed interaction effects, with both maskers significantly reducing PAY compared to ANC alone.【6】 B-CLEAN-SC: CLEAN-SC for broadband sources标题：B-CLEAN-SC：宽带源的CLEAN-SC链接：https://arxiv.org/abs/2307.06181作者：Armin Goudarzi机构：German Aerospace Center (DLR), G¨ottingen, Germany摘要：本文介绍了B-CLEAN-SC，CLEAN-SC的宽带源的变化。与CLEAN-SC不同，B-CLEAN-SC处理频率间隔，CLEAN-SC对每个频率的波束成形图进行“去卷积”。B-CLEAN-SC不是在最大电平的位置处执行去卷积迭代，而是在过频率平均最大值的位置处执行去卷积迭代以改进位置估计。该方法进行了验证，并与标准CLEAN-SC合成的情况下，和现实世界的实验，宽和窄带源。它改善了在低频和高频的源重建并抑制噪声，同时它只增加了对存储器的需求，而不是计算工作量。摘要：This paper presents B-CLEAN-SC, a variation of CLEAN-SC for broadband sources. Opposed to CLEAN-SC, which ``deconvolves'' the beamforming map for each frequency individually, B-CLEAN-SC processes frequency intervals. Instead of performing a deconvolution iteration at the location of the maximum level, B-CLEAN-SC performs it at the location of the over-frequency-averaged maximum to improve the location estimation. The method is validated and compared to standard CLEAN-SC on synthetic cases, and real-world experiments, for broad- and narrowband sources. It improves the source reconstruction at low and high frequencies and suppresses noise, while it only increases the need for memory but not computational effort.【7】 Can Large Language Models Aid in Annotating Speech Emotional Data? Uncovering New Frontiers标题：大型语言模型能帮助注释语音情感数据吗？发现新的前沿链接：https://arxiv.org/abs/2307.06090作者：Siddique Latif,Muhammad Usama,Mohammad Ibrahim Malik,Björn W. Schuller机构：Bj¨orn W. Schuller, Fellow, IEEE备注：Under review摘要：尽管最近在语音情感识别（SER）模型方面取得了进展，但最先进的深度学习（DL）方法面临着注释数据的有限可用性的挑战。大型语言模型（LLM）彻底改变了我们对自然语言的理解，引入了扩展语言，语音和视觉理解的新兴属性。本文探讨了潜在的LLM注释丰富的语音数据，旨在提高国家的最先进的SER。我们使用公开的语音情感分类数据集在各种设置中评估这种能力。利用ChatGPT，我们实验证明了LLM在语音情感数据注释中的重要作用。我们的评估包括单镜头和几个镜头的情况下，揭示SER的性能变化。值得注意的是，我们通过数据增强实现了改进的结果，将ChatGPT注释的样本纳入现有数据集。我们的工作揭示了语音情感分类的新领域，突出了LLM在这一领域的重要性。摘要：Despite recent advancements in speech emotion recognition (SER) models, state-of-the-art deep learning (DL) approaches face the challenge of the limited availability of annotated data. Large language models (LLMs) have revolutionised our understanding of natural language, introducing emergent properties that broaden comprehension in language, speech, and vision. This paper examines the potential of LLMs to annotate abundant speech data, aiming to enhance the state-of-the-art in SER. We evaluate this capability across various settings using publicly available speech emotion classification datasets. Leveraging ChatGPT, we experimentally demonstrate the promising role of LLMs in speech emotion data annotation. Our evaluation encompasses single-shot and few-shots scenarios, revealing performance variability in SER. Notably, we achieve improved results through data augmentation, incorporating ChatGPT-annotated samples into existing datasets. Our work uncovers new frontiers in speech emotion classification, highlighting the increasing significance of LLMs in this field moving forward.【8】 Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognition标题：用于多语言和代码切换语音识别的专家的语言路由混合链接：https://arxiv.org/abs/2307.05956作者：Wenxuan Wang,Guodong Ma,Yuke Li,Binbin Du机构：NetEase Yidun AI Lab, Hangzhou, China备注：To appear in Proc. INTERSPEECH 2023, August 20-24, 2023, Dublin, Ireland摘要：单语和语码转换的多语言语音识别是一项具有挑战性的任务。近年来，基于专家混合算法（Mixture of Experts，MoE）的多语种和语码转换ASR研究取得了很大进展，但随着支持语言的增加，计算复杂度也随之增加。在这项工作中，我们提出了一个计算效率的网络名为语言路由混合专家（LR-MoE）的多语言和代码交换ASR。LR-MoE通过混合语言专家（MLE）提取特定于语言的表示，该混合语言专家（MLE）由逐帧语言路由机制引导学习。权重共享的帧级语言识别（LID）网络被联合训练为每个MoE层的共享预路由器。实验结果表明，该方法在计算效率相当的情况下，显著提高了多语种和语码切换语音识别性能。摘要：Multilingual speech recognition for both monolingual and code-switching speech is a challenging task. Recently, based on the Mixture of Experts (MoE), many works have made good progress in multilingual and code-switching ASR, but present huge computational complexity with the increase of supported languages. In this work, we propose a computation-efficient network named Language-Routing Mixture of Experts (LR-MoE) for multilingual and code-switching ASR. LR-MoE extracts language-specific representations through the Mixture of Language Experts (MLE), which is guided to learn by a frame-wise language routing mechanism. The weight-shared frame-level language identification (LID) network is jointly trained as the shared pre-router of each MoE layer. Experiments show that the proposed method significantly improves multilingual and code-switching speech recognition performances over baseline with comparable computational efficiency.【9】 SnakeSynth: New Interactions for Generative Audio Synthesis标题：SnakeSynth：生成性音频合成的新交互链接：https://arxiv.org/abs/2307.05830作者：Eric Easthope机构：University of British Columbia, Vancouver, British Columbia, Canada摘要：我介绍了“SnakeSynth”，这是一个基于Web的轻量级音频合成器，它结合了由深度生成模型生成的音频和实时连续二维（2D）输入，通过2D交互手势创建和控制可变长度的生成声音。交互手势是触摸和移动兼容的，类似于弹奏、鞠躬和弹拨乐器控制。点击和拖放手势直接控制音频播放长度，我表明，声音的长度和强度调制的互动与可编程的二维坐标网格。利用Google的TensorFlow.js中基于浏览器的音频和硬件加速的速度和普遍性，我们生成具有实时交互性的时变高保真声音。SnakeSynth在模型训练期间遇到的声音之间自适应地再现和插值，特别是没有长时间的训练，我简要讨论了深度生成模型作为音乐表达的交互式范例的可能未来。摘要：I present "SnakeSynth," a web-based lightweight audio synthesizer that combines audio generated by a deep generative model and real-time continuous two-dimensional (2D) input to create and control variable-length generative sounds through 2D interaction gestures. Interaction gestures are touch and mobile-compatible with analogies to strummed, bowed, and plucked musical instrument controls. Point-and-click and drag-and-drop gestures directly control audio playback length and I show that sound length and intensity are modulated by interactions with a programmable 2D coordinate grid. Leveraging the speed and ubiquity of browser-based audio and hardware acceleration in Google's TensorFlow.js we generate time-varying high-fidelity sounds with real-time interactivity. SnakeSynth adaptively reproduces and interpolates between sounds encountered during model training, notably without long training times, and I briefly discuss possible futures for deep generative models as an interactive paradigm for musical expression.【10】 Collaborative Song Dataset (CoSoD): An annotated dataset of multi-artist collaborations in popular music标题：协作歌曲数据集(CoSoD)：流行音乐中多个艺术家合作的带注释的数据集链接：https://arxiv.org/abs/2307.05588作者：Michèle Duguay,Kate Mancey,Johanna Devaney机构：Department of Music, Harvard University, Brooklyn College and Graduate Center, City University of New York备注：To be published in the Proceedings of the 24th International Society for Music Information Retrieval Conference (ISMIR)摘要：协作歌曲数据集（CoSoD）是2010-2019年Billboard“Hot 100”年终排行榜中331位多艺术家合作的语料库。语料库被注释有正式部分、声音产生的方面（包括混响、分层、平移和表演者的性别）和相关元数据。CoSoD通过专注于独立行为之间的音乐合作来补充其他流行音乐数据集。除了促进歌曲形式和声乐制作的研究，CoSoD允许性别的深入研究，因为它涉及到各种音色，音高和音乐合作中的正式参数。在本文中，我们详细介绍了数据集的内容，并概述了注释过程。我们还提出了一个实验，使用CoSoD，探讨如何使用混响，分层和平移与艺术家的性别。在这个实验中，我们发现，男性的声音平均处理混响较少，在立体声组合中占据的位置比女性的声音更窄。摘要：The Collaborative Song Dataset (CoSoD) is a corpus of 331 multi-artist collaborations from the 2010-2019 Billboard "Hot 100" year-end charts. The corpus is annotated with formal sections, aspects of vocal production (including reverberation, layering, panning, and gender of the performers), and relevant metadata. CoSoD complements other popular music datasets by focusing exclusively on musical collaborations between independent acts. In addition to facilitating the study of song form and vocal production, CoSoD allows for the in-depth study of gender as it relates to various timbral, pitch, and formal parameters in musical collaborations. In this paper, we detail the contents of the dataset and outline the annotation process. We also present an experiment using CoSoD that examines how the use of reverberation, layering, and panning are related to the gender of the artist. In this experiment, we find that men's voices are on average treated with less reverberation and occupy a more narrow position in the stereo mix than women's voices.【11】 The Ethical Implications of Generative Audio Models: A Systematic Literature Review标题：生成性音频模型的伦理意蕴：系统的文献综述链接：https://arxiv.org/abs/2307.05527
作者：Julia Barnett机构：Northwestern University, Evanston, IL, USA备注：In proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES '23). 10 pages, 1 figure摘要：生成音频模型通常将其应用集中在音乐和语音生成中，最近的模型在其音频输出中具有类似人类的质量。本文对生成音频模型领域的884篇论文进行了系统的文献回顾，以量化该领域的研究人员考虑潜在的负面影响的程度，并确定该领域研究人员需要考虑的伦理影响类型。虽然65%的生成音频研究论文指出了他们的工作的积极潜在影响，但只有不到10%的论文讨论了任何负面影响。考虑负面影响的论文比例如此之小，尤其令人担忧，因为少数论文所揭示的问题正在引起严重的道德影响和与更广泛领域相关的担忧，例如欺诈，深度伪造和侵犯版权的可能性。通过量化生成音频研究中缺乏伦理考虑并确定潜在危害的关键领域，本文为该领域未来的工作奠定了基础，以便在关键时刻指导更认真的研究。摘要：Generative audio models typically focus their applications in music and speech generation, with recent models having human-like quality in their audio output. This paper conducts a systematic literature review of 884 papers in the area of generative audio models in order to both quantify the degree to which researchers in the field are considering potential negative impacts and identify the types of ethical implications researchers in this area need to consider. Though 65% of generative audio research papers note positive potential impacts of their work, less than 10% discuss any negative impacts. This jarringly small percentage of papers considering negative impact is particularly worrying because the issues brought to light by the few papers doing so are raising serious ethical implications and concerns relevant to the broader field such as the potential for fraud, deep-fakes, and copyright infringement. By quantifying this lack of ethical consideration in generative audio research and identifying key areas of potential harm, this paper lays the groundwork for future work in the field at a critical point in time in order to guide more conscientious research as this field progresses.
机器翻译，仅供参考
每日语音速递