今日论文合集：cs.SD语音9篇，eess.AS音频处理10篇。
cs.SD语音

【1】 Anomalous Sound Detection using Audio Representation with Machine ID based Contrastive Learning Pretraining标题：基于机器ID的对比学习预训练音频表示法检测异常音链接：http://arxiv.org/pdf/2304.03588v1

作者：Jian Guan,Feiyang Xiao,Youde Liu,Qiaoxi Zhu,Wenwu Wang机构：Group of Intelligent Signal Processing, College of Computer Science and Technology, Harbin Engineering University, Harbin, China, School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China摘要：用于异常声音检测的现有对比学习方法通过使用样本的增强之间的对比度（例如，具有时间或频率掩蔽）。然而，由于缺乏机器声音的物理特性，它们可能会被增强的数据所偏置，从而限制了检测性能。本文使用对比学习来细化每个机器ID的音频表示，而不是每个音频样本。所提出的两阶段方法使用对比学习来通过结合机器ID和自监督ID分类器来预训练音频表示模型以微调所学习的模型。实验表明，该方法优于使用对比学习或自学习的最新方法。监督分类在DCASE 2020挑战任务2数据集上的整体异常检测性能和稳定性。摘要：Existing contrastive learning methods for anomalous sound detection refine the audio representation of each audio sample by using the contrast between the samples' augmentations (e.g., with time or frequency masking). However, they might be biased by the augmented data, due to the lack of physical properties of machine sound, thereby limiting the detection performance. This paper uses contrastive learning to refine audio representations for each machine ID, rather than for each audio sample. The proposed two-stage method uses contrastive learning to pretrain the audio representation model by incorporating machine ID and a self-supervised ID classifier to fine-tune the learnt model, while enhancing the relation between audio features from the same ID. Experiments show that our method outperforms the state-of-the-art methods using contrastive learning or self-supervised classification in overall anomaly detection performance and stability on DCASE 2020 Challenge Task2 dataset.

【2】 Graph Attention for Automated Audio Captioning标题：自动音频字幕的图形注意链接：http://arxiv.org/pdf/2304.03586v1

作者：Feiyang Xiao,Jian Guan,Qiaoxi Zhu,Wenwu Wang摘要：现有技术的音频字幕方法通常使用具有预训练的音频神经网络（PANN）的编码器-解码器结构作为用于特征提取的编码器。然而，PANN中使用的卷积运算在捕获音频信号内的长时间依赖性方面受到限制，从而导致音频字幕的潜在性能下降。这封信提出了一种新的方法，使用图形注意力（GraphAC）的编码器-解码器为基础的音频字幕。在编码器中，在PANN之后引入图形注意模块，以通过邻接图来学习上下文关联（即，不同时间帧上的音频特征之间的依赖性），并且使用前k掩码来减轻来自噪声节点的干扰。学习的上下文关联导致更有效的特征表示与特征节点聚合。结果，解码器可以基于从音频信号学习的上下文关联来预测关于声学场景和事件的重要语义信息。实验结果表明，GraphAC优于PANN作为编码器的最先进的方法，这要归功于将图形注意力模块并入编码器中以捕获音频信号内的长时间依赖性。源代码可在www.example.com获得https: github.com LittleFlyingSheep GraphAC。摘要：State-of-the-art audio captioning methods typically use the encoder-decoder structure with pretrained audio neural networks (PANNs) as encoders for feature extraction. However, the convolution operation used in PANNs is limited in capturing the long-time dependencies within an audio signal, thereby leading to potential performance degradation in audio captioning. This letter presents a novel method using graph attention (GraphAC) for encoder-decoder based audio captioning. In the encoder, a graph attention module is introduced after the PANNs to learn contextual association (i.e. the dependency among the audio features over different time frames) through an adjacency graph, and a top-k mask is used to mitigate the interference from noisy nodes. The learnt contextual association leads to a more effective feature representation with feature node aggregation. As a result, the decoder can predict important semantic information about the acoustic scene and events based on the contextual associations learned from the audio signal. Experimental results show that GraphAC outperforms the state-of-the-art methods with PANNs as the encoders, thanks to the incorporation of the graph attention module into the encoder for capturing the long-time dependencies within the audio signal. The source code is available at https: github.com LittleFlyingSheep GraphAC.

【3】 On-site Noise Exposure technique for noise-robust machine fault classification标题：现场噪声暴露技术在机器故障分类中的应用链接：http://arxiv.org/pdf/2304.03522v1

作者：Wonjun Yi,Jung-Woo Choi机构：School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea备注：Presented at 24th International Congress on Acoustics (ICA) 2022摘要：故障声音的现场分类是机器健康监测和诊断中的重要问题。然而，在诸如工厂的嘈杂环境中，机器声音总是与环境噪声混合，并且当机器不运行时可能存在仅噪声时段。因此，基于深度神经网络（DNN）的故障分类器必须能够区分噪声和机器声音，并且对混合噪声具有鲁棒性。为了解决这些问题，我们研究了现场噪声暴露（ONE），将DNN模型暴露在机器运行的相同环境中记录的噪声中。与离群值暴露技术一样，噪声暴露训练DNN分类器以针对仅噪声数据产生均匀的预测概率分布。在推理期间，由ONE训练的DNN分类器输出最大softmax概率作为噪声分数，并确定仅噪声时段。我们混合机器声音和ToyADMOS2数据集的噪声来模拟高噪声数据。评估由ONE训练的基于ResNet的分类器，并与由其他分布外检测技术训练的分类器进行比较。测试结果表明，将模型暴露于现场噪声可以使模型比使用其他噪声或检测技术更鲁棒。摘要：In-situ classification of faulty sounds is an important issue in machine health monitoring and diagnosis. However, in a noisy environment such as a factory, machine sound is always mixed up with environmental noises, and noise-only periods can exist when a machine is not in operation. Therefore, a deep neural network (DNN)-based fault classifier has to be able to distinguish noise from machine sound and be robust to mixed noises. To deal with these problems, we investigate on-site noise exposure (ONE) that exposes a DNN model to the noises recorded in the same environment where the machine operates. Like the outlier exposure technique, noise exposure trains a DNN classifier to produce a uniform predicted probability distribution against noise-only data. During inference, the DNN classifier trained by ONE outputs the maximum softmax probability as the noise score and determines the noise-only period. We mix machine sound and noises of the ToyADMOS2 dataset to simulate highly noisy data. A ResNet-based classifier trained by ONE is evaluated and compared with those trained by other out-of-distribution detection techniques. The test results show that exposing a model to on-site noises can make a model more robust than using other noises or detection techniques.

【4】 Adoption of AI Technology in the Music Mixing Workflow: An Investigation标题：人工智能技术在混音工作流中的应用研究链接：http://arxiv.org/pdf/2304.03407v1

作者：Soumya Sai Vanka,Maryam Safi,Jean-Baptiste Rolland,George Fazekas机构：Queen Mary University of London, London, UK, Steinberg Media Technologies GmbH, Hamburg, Germany备注：To be published at the 154th AES Europe Convention, 2023 in Helsinki, Finland摘要：人工智能（AI）技术在音乐行业的整合正在推动音乐创作、制作和混合方式的重大变革。本研究调查了AI在混合工作流中的现状及其在不同用户群体中的采用情况。本研究透过半结构式访谈、问卷调查及网路论坛分析，确认三个使用者群，分别为业馀使用者、专业使用者及专业使用者。我们的研究结果表明，虽然人工智能混合工具可以简化过程并为业余爱好者提供不错的结果，但专业人士寻求精确的控制和定制选项，而专业人士除了辅助和协作技术外还希望控制和定制选项。该研究为不同用户群体设计有效的AI混合工具提供了策略，并概述了未来的发展方向。摘要：The integration of artificial intelligence (AI) technology in the music industry is driving a significant change in the way music is being composed, produced and mixed. This study investigates the current state of AI in the mixing workflows and its adoption by different user groups. Through semi-structured interviews, a questionnaire-based study, and analyzing web forums, the study confirms three user groups comprising amateurs, pro-ams, and professionals. Our findings show that while AI mixing tools can simplify the process and provide decent results for amateurs, pro-ams seek precise control and customization options, while professionals desire control and customization options in addition to assistive and collaborative technologies. The study provides strategies for designing effective AI mixing tools for different user groups and outlines future directions.

【5】 DSVAE: Interpretable Disentangled Representation for Synthetic Speech Detection标题：DSVAE：用于合成语音检测的可解释解缠表示链接：http://arxiv.org/pdf/2304.03323v1

作者：Amit Kumar Singh Yadav,Kratika Bhagtani,Ziyue Xiang,Paolo Bestagini,Stefano Tubaro,Edward J. Delp机构：†Video and Image Processing Lab (VIPER), School of Electrical and Computer Engineering, Purdue University, West Lafayette, Indiana, USA, ‡Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy摘要：用于生成与从人类说话者记录的语音在感知上不可区分的高质量合成语音信号的工具是容易获得的。已经提出了几种用于检测合成语音的方法。这些方法中的许多方法将深度学习方法用作黑盒，而不为它们所做的决策提供推理。这限制了这些方法的可解释性。在本文中，我们提出了解纠缠频谱图变分自动编码器（DSVAE），这是一个两阶段的训练变分自动编码器，处理语音频谱图使用解纠缠表示学习生成可解释的表示的语音信号检测合成语音。DSVAE还创建了一个激活图，以突出显示区分合成和真实人类语音信号的频谱图区域。我们使用ASVspoof2019数据集评估了从DSVAE获得的表示。实验结果表明，在11个未知语音合成器中，我们对6个已知语音合成器和10个未知语音合成器的合成语音检测准确率都在98%以上。我们还可视化表示从DSVAE获得17个不同的语音合成器，并验证它们确实是可解释的，并区分真正的合成语音从每个合成器。摘要：Tools to generate high quality synthetic speech signal that is perceptually indistinguishable from speech recorded from human speakers are easily available. Several approaches have been proposed for detecting synthetic speech. Many of these approaches use deep learning methods as a black box without providing reasoning for the decisions they make. This limits the interpretability of these approaches. In this paper, we propose Disentangled Spectrogram Variational Auto Encoder (DSVAE) which is a two staged trained variational autoencoder that processes spectrograms of speech using disentangled representation learning to generate interpretable representations of a speech signal for detecting synthetic speech. DSVAE also creates an activation map to highlight the spectrogram regions that discriminate synthetic and bona fide human speech signals. We evaluated the representations obtained from DSVAE using the ASVspoof2019 dataset. Our experimental results show high accuracy ( 98%) on detecting synthetic speech from 6 known and 10 out of 11 unknown speech synthesizers. We also visualize the representation obtained from DSVAE for 17 different speech synthesizers and verify that they are indeed interpretable and discriminate bona fide and synthetic speech from each of the synthesizers.

【6】 Automatic Detection of Reactions to Music via Earable Sensing标题：通过听觉感知自动检测对音乐的反应链接：http://arxiv.org/pdf/2304.03295v1

作者：Euihyoek Lee,Chulhong Min,Jeaseung Lee,Jin Yu,Seungwoo Kang机构：JAESEUNG LEE, KOREATECH, Republic of Korea摘要：本文介绍了一种新颖的系统--GrooveMeter，它通过可听见的感知来自动检测对音乐的声音和动作反应，并支持音乐参与度感知应用。为此，我们使用智能耳塞作为传感设备，这已经被广泛用于音乐收听，并通过利用耳塞上的惯性测量单元（IMU）和麦克风来设计反应检测技术。为了探索日常音乐听的情况下的反应，我们收集了第一种数据集MusicReactionSet，其中包含926分钟长的IMU和30名参与者的音频数据。通过数据集，我们发现了一组独特的挑战，即使用音频和运动传感准确和鲁棒地检测音乐聆听反应。我们设计了复杂的处理管道，使反应检测准确高效。我们提出了一个全面的评估，以检查反应检测和系统成本的性能。它表明，GrooveMeter实现了宏观F1分数为0.89的声音反应和0.81的运动反应留一个主题的交叉验证。更重要的是，与其他方法相比，GrooveMeter显示出更高的准确性和鲁棒性。我们还表明，我们的过滤方法减少了50%或更多的能量开销。最后，我们通过案例研究展示了潜在的用例。摘要：We present GrooveMeter, a novel system that automatically detects vocal and motion reactions to music via earable sensing and supports music engagement-aware applications. To this end, we use smart earbuds as sensing devices, which are already widely used for music listening, and devise reaction detection techniques by leveraging an inertial measurement unit (IMU) and a microphone on earbuds. To explore reactions in daily music-listening situations, we collect the first kind of dataset, MusicReactionSet, containing 926-minute-long IMU and audio data with 30 participants. With the dataset, we discover a set of unique challenges in detecting music listening reactions accurately and robustly using audio and motion sensing. We devise sophisticated processing pipelines to make reaction detection accurate and efficient. We present a comprehensive evaluation to examine the performance of reaction detection and system cost. It shows that GrooveMeter achieves the macro F1 scores of 0.89 for vocal reaction and 0.81 for motion reaction with leave-one-subject-out cross-validation. More importantly, GrooveMeter shows higher accuracy and robustness compared to alternative methods. We also show that our filtering approach reduces 50% or more of the energy overhead. Finally, we demonstrate the potential use cases through a case study.

【7】 A2D: Anywhere Anytime Drumming标题：A2D：随时随地打鼓链接：http://arxiv.org/pdf/2304.03289v1

作者：Harel Yadid,Almog Algranti,Mark Levin,Ayal Taitler机构：Technion – Israel Institute of Technology, Haifa, Israel, University of Toronto, Department of Mechanical and and Industrial Engineering, Toronto, Ontario, Canada摘要：鼓包，只有大约100年的历史，是流行音乐，摇滚乐和爵士乐等许多音乐流派中的流行乐器。然而，拥有一个套件的道路是昂贵的，无论是在财政上还是在空间上。此外，鼓是更难以走动相比，其他文书，因为他们不适合到一个单一的袋。我们提出了一个无鼓的方法，只使用两根棍子和智能手机或网络摄像头提供一个空气鼓的经验。检测算法将深度学习工具与跟踪方法相结合，以增强用户体验。基于定量和定性测试与人在循环中，我们表明，我们的系统有零失误的初学者水平发挥和忽略不计的失误，先进的水平发挥。此外，我们有限的人体试验为未来的研究提供了潜在的方向。摘要：The drum kit, which has only been around for around 100 years, is a popular instrument in many music genres such as pop, rock, and jazz. However, the road to owning a kit is expensive, both financially and space-wise. Also, drums are more difficult to move around compared to other instruments, as they do not fit into a single bag. We propose a no-drums approach that uses only two sticks and a smartphone or a webcam to provide an air-drumming experience. The detection algorithm combines deep learning tools with tracking methods for an enhanced user experience. Based on both quantitative and qualitative testing with humans-in-the-loop, we show that our system has zero misses for beginner level play and negligible misses for advanced level play. Additionally, our limited human trials suggest potential directions for future research.

【8】 Margin-Mixup: A Method for Robust Speaker Verification in Multi-Speaker Audio标题：边缘混合：一种多说话人音频中的稳健说话人确认方法链接：http://arxiv.org/pdf/2304.03515v1

作者：Jenthe Thienpondt,Nilesh Madhu,Kris Demuynck机构：IDLab, Department of Electronics and Information Systems, Ghent University - imec, Belgium备注：proceedings of ICASSP 2023摘要：本文研究了多个说话人重叠情况下的说话人确认问题。大多数说话人确认系统都是在假设给定音频段中存在单个说话人的情况下设计的。然而，在现实世界中，这种假设并不总是成立的。在本文中，我们证明了目前的说话人验证系统是不强大的音频与显着的扬声器重叠。为了缓解这个问题，我们提出了边缘混合，一个简单的训练策略，可以很容易地通过现有的说话人验证管道，使所得到的说话人嵌入鲁棒性对多扬声器音频。与其他方法相比，边缘混合不需要改变常规的说话人验证架构，同时获得更好的结果。在我们的多扬声器测试集的基础上VoxCeleb1，所提出的边际混合策略提高了EER平均44.4%，相对于我们的国家的最先进的扬声器验证基线系统。摘要：This paper is concerned with the task of speaker verification on audio with multiple overlapping speakers. Most speaker verification systems are designed with the assumption of a single speaker being present in a given audio segment. However, in a real-world setting this assumption does not always hold. In this paper, we demonstrate that current speaker verification systems are not robust against audio with noticeable speaker overlap. To alleviate this issue, we propose margin-mixup, a simple training strategy that can easily be adopted by existing speaker verification pipelines to make the resulting speaker embeddings robust against multi-speaker audio. In contrast to other methods, margin-mixup requires no alterations to regular speaker verification architectures, while attaining better results. On our multi-speaker test set based on VoxCeleb1, the proposed margin-mixup strategy improves the EER on average with 44.4% relative to our state-of-the-art speaker verification baseline systems.

【9】 To Wake-up or Not to Wake-up: Reducing Keyword False Alarm by Successive Refinement标题：唤醒还是不唤醒：通过逐级细化减少关键词虚警链接：http://arxiv.org/pdf/2304.03416v1

作者：Yashas Malur Saidutta,Rakshith Sharma Srinivasa,Ching-Hua Lee,Chouchang Yang,Yilin Shen,Hongxia Jin机构：Samsung Research America, Mountain View, CA, USA, ©, IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including备注：Accepted for publication in ICASSP 2023摘要：关键词定位系统连续地处理音频流以检测关键词。在设计这样的系统中最具挑战性的任务之一是减少当系统错误地注册关键字而不管关键字没有被说出时发生的假警报（FA）。在本文中，我们提出了一个简单而优雅的解决方案，这个问题，从法律的全概率。我们表明，现有的深度关键字发现机制可以通过连续细化来改进，其中系统首先对输入音频是否是语音进行分类，其次是输入是否是关键字，最后对说出的关键字进行分类。我们表明，在多个模型的大小范围从13K参数到2.41M参数，连续细化技术减少FA的一个因素，8在域内举行了FA数据，并高达7的因素，在域外（OOD）FA数据。此外，我们提出的方法是“即插即用”，可以应用于任何深度关键字定位模型。摘要：Keyword spotting systems continuously process audio streams to detect keywords. One of the most challenging tasks in designing such systems is to reduce False Alarm (FA) which happens when the system falsely registers a keyword despite the keyword not being uttered. In this paper, we propose a simple yet elegant solution to this problem that follows from the law of total probability. We show that existing deep keyword spotting mechanisms can be improved by Successive Refinement, where the system first classifies whether the input audio is speech or not, followed by whether the input is keyword-like or not, and finally classifies which keyword was uttered. We show across multiple models with size ranging from 13K parameters to 2.41M parameters, the successive refinement technique reduces FA by up to a factor of 8 on in-domain held-out FA data, and up to a factor of 7 on out-of-domain (OOD) FA data. Further, our proposed approach is "plug-and-play" and can be applied to any deep keyword spotting model.
eess.AS音频处理

【1】 An unsupervised segmentation of vocal breath sounds标题：一种无监督的声音呼吸音分割方法链接：http://arxiv.org/pdf/2304.03758v1

作者：Shivani Yadav,Dipanjan Gope,Uma Maheswari K.,Prasanta Kumar Ghosh摘要：呼吸是人类生存必不可少的一部分，它承载着一个人生理和心理状态的信息。一般来说，呼吸边界在用于任何任务之前由专家标记。针对在口中记录的呼吸音（在本工作中也称为有声呼吸音（VBS）），提出了一种用于呼吸边界检测的无监督算法。在此工作中使用在口处记录的呼吸声音，因为它们比气管呼吸声音和肺呼吸声音容易且无接触地记录。呼吸信号能量的周期性性质用于分割呼吸边界。使用具有呼吸阶段的数量（$P$）和呼吸阶段持续时间（$d$）的先验信息的动态编程来找到边界。在这项工作中，预测来自具有307次呼吸的60名受试者（31名健康受试者，29名患者）的367个呼吸边界。利用所提出的方法，得到了M（89美元），I（13美元），D（11美元）和S（79美元），比本工作中使用的基线具有更好的性能。甚至哮喘和健康受试者之间的分类性能，使用所提出的方法估计的边界与地面实况边界。摘要：Breathing is an essential part of human survival, which carries information about a person's physiological and psychological state. Generally, breath boundaries are marked by experts before using for any task. An unsupervised algorithm for breath boundary detection has been proposed for breath sounds recorded at the mouth also referred as vocal breath sounds (VBS) in this work. Breath sounds recorded at the mouth are used in this work because they are easy and contactless to record than tracheal breath sounds and lung breath sounds. The periodic nature of breath signal energy is used to segment the breath boundaries. Dynamic programming with the prior information of the number of breath phases($P$) and breath phase duration($d$) is used to find the boundaries. In this work, 367 breath boundaries from 60 subjects (31 healthy, 29 patients) having 307 breaths are predicted. With the proposed method, M ($89 %$), I ($13 %$), D ($11 %$) and S ($79 %$) is found. The proposed method shows better performance than the baselines used in this work. Even the classification performance between asthmatic and healthy subjects using estimated boundaries by the proposed method is comparable with the ground truth boundaries.

【2】 Margin-Mixup: A Method for Robust Speaker Verification in Multi-Speaker Audio标题：边缘混合：一种多说话人音频中的稳健说话人确认方法链接：http://arxiv.org/pdf/2304.03515v1

*与cs.SD语音【8】为同一篇

作者：Jenthe Thienpondt,Nilesh Madhu,Kris Demuynck机构：IDLab, Department of Electronics and Information Systems, Ghent University - imec, Belgium备注：proceedings of ICASSP 2023摘要：本文研究了多个说话人重叠情况下的说话人确认问题。大多数说话人确认系统都是在假设给定音频段中存在单个说话人的情况下设计的。然而，在现实世界中，这种假设并不总是成立的。在本文中，我们证明了目前的说话人验证系统是不强大的音频与显着的扬声器重叠。为了缓解这个问题，我们提出了边缘混合，一个简单的训练策略，可以很容易地通过现有的说话人验证管道，使所得到的说话人嵌入鲁棒性对多扬声器音频。与其他方法相比，边缘混合不需要改变常规的说话人验证架构，同时获得更好的结果。在我们的多扬声器测试集的基础上VoxCeleb1，所提出的边际混合策略提高了EER平均44.4%，相对于我们的国家的最先进的扬声器验证基线系统。摘要：This paper is concerned with the task of speaker verification on audio with multiple overlapping speakers. Most speaker verification systems are designed with the assumption of a single speaker being present in a given audio segment. However, in a real-world setting this assumption does not always hold. In this paper, we demonstrate that current speaker verification systems are not robust against audio with noticeable speaker overlap. To alleviate this issue, we propose margin-mixup, a simple training strategy that can easily be adopted by existing speaker verification pipelines to make the resulting speaker embeddings robust against multi-speaker audio. In contrast to other methods, margin-mixup requires no alterations to regular speaker verification architectures, while attaining better results. On our multi-speaker test set based on VoxCeleb1, the proposed margin-mixup strategy improves the EER on average with 44.4% relative to our state-of-the-art speaker verification baseline systems.

【3】 To Wake-up or Not to Wake-up: Reducing Keyword False Alarm by Successive Refinement标题：唤醒还是不唤醒：通过逐级细化减少关键词虚警链接：http://arxiv.org/pdf/2304.03416v1

*与cs.SD语音【9】为同一篇

作者：Yashas Malur Saidutta,Rakshith Sharma Srinivasa,Ching-Hua Lee,Chouchang Yang,Yilin Shen,Hongxia Jin机构：Samsung Research America, Mountain View, CA, USA, ©, IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including备注：Accepted for publication in ICASSP 2023摘要：关键词定位系统连续地处理音频流以检测关键词。在设计这样的系统中最具挑战性的任务之一是减少当系统错误地注册关键字而不管关键字没有被说出时发生的假警报（FA）。在本文中，我们提出了一个简单而优雅的解决方案，这个问题，从法律的全概率。我们表明，现有的深度关键字发现机制可以通过连续细化来改进，其中系统首先对输入音频是否是语音进行分类，其次是输入是否是关键字，最后对说出的关键字进行分类。我们表明，在多个模型的大小范围从13K参数到2.41M参数，连续细化技术减少FA的一个因素，8在域内举行了FA数据，并高达7的因素，在域外（OOD）FA数据。此外，我们提出的方法是“即插即用”，可以应用于任何深度关键字定位模型。摘要：Keyword spotting systems continuously process audio streams to detect keywords. One of the most challenging tasks in designing such systems is to reduce False Alarm (FA) which happens when the system falsely registers a keyword despite the keyword not being uttered. In this paper, we propose a simple yet elegant solution to this problem that follows from the law of total probability. We show that existing deep keyword spotting mechanisms can be improved by Successive Refinement, where the system first classifies whether the input audio is speech or not, followed by whether the input is keyword-like or not, and finally classifies which keyword was uttered. We show across multiple models with size ranging from 13K parameters to 2.41M parameters, the successive refinement technique reduces FA by up to a factor of 8 on in-domain held-out FA data, and up to a factor of 7 on out-of-domain (OOD) FA data. Further, our proposed approach is "plug-and-play" and can be applied to any deep keyword spotting model.

【4】 Anomalous Sound Detection using Audio Representation with Machine ID based Contrastive Learning Pretraining标题：基于机器ID的对比学习预训练音频表示法检测异常音链接：http://arxiv.org/pdf/2304.03588v1

*与cs.SD语音【1】为同一篇

作者：Jian Guan,Feiyang Xiao,Youde Liu,Qiaoxi Zhu,Wenwu Wang机构：Group of Intelligent Signal Processing, College of Computer Science and Technology, Harbin Engineering University, Harbin, China, School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China摘要：用于异常声音检测的现有对比学习方法通过使用样本的增强之间的对比度（例如，具有时间或频率掩蔽）。然而，由于缺乏机器声音的物理特性，它们可能会被增强的数据所偏置，从而限制了检测性能。本文使用对比学习来细化每个机器ID的音频表示，而不是每个音频样本。所提出的两阶段方法使用对比学习来通过结合机器ID和自监督ID分类器来预训练音频表示模型以微调所学习的模型。实验表明，该方法优于使用对比学习或自学习的最新方法。监督分类在DCASE 2020挑战任务2数据集上的整体异常检测性能和稳定性。摘要：Existing contrastive learning methods for anomalous sound detection refine the audio representation of each audio sample by using the contrast between the samples' augmentations (e.g., with time or frequency masking). However, they might be biased by the augmented data, due to the lack of physical properties of machine sound, thereby limiting the detection performance. This paper uses contrastive learning to refine audio representations for each machine ID, rather than for each audio sample. The proposed two-stage method uses contrastive learning to pretrain the audio representation model by incorporating machine ID and a self-supervised ID classifier to fine-tune the learnt model, while enhancing the relation between audio features from the same ID. Experiments show that our method outperforms the state-of-the-art methods using contrastive learning or self-supervised classification in overall anomaly detection performance and stability on DCASE 2020 Challenge Task2 dataset.

【5】 Graph Attention for Automated Audio Captioning标题：自动音频字幕的图形注意链接：http://arxiv.org/pdf/2304.03586v1

*与cs.SD语音【2】为同一篇

作者：Feiyang Xiao,Jian Guan,Qiaoxi Zhu,Wenwu Wang摘要：现有技术的音频字幕方法通常使用具有预训练的音频神经网络（PANN）的编码器-解码器结构作为用于特征提取的编码器。然而，PANN中使用的卷积运算在捕获音频信号内的长时间依赖性方面受到限制，从而导致音频字幕的潜在性能下降。这封信提出了一种新的方法，使用图形注意力（GraphAC）的编码器-解码器为基础的音频字幕。在编码器中，在PANN之后引入图形注意模块，以通过邻接图来学习上下文关联（即，不同时间帧上的音频特征之间的依赖性），并且使用前k掩码来减轻来自噪声节点的干扰。学习的上下文关联导致更有效的特征表示与特征节点聚合。结果，解码器可以基于从音频信号学习的上下文关联来预测关于声学场景和事件的重要语义信息。实验结果表明，GraphAC优于PANN作为编码器的最先进的方法，这要归功于将图形注意力模块并入编码器中以捕获音频信号内的长时间依赖性。源代码可在www.example.com获得https: github.com LittleFlyingSheep GraphAC。摘要：State-of-the-art audio captioning methods typically use the encoder-decoder structure with pretrained audio neural networks (PANNs) as encoders for feature extraction. However, the convolution operation used in PANNs is limited in capturing the long-time dependencies within an audio signal, thereby leading to potential performance degradation in audio captioning. This letter presents a novel method using graph attention (GraphAC) for encoder-decoder based audio captioning. In the encoder, a graph attention module is introduced after the PANNs to learn contextual association (i.e. the dependency among the audio features over different time frames) through an adjacency graph, and a top-k mask is used to mitigate the interference from noisy nodes. The learnt contextual association leads to a more effective feature representation with feature node aggregation. As a result, the decoder can predict important semantic information about the acoustic scene and events based on the contextual associations learned from the audio signal. Experimental results show that GraphAC outperforms the state-of-the-art methods with PANNs as the encoders, thanks to the incorporation of the graph attention module into the encoder for capturing the long-time dependencies within the audio signal. The source code is available at https: github.com LittleFlyingSheep GraphAC.

【6】 On-site Noise Exposure technique for noise-robust machine fault classification标题：现场噪声暴露技术在机器故障分类中的应用链接：http://arxiv.org/pdf/2304.03522v1

*与cs.SD语音【3】为同一篇

作者：Wonjun Yi,Jung-Woo Choi机构：School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Republic of Korea备注：Presented at 24th International Congress on Acoustics (ICA) 2022摘要：故障声音的现场分类是机器健康监测和诊断中的重要问题。然而，在诸如工厂的嘈杂环境中，机器声音总是与环境噪声混合，并且当机器不运行时可能存在仅噪声时段。因此，基于深度神经网络（DNN）的故障分类器必须能够区分噪声和机器声音，并且对混合噪声具有鲁棒性。为了解决这些问题，我们研究了现场噪声暴露（ONE），将DNN模型暴露在机器运行的相同环境中记录的噪声中。与离群值暴露技术一样，噪声暴露训练DNN分类器以针对仅噪声数据产生均匀的预测概率分布。在推理期间，由ONE训练的DNN分类器输出最大softmax概率作为噪声分数，并确定仅噪声时段。我们混合机器声音和ToyADMOS2数据集的噪声来模拟高噪声数据。评估由ONE训练的基于ResNet的分类器，并与由其他分布外检测技术训练的分类器进行比较。测试结果表明，将模型暴露于现场噪声可以使模型比使用其他噪声或检测技术更鲁棒。摘要：In-situ classification of faulty sounds is an important issue in machine health monitoring and diagnosis. However, in a noisy environment such as a factory, machine sound is always mixed up with environmental noises, and noise-only periods can exist when a machine is not in operation. Therefore, a deep neural network (DNN)-based fault classifier has to be able to distinguish noise from machine sound and be robust to mixed noises. To deal with these problems, we investigate on-site noise exposure (ONE) that exposes a DNN model to the noises recorded in the same environment where the machine operates. Like the outlier exposure technique, noise exposure trains a DNN classifier to produce a uniform predicted probability distribution against noise-only data. During inference, the DNN classifier trained by ONE outputs the maximum softmax probability as the noise score and determines the noise-only period. We mix machine sound and noises of the ToyADMOS2 dataset to simulate highly noisy data. A ResNet-based classifier trained by ONE is evaluated and compared with those trained by other out-of-distribution detection techniques. The test results show that exposing a model to on-site noises can make a model more robust than using other noises or detection techniques.

【7】 Adoption of AI Technology in the Music Mixing Workflow: An Investigation标题：人工智能技术在混音工作流中的应用研究链接：http://arxiv.org/pdf/2304.03407v1

*与cs.SD语音【4】为同一篇

作者：Soumya Sai Vanka,Maryam Safi,Jean-Baptiste Rolland,George Fazekas机构：Queen Mary University of London, London, UK, Steinberg Media Technologies GmbH, Hamburg, Germany备注：To be published at the 154th AES Europe Convention, 2023 in Helsinki, Finland摘要：人工智能（AI）技术在音乐行业的整合正在推动音乐创作、制作和混合方式的重大变革。本研究调查了AI在混合工作流中的现状及其在不同用户群体中的采用情况。本研究透过半结构式访谈、问卷调查及网路论坛分析，确认三个使用者群，分别为业馀使用者、专业使用者及专业使用者。我们的研究结果表明，虽然人工智能混合工具可以简化过程并为业余爱好者提供不错的结果，但专业人士寻求精确的控制和定制选项，而专业人士除了辅助和协作技术外还希望控制和定制选项。该研究为不同用户群体设计有效的AI混合工具提供了策略，并概述了未来的发展方向。摘要：The integration of artificial intelligence (AI) technology in the music industry is driving a significant change in the way music is being composed, produced and mixed. This study investigates the current state of AI in the mixing workflows and its adoption by different user groups. Through semi-structured interviews, a questionnaire-based study, and analyzing web forums, the study confirms three user groups comprising amateurs, pro-ams, and professionals. Our findings show that while AI mixing tools can simplify the process and provide decent results for amateurs, pro-ams seek precise control and customization options, while professionals desire control and customization options in addition to assistive and collaborative technologies. The study provides strategies for designing effective AI mixing tools for different user groups and outlines future directions.

【8】 DSVAE: Interpretable Disentangled Representation for Synthetic Speech Detection标题：DSVAE：用于合成语音检测的可解释解缠表示链接：http://arxiv.org/pdf/2304.03323v1

*与cs.SD语音【5】为同一篇

作者：Amit Kumar Singh Yadav,Kratika Bhagtani,Ziyue Xiang,Paolo Bestagini,Stefano Tubaro,Edward J. Delp机构：†Video and Image Processing Lab (VIPER), School of Electrical and Computer Engineering, Purdue University, West Lafayette, Indiana, USA, ‡Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy摘要：用于生成与从人类说话者记录的语音在感知上不可区分的高质量合成语音信号的工具是容易获得的。已经提出了几种用于检测合成语音的方法。这些方法中的许多方法将深度学习方法用作黑盒，而不为它们所做的决策提供推理。这限制了这些方法的可解释性。在本文中，我们提出了解纠缠频谱图变分自动编码器（DSVAE），这是一个两阶段的训练变分自动编码器，处理语音频谱图使用解纠缠表示学习生成可解释的表示的语音信号检测合成语音。DSVAE还创建了一个激活图，以突出显示区分合成和真实人类语音信号的频谱图区域。我们使用ASVspoof2019数据集评估了从DSVAE获得的表示。实验结果表明，在11个未知语音合成器中，我们对6个已知语音合成器和10个未知语音合成器的合成语音检测准确率都在98%以上。我们还可视化表示从DSVAE获得17个不同的语音合成器，并验证它们确实是可解释的，并区分真正的合成语音从每个合成器。摘要：Tools to generate high quality synthetic speech signal that is perceptually indistinguishable from speech recorded from human speakers are easily available. Several approaches have been proposed for detecting synthetic speech. Many of these approaches use deep learning methods as a black box without providing reasoning for the decisions they make. This limits the interpretability of these approaches. In this paper, we propose Disentangled Spectrogram Variational Auto Encoder (DSVAE) which is a two staged trained variational autoencoder that processes spectrograms of speech using disentangled representation learning to generate interpretable representations of a speech signal for detecting synthetic speech. DSVAE also creates an activation map to highlight the spectrogram regions that discriminate synthetic and bona fide human speech signals. We evaluated the representations obtained from DSVAE using the ASVspoof2019 dataset. Our experimental results show high accuracy ( 98%) on detecting synthetic speech from 6 known and 10 out of 11 unknown speech synthesizers. We also visualize the representation obtained from DSVAE for 17 different speech synthesizers and verify that they are indeed interpretable and discriminate bona fide and synthetic speech from each of the synthesizers.

【9】 Automatic Detection of Reactions to Music via Earable Sensing标题：通过听觉感知自动检测对音乐的反应链接：http://arxiv.org/pdf/2304.03295v1

*与cs.SD语音【6】为同一篇

作者：Euihyoek Lee,Chulhong Min,Jeaseung Lee,Jin Yu,Seungwoo Kang机构：JAESEUNG LEE, KOREATECH, Republic of Korea摘要：本文介绍了一种新颖的系统--GrooveMeter，它通过可听见的感知来自动检测对音乐的声音和动作反应，并支持音乐参与度感知应用。为此，我们使用智能耳塞作为传感设备，这已经被广泛用于音乐收听，并通过利用耳塞上的惯性测量单元（IMU）和麦克风来设计反应检测技术。为了探索日常音乐听的情况下的反应，我们收集了第一种数据集MusicReactionSet，其中包含926分钟长的IMU和30名参与者的音频数据。通过数据集，我们发现了一组独特的挑战，即使用音频和运动传感准确和鲁棒地检测音乐聆听反应。我们设计了复杂的处理管道，使反应检测准确高效。我们提出了一个全面的评估，以检查反应检测和系统成本的性能。它表明，GrooveMeter实现了宏观F1分数为0.89的声音反应和0.81的运动反应留一个主题的交叉验证。更重要的是，与其他方法相比，GrooveMeter显示出更高的准确性和鲁棒性。我们还表明，我们的过滤方法减少了50%或更多的能量开销。最后，我们通过案例研究展示了潜在的用例。摘要：We present GrooveMeter, a novel system that automatically detects vocal and motion reactions to music via earable sensing and supports music engagement-aware applications. To this end, we use smart earbuds as sensing devices, which are already widely used for music listening, and devise reaction detection techniques by leveraging an inertial measurement unit (IMU) and a microphone on earbuds. To explore reactions in daily music-listening situations, we collect the first kind of dataset, MusicReactionSet, containing 926-minute-long IMU and audio data with 30 participants. With the dataset, we discover a set of unique challenges in detecting music listening reactions accurately and robustly using audio and motion sensing. We devise sophisticated processing pipelines to make reaction detection accurate and efficient. We present a comprehensive evaluation to examine the performance of reaction detection and system cost. It shows that GrooveMeter achieves the macro F1 scores of 0.89 for vocal reaction and 0.81 for motion reaction with leave-one-subject-out cross-validation. More importantly, GrooveMeter shows higher accuracy and robustness compared to alternative methods. We also show that our filtering approach reduces 50% or more of the energy overhead. Finally, we demonstrate the potential use cases through a case study.

【10】 A2D: Anywhere Anytime Drumming标题：A2D：随时随地打鼓链接：http://arxiv.org/pdf/2304.03289v1

*与cs.SD语音【7】为同一篇

作者：Harel Yadid,Almog Algranti,Mark Levin,Ayal Taitler机构：Technion – Israel Institute of Technology, Haifa, Israel, University of Toronto, Department of Mechanical and and Industrial Engineering, Toronto, Ontario, Canada摘要：鼓包，只有大约100年的历史，是流行音乐，摇滚乐和爵士乐等许多音乐流派中的流行乐器。然而，拥有一个套件的道路是昂贵的，无论是在财政上还是在空间上。此外，鼓是更难以走动相比，其他文书，因为他们不适合到一个单一的袋。我们提出了一个无鼓的方法，只使用两根棍子和智能手机或网络摄像头提供一个空气鼓的经验。检测算法将深度学习工具与跟踪方法相结合，以增强用户体验。基于定量和定性测试与人在循环中，我们表明，我们的系统有零失误的初学者水平发挥和忽略不计的失误，先进的水平发挥。此外，我们有限的人体试验为未来的研究提供了潜在的方向。摘要：The drum kit, which has only been around for around 100 years, is a popular instrument in many music genres such as pop, rock, and jazz. However, the road to owning a kit is expensive, both financially and space-wise. Also, drums are more difficult to move around compared to other instruments, as they do not fit into a single bag. We propose a no-drums approach that uses only two sticks and a smartphone or a webcam to provide an air-drumming experience. The detection algorithm combines deep learning tools with tracking methods for an enhanced user experience. Based on both quantitative and qualitative testing with humans-in-the-loop, we show that our system has zero misses for beginner level play and negligible misses for advanced level play. Additionally, our limited human trials suggest potential directions for future research.

机器翻译，仅供参考