今日论文合集：cs.SD语音2篇，eess.AS音频处理7篇。
cs.SD语音
【1】 Self-Supervised Music Source Separation Using Vector-Quantized Source  Category Estimates标题：基于矢量量化信源类别估计的自监督音乐信源分离链接：https://arxiv.org/abs/2311.13058
作者：Marco Pasini,Stefan Lattner,George Fazekas备注：4 pages, 2 figures, 1 table; Accepted at the 37th Conference on Neural Information Processing Systems (2023), Machine Learning for Audio Workshop摘要：音乐源分离的重点是从复合音轨中提取不同的声音元素。从历史上看，许多方法都是基于监督学习，需要标记数据，这有时会限制其多样性。最近的方法已经深入研究了利用一个或多个音频样本来帮助分离的N-shot技术。然而，这些方法中的一些方法的挑战是在推理期间需要音频查询，使得它们不太适合具有不同音色和效果的流派。本文提供了一个自我监督的音乐源分离系统，消除了在推理时间的音频查询的需要的概念验证。在训练阶段，虽然它采用了基于查询的方法，但我们引入了一种修改，用矢量量化（VQ）表示代替查询音频的连续嵌入。通过VQ的码本大小确定的多达N个类的端到端训练，该模型旨在有效地对乐器类进行分类。在推理过程中，输入被划分为N个源，根据混合的乐器组成，一些源可能未被利用。这种方法提出了一种替代途径，考虑跨不同音乐流派的源分离。我们在线提供示例和其他结果。摘要：Music source separation is focused on extracting distinct sonic elements from composite tracks. Historically, many methods have been grounded in supervised learning, necessitating labeled data, which is occasionally constrained in its diversity. More recent methods have delved into N-shot techniques that utilize one or more audio samples to aid in the separation. However, a challenge with some of these methods is the necessity for an audio query during inference, making them less suited for genres with varied timbres and effects. This paper offers a proof-of-concept for a self-supervised music source separation system that eliminates the need for audio queries at inference time. In the training phase, while it adopts a query-based approach, we introduce a modification by substituting the continuous embedding of query audios with Vector Quantized (VQ) representations. Trained end-to-end with up to N classes as determined by the VQ's codebook size, the model seeks to effectively categorise instrument classes. During inference, the input is partitioned into N sources, with some potentially left unutilized based on the mix's instrument makeup. This methodology suggests an alternative avenue for considering source separation across diverse music genres. We provide examples and additional results online.
【2】 Spatial Audio and Individualized HRTFs using a Convolutional Neural  Network (CNN)标题：利用卷积神经网络(CNN)实现空间音频和个性化HRTF链接：https://arxiv.org/abs/2311.13397
作者：Ludovic Pirard摘要：空间音频和三维声音渲染技术在沉浸式音频体验中起着关键和重要的作用。头部相关传递函数（Head Related Transfer Functions，HRTF）是一种声学滤波器，它代表了声音如何与个人独特的头部和耳朵解剖结构相互作用。使用符合受试者解剖特征的HRTF对于确保个性化和独特的空间体验至关重要。本文提出了一种基于卷积神经网络（CNN）从人耳图像中自动提取人体测量特征的HRTF个性化方法。首先，实现并测试了CNN，以评估机器学习在耳图像上定位地标的性能。I-BUG数据集，包含相应的55个标志的耳朵图像，被用来训练和测试的神经网络。随后，选择12个相关标志，以对应于由HUTUBS数据库建立的7个特定人体测量值。这些地标用作像素距离计算的参考，以便从耳朵图像中检索人体测量结果。一旦从耳朵图像中提取了以像素为单位的7个距离，则使用转换因子将它们转换为厘米，实施最佳匹配方法向量，计算116只耳朵的数据库中的每组的欧几里得距离，其中它们的对应的7个人体测量值由HUTUBS数据库提供。可以识别人体测量的最接近匹配，并且可以获得相应的HRTF集合以用于个性化使用。该方法是在其有效性，而不是结果的准确性进行评估。每个阶段的概念范围都已得到验证和证实，可以正确运作。审查和挑战过程中的各个步骤和可用元素，以定义为所需任务设计的更大的算法实体。摘要：Spatial audio and 3-Dimensional sound rendering techniques play a pivotal and essential role in immersive audio experiences. Head-Related Transfer Functions (HRTFs) are acoustic filters which represent how sound interacts with an individual's unique head and ears anatomy. The use of HRTFs compliant to the subjects anatomical traits is crucial to ensure a personalized and unique spatial experience. This work proposes the implementation of an HRTF individualization method based on anthropometric features automatically extracted from ear images using a Convolutional Neural Network (CNN). Firstly, a CNN is implemented and tested to assess the performance of machine learning on positioning landmarks on ear images. The I-BUG dataset, containing ear images with corresponding 55 landmarks, was used to train and test the neural network. Subsequently, 12 relevant landmarks were selected to correspond to 7 specific anthropometric measurements established by the HUTUBS database. These landmarks serve as a reference for distance computation in pixels in order to retrieve the anthropometric measurements from the ear images. Once the 7 distances in pixels are extracted from the ear image, they are converted in centimetres using conversion factors, a best match method vector is implemented computing the Euclidean distance for each set in a database of 116 ears with their corresponding 7 anthropometric measurements provided by the HUTUBS database. The closest match of anthropometry can be identified and the corresponding set of HRTFs can be obtained for personnalized use. The method is evaluated in its validity instead of the accuracy of the results. The conceptual scope of each stage has been verified and substantiated to function correctly. The various steps and the available elements in the process are reviewed and challenged to define a greater algorithm entity designed for the desired task.
eess.AS音频处理
【1】 Sparsity-Driven EEG Channel Selection for Brain-Assisted Speech  Enhancement标题：脑辅助语音增强中稀疏驱动的脑电通道选择链接：https://arxiv.org/abs/2311.13436
作者：Jie Zhang,Qing-Tian Xu,Zhen-Hua Ling备注：arXiv admin note: text overlap with arXiv:2305.09994摘要：语音增强技术作为提高语音质量的前端技术被广泛应用于许多音频系统中，但在没有说话人身份先验信息的情况下，提取多说话人的目标语音仍然是一个难题。听觉注意解码表明，听者的脑电（EEG）可以内隐地揭示出被关注的说话人。因此，在这项工作中，我们提出了一种新的端到端的脑辅助语音增强网络（BASEN），它结合了听众的EEG信号，并采用了时间卷积网络与卷积多层交叉注意模块融合EEG音频特征。考虑到具有稀疏通道的EEG帽具有多种益处，并且在实践中许多电极可能贡献很少，我们进一步提出了两种通道选择方法，称为残差Gumbel选择和卷积正则化选择。它们分别致力于解决训练不稳定和重复信道选择的问题。在公共数据集上的实验结果表明，所提出的基线BASEN优于现有的方法。所提出的通道选择方法可以显着减少信息EEG通道的数量，对性能的影响可以忽略不计。摘要：Speech enhancement is widely used as a front-end to improve the speech quality in many audio systems, while it is still hard to extract the target speech in multi-talker conditions without prior information on the speaker identity. It was shown by auditory attention decoding that the attended speaker can be revealed by the electroencephalogram (EEG) of the listener implicitly. In this work, we therefore propose a novel end-to-end brain-assisted speech enhancement network (BASEN), which incorporates the listeners' EEG signals and adopts a temporal convolutional network together with a convolutional multi-layer cross attention module to fuse EEG-audio features. Considering that an EEG cap with sparse channels exhibits multiple benefits and in practice many electrodes might contribute marginally, we further propose two channel selection methods, called residual Gumbel selection and convolutional regularization selection. They are dedicated to tackling the issues of training instability and duplicated channel selections, respectively. Experimental results on a public dataset show the superiority of the proposed baseline BASEN over existing approaches. The proposed channel selection methods can significantly reduce the amount of informative EEG channels with a negligible impact on the performance.
【2】 Spatial Audio and Individualized HRTFs using a Convolutional Neural  Network (CNN)标题：使用卷积神经网络（CNN）的空间音频和个性化HRTF链接：https://arxiv.org/abs/2311.13397
作者：Ludovic Pirard摘要：空间音频和三维声音渲染技术在沉浸式音频体验中起着关键和重要的作用。头部相关传递函数（Head Related Transfer Functions，HRTF）是一种声学滤波器，它代表了声音如何与个人独特的头部和耳朵解剖结构相互作用。使用符合受试者解剖特征的HRTF对于确保个性化和独特的空间体验至关重要。本文提出了一种基于卷积神经网络（CNN）从人耳图像中自动提取人体测量特征的HRTF个性化方法。首先，实现并测试了CNN，以评估机器学习在耳图像上定位地标的性能。I-BUG数据集，包含相应的55个标志的耳朵图像，被用来训练和测试的神经网络。随后，选择12个相关标志，以对应于由HUTUBS数据库建立的7个特定人体测量值。这些地标用作像素距离计算的参考，以便从耳朵图像中检索人体测量结果。一旦从耳朵图像中提取了以像素为单位的7个距离，则使用转换因子将它们转换为厘米，实施最佳匹配方法向量，计算116只耳朵的数据库中的每组的欧几里得距离，其中它们的对应的7个人体测量值由HUTUBS数据库提供。可以识别人体测量的最接近匹配，并且可以获得相应的HRTF集合以用于个性化使用。该方法是在其有效性，而不是结果的准确性进行评估。每个阶段的概念范围都已得到验证和证实，可以正确运作。审查和挑战过程中的各个步骤和可用元素，以定义为所需任务设计的更大的算法实体。摘要：Spatial audio and 3-Dimensional sound rendering techniques play a pivotal and essential role in immersive audio experiences. Head-Related Transfer Functions (HRTFs) are acoustic filters which represent how sound interacts with an individual's unique head and ears anatomy. The use of HRTFs compliant to the subjects anatomical traits is crucial to ensure a personalized and unique spatial experience. This work proposes the implementation of an HRTF individualization method based on anthropometric features automatically extracted from ear images using a Convolutional Neural Network (CNN). Firstly, a CNN is implemented and tested to assess the performance of machine learning on positioning landmarks on ear images. The I-BUG dataset, containing ear images with corresponding 55 landmarks, was used to train and test the neural network. Subsequently, 12 relevant landmarks were selected to correspond to 7 specific anthropometric measurements established by the HUTUBS database. These landmarks serve as a reference for distance computation in pixels in order to retrieve the anthropometric measurements from the ear images. Once the 7 distances in pixels are extracted from the ear image, they are converted in centimetres using conversion factors, a best match method vector is implemented computing the Euclidean distance for each set in a database of 116 ears with their corresponding 7 anthropometric measurements provided by the HUTUBS database. The closest match of anthropometry can be identified and the corresponding set of HRTFs can be obtained for personnalized use. The method is evaluated in its validity instead of the accuracy of the results. The conceptual scope of each stage has been verified and substantiated to function correctly. The various steps and the available elements in the process are reviewed and challenged to define a greater algorithm entity designed for the desired task.
【3】 Performance Analysis Of Binaural Signal Matching (BSM) in the  Time-Frequency Domain标题：双耳信号匹配(BSM)时频域性能分析链接：https://arxiv.org/abs/2311.13390
作者：Ami Berger,Vladimir Tourbabin,Jacob Donley,Zamir Ben-Hur,Boaz Rafaely备注：None摘要：随着在电话会议、娱乐和虚拟现实中的应用如雨后春笋般涌现，空间音频的捕获和再现变得越来越流行。许多双耳再现方法已被开发和广泛研究的球形和其他专门设计的阵列。然而，最近越来越受欢迎的可穿戴和移动阵列需要开发用于这些阵列的双耳再现方法。一种这样的方法是双耳信号匹配（BSM）。然而，到目前为止，这种方法只研究了固定的匹配滤波器设计的长音频记录。为了使BSM方法更好地适应动态环境，本文在时频域分析了参数化声场的BSM。本文提出了一个被分解成其直接和混响分量的声场上实现的BSM方法的结果，并将此实现与整个声场计算的BSM进行比较，以比较在模拟环境中混响语音的双耳再现的性能。摘要：The capture and reproduction of spatial audio is becoming increasingly popular, with the mushrooming of applications in teleconferencing, entertainment and virtual reality. Many binaural reproduction methods have been developed and studied extensively for spherical and other specially designed arrays. However, the recent increased popularity of wearable and mobile arrays requires the development of binaural reproduction methods for these arrays. One such method is binaural signal matching (BSM). However, to date this method has only been investigated with fixed matched filters designed for long audio recordings. With the aim of making the BSM method more adaptive to dynamic environments, this paper analyzes BSM with a parameterized sound-field in the time-frequency domain. The paper presents results of implementing the BSM method on a sound-field that was decomposed into its direct and reverberant components, and compares this implementation with the BSM computed for the entire sound-field, to compare performance for binaural reproduction of reverberant speech in a simulated environment.
【4】 Deep Audio Zooming: Beamwidth-Controllable Neural Beamformer标题：深度音频变焦：波束宽度可控的神经波束形成器链接：https://arxiv.org/abs/2311.13075
作者：Meng Yu,Dong Yu备注：6 pages, 5 figures摘要：音频缩放是一种信号处理技术，能够选择性地聚焦和增强来自指定区域的声音信号，衰减其他声音信号。虽然传统的波束成形和神经波束成形技术以创建定向阵列为中心，需要指定单一目标方向，但它们通常忽略了定义角度区域的视场（FOV）的概念。在本文中，我们提出了一个简单而有效的FOV功能，合并用户定义的字段内的所有方向属性。同时，我们还引入了一个反FOV功能，用于捕获所需视野之外的方向性方面。这些进步确保了精确的声音捕获，特别是强调了FOV的边界，并保证了在用户定义的范围内增强所有所需声源的捕获。实验结果表明，引入的角度FOV功能和它的无缝纳入到一个低功耗的子带模型，适合于实时应用的有效性？选项。摘要：Audio zooming, a signal processing technique, enables selective focusing and enhancement of sound signals from a specified region, attenuating others. While traditional beamforming and neural beamforming techniques, centered on creating a directional array, necessitate the designation of a singular target direction, they often overlook the concept of a field of view (FOV), that defines an angular area. In this paper, we proposed a simple yet effective FOV feature, amalgamating all directional attributes within the user-defined field. In conjunction, we've introduced a counter FOV feature capturing directional aspects outside the desired field. Such advancements ensure refined sound capture, particularly emphasizing the FOV's boundaries, and guarantee the enhanced capture of all desired sound sources inside the user-defined field. The results from the experiment demonstrate the efficacy of the introduced angular FOV feature and its seamless incorporation into a low-power subband model suited for real-time applica?tions.
【5】 FedCPC: An Effective Federated Contrastive Learning Method for Privacy  Preserving Early-Stage Alzheimer's Speech Detection标题：FedCPC：一种用于阿尔茨海默病早期语音检测的有效隐私保护联合对比学习方法链接：https://arxiv.org/abs/2311.13043
作者：Wenqing Wei,Zhengdong Yang,Yuan Gao,Jiyi Li,Chenhui Chu,Shogo Okada,Sheng Li备注：accepted in IEEE-ASRU2023摘要：None摘要：The early-stage Alzheimer's disease (AD) detection has been considered an important field of medical studies. Like traditional machine learning methods, speech-based automatic detection also suffers from data privacy risks because the data of specific patients are exclusive to each medical institution. A common practice is to use federated learning to protect the patients' data privacy. However, its distributed learning process also causes performance reduction. To alleviate this problem while protecting user privacy, we propose a federated contrastive pre-training (FedCPC) performed before federated training for AD speech detection, which can learn a better representation from raw data and enables different clients to share data in the pre-training and training stages. Experimental results demonstrate that the proposed methods can achieve satisfactory performance while preserving data privacy.
【6】 Self-Supervised Music Source Separation Using Vector-Quantized Source  Category Estimates标题：基于矢量量化信源类别估计的自监督音乐信源分离链接：https://arxiv.org/abs/2311.13058
作者：Marco Pasini,Stefan Lattner,George Fazekas备注：4 pages, 2 figures, 1 table; Accepted at the 37th Conference on Neural Information Processing Systems (2023), Machine Learning for Audio Workshop摘要：None摘要：Music source separation is focused on extracting distinct sonic elements from composite tracks. Historically, many methods have been grounded in supervised learning, necessitating labeled data, which is occasionally constrained in its diversity. More recent methods have delved into N-shot techniques that utilize one or more audio samples to aid in the separation. However, a challenge with some of these methods is the necessity for an audio query during inference, making them less suited for genres with varied timbres and effects. This paper offers a proof-of-concept for a self-supervised music source separation system that eliminates the need for audio queries at inference time. In the training phase, while it adopts a query-based approach, we introduce a modification by substituting the continuous embedding of query audios with Vector Quantized (VQ) representations. Trained end-to-end with up to N classes as determined by the VQ's codebook size, the model seeks to effectively categorise instrument classes. During inference, the input is partitioned into N sources, with some potentially left unutilized based on the mix's instrument makeup. This methodology suggests an alternative avenue for considering source separation across diverse music genres. We provide examples and additional results online.
【7】 A versatile circuit for emulating active biological dendrites applied to  sound localisation and neuron imitation标题：一种用于声音定位和神经元模拟的多功能生物树突模拟电路链接：https://arxiv.org/abs/2311.12861
作者：Daniel John Mannion备注：13 pages. 6 Figues in main text, 1 figure in supplementary materials摘要：None摘要：Sophisticated machine learning struggles to transition onto battery-operated devices due to the high-power consumption of neural networks. Researchers have turned to neuromorphic engineering, inspired by biological neural networks, for more efficient solutions. While previous research focused on artificial neurons and synapses, an essential component has been overlooked: dendrites. Dendrites transmit inputs from synapses to the neuron's soma, applying both passive and active transformations. However, neuromorphic circuits replace these sophisticated computational channels with metallic interconnects. In this study, we introduce a versatile circuit that emulates a segment of a dendrite which exhibits gain, introduces delays, and performs integration. We show how sound localisation - a biological example of dendritic computation - is not possible with the existing passive dendrite circuits but can be achieved using this proposed circuit. We also find that dendrites can form bursting neurons. This significant discovery suggests the potential to fabricate neural networks solely comprised of dendrite circuits.
机器翻译，仅供参考