今天跟大家分享一篇语音相关的论文合集：cs.SD语音3篇，eess.AS音频处理5篇。
【1】 The BUCEA Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2022
标题：用于2022年VoxCeleb说话人识别挑战赛的BUCEA说话人二值化系统
链接：https://arxiv.org/abs/2209.09635
作者：Ruohua Zhou,Yuxuan Du,Chenlei Hu机构：School of Electrical and Information Engineering, Beijing University of Civil Engineering and Architecture, China摘要：本文介绍了2022年VoxCeleb说话人识别挑战赛的BUCEA说话人识别系统。Voxsrc-22提供了VoxConverse的开发集和测试集，我们主要使用VoxConverse的测试集进行参数调整。该系统由语音活动检测、说话人嵌入提取、聚类方法、重叠语音检测和结果融合等模块组成。在不考虑重叠的情况下，将Dover-LAP（Diarization Output Voting Error Reduction）方法应用于系统融合，最后进行重叠语音的检测和处理。我们的最佳系统在VoxSRC 2022评估集上分别实现了5.48%的二值化错误率（DER）和32.1%的Jaccard错误率（JER）。摘要：This paper describes the BUCEA speaker diarization system for the 2022 VoxCeleb Speaker Recognition Challenge. Voxsrc-22 provides the development set and test set of VoxConverse, and we mainly use the test set of VoxConverse for parameter adjustment. Our system consists of several modules, including speech activity detection (VAD), speaker embedding extractor, clustering methods, overlapping speech detection (OSD), and result fusion. Without considering overlap, the Dover-LAP (short for Diarization Output Voting Error Reduction) method was applied to system fusion, and overlapping speech detection and processing were finally carried out. Our best system achieves a diarization error rate (DER) of 5.48% and a Jaccard error rate (JER) of 32.1% on the VoxSRC 2022 evaluation set respectively.
【2】 A Closer Look at Weakly-Supervised Audio-Visual Source Localization
标题：弱监督视听信源定位算法研究
链接：https://arxiv.org/abs/2209.09634
作者：Shentong Mo,Pedro Morgado机构：Carnegie Mellon University, University of Wisconsin-Madison摘要：音视频源定位是一项具有挑战性的任务，其目的是预测视频中的视觉声源的位置。由于收集探测对象的地面实况注释可能是昂贵的，因此近年来已经提出了大量的弱监督定位方法，这些方法可以通过利用音频和视频信号的自然共现来从没有边界框注释的数据集学习。尽管人们对此很感兴趣，但流行的评估协议有两个主要缺陷。首先，它们允许使用完全注释的数据集来执行早期停止，因此显著增加了训练所需的注释工作。第二，当前的评估度量假设在所有时间都存在声源。当然，这是一个不切实际的假设，因此需要更好的度量来捕获模型在没有可见声源的（负）样本上的性能。为了实现这一目标，我们扩展了常用基准测试集Flickr SoundNet和VGG-Sound Sources，以便包括负面样本，并使用平衡定位准确性和召回率的指标来衡量性能。使用新的协议，我们对现有方法进行了广泛的评估，发现大多数现有工作不能识别阴性结果，并且存在明显的过拟合问题（严重依赖于早期停止以获得最佳结果）。我们还提出了一种新的视觉声源定位方法，解决了这两个问题。特别地，我们发现，通过极端的视觉丢失和动量编码器的使用，所提出的方法有效地克服了过拟合，并在Flickr SoundNet和VGG-Sound Source上建立了新的最先进的性能。代码和预先培训的模型可在www.example.com上获得https://github.com/stoneMo/SLAVC。摘要：Audio-visual source localization is a challenging task that aims to predict the location of visual sound sources in a video. Since collecting ground-truth annotations of sounding objects can be costly, a plethora of weakly-supervised localization methods that can learn from datasets with no bounding-box annotations have been proposed in recent years, by leveraging the natural co-occurrence of audio and visual signals. Despite significant interest, popular evaluation protocols have two major flaws. First, they allow for the use of a fully annotated dataset to perform early stopping, thus significantly increasing the annotation effort required for training. Second, current evaluation metrics assume the presence of sound sources at all times. This is of course an unrealistic assumption, and thus better metrics are necessary to capture the model's performance on (negative) samples with no visible sound sources. To accomplish this, we extend the test set of popular benchmarks, Flickr SoundNet and VGG-Sound Sources, in order to include negative samples, and measure performance using metrics that balance localization accuracy and recall. Using the new protocol, we conducted an extensive evaluation of prior methods, and found that most prior works are not capable of identifying negatives and suffer from significant overfitting problems (rely heavily on early stopping for best results). We also propose a new approach for visual sound source localization that addresses both these problems. In particular, we found that, through extreme visual dropout and the use of momentum encoders, the proposed approach combats overfitting effectively, and establishes a new state-of-the-art performance on both Flickr SoundNet and VGG-Sound Source. Code and pre-trained models are available at https://github.com/stoneMo/SLAVC.
【3】 Scaling and compressing melodies using geometric similarity measures
标题：使用几何相似性度量对旋律进行缩放和压缩
链接：https://arxiv.org/abs/2209.09621
作者：Luis Evaristo Caraballo,José Miguel Díaz-Báñez,Fabio Rodríguez,Vanesa Sánchez-Canales,Inmaculada Ventura机构：Department of Applied Mathematics II, University of Seville摘要：旋律相似性度量是音乐信息检索中的一个关键问题。本文利用几何匹配技术来度量两个旋律之间的相似性。我们将音乐表示为欧几里得平面上的点集或水平线段集，并提出了有效的算法来解决优化问题，这是由对旋律的两种操作引起的;线性缩放和音频压缩。在缩放问题中，输入的查询旋律被向前缩放，直到查询和参考旋律之间的相似性度量被最小化。压缩问题要求给定旋律的音符子集，使得所选音符和参考旋律之间的匹配成本最小化。摘要：Melodic similarity measurement is of key importance in music information retrieval. In this paper, we use geometric matching techniques to measure the similarity between two melodies. We represent music as sets of points or sets of horizontal line segments in the Euclidean plane and propose efficient algorithms for optimization problems inspired in two operations on melodies; linear scaling and audio compression. In the scaling problem, an incoming query melody is scaled forward until the similarity measure between the query and a reference melody is minimized. The compression problem asks for a subset of notes of a given melody such that the matching cost between the selected notes and the reference melody is minimized.
【1】 ESPnet-ONNX: Bridging a Gap Between Research and Production
标题：ESPnet-ONNX：弥合研究与生产之间的鸿沟
链接：https://arxiv.org/abs/2209.09756
作者：Masao Someki,Yosuke Higuchi,Tomoki Hayashi,Shinji Watanabe机构：‡ Nagoya University, § Human Dataware Lab Co., Ltd., Nagoya, Japan备注：Accepted to APSIPA ASC 2022摘要：在深度学习领域，研究人员往往专注于发明新颖的神经网络模型和改进基准测试。相比之下，应用程序开发者感兴趣的是使模型适合于实际产品，这涉及优化模型以进行更快的推断以及使模型适应于各种平台（例如，C++和Python语言）。本文以语音处理工具ESPnet为例，建立了一个基于PyTorch的面向研究的部署模型优化过程，以弥补两者之间的差距。我们在ESPnet中引入了不同的技术，包括将模型转换为ONNX格式、融合图中的节点以及量化参数，这些技术可使各种任务的速度提高约1.3-2$\times$（即：ASR、TTS、语音翻译和口语理解），同时无需任何额外培训即可保持其性能。我们的ESPnet-ONNX将在www.example.com上公开提供https://github.com/espnet/espnet_onnx摘要：In the field of deep learning, researchers often focus on inventing novel neural network models and improving benchmarks. In contrast, application developers are interested in making models suitable for actual products, which involves optimizing a model for faster inference and adapting a model to various platforms (e.g., C++ and Python). In this work, to fill the gap between the two, we establish an effective procedure for optimizing a PyTorch-based research-oriented model for deployment, taking ESPnet, a widely used toolkit for speech processing, as an instance. We introduce different techniques to ESPnet, including converting a model into an ONNX format, fusing nodes in a graph, and quantizing parameters, which lead to approximately 1.3-2$\times$ speedup in various tasks (i.e., ASR, TTS, speech translation, and spoken language understanding) while keeping its performance without any additional training. Our ESPnet-ONNX will be publicly available at https://github.com/espnet/espnet_onnx
【2】 A Combined Model for Noise Reduction of Lung Sound Signals Based on Empirical Mode Decomposition and Artificial Neural Network
标题：基于经验模式分解和人工神经网络的肺音信号降噪组合模型
链接：https://arxiv.org/abs/2209.09512
作者：Mozhde Firoozi Pouyani,Mansour Vali,Mohammad Amin Ghasemi机构：Pouyani, Firoozi, Mozhde, K. N. Toosi University of Technology, Department of Electrical Engineering, Department of Electrical and Computer Engineering, Tarbiat Modares University, Tehran, Iran摘要：近年来，肺音（Lung Sound，LS）信号的计算机分析已被提出作为分析肺的状态的工具，但一直存在主要挑战，包括LS被来自不同强度的不同来源的环境噪声污染。LS信号降噪的常用方法之一是基于信号的离散小波变换（DWT）系数或经验模态分解（EMD）的阈值化，然而，在这些方法中，需要计算信噪比值以确定合适的阈值来去除噪声。针对这一问题，提出了一种基于EMD和人工神经网络（ANN）的组合模型，分别采用0、5、10、15、20 dB的信噪比进行训练。该模型不需要阈值处理，甚至不需要估计信噪比，就能去除-2 ~20dB范围内的白噪声和粉红噪声，同时很好地保留了LS信号的主要内容。将该方法与EMD自定义方法进行了比较，从信噪比、拟合准则等方面验证了该方法的绝对优越性。例如，在SNR = 0 dB时，组合方法对白噪声和粉红噪声的SNR分别提高了9.41和8.23dB，而EMD-Custom方法的相应值分别为5.89和4.31dB。摘要：Computer analysis of Lung Sound (LS) signals has been proposed in recent years as a tool to analyze the lungs' status but there have always been main challenges, including the contamination of LS with environmental noises, which come from different sources of unlike intensities. One of the common methods in noise reduction of LS signals is based on thresholding on Discrete Wavelet Transform (DWT) coefficients or Empirical Mode Decomposition (EMD) of the signal, however, in these methods, it is necessary to calculate the SNR value to determine the appropriate threshold for noise removal. To solve this problem, a combined model based on EMD and Artificial Neural Network (ANN) trained with different SNRs (0, 5, 10, 15, and 20dB) is proposed in this research. The model can denoise white and pink noises in the range of -2 to 20dB without thresholding or even estimating SNR, and at the same time, keep the main content of the LS signal well. The proposed method is also compared with the EMD-custom method, and the results obtained from the SNR, and fit criteria indicate the absolute superiority of the proposed method. For example, at SNR = 0dB, the combined method can improve the SNR by 9.41 and 8.23dB for white and pink noises, respectively, while the corresponding values are respectively 5.89 and 4.31dB for the EMD-Custom method.
【3】 Relaxed Attention for Transformer Models
标题：放松对Transformer模型的关注
链接：https://arxiv.org/abs/2209.09735
作者：Timo Lohrenz,Björn Möller,Zhengyang Li,Tim Fingscheidt机构：Technische Universität Braunschweig, Germany摘要：基于全注意力的Transformer架构的强大建模能力经常导致过拟合，并且对于自然语言处理任务，导致自回归转换器解码器中的隐式学习的内部语言模型，使得外部语言模型的集成复杂化。在本文中，我们探讨了放松注意力，这是一种简单且易于实现的注意力权重平滑方法，对一般的Transformer架构进行了两方面的改进：首先，放松注意力在应用于编码器中的自注意力层时提供正则化。第二，我们表明，它自然地支持外部语言模型的集成，因为它通过放松解码器中的交叉注意力来抑制隐式学习的内部语言模型。我们证明了放松注意力在几项任务中的好处，并结合最近的基准方法进行了明显的改进。具体而言，我们在最大的公共唇读LRS 3基准测试中以26.31%的单词错误率超过了之前26.90%的单词错误率的最高性能，并且在没有外部语言模型和几乎没有额外模型参数的情况下，我们在IWSLT 14（DE$\rightarrow$EN）机器翻译任务中获得了37.67的最高性能BLEU分数。代码和模型将公开发布。摘要：The powerful modeling capabilities of all-attention-based transformer architectures often cause overfitting and - for natural language processing tasks - lead to an implicitly learned internal language model in the autoregressive transformer decoder complicating the integration of external language models. In this paper, we explore relaxed attention, a simple and easy-to-implement smoothing of the attention weights, yielding a two-fold improvement to the general transformer architecture: First, relaxed attention provides regularization when applied to the self-attention layers in the encoder. Second, we show that it naturally supports the integration of an external language model as it suppresses the implicitly learned internal language model by relaxing the cross attention in the decoder. We demonstrate the benefit of relaxed attention across several tasks with clear improvement in combination with recent benchmark approaches. Specifically, we exceed the former state-of-the-art performance of 26.90% word error rate on the largest public lip-reading LRS3 benchmark with a word error rate of 26.31%, as well as we achieve a top-performing BLEU score of 37.67 on the IWSLT14 (DE$\rightarrow$EN) machine translation task without external language models and virtually no additional model parameters. Code and models will be made publicly available.
【4】 The BUCEA Speaker Diarization System for the VoxCeleb Speaker Recognition Challenge 2022
标题：用于2022年VoxCeleb说话人识别挑战赛的BUCEA说话人二值化系统
链接：https://arxiv.org/abs/2209.09635
* 与cs.SD语音【1】为同一篇
作者：Ruohua Zhou,Yuxuan Du,Chenlei Hu机构：School of Electrical and Information Engineering, Beijing University of Civil Engineering and Architecture, China摘要：本文介绍了2022年VoxCeleb说话人识别挑战赛的BUCEA说话人识别系统。Voxsrc-22提供了VoxConverse的开发集和测试集，我们主要使用VoxConverse的测试集进行参数调整。该系统由语音活动检测、说话人嵌入提取、聚类方法、重叠语音检测和结果融合等模块组成。在不考虑重叠的情况下，将Dover-LAP（Diarization Output Voting Error Reduction）方法应用于系统融合，最后进行重叠语音的检测和处理。我们的最佳系统在VoxSRC 2022评估集上分别实现了5.48%的二值化错误率（DER）和32.1%的Jaccard错误率（JER）。摘要：This paper describes the BUCEA speaker diarization system for the 2022 VoxCeleb Speaker Recognition Challenge. Voxsrc-22 provides the development set and test set of VoxConverse, and we mainly use the test set of VoxConverse for parameter adjustment. Our system consists of several modules, including speech activity detection (VAD), speaker embedding extractor, clustering methods, overlapping speech detection (OSD), and result fusion. Without considering overlap, the Dover-LAP (short for Diarization Output Voting Error Reduction) method was applied to system fusion, and overlapping speech detection and processing were finally carried out. Our best system achieves a diarization error rate (DER) of 5.48% and a Jaccard error rate (JER) of 32.1% on the VoxSRC 2022 evaluation set respectively.
【5】 Scaling and compressing melodies using geometric similarity measures
标题：使用几何相似性度量对旋律进行缩放和压缩
链接：https://arxiv.org/abs/2209.09621
* 与cs.SD语音【3】为同一篇
作者：Luis Evaristo Caraballo,José Miguel Díaz-Báñez,Fabio Rodríguez,Vanesa Sánchez-Canales,Inmaculada Ventura机构：Department of Applied Mathematics II, University of Seville摘要：旋律相似性度量是音乐信息检索中的一个关键问题。本文利用几何匹配技术来度量两个旋律之间的相似性。我们将音乐表示为欧几里得平面上的点集或水平线段集，并提出了有效的算法来解决优化问题，这是由对旋律的两种操作引起的;线性缩放和音频压缩。在缩放问题中，输入的查询旋律被向前缩放，直到查询和参考旋律之间的相似性度量被最小化。压缩问题要求给定旋律的音符子集，使得所选音符和参考旋律之间的匹配成本最小化。摘要：Melodic similarity measurement is of key importance in music information retrieval. In this paper, we use geometric matching techniques to measure the similarity between two melodies. We represent music as sets of points or sets of horizontal line segments in the Euclidean plane and propose efficient algorithms for optimization problems inspired in two operations on melodies; linear scaling and audio compression. In the scaling problem, an incoming query melody is scaled forward until the similarity measure between the query and a reference melody is minimized. The compression problem asks for a subset of notes of a given melody such that the matching cost between the selected notes and the reference melody is minimized.
机器翻译，仅供参考
每日语音速递