语音/音频处理学术速递[2.10]

今日论文合集：cs.SD语音4篇，eess.AS音频处理4篇。

cs.SD语音

【1】 Incorporating Total Variation Regularization in the design of an intelligent Query by Humming system

标题：基于全变差正则化的哼唱智能查询系统设计

链接：https://arxiv.org/abs/2302.04577

作者：Shivangi Ranjan,Vishal Srivastava

机构：Indian Institute of Technology Kanpur

摘要：哼唱查询（QBH）系统构成音乐信息检索的一种特殊情况，其中输入是用户哼唱的旋律，输出是包含该旋律的原始歌曲。典型的QBH系统由旋律提取和候选旋律检索两部分组成。对于旋律提取，准确的音符转录是关键使能技术。然而，当前的转录方法不能明确地捕捉旋律和解决用户哼唱的查询中的不准确性。本文将全变分正则化（TVR）引入查询去噪。这种方法解决了用户在哼唱时的错误，而没有丢失有意义的数据，并且可靠地捕获了潜在的旋律。对于候选旋律检索，我们采用深度学习方法，使用全卷积神经网络进行时间序列分类。经训练的网络将传入查询分类为属于目标歌曲之一。对于我们的实验，我们使用Roger Jang的MIR-QBSH数据集，它是标准MIREX数据集。我们证明了在训练集中包含TVR去噪查询将系统的总体准确率提高到93%，这高于其他最先进的QBH系统。

摘要：A Query-By-Humming (QBH) system constitutes a particular case of music information retrieval where the input is a user-hummed melody and the output is the original song which contains that melody. A typical QBH system consists of melody extraction and candidate melody retrieval. For melody extraction, accurate note transcription is the key enabling technology. However, current transcription methods are unable to definitively capture the melody and address inaccuracies in user-hummed queries. In this paper, we incorporate Total Variation Regularization (TVR) to denoise queries. This approach accounts for user error in humming without loss of meaningful data and reliably captures the underlying melody. For candidate melody retrieval, we employ a deep learning approach to time series classification using a Fully Convolutional Neural Network. The trained network classifies the incoming query as belonging to one of the target songs. For our experiments, we use Roger Jang's MIR-QBSH dataset which is the standard MIREX dataset. We demonstrate that inclusion of TVR denoised queries in the training set enhances the overall accuracy of the system to 93% which is higher than other state-of-the-art QBH systems.

【2】 Joint Acoustic Echo Cancellation and Speech Dereverberation Using Kalman filters

标题：基于卡尔曼滤波的声学回波消除和语音去混响联合算法

链接：https://arxiv.org/abs/2302.04469

作者：Ziteng Wang,Yueyue Na,Biao Tian,Qiang Fu

机构：Alibaba Group, China

摘要：提出了一种短时傅里叶变换域的回声消除（AEC）和语音去混响（DR）联合算法。混响麦克风信号使用自回归（AR）模型来描述。AR系数和扬声器到麦克风的声学传递函数（ATF）被认为是时变的，并且使用一阶马尔可夫过程同时建模。这导致了一种解决方案，其中可以使用卡尔曼滤波器来最优地估计这些参数。实验结果表明，该算法在语音质量和回声抑制性能方面均优于传统的AEC和DR算法以及一种基于半盲源分离的联合DRAEC算法。

摘要：This paper proposes a joint acoustic echo cancellation (AEC) and speech dereverberation (DR) algorithm in the short-time Fourier transform domain. The reverberant microphone signals are described using an auto-regressive (AR) model. The AR coefficients and the loudspeaker-to-microphone acoustic transfer functions (ATFs) are considered time-varying and are modeled simultaneously using a first-order Markov process. This leads to a solution where these parameters can be optimally estimated using Kalman filters. It is shown that the proposed algorithm outperforms vanilla solutions that solve AEC and DR sequentially and one state-of-the-art joint DRAEC algorithm based on semi-blind source separation, in terms of both speech quality and echo reduction performance.

【3】 ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Models

标题：Ernie-Music：基于扩散模型的文本到波形音乐生成

链接：https://arxiv.org/abs/2302.04456

作者：Pengfei Zhu,Chao Pang,Shuohuan Wang,Yekun Chai,Yu Sun,Hao Tian,Hua Wu

机构：Baidu Inc.

摘要：近年来，在使用扩散模型的图像和语音生成中已经变得越来越普及。然而，从自由形式的文本提示直接生成音乐波形仍处于探索阶段。在这篇论文中，我们提出了第一个使用扩散模型可以接收任意文本的文本到波形的音乐生成模型。我们将自由形式的文本提示作为条件来指导扩散模型的波形生成过程。为了解决缺乏文本-音乐平行数据的问题，我们从互联网上收集了一个文本-音乐对数据集。此外，我们比较了两种条件文本提示格式（音乐标签和自由文本）的效果，证明了我们的方法在文本-音乐相关性方面的优越性能。我们进一步证明了我们在波形域中生成的音乐在多样性、质量和文本-音乐相关性方面大大优于以前的作品。

摘要：In recent years, there has been an increased popularity in image and speech generation using diffusion models. However, directly generating music waveforms from free-form text prompts is still under-explored. In this paper, we propose the first text-to-waveform music generation model that can receive arbitrary texts using diffusion models. We incorporate the free-form textual prompt as the condition to guide the waveform generation process of diffusion models. To solve the problem of lacking such text-music parallel data, we collect a dataset of text-music pairs from the Internet with weak supervision. Besides, we compare the effect of two prompt formats of conditioning texts (music tags and free-form texts) and prove the superior performance of our method in terms of text-music relevance. We further demonstrate that our generated music in the waveform domain outperforms previous works by a large margin in terms of diversity, quality, and text-music relevance.

【4】 Short-Term Memory Convolutions

标题：短时记忆卷曲

链接：https://arxiv.org/abs/2302.04331

作者：Grzegorz Stefański,Krzysztof Arendt,Paweł Daniluk,Bartłomiej Jasik,Artur Szumaczuk

机构：Samsung R&D Institute Poland

备注：ICLR 2023

摘要：时间序列信号的实时处理是许多实际应用中的关键问题。实时处理的思想在音频域中尤其重要，因为人类对声音的感知对感知信号中的任何类型的干扰都是敏感的，尤其是听觉和视觉模态之间的滞后。深度学习（DL）模型的兴起使信号处理领域变得更加复杂。虽然与标准DSP方法相比，它们通常具有更高的质量，但这一优势因更高的延迟而减弱。在这项工作中，我们提出了一种新的方法来最小化推理时间延迟和内存消耗，称为短期内存卷积（STMC）及其转置对应。STMC的主要优点是与长短时记忆（LSTM）网络相比具有低延迟。此外，基于STMC的模型的训练更快且更稳定，因为该方法仅基于卷积神经网络（CNN）。在本研究中，我们展示了这种解决方案在语音分离任务的U-Net模型和声学场景分类（ASC）任务的GhostNet模型中的应用。在语音分离的情况下，我们在不影响输出质量的情况下实现了推理时间的5倍减少和延迟的2倍减少。ASC任务的推理速度提高了4倍，同时保持了原有的推理精度。

摘要：The real-time processing of time series signals is a critical issue for many real-life applications. The idea of real-time processing is especially important in audio domain as the human perception of sound is sensitive to any kind of disturbance in perceived signals, especially the lag between auditory and visual modalities. The rise of deep learning (DL) models complicated the landscape of signal processing. Although they often have superior quality compared to standard DSP methods, this advantage is diminished by higher latency. In this work we propose novel method for minimization of inference time latency and memory consumption, called Short-Term Memory Convolution (STMC) and its transposed counterpart. The main advantage of STMC is the low latency comparable to long short-term memory (LSTM) networks. Furthermore, the training of STMC-based models is faster and more stable as the method is based solely on convolutional neural networks (CNNs). In this study we demonstrate an application of this solution to a U-Net model for a speech separation task and GhostNet model in acoustic scene classification (ASC) task. In case of speech separation we achieved a 5-fold reduction in inference time and a 2-fold reduction in latency without affecting the output quality. The inference time for ASC task was up to 4 times faster while preserving the original accuracy.

eess.AS音频处理