今日论文合集：cs.SD语音7篇，eess.AS音频处理7篇。
cs.SD语音

【1】 GTR-CTRL: Instrument and Genre Conditioning for Guitar-Focused Music  Generation with Transformers

标题：GTR-CTRL：以Transformer为重点的吉他音乐生成的乐器和流派调节

链接：https://arxiv.org/abs/2302.05393

作者：Pedro Sarmento,Adarsh Kumar,Yu-Hua Chen,CJ Carr,Zack Zukowski,Mathieu Barthet
机构：Queen Mary University of London, United Kingdom,  Indian Institute of Technology Kharagpur, India,  National Taiwan University, Taiwan
备注：None
摘要：近年来，利用深度学习技术生成符号音乐的技术得到了稳步的发展。大多数关于这个主题的作品都集中在MIDI表示上，但很少关注使用吉他制表符（tab）来生成符号音乐，这些制表符可以用来编码多个乐器。除了节奏和音高之外，标签还包括关于有烦恼的弦乐器的表现技巧和指法的信息。在这项工作中，我们使用DadaGP数据集生成吉他标签音乐，这是一个超过26 k首GuitarPro和令牌格式歌曲的语料库。我们介绍了一些方法来调节Transformer-XL深度学习模型，以根据所需乐器（inst-CTRL）和流派（genre-CTRL）生成吉他谱（GTR-CTRL）。在训练语料库中的每首歌曲的开头附加特殊控制标记。我们评估了模型在有条件和无条件下的性能。我们提出了仪器存在度量来评估inst-CTRL模型对给定仪器提示的响应。我们为下游类型分类训练了BERT模型，并用它来评估用类型CTRL模型获得的结果。统计分析证明了条件模型和非条件模型之间的显著差异。总的来说，结果表明GTR-CTRL方法比无条件模型为吉他为中心的符号音乐生成提供了更大的灵活性和控制。
摘要：Recently, symbolic music generation with deep learning techniques has witnessed steady improvements. Most works on this topic focus on MIDI representations, but less attention has been paid to symbolic music generation using guitar tablatures (tabs) which can be used to encode multiple instruments. Tabs include information on expressive techniques and fingerings for fretted string instruments in addition to rhythm and pitch. In this work, we use the DadaGP dataset for guitar tab music generation, a corpus of over 26k songs in GuitarPro and token formats. We introduce methods to condition a Transformer-XL deep learning model to generate guitar tabs (GTR-CTRL) based on desired instrumentation (inst-CTRL) and genre (genre-CTRL). Special control tokens are appended at the beginning of each song in the training corpus. We assess the performance of the model with and without conditioning. We propose instrument presence metrics to assess the inst-CTRL model's response to a given instrumentation prompt. We trained a BERT model for downstream genre classification and used it to assess the results obtained with the genre-CTRL model. Statistical analyses evidence significant differences between the conditioned and unconditioned models. Overall, results indicate that the GTR-CTRL methods provide more flexibility and control for guitar-focused symbolic music generation than an unconditioned model.

【2】 PATCorrect: Non-autoregressive Phoneme-augmented Transformer for ASR  Error Correction

标题：用于ASR纠错的非自回归音素增强型转换器

链接：https://arxiv.org/abs/2302.05040

作者：Ziji Zhang,Zhehui Wang,Rajesh Kamma,Sharanya Eswaran,Narayanan Sadagopan
机构：Stony Brook University, Amazon
备注：12 pages, 1 figure
摘要：自动语音识别（ASR）系统产生的语音到文本错误会对依赖ASR转录的下游模型产生负面影响。语言纠错模型作为一种后处理文本编辑方法，近年来已经被开发用于精炼源句子。然而，满足工业级生产系统的低等待时间要求的用于校正ASR转录中的错误的有效模型还没有被很好地研究。在这项工作中，我们提出了一种新的非自回归（NAR）纠错方法，通过降低字错误率（WER）来提高转录质量，并在不同的上游ASR系统中实现鲁棒性能。我们的方法使用嵌入发音信息的音素编码器来增强Transformer模型的文本编码。通过多模态融合将音素编码器和文本编码器的表示进行组合，然后输入长度标记预测器以预测目标序列长度。联合编码器还向NAR解码器中的注意机制提供输入。我们在2个公共英语语料库上对3个具有不同语音到文本转录质量的开源ASR系统及其错误转录进行了实验。实验结果表明，在英语语料库上，我们的PATCorrect（Phoneme Augmented Transformer for ASR error Correction）在不同上游ASR系统上的纠错性能始终优于最先进的NAR纠错方法。例如，PATCorrect在3个ASR系统上实现了平均11.62%的WER减少（WER），相比之下，使用仅文本模态的其他方法实现了9.46%的WER减少，并且还实现了与其他NAR模型相当的数十毫秒级的推理延迟，特别是在GPU硬件上，同时在Common Voice和LibriSpeech数据集上仍然比自回归模型快4.2 - 6.7倍。
摘要：Speech-to-text errors made by automatic speech recognition (ASR) system negatively impact downstream models relying on ASR transcriptions. Language error correction models as a post-processing text editing approach have been recently developed for refining the source sentences. However, efficient models for correcting errors in ASR transcriptions that meet the low latency requirements of industrial grade production systems have not been well studied. In this work, we propose a novel non-autoregressive (NAR) error correction approach to improve the transcription quality by reducing word error rate (WER) and achieve robust performance across different upstream ASR systems. Our approach augments the text encoding of the Transformer model with a phoneme encoder that embeds pronunciation information. The representations from phoneme encoder and text encoder are combined via multi-modal fusion before feeding into the length tagging predictor for predicting target sequence lengths. The joint encoders also provide inputs to the attention mechanism in the NAR decoder. We experiment on 3 open-source ASR systems with varying speech-to-text transcription quality and their erroneous transcriptions on 2 public English corpus datasets. Results show that our PATCorrect (Phoneme Augmented Transformer for ASR error Correction) consistently outperforms state-of-the-art NAR error correction method on English corpus across different upstream ASR systems. For example, PATCorrect achieves 11.62% WER reduction (WERR) averaged on 3 ASR systems compared to 9.46% WERR achieved by other method using text only modality and also achieves an inference latency comparable to other NAR models at tens of millisecond scale, especially on GPU hardware, while still being 4.2 - 6.7x times faster than autoregressive models on Common Voice and LibriSpeech datasets.

【3】 Hypernetworks build Implicit Neural Representations of Sounds

标题：超网络构建声音的隐式神经表示

链接：https://arxiv.org/abs/2302.04959

作者：Filip Szatkowski,Karol J. Piczak,Przemtsław Spurek,Jacek Tabor,Tomasz Trzciński
机构：INRs can be used to synthesise data (Zuiderveld 1Warsaw University of Technology,  JagiellonianUniversity
摘要：隐式神经表示（INR）如今被用于表示跨越各种现实生活应用的多媒体信号，包括图像超分辨率、图像压缩或3D渲染。利用INR的现有方法主要集中在视觉数据上，因为它们对其他模态（例如音频）的应用由于基于图像的INR模型的架构属性中存在的归纳偏差而不是微不足道的。为了解决这一局限性，我们引入了HyperSound，这是第一种生成音频样本INR的元学习方法，它利用超网络来泛化训练中观察到的样本。我们的方法重建音频样本的质量可与其他最先进的模型相媲美，并提供了一个可行的替代方案，用于音频处理的深度神经网络中使用的当代声音表示，如声谱图。
摘要：Implicit Neural Representations (INRs) are nowadays used to represent multimedia signals across various real-life applications, including image super-resolution, image compression, or 3D rendering. Existing methods that leverage INRs are predominantly focused on visual data, as their application to other modalities, such as audio, is nontrivial due to the inductive biases present in architectural attributes of image-based INR models. To address this limitation, we introduce HyperSound, the first meta-learning approach to produce INRs for audio samples that leverages hypernetworks to generalize beyond samples observed in training. Our approach reconstructs audio samples with quality comparable to other state-of-the-art models and provides a viable alternative to contemporary sound representations used in deep neural networks for audio processing, such as spectrograms.

【4】 The LuViRA Dataset: Measurement Description

标题：LuViRA数据集：测量说明

链接：https://arxiv.org/abs/2302.05309

作者：Ilayda Yaman,Guoda Tian,Martin Larsson,Patrik Persson,Michiel Sandra,Alexander Dürr,Erik Tegler,Nikhil Challa,Henrik Garde,Fredrik Tufvesson,Kalle Åström,Ove Edfors,Steffen Malkowsky,Liang Liu
机构：Lund University
备注：7 pages, 7 figures
摘要：我们提供了一个数据集来评估定位算法，该算法利用视觉、音频和无线电传感器：隆德大学视觉、无线电和音频（LuViRA）数据集。该数据集包括RGB图像、对应的深度图、IMU读数、大规模MIMO信道探测器和用户设备之间的信道响应、由12个麦克风记录的音频以及0.5mm精度的6DoF姿态地面实况。我们同步这些传感器以确保所有数据同时记录。摄像机、扬声器和发射天线被放置在一个缓慢移动的服务机器人的顶部，记录了88条轨迹。每个轨迹包括20至50秒的记录传感器数据和地面实况标签。来自不同传感器的数据可以单独地或联合地用于执行定位任务，并且运动捕获系统用于验证由定位算法获得的结果。该数据集的主要目的是使融合最常用的传感器用于定位任务的研究成为可能。然而，整个数据集或其某些部分也可用于其它研究领域，例如信道估计、图像分类等。融合传感器数据可导致增加的定位准确性和可靠性，以及减少的延迟和功耗。所创建的数据集将在稍后公布。
摘要：We present a dataset to evaluate localization algorithms, which utilizes vision, audio, and radio sensors: the Lund University Vision, Radio, and Audio (LuViRA) Dataset. The dataset includes RGB images, corresponding depth maps, IMU readings, channel response between a massive MIMO channel sounder and a user equipment, audio recorded by 12 microphones, and 0.5 mm accurate 6DoF pose ground truth. We synchronize these sensors to make sure that all data are recorded simultaneously. A camera, speaker, and transmit antenna are placed on top of a slowly moving service robot and 88 trajectories are recorded. Each trajectory includes 20 to 50 seconds of recorded sensor data and ground truth labels. The data from different sensors can be used separately or jointly to conduct localization tasks and a motion capture system is used to verify the results obtained by the localization algorithms. The main aim of this dataset is to enable research on fusing the most commonly used sensors for localization tasks. However, the full dataset or some parts of it can also be used for other research areas such as channel estimation, image classification, etc. Fusing sensor data can lead to increased localization accuracy and reliability, as well as decreased latency and power consumption. The created dataset will be made public at a later date.

【5】 Spoken language change detection inspired by speaker change detection

标题：受说话人变化检测启发的口语变化检测

链接：https://arxiv.org/abs/2302.05265

作者：Jagabandhu Mishra,S. R. Mahadeva Prasanna
机构：Department of Electrical Engineering, IIT Dharwad, Dharwad , Karnataka, India.
摘要：口语变化检测（LCD）是指识别语码转换话语中的语言转换。类似地，识别多说话者话语中的说话者转换被称为说话者改变检测（SCD）。由于两者在任务方面相似，因此为SCD任务开发的架构/框架可能适用于LCD任务。因此，本工作的目的是开发受SCD启发的LCD系统。最初，LCD和SCD均由人执行。该研究表明，与SCD相比，人类需要（a）更长的变化点持续时间和（b）语言特异性既往暴露。通过增加无监督的基于距离的方法的分析窗口长度来合并更大的持续时间要求。这导致了29.1%和2.4%的相对性能改进，并且先验语言知识分别在合成和实际码交换数据集上提供了31.63%和14.27%的相对性能改进。实际数据集和合成数据集之间的性能差异主要是由于单语片段持续时间分布的差异。
摘要：Spoken language change detection (LCD) refers to identifying the language transitions in a code-switched utterance. Similarly, identifying the speaker transitions in a multispeaker utterance is known as speaker change detection (SCD). Since tasks-wise both are similar, the architecture/framework developed for the SCD task may be suitable for the LCD task. Hence, the aim of the present work is to develop LCD systems inspired by SCD. Initially, both LCD and SCD are performed by humans. The study suggests humans require (a) a larger duration around the change point and (b) language-specific prior exposure, for performing LCD as compared to SCD. The larger duration requirement is incorporated by increasing the analysis window length of the unsupervised distance-based approach. This leads to a relative performance improvement of 29.1% and 2.4%, and a priori language knowledge provides a relative improvement of 31.63% and 14.27% on the synthetic and practical codeswitched datasets, respectively. The performance difference between the practical and synthetic datasets is mostly due to differences in the distribution of the monolingual segment duration.

【6】 Cross-Corpora Spoken Language Identification with Domain Diversification  and Generalization

标题：领域多样化和泛化的跨语料库口语识别

链接：https://arxiv.org/abs/2302.05110

作者：Spandan Dey,Md Sahidullah,Goutam Saha
机构：Department of Electronics & Electrical Communication Engineering, Indian Institute of Technology Kharagpur, Kharagpur, India-, Universit´e de Lorraine, CNRS, Inria, LORIA, F-, Nancy, France
备注：Accepted for publication in Elsevier Computer Speech & Language
摘要：本研究针对低资源环境下口语辨识的跨语料库泛化问题。我们已经在印度LID的背景下进行了实验，发现由于语料库依赖的非语言偏见，跨语料库泛化效果非常差。我们对这项工作的贡献是双重的。首先，我们提出了域多样化，它使用不同的音频数据增强方法使有限的训练数据多样化。然后提出了最大多样性感知级联增强的概念，并优化了增强倍数以有效地分散训练数据。其次，我们引入了域泛化的思想，将增广方法看作伪域。为此，我们研究了领域不变和领域感知两种方法。我们的LID系统基于最先进的强调信道关注、传播和基于聚合的时延神经网络（ECAPA-TDNN）架构。我们使用三个广泛使用的语料库进行了广泛的实验，用于印度LID研究。此外，我们在野外收集的VoxLingua 107语料库的印度子集上对我们提出的方法进行了最终的盲评估。实验结果表明，与常用的简单扩展方法相比，提出的领域多样化方法具有更好的应用前景.研究还表明，领域泛化比领域多元化更有效。我们还注意到领域感知学习对于相同语料库的LID表现更好，而领域不变学习更适合跨语料库的泛化。与基本的ECAPA-TDNN相比，其领域不变扩展使跨语料库的EER提高了5.23%。相比之下，所提出的领域感知扩展也提高了相同语料库测试场景的性能。
摘要：This work addresses the cross-corpora generalization issue for the low-resourced spoken language identification (LID) problem. We have conducted the experiments in the context of Indian LID and identified strikingly poor cross-corpora generalization due to corpora-dependent non-lingual biases. Our contribution to this work is twofold. First, we propose domain diversification, which diversifies the limited training data using different audio data augmentation methods. We then propose the concept of maximally diversity-aware cascaded augmentations and optimize the augmentation fold-factor for effective diversification of the training data. Second, we introduce the idea of domain generalization considering the augmentation methods as pseudo-domains. Towards this, we investigate both domain-invariant and domain-aware approaches. Our LID system is based on the state-of-the-art emphasized channel attention, propagation, and aggregation based time delay neural network (ECAPA-TDNN) architecture. We have conducted extensive experiments with three widely used corpora for Indian LID research. In addition, we conduct a final blind evaluation of our proposed methods on the Indian subset of VoxLingua107 corpus collected in the wild. Our experiments demonstrate that the proposed domain diversification is more promising over commonly used simple augmentation methods. The study also reveals that domain generalization is a more effective solution than domain diversification. We also notice that domain-aware learning performs better for same-corpora LID, whereas domain-invariant learning is more suitable for cross-corpora generalization. Compared to basic ECAPA-TDNN, its proposed domain-invariant extensions improve the cross-corpora EER up to 5.23%. In contrast, the proposed domain-aware extensions also improve performance for same-corpora test scenarios.

【7】 A Composite T60 Regression and Classification Approach for Speech  Dereverberation

标题：一种用于语音去混响的复合T60回归和分类方法

链接：https://arxiv.org/abs/2302.04932

作者：Yuying Li,Yuchen Liu,Donald S. Williamson
机构：Member, IEEE
摘要：去混响通常直接在混响音频信号上执行，而不需要声学环境的知识。然而，混响时间T60是反映混响如何影响信号的基本声学因素。在这项工作中，我们建议执行去混响，同时利用来自环境的关键声学信息。更具体地说，我们开发了一种联合学习方法，该方法使用复合T60模块和单独的去混响模块来同时执行混响时间估计和去混响。混响时间模块在微调期间向去混响模块提供关键特征。我们在模拟和真实环境中评估了我们的方法，并与几种方法进行了比较。结果表明，该组合框架提高了环境中的性能。
摘要：Dereverberation is often performed directly on the reverberant audio signal, without knowledge of the acoustic environment. Reverberation time, T60, however, is an essential acoustic factor that reflects how reverberation may impact a signal. In this work, we propose to perform dereverberation while leveraging key acoustic information from the environment. More specifically, we develop a joint learning approach that uses a composite T60 module and a separate dereverberation module to simultaneously perform reverberation time estimation and dereverberation. The reverberation time module provides key features to the dereverberation module during fine tuning. We evaluate our approach in simulated and real environments, and compare against several approaches. The results show that this composite framework improves performance in environments.
eess.AS音频处理

【1】 The LuViRA Dataset: Measurement Description

标题：LuViRA数据集：测量说明

链接：https://arxiv.org/abs/2302.05309

*与cs.SD语音【4】为同一篇

作者：Ilayda Yaman,Guoda Tian,Martin Larsson,Patrik Persson,Michiel Sandra,Alexander Dürr,Erik Tegler,Nikhil Challa,Henrik Garde,Fredrik Tufvesson,Kalle Åström,Ove Edfors,Steffen Malkowsky,Liang Liu
机构：Lund University
备注：7 pages, 7 figures
摘要：我们提供了一个数据集来评估定位算法，该算法利用视觉、音频和无线电传感器：隆德大学视觉、无线电和音频（LuViRA）数据集。该数据集包括RGB图像、对应的深度图、IMU读数、大规模MIMO信道探测器和用户设备之间的信道响应、由12个麦克风记录的音频以及0.5mm精度的6DoF姿态地面实况。我们同步这些传感器以确保所有数据同时记录。摄像机、扬声器和发射天线被放置在一个缓慢移动的服务机器人的顶部，记录了88条轨迹。每个轨迹包括20至50秒的记录传感器数据和地面实况标签。来自不同传感器的数据可以单独地或联合地用于执行定位任务，并且运动捕获系统用于验证由定位算法获得的结果。该数据集的主要目的是使融合最常用的传感器用于定位任务的研究成为可能。然而，整个数据集或其某些部分也可用于其它研究领域，例如信道估计、图像分类等。融合传感器数据可导致增加的定位准确性和可靠性，以及减少的延迟和功耗。所创建的数据集将在稍后公布。
摘要：We present a dataset to evaluate localization algorithms, which utilizes vision, audio, and radio sensors: the Lund University Vision, Radio, and Audio (LuViRA) Dataset. The dataset includes RGB images, corresponding depth maps, IMU readings, channel response between a massive MIMO channel sounder and a user equipment, audio recorded by 12 microphones, and 0.5 mm accurate 6DoF pose ground truth. We synchronize these sensors to make sure that all data are recorded simultaneously. A camera, speaker, and transmit antenna are placed on top of a slowly moving service robot and 88 trajectories are recorded. Each trajectory includes 20 to 50 seconds of recorded sensor data and ground truth labels. The data from different sensors can be used separately or jointly to conduct localization tasks and a motion capture system is used to verify the results obtained by the localization algorithms. The main aim of this dataset is to enable research on fusing the most commonly used sensors for localization tasks. However, the full dataset or some parts of it can also be used for other research areas such as channel estimation, image classification, etc. Fusing sensor data can lead to increased localization accuracy and reliability, as well as decreased latency and power consumption. The created dataset will be made public at a later date.

【2】 Spoken language change detection inspired by speaker change detection

标题：受说话人变化检测启发的口语变化检测

链接：https://arxiv.org/abs/2302.05265

*与cs.SD语音【5】为同一篇

作者：Jagabandhu Mishra,S. R. Mahadeva Prasanna
机构：Department of Electrical Engineering, IIT Dharwad, Dharwad , Karnataka, India.
摘要：None
摘要：Spoken language change detection (LCD) refers to identifying the language transitions in a code-switched utterance. Similarly, identifying the speaker transitions in a multispeaker utterance is known as speaker change detection (SCD). Since tasks-wise both are similar, the architecture/framework developed for the SCD task may be suitable for the LCD task. Hence, the aim of the present work is to develop LCD systems inspired by SCD. Initially, both LCD and SCD are performed by humans. The study suggests humans require (a) a larger duration around the change point and (b) language-specific prior exposure, for performing LCD as compared to SCD. The larger duration requirement is incorporated by increasing the analysis window length of the unsupervised distance-based approach. This leads to a relative performance improvement of 29.1% and 2.4%, and a priori language knowledge provides a relative improvement of 31.63% and 14.27% on the synthetic and practical codeswitched datasets, respectively. The performance difference between the practical and synthetic datasets is mostly due to differences in the distribution of the monolingual segment duration.

【3】 Cross-Corpora Spoken Language Identification with Domain Diversification  and Generalization

标题：领域多样化和泛化的跨语料库口语识别

链接：https://arxiv.org/abs/2302.05110

*与cs.SD语音【6】为同一篇

作者：Spandan Dey,Md Sahidullah,Goutam Saha
机构：Department of Electronics & Electrical Communication Engineering, Indian Institute of Technology Kharagpur, Kharagpur, India-, Universit´e de Lorraine, CNRS, Inria, LORIA, F-, Nancy, France
备注：Accepted for publication in Elsevier Computer Speech & Language
摘要：本研究针对低资源环境下口语辨识的跨语料库泛化问题。我们已经在印度LID的背景下进行了实验，发现由于语料库依赖的非语言偏见，跨语料库泛化效果非常差。我们对这项工作的贡献是双重的。首先，我们提出了域多样化，它使用不同的音频数据增强方法使有限的训练数据多样化。然后提出了最大多样性感知级联增强的概念，并优化了增强倍数以有效地分散训练数据。其次，我们引入了域泛化的思想，将增广方法看作伪域。为此，我们研究了领域不变和领域感知两种方法。我们的LID系统基于最先进的强调信道关注、传播和基于聚合的时延神经网络（ECAPA-TDNN）架构。我们使用三个广泛使用的语料库进行了广泛的实验，用于印度LID研究。此外，我们在野外收集的VoxLingua 107语料库的印度子集上对我们提出的方法进行了最终的盲评估。实验结果表明，与常用的简单扩展方法相比，提出的领域多样化方法具有更好的应用前景.研究还表明，领域泛化比领域多元化更有效。我们还注意到领域感知学习对于相同语料库的LID表现更好，而领域不变学习更适合跨语料库的泛化。与基本的ECAPA-TDNN相比，其领域不变扩展使跨语料库的EER提高了5.23%。相比之下，所提出的领域感知扩展也提高了相同语料库测试场景的性能。
摘要：This work addresses the cross-corpora generalization issue for the low-resourced spoken language identification (LID) problem. We have conducted the experiments in the context of Indian LID and identified strikingly poor cross-corpora generalization due to corpora-dependent non-lingual biases. Our contribution to this work is twofold. First, we propose domain diversification, which diversifies the limited training data using different audio data augmentation methods. We then propose the concept of maximally diversity-aware cascaded augmentations and optimize the augmentation fold-factor for effective diversification of the training data. Second, we introduce the idea of domain generalization considering the augmentation methods as pseudo-domains. Towards this, we investigate both domain-invariant and domain-aware approaches. Our LID system is based on the state-of-the-art emphasized channel attention, propagation, and aggregation based time delay neural network (ECAPA-TDNN) architecture. We have conducted extensive experiments with three widely used corpora for Indian LID research. In addition, we conduct a final blind evaluation of our proposed methods on the Indian subset of VoxLingua107 corpus collected in the wild. Our experiments demonstrate that the proposed domain diversification is more promising over commonly used simple augmentation methods. The study also reveals that domain generalization is a more effective solution than domain diversification. We also notice that domain-aware learning performs better for same-corpora LID, whereas domain-invariant learning is more suitable for cross-corpora generalization. Compared to basic ECAPA-TDNN, its proposed domain-invariant extensions improve the cross-corpora EER up to 5.23%. In contrast, the proposed domain-aware extensions also improve performance for same-corpora test scenarios.

【4】 A Composite T60 Regression and Classification Approach for Speech  Dereverberation

标题：一种用于语音去混响的复合T60回归和分类方法

链接：https://arxiv.org/abs/2302.04932

*与cs.SD语音【7】为同一篇

作者：Yuying Li,Yuchen Liu,Donald S. Williamson
机构：Member, IEEE
摘要：去混响通常直接在混响音频信号上执行，而不需要声学环境的知识。然而，混响时间T60是反映混响如何影响信号的基本声学因素。在这项工作中，我们建议执行去混响，同时利用来自环境的关键声学信息。更具体地说，我们开发了一种联合学习方法，该方法使用复合T60模块和单独的去混响模块来同时执行混响时间估计和去混响。混响时间模块在微调期间向去混响模块提供关键特征。我们在模拟和真实环境中评估了我们的方法，并与几种方法进行了比较。结果表明，该组合框架提高了环境中的性能。
摘要：Dereverberation is often performed directly on the reverberant audio signal, without knowledge of the acoustic environment. Reverberation time, T60, however, is an essential acoustic factor that reflects how reverberation may impact a signal. In this work, we propose to perform dereverberation while leveraging key acoustic information from the environment. More specifically, we develop a joint learning approach that uses a composite T60 module and a separate dereverberation module to simultaneously perform reverberation time estimation and dereverberation. The reverberation time module provides key features to the dereverberation module during fine tuning. We evaluate our approach in simulated and real environments, and compare against several approaches. The results show that this composite framework improves performance in environments.

【5】 GTR-CTRL: Instrument and Genre Conditioning for Guitar-Focused Music  Generation with Transformers

标题：GTR-CTRL：以Transformer为重点的吉他音乐生成的乐器和流派调节

链接：https://arxiv.org/abs/2302.05393

*与cs.SD语音【5】为同一篇

作者：Pedro Sarmento,Adarsh Kumar,Yu-Hua Chen,CJ Carr,Zack Zukowski,Mathieu Barthet
机构：Queen Mary University of London, United Kingdom,  Indian Institute of Technology Kharagpur, India,  National Taiwan University, Taiwan
备注：None
摘要：近年来，利用深度学习技术生成符号音乐的技术得到了稳步的发展。大多数关于这个主题的作品都集中在MIDI表示上，但很少关注使用吉他制表符（tab）来生成符号音乐，这些制表符可以用来编码多个乐器。除了节奏和音高之外，标签还包括关于有烦恼的弦乐器的表现技巧和指法的信息。在这项工作中，我们使用DadaGP数据集生成吉他标签音乐，这是一个超过26 k首GuitarPro和令牌格式歌曲的语料库。我们介绍了一些方法来调节Transformer-XL深度学习模型，以根据所需乐器（inst-CTRL）和流派（genre-CTRL）生成吉他谱（GTR-CTRL）。在训练语料库中的每首歌曲的开头附加特殊控制标记。我们评估了模型在有条件和无条件下的性能。我们提出了仪器存在度量来评估inst-CTRL模型对给定仪器提示的响应。我们为下游类型分类训练了BERT模型，并用它来评估用类型CTRL模型获得的结果。统计分析证明了条件模型和非条件模型之间的显著差异。总的来说，结果表明GTR-CTRL方法比无条件模型为吉他为中心的符号音乐生成提供了更大的灵活性和控制。
 摘要：Recently, symbolic music generation with deep learning techniques has witnessed steady improvements. Most works on this topic focus on MIDI representations, but less attention has been paid to symbolic music generation using guitar tablatures (tabs) which can be used to encode multiple instruments. Tabs include information on expressive techniques and fingerings for fretted string instruments in addition to rhythm and pitch. In this work, we use the DadaGP dataset for guitar tab music generation, a corpus of over 26k songs in GuitarPro and token formats. We introduce methods to condition a Transformer-XL deep learning model to generate guitar tabs (GTR-CTRL) based on desired instrumentation (inst-CTRL) and genre (genre-CTRL). Special control tokens are appended at the beginning of each song in the training corpus. We assess the performance of the model with and without conditioning. We propose instrument presence metrics to assess the inst-CTRL model's response to a given instrumentation prompt. We trained a BERT model for downstream genre classification and used it to assess the results obtained with the genre-CTRL model. Statistical analyses evidence significant differences between the conditioned and unconditioned models. Overall, results indicate that the GTR-CTRL methods provide more flexibility and control for guitar-focused symbolic music generation than an unconditioned model.

【6】 PATCorrect: Non-autoregressive Phoneme-augmented Transformer for ASR  Error Correction

标题：用于ASR纠错的非自回归音素增强型转换器

链接：https://arxiv.org/abs/2302.05040

*与cs.SD语音【2】为同一篇

作者：Ziji Zhang,Zhehui Wang,Rajesh Kamma,Sharanya Eswaran,Narayanan Sadagopan
机构：Stony Brook University, Amazon
备注：12 pages, 1 figure
摘要：None
摘要：Speech-to-text errors made by automatic speech recognition (ASR) system negatively impact downstream models relying on ASR transcriptions. Language error correction models as a post-processing text editing approach have been recently developed for refining the source sentences. However, efficient models for correcting errors in ASR transcriptions that meet the low latency requirements of industrial grade production systems have not been well studied. In this work, we propose a novel non-autoregressive (NAR) error correction approach to improve the transcription quality by reducing word error rate (WER) and achieve robust performance across different upstream ASR systems. Our approach augments the text encoding of the Transformer model with a phoneme encoder that embeds pronunciation information. The representations from phoneme encoder and text encoder are combined via multi-modal fusion before feeding into the length tagging predictor for predicting target sequence lengths. The joint encoders also provide inputs to the attention mechanism in the NAR decoder. We experiment on 3 open-source ASR systems with varying speech-to-text transcription quality and their erroneous transcriptions on 2 public English corpus datasets. Results show that our PATCorrect (Phoneme Augmented Transformer for ASR error Correction) consistently outperforms state-of-the-art NAR error correction method on English corpus across different upstream ASR systems. For example, PATCorrect achieves 11.62% WER reduction (WERR) averaged on 3 ASR systems compared to 9.46% WERR achieved by other method using text only modality and also achieves an inference latency comparable to other NAR models at tens of millisecond scale, especially on GPU hardware, while still being 4.2 - 6.7x times faster than autoregressive models on Common Voice and LibriSpeech datasets.

【7】 Hypernetworks build Implicit Neural Representations of Sounds

标题：超网络构建声音的隐式神经表示

链接：https://arxiv.org/abs/2302.04959

*与cs.SD语音【3】为同一篇

作者：Filip Szatkowski,Karol J. Piczak,Przemtsław Spurek,Jacek Tabor,Tomasz Trzciński
机构：INRs can be used to synthesise data (Zuiderveld 1Warsaw University of Technology,  JagiellonianUniversity
摘要：隐式神经表示（INR）如今被用于表示跨越各种现实生活应用的多媒体信号，包括图像超分辨率、图像压缩或3D渲染。利用INR的现有方法主要集中在视觉数据上，因为它们对其他模态（例如音频）的应用由于基于图像的INR模型的架构属性中存在的归纳偏差而不是微不足道的。为了解决这一局限性，我们引入了HyperSound，这是第一种生成音频样本INR的元学习方法，它利用超网络来泛化训练中观察到的样本。我们的方法重建音频样本的质量可与其他最先进的模型相媲美，并提供了一个可行的替代方案，用于音频处理的深度神经网络中使用的当代声音表示，如声谱图。
摘要：Implicit Neural Representations (INRs) are nowadays used to represent multimedia signals across various real-life applications, including image super-resolution, image compression, or 3D rendering. Existing methods that leverage INRs are predominantly focused on visual data, as their application to other modalities, such as audio, is nontrivial due to the inductive biases present in architectural attributes of image-based INR models. To address this limitation, we introduce HyperSound, the first meta-learning approach to produce INRs for audio samples that leverages hypernetworks to generalize beyond samples observed in training. Our approach reconstructs audio samples with quality comparable to other state-of-the-art models and provides a viable alternative to contemporary sound representations used in deep neural networks for audio processing, such as spectrograms.

机器翻译，仅供参考