【1】 Heart Abnormality Detection from Heart Sound Signals using MFCC Feature and Dual Stream Attention Based Network标题：基于MFCC特征和双流注意网络的心音信号心脏异常检测链接：https://arxiv.org/abs/2211.09751
作者：Nayeeb Rashid,Swapnil Saha,Mohseu Rashid Subah,Rizwan Ahmed Robin,Syed Mortuza Hasan Fahim,Shahed Ahmed,Talha Ibn Mahmud机构：Department of Electrical and Electronic Engineering (EEE), Bangladesh University of Engineering and Technology (BUET), Dhaka - , Bangladesh.摘要：心血管疾病是当今世界死亡的主要原因之一，心脏疾病的早期筛查在预防心血管疾病中发挥着至关重要的作用。心音信号是心脏状况的主要指标之一，并且可以用于检测心脏中的异常。心音信号的采集是无创的、成本有效的并且需要最少的设备。但目前从心音信号中检测心脏异常在很大程度上依赖于医生的专业知识和经验。因此，从心音信号中检测心脏异常的自动检测系统对于生活在不发达地区的人们来说是一笔巨大的财富。本文提出了一种基于深度学习的新型注意机制双流网络，该网络同时使用原始心音信号和MFCC特征来检测患者心脏状况的异常。深度神经网络具有使用原始心音信号的卷积流和使用信号的MFCC特征的循环流。使用新颖的注意力网络将来自这两个流的特征合并在一起，并使其通过分类网络。该模型在最大的公开可用的心音信号数据集上训练，获得了87.11的准确度、82.41的灵敏度、91.8的特异性和87.12的MACC。摘要：Cardiovascular diseases are one of the leading cause of death in today's world and early screening of heart condition plays a crucial role in preventing them. The heart sound signal is one of the primary indicator of heart condition and can be used to detect abnormality in the heart. The acquisition of heart sound signal is non-invasive, cost effective and requires minimum equipment. But currently the detection of heart abnormality from heart sound signal depends largely on the expertise and experience of the physician. As such an automatic detection system for heart abnormality detection from heart sound signal can be a great asset for the people living in underdeveloped areas. In this paper we propose a novel deep learning based dual stream network with attention mechanism that uses both the raw heart sound signal and the MFCC features to detect abnormality in heart condition of a patient. The deep neural network has a convolutional stream that uses the raw heart sound signal and a recurrent stream that uses the MFCC features of the signal. The features from these two streams are merged together using a novel attention network and passed through the classification network. The model is trained on the largest publicly available dataset of PCG signal and achieves an accuracy of 87.11, sensitivity of 82.41, specificty of 91.8 and a MACC of 87.12.
【2】 Stutter-TTS: Controlled Synthesis and Improved Recognition of Stuttered Speech标题：卡顿-TTS：受控合成和改进的卡顿语音识别链接：https://arxiv.org/abs/2211.09731
作者：Xin Zhang,Iván Vallés-Pérez,Andreas Stolcke,Chengzhu Yu,Jasha Droppo,Olabanji Shonibare,Roberto Barra-Chicote,Venkatesh Ravichandran机构：Amazon Alexa AI备注：None摘要：口吃是一种语言障碍，其中自然的语言流被音节、单词和短语的阻塞、重复或延长所打断。现有的大多数自动语音识别（ASR）接口在处理口吃语音时性能较差，主要是由于缺乏匹配的训练数据。因此，具有口吃的语音的合成提供了改善这种类型的语音的ASR的机会。我们描述了口吃TTS，一种能够合成不同类型口吃话语的端到端神经文本到语音模型。我们开发了一个简单而有效的韵律控制策略，在训练过程中将附加标记引入源文本中以表示特定的口吃特征。通过选择口吃标记的位置，Stutter-TTS允许对合成话语中口吃发生的位置进行单词级控制。我们能够以高准确度合成口吃事件（F1分数在0.63和0.84之间，取决于口吃类型）。通过对合成口吃语音的ASR模型进行微调，我们能够将口吃话语的单词错误相对减少5.7%，而对于流畅话语仅具有较小（〈0.2%相对）的降级。摘要：Stuttering is a speech disorder where the natural flow of speech is interrupted by blocks, repetitions or prolongations of syllables, words and phrases. The majority of existing automatic speech recognition (ASR) interfaces perform poorly on utterances with stutter, mainly due to lack of matched training data. Synthesis of speech with stutter thus presents an opportunity to improve ASR for this type of speech. We describe Stutter-TTS, an end-to-end neural text-to-speech model capable of synthesizing diverse types of stuttering utterances. We develop a simple, yet effective prosody-control strategy whereby additional tokens are introduced into source text during training to represent specific stuttering characteristics. By choosing the position of the stutter tokens, Stutter-TTS allows word-level control of where stuttering occurs in the synthesized utterance. We are able to synthesize stutter events with high accuracy (F1-scores between 0.63 and 0.84, depending on stutter type). By fine-tuning an ASR model on synthetic stuttered speech we are able to reduce word error by 5.7% relative on stuttered utterances, with only minor (<0.2% relative) degradation for fluent utterances.
【3】 Design Considerations For Hypothesis Rejection Modules In Spoken Language Understanding Systems标题：口语理解系统中假设拒绝模块的设计考虑链接：https://arxiv.org/abs/2211.09711
作者：Aman Alok,Rahul Gupta,Shankar Ananthakrishnan备注：5 pages. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)摘要：口语理解（SLU）系统通常由一组机器学习模型组成，这些模型联合操作以产生SLU假设。然后，将生成的假设发送到下游组件以进行进一步操作。然而，期望在向下游发送不正确的假设之前丢弃该假设。在本文中，我们提出了两种SLU假设拒绝模块的设计：（i）对域特定SLU假设执行拒绝的方案R1，以及（ii）对从整个SLU系统生成的假设执行拒绝的方案R2。两种方案中的假设拒绝模块基于从指向SLU系统的话语中提取的特征、相关联的SLU假设和SLU置信度分数来拒绝/接受假设。我们的实验表明，两种方案产生类似的结果（方案R1：2.5% FRR@4.5% FAR，方案R2：2.5% FRR@4.6% FAR），性能最佳的系统使用所有可用功能。我们认为，虽然可以选择其中一种抑制方案，但它们存在一些内在差异，在进行选择时需要考虑这些差异。此外，我们还将ASR功能整合到抑制模块中（在3.8% FAR时获得1.9% FRR），并分析改进情况。摘要：Spoken Language Understanding (SLU) systems typically consist of a set of machine learning models that operate in conjunction to produce an SLU hypothesis. The generated hypothesis is then sent to downstream components for further action. However, it is desirable to discard an incorrect hypothesis before sending it downstream. In this work, we present two designs for SLU hypothesis rejection modules: (i) scheme R1 that performs rejection on domain specific SLU hypothesis and, (ii) scheme R2 that performs rejection on hypothesis generated from the overall SLU system. Hypothesis rejection modules in both schemes reject/accept a hypothesis based on features drawn from the utterance directed to the SLU system, the associated SLU hypothesis and SLU confidence score. Our experiments suggest that both the schemes yield similar results (scheme R1: 2.5% FRR @ 4.5% FAR, scheme R2: 2.5% FRR @ 4.6% FAR), with the best performing systems using all the available features. We argue that while either of the rejection schemes can be chosen over the other, they carry some inherent differences which need to be considered while making this choice. Additionally, we incorporate ASR features in the rejection module (obtaining an 1.9% FRR @ 3.8% FAR) and analyze the improvements.
【4】 Listen, denoise, action! Audio-driven motion synthesis with diffusion models标题：听着，丹尼斯，开拍！基于扩散模型的音频驱动运动合成链接：https://arxiv.org/abs/2211.09707
作者：Simon Alexanderson,Rajmund Nagy,Jonas Beskow,Gustav Eje Henter机构：Division of Speech, Music and Hearing, KTH Royal Institute of Technology, Stockholm, Sweden备注：15 pages, 6 figures摘要：扩散模型作为一种表达能力强、可有效训练的概率模型，受到了广泛的关注。我们表明，这些模型非常适合于合成与音频共同出现的人体运动，例如共同语音手势，因为给定音频，运动是复杂的和高度模糊的，需要概率描述。具体来说，我们采用DiffWave架构来建模3D姿势序列，用Conformers代替膨胀卷积以提高准确性。我们还演示了对运动风格的控制，使用无分类器指导来调整风格表达的强度。在Trinity Speech-Gesture和ZeroEGGS数据集上进行的手势生成实验表明，该方法能够获得一流的运动质量，并具有不同的风格，其表达可以或多或少地变得明显。我们还使用相同的模型架构合成舞蹈运动和路径驱动的移动。最后，我们扩展了指导过程，以一种对合成任务有吸引力的方式执行风格插值，并与专家产品模型有联系，我们认为这是一个独立的贡献。有关视频示例，请访问https://www.speech.kth.se/research/listen-denoise-action/摘要：Diffusion models have experienced a surge of interest as highly expressive yet efficiently trainable probabilistic models. We show that these models are an excellent fit for synthesising human motion that co-occurs with audio, for example co-speech gesticulation, since motion is complex and highly ambiguous given audio, calling for a probabilistic description. Specifically, we adapt the DiffWave architecture to model 3D pose sequences, putting Conformers in place of dilated convolutions for improved accuracy. We also demonstrate control over motion style, using classifier-free guidance to adjust the strength of the stylistic expression. Gesture-generation experiments on the Trinity Speech-Gesture and ZeroEGGS datasets confirm that the proposed method achieves top-of-the-line motion quality, with distinctive styles whose expression can be made more or less pronounced. We also synthesise dance motion and path-driven locomotion using the same model architecture. Finally, we extend the guidance procedure to perform style interpolation in a manner that is appealing for synthesis tasks and has connections to product-of-experts models, a contribution we believe is of independent interest. Video examples are available at https://www.speech.kth.se/research/listen-denoise-action/
【5】 Towards Building Text-To-Speech Systems for the Next Billion Users标题：为下一个十亿用户打造文语转换系统链接：https://arxiv.org/abs/2211.09536
作者：Gokul Karthik Kumar,Praveen S V,Pratyush Kumar,Mitesh M. Khapra,Karthik Nandakumar机构：AI,Bharat, India, Indian Institute of Technology Madras (IITM), India, Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), UAE, Microsoft Research, India备注：Under review in ICASSP 2023. First two authors contributed equally摘要：基于深度学习的文本到语音（TTS）系统随着模型架构、训练方法和跨说话者和语言的泛化的进步而迅速发展。然而，这些进展对于印度语言语音合成还没有被彻底研究。考虑到印度语言的数量和多样性，相对较低的资源可用性，以及尚未测试的神经TTS的各种进步，这种研究在计算上是昂贵的。在这篇论文中，我们评估了声学模型、声码器、补充损失函数、训练时间表以及达罗毗荼语和印度-雅利安语的说话人和语言多样性的选择。在此基础上，我们确定了FastPitch和HiFi-GAN V1的单语模型，在男性和女性说话人上联合训练以获得最佳性能。通过这种设置，我们训练和评估了13种语言的TTS模型，发现我们的模型在所有语言中都比现有模型有了显著的改进，如平均意见得分所测量的。我们开放Bhashini平台上的所有模型。摘要：Deep learning based text-to-speech (TTS) systems have been evolving rapidly with advances in model architectures, training methodologies, and generalization across speakers and languages. However, these advances have not been thoroughly investigated for Indian language speech synthesis. Such investigation is computationally expensive given the number and diversity of Indian languages, relatively lower resource availability, and the diverse set of advances in neural TTS that remain untested. In this paper, we evaluate the choice of acoustic models, vocoders, supplementary loss functions, training schedules, and speaker and language diversity for Dravidian and Indo-Aryan languages. Based on this, we identify monolingual models with FastPitch and HiFi-GAN V1, trained jointly on male and female speakers to perform the best. With this setup, we train and evaluate TTS models for 13 languages and find our models to significantly improve upon existing models in all languages as measured by mean opinion scores. We open-source all models on the Bhashini platform.
【6】 Hey ASR System! Why Aren't You More Inclusive? Automatic Speech Recognition Systems' Bias and Proposed Bias Mitigation Techniques. A Literature Review标题：嘿ASR系统！你为什么不更包容一些呢？自动语音识别系统的偏差和提出的偏差缓解技术。文献综述链接：https://arxiv.org/abs/2211.09511
作者：Mikel K. Ngueajio,Gloria Washington机构：Department of Computer Science, Howard University, Washington DC , USA备注：In press at HCI International 2022 - Late Breaking Papers: Interacting with eXtended Reality and Artificial Intelligence, LNCS 13518摘要：言语是人类之间进行交流的基本手段。人工智能和复杂的语音技术的出现导致了基于人机交互的快速增长，这主要是由自动语音识别（ASR）系统推动的。ASR系统通常采用音频形式的人类语音并将其转换为单词，但对于一些用户，它不能解码语音，并且任何输出文本都充满了人类读者无法理解的错误。这些系统并非对每个人都同样有效，实际上会妨碍某些用户的工作效率。在这篇文章中，我们介绍了针对性别、种族、病人和残疾人的ASR偏见的研究，同时探讨了提出ASR去偏见技术以减轻这些歧视的研究。我们还讨论了设计更易访问和包容性更强的ASR技术的技术。对于调查的每种方法，我们还提供了所用调查和方法、所用ASR系统和语料库以及研究结果的总结，并强调了它们的优势和/或劣势。最后，我们提出未来的机会，为自然语言处理研究人员探索下一个层次的创造ASR技术。摘要：Speech is the fundamental means of communication between humans. The advent of AI and sophisticated speech technologies have led to the rapid proliferation of human-to-computer-based interactions, fueled primarily by Automatic Speech Recognition (ASR) systems. ASR systems normally take human speech in the form of audio and convert it into words, but for some users, it cannot decode the speech, and any output text is filled with errors that are incomprehensible to the human reader. These systems do not work equally for everyone and actually hinder the productivity of some users. In this paper, we present research that addresses ASR biases against gender, race, and the sick and disabled, while exploring studies that propose ASR debiasing techniques for mitigating these discriminations. We also discuss techniques for designing a more accessible and inclusive ASR technology. For each approach surveyed, we also provide a summary of the investigation and methods applied, the ASR systems and corpora used, and the research findings, and highlight their strengths and/or weaknesses. Finally, we propose future opportunities for Natural Language Processing researchers to explore in the next level creation of ASR technologies.
【7】 Adaptive Representations of Sound for Automatic Insect Recognition标题：用于昆虫自动识别的声音自适应表示法链接：https://arxiv.org/abs/2211.09503
作者：Marius Faiß机构：Evolutionary Biology, Naturalis Biodiversity Center, Evolutionary Ecology, Darwinweg , CR Leiden, The Netherlands, Supervisor: Prof. Dan Stowell, ,, – ,,, EC, Contact: Prof. Dan Stowell, Format: Plos Computational Biology备注：30 pages, 9 figures Dataset: this https URL摘要：昆虫是我们生态系统不可或缺的一部分。这些通常体型小、易躲避的动物对周围环境有很大影响，提供了目前生物多样性和授粉职责的很大一部分，形成了食物链和许多生物和生态过程的基础。由于人类的影响，人口数量和生物多样性随着时间的推移迅速下降。监测这种下降情况对于有效实施保护措施变得越来越重要。但是，监测方法往往是侵入性的，时间和资源密集，并容易产生各种偏见。许多昆虫物种产生特征性的交配声音，这些声音可以容易地被检测和记录，而不需要大的成本或努力。使用深度学习方法，可以自动检测和分类来自现场记录的昆虫声音，以监测生物多样性和物种分布范围。在这个项目中，我使用现有的昆虫声音数据集（直翅目和蝉科）和机器学习方法来实现这一点，并评估它们在昆虫声学监测方面的潜力。我比较了传统的基于频谱图的深度学习方法与新的自适应和基于波形的方法LEAF的性能。通过在训练过程中调整特征提取参数，基于波形的前端获得了比Mel谱图前端更好的分类性能。这一结果对于未来实现用于昆虫声音自动识别的深度学习技术是令人鼓舞的，特别是在更大的数据集可用的情况下。摘要：Insects are an integral part of our ecosystem. These often small and evasive animals have a big impact on their surroundings, providing a large part of the present biodiversity and pollination duties, forming the foundation of the food chain and many biological and ecological processes. Due to factors of human influence, population numbers and biodiversity have been rapidly declining with time. Monitoring this decline has become increasingly important for conservation measures to be effectively implemented. But monitoring methods are often invasive, time and resource intense, and prone to various biases. Many insect species produce characteristic mating sounds that can easily be detected and recorded without large cost or effort. Using deep learning methods, insect sounds from field recordings could be automatically detected and classified to monitor biodiversity and species distribution ranges. In this project, I implement this using existing datasets of insect sounds (Orthoptera and Cicadidae) and machine learning methods and evaluate their potential for acoustic insect monitoring. I compare the performance of the conventional spectrogram-based deep learning method against the new adaptive and waveform-based approach LEAF. The waveform-based frontend achieved significantly better classification performance than the Mel-spectrogram frontend by adapting its feature extraction parameters during training. This result is encouraging for future implementations of deep learning technology for automatic insect sound recognition, especially if larger datasets become available.
【8】 Back-Translation-Style Data Augmentation for Mandarin Chinese Polyphone Disambiguation标题：汉语多音节消解中的回译式数据增强链接：https://arxiv.org/abs/2211.09495
作者：Chunyu Qiang,Peng Yang,Hao Che,Jinba Xiao,Xiaorui Wang,Zhongyuan Wang机构：Kwai, Beijing, P.R. China备注：Published to APSIPA ASC 2022摘要：汉字形音转换（G2P）在汉语文语转换（TTS）系统中占有重要地位，其中多音字消歧是最大的挑战之一。现有的多音字消歧模型大多是在人工标注的数据集上训练的，而公开的多音字消歧数据集非常缺乏。本文提出了一种简单的反译式数据扩充方法，利用大量未标注文本数据进行汉语多音字消歧。受机器翻译领域提出的回译技术的启发，我们建立了一个字形到音素（G2P）模型来预测多音字的发音，以及一个音素到字形（P2G）模型来预测发音到文本。同时，提出了基于窗口的匹配策略和多模型评分策略来判断伪标签的正确性。针对训练集中典型多音字分布不均衡或数据不足的问题，设计了一种数据均衡策略，以提高多音字的准确率。实验结果验证了所提出的反翻译式数据扩充方法的有效性。摘要：Conversion of Chinese Grapheme-to-Phoneme (G2P) plays an important role in Mandarin Chinese Text-To-Speech (TTS) systems, where one of the biggest challenges is the task of polyphone disambiguation. Most of the previous polyphone disambiguation models are trained on manually annotated datasets, and publicly available datasets for polyphone disambiguation are scarce. In this paper we propose a simple back-translation-style data augmentation method for mandarin Chinese polyphone disambiguation, utilizing a large amount of unlabeled text data. Inspired by the back-translation technique proposed in the field of machine translation, we build a Grapheme-to-Phoneme (G2P) model to predict the pronunciation of polyphonic character, and a Phoneme-to-Grapheme (P2G) model to predict pronunciation into text. Meanwhile, a window-based matching strategy and a multi-model scoring strategy are proposed to judge the correctness of the pseudo-label. We design a data balance strategy to improve the accuracy of some typical polyphonic characters in the training set with imbalanced distribution or data scarcity. The experimental result shows the effectiveness of the proposed back-translation-style data augmentation method.
【9】 LongFNT: Long-form Speech Recognition with Factorized Neural Transducer标题：LongFNT：基于因式分解神经传感器的长格式语音识别链接：https://arxiv.org/abs/2211.09412
作者：Xun Gong,Yu Wu,Jinyu Li,Shujie Liu,Rui Zhao,Xie Chen,Yanmin Qian机构： MoE Key Lab of Artificial Intelligence, AI Institute, X-LANCE Lab, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Microsoft备注：Submitted to ICASSP2023摘要：传统的自动语音识别（ASR）系统通常只关注单个语音，而没有考虑包含有用历史信息的长语音，这在实际应用中更具有实用性。在我们的初步实验中，简单地关注普通神经转换器模型的较长转录历史并没有显示出太大的收益，因为预测网络不是一个纯粹的语言模型。这促使我们利用因子化的神经转换器结构，其中包含一个真实的语言模型，词汇预测器。我们提出了{LongFNT-Text}结构，该结构将句子级长形式特征直接与词汇预测器的输出融合，然后将单词级长形式特征嵌入到词汇预测器中，并使用预先训练的上下文编码器RoBERTa进一步提高性能。此外，我们提出了{LongFNT}架构，将长格式语音延伸至原始语音输入，并取得最佳的效能。在LibriSpeech和GigaSpeech语料库上的实验结果表明，LongFNT算法的相对误词率分别降低了19%和12%。摘要：Traditional automatic speech recognition~(ASR) systems usually focus on individual utterances, without considering long-form speech with useful historical information, which is more practical in real scenarios. Simply attending longer transcription history for a vanilla neural transducer model shows no much gain in our preliminary experiments, since the prediction network is not a pure language model. This motivates us to leverage the factorized neural transducer structure, containing a real language model, the vocabulary predictor. We propose the {LongFNT-Text} architecture, which fuses the sentence-level long-form features directly with the output of the vocabulary predictor and then embeds token-level long-form features inside the vocabulary predictor, with a pre-trained contextual encoder RoBERTa to further boost the performance. Moreover, we propose the {LongFNT} architecture by extending the long-form speech to the original speech input and achieve the best performance. The effectiveness of our LongFNT approach is validated on LibriSpeech and GigaSpeech corpora with 19% and 12% relative word error rate~(WER) reduction, respectively.
【10】 NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesis标题：Nansy++：具有神经分析和合成功能的统一语音合成链接：https://arxiv.org/abs/2211.09407
作者：Hyeong-Seok Choi,Jinhyeok Yang,Juheon Lee,Hyeongju Kim机构：Seoul National University, Supertone, Inc.备注：Submitted to ICLR 2023摘要：尽管语音合成的各种应用共同产生“语音”作为输出，但是它们已经被独立地开发。此外，大多数语音合成模型仍然需要大量与注释标签配对的音频数据（例如，文本转录和乐谱）用于训练。为此，我们提出了一个统一的框架，合成和操纵语音信号的分析特征，称为NANSY++。NANSY++的主干网络以自我监督的方式进行训练，不需要任何与音频配对的注释。在训练主干网络之后，我们通过对每个任务所需的分析功能进行部分建模，有效地处理了四种语音应用-即语音转换、文本到语音、歌声合成和语音设计。大量实验表明，该框架具有可控性强、数据效率高、训练收敛快等优点，同时能够提供高质量的合成结果.音频样本：tinyurl.com/8tnsy3uc。摘要：Various applications of voice synthesis have been developed independently despite the fact that they generate "voice" as output in common. In addition, most of the voice synthesis models still require a large number of audio data paired with annotated labels (e.g., text transcription and music score) for training. To this end, we propose a unified framework of synthesizing and manipulating voice signals from analysis features, dubbed NANSY++. The backbone network of NANSY++ is trained in a self-supervised manner that does not require any annotations paired with audio. After training the backbone network, we efficiently tackle four voice applications - i.e. voice conversion, text-to-speech, singing voice synthesis, and voice designing - by partially modeling the analysis features required for each task. Extensive experiments show that the proposed framework offers competitive advantages such as controllability, data efficiency, and fast training convergence, while providing high quality synthesis. Audio samples: tinyurl.com/8tnsy3uc.
【11】 ComMU: Dataset for Combinatorial Music Generation标题：COMU：用于组合音乐生成的数据集链接：https://arxiv.org/abs/2211.09385
作者：Lee Hyun,Taehyun Kim,Hyolim Kang,Minjoo Ki,Hyeonchan Hwang,Kwanho Park,Sharang Han,Seon Joo Kim机构：Pozalabs, Yonsei Univ.备注：19 pages, 12 figures摘要：自动音乐创作的商业采用需要生成适合于期望的环境（例如，用于浪漫电影、动作游戏、餐馆等的音乐）。本文介绍了组合音乐生成技术，它是一种根据给定的条件生成变化的背景音乐的新技术。组合音乐生成创建具有丰富音乐元数据的短音乐样本，并将它们组合以产生完整的音乐。此外，我们还引入了第一个由短音乐样本及其对应的12个音乐元数据组成的符号音乐数据集ComMU，用于组合音乐生成。ComMU的显著特性是（1）数据集由专业作曲家手动构建，具有诱导规律性的客观准则，以及（2）它具有包含作曲家意图的12个音乐元数据。实验结果表明，仅使用元数据就可以生成多样的高质量音乐，而我们独特的元数据，如音轨角色和扩展和弦质量，提高了自动作曲的能力。我们强烈建议在阅读本文之前先观看我们的视频（https：//pozalabs.github.io/ComMU）。摘要：Commercial adoption of automatic music composition requires the capability of generating diverse and high-quality music suitable for the desired context (e.g., music for romantic movies, action games, restaurants, etc.). In this paper, we introduce combinatorial music generation, a new task to create varying background music based on given conditions. Combinatorial music generation creates short samples of music with rich musical metadata, and combines them to produce a complete music. In addition, we introduce ComMU, the first symbolic music dataset consisting of short music samples and their corresponding 12 musical metadata for combinatorial music generation. Notable properties of ComMU are that (1) dataset is manually constructed by professional composers with an objective guideline that induces regularity, and (2) it has 12 musical metadata that embraces composers' intentions. Our results show that we can generate diverse high-quality music only with metadata, and that our unique metadata such as track-role and extended chord quality improves the capacity of the automatic composition. We highly recommend watching our video before reading the paper (https://pozalabs.github.io/ComMU).
【12】 Token-level Speaker Change Detection Using Speaker Difference and Speech Content via Continuous Integrate-and-fire标题：基于说话人差异和语音内容的连续积分点火令牌级说话人变化检测链接：https://arxiv.org/abs/2211.09381
作者：Zhiyun Fan,Zhenlin Liang,Linhao Dong,Yi Liu,Shiyu Zhou,Meng Cai,Jun Zhang,Zejun Ma,Bo Xu机构：Institute of Automation, Chinese Academy of Sciences, China, School of Artificial Intelligence, University of Chinese Academy of Sciences, China, Bytedance AI LAB摘要：在诸如会议和会话的多讲话者场景中，通常需要语音处理系统来分割音频，然后转录每个分割。这两个阶段分别由说话人变化检测（SCD）和自动语音识别（ASR）解决。大多数现有的SCD系统仅依赖于说话人信息而忽略了语音内容的重要性。本文提出了一种同时考虑说话人差异和语音内容的SCD系统。通过连续积分触发机制将这两个线索转换为表征级表示，然后将其组合以检测表征声学边界上的说话人变化。我们在公开的真实会议记录数据集AISHELL-4上评估了我们的方法的性能。实验结果表明，该方法比竞争性的帧级基线系统的等覆盖率-纯度（ECP）提高了2.45%。此外，我们还证明了语音内容和说话人差异对SCD任务的重要性，以及在令牌声学边界上进行SCD与逐帧进行SCD相比的优势。摘要：In multi-talker scenarios such as meetings and conversations, speech processing systems are usually required to segment the audio and then transcribe each segmentation. These two stages are addressed separately by speaker change detection (SCD) and automatic speech recognition (ASR). Most previous SCD systems rely solely on speaker information and ignore the importance of speech content. In this paper, we propose a novel SCD system that considers both cues of speaker difference and speech content. These two cues are converted into token-level representations by the continuous integrate-and-fire (CIF) mechanism and then combined for detecting speaker changes on the token acoustic boundaries. We evaluate the performance of our approach on a public real-recorded meeting dataset, AISHELL-4. The experiment results show that our method outperforms a competitive frame-level baseline system by 2.45% equal coverage-purity (ECP). In addition, we demonstrate the importance of speech content and speaker difference to the SCD task, and the advantages of conducting SCD on the token acoustic boundaries compared with conducting SCD frame by frame.
【13】 Balanced Deep CCA for Bird Vocalization Detection标题：用于鸟类发声检测的平衡深度CCA链接：https://arxiv.org/abs/2211.09376
作者：Sumit Kumar,B. Anshuman,Linus Ruettimann,Richard H. R. Hahnloser,Vipul Arora机构：Department of Electrical Engineering, Indian Institute of Technology, Kanpur, India., Institute of Neuroinformatics, University of Z¨urich and ETH Z¨urich, Z¨urich, Switzerland.摘要：当事件由两个不同的模态而不是仅一个模态捕获时，事件检测得到改善。但是，在多个模态上训练检测系统是具有挑战性的，特别是当存在大量未标记数据但有限量的标记数据时。我们开发了一种新颖的用于多模态数据的自监督学习技术，该技术学习同时记录的麦克风（声音）信号和加速度计（身体振动）信号之间的（隐藏）相关性。本研究的主要目标是在标记数据稀少且感兴趣的音频事件（鸣禽发声）稀疏时，学习与下游事件检测任务中的高性能相关的有用嵌入。我们的方法基于深度典型相关分析（DCCA），它遭受事件稀疏性。我们首先从标记数据中学习数据采样模型，并对它产生的输出应用DCCA，从而克服了正标记的稀疏性。与经典DCCA相比，该方法提高了无监督嵌入在下游有监督音频检测任务中的性能。由于数据标签经常是不平衡的，我们的方法可能在低资源场景中具有广泛的实用性。摘要：Event detection improves when events are captured by two different modalities rather than just one. But to train detection systems on multiple modalities is challenging, in particular when there is abundance of unlabelled data but limited amounts of labeled data. We develop a novel self-supervised learning technique for multi-modal data that learns (hidden) correlations between simultaneously recorded microphone (sound) signals and accelerometer (body vibration) signals. The key objective of this work is to learn useful embeddings associated with high performance in downstream event detection tasks when labeled data is scarce and the audio events of interest (songbird vocalizations) are sparse. We base our approach on deep canonical correlation analysis (DCCA) that suffers from event sparseness. We overcome the sparseness of positive labels by first learning a data sampling model from the labelled data and by applying DCCA on the output it produces. This method that we term balanced DCCA (b-DCCA) improves the performance of the unsupervised embeddings on the downstream supervised audio detection task compared to classsical DCCA. Because data labels are frequently imbalanced, our method might be of broad utility in low-resource scenarios.
【14】 Low-Resource Mongolian Speech Synthesis Based on Automatic Prosody Annotation标题：基于韵律自动标注的低资源蒙古语语音合成链接：https://arxiv.org/abs/2211.09365
作者：Xin Yuan,Robin Feng,Mingming Ye机构：Ye[,−,−,−,], VXI, China备注：submitted to NCMMSC2022摘要：虽然基于深度学习的文本到语音（TTS）模型（如VTS）已经显示出了出色的效果，但它们通常需要大量高质量的〈text，audio〉对来训练，而收集这些数据的成本很高。迄今为止，世界上大多数语言仍然缺乏开发TTS系统所需的训练数据。针对低资源蒙古语语音合成面临的两个问题，提出了两种改进方法：a）鉴于缺乏高质量的数据对，难以对从语言特征到声学特征的映射问题进行建模。利用预先训练好的VITS模型和迁移学习方法进行了改进。2）针对标注信息量少的问题，提出了一种自动韵律标注方法，对文本及其对应语音的韵律信息进行标注，从而提高低资源蒙古语的自然度和可懂度。通过实证研究，本文提出的方法的N-MOS为4.195，I-MOS为4.228。摘要：While deep learning-based text-to-speech (TTS) models such as VITS have shown excellent results, they typically require a sizable set of high-qualitypairs to train, which is expensive to collect. So far, most languages in the world still lack the training data needed to develop TTS systems. This paper proposes two improvement methods for the two problems faced by low-resource Mongolian speech synthesis: a) In view of the lack of high-qualitypairs of data, it is difficult to model the mapping problem from linguistic features to acoustic features. Improvements are made using pre-trained VITS model and transfer learning methods. b) In view of the problem of less labeled information, this paper proposes to use an automatic prosodic annotation method to label the prosodic information of text and corresponding speech, thereby improving the naturalness and intelligibility of low-resource Mongolian language. Through empirical research, the N-MOS of the method proposed in this paper is 4.195, and the I-MOS is 4.228.
【15】 Privacy against Real-Time Speech Emotion Detection via Acoustic Adversarial Evasion of Machine Learning标题：基于机器学习的声学对抗性回避的隐私对抗实时语音情感检测链接：https://arxiv.org/abs/2211.09273
作者：Brian Testa,Yi Xiao,Avery Gump,Asif Salekin机构： Syracuse University摘要：情绪监控是一个新兴的领域，具有广泛的隐私问题。这些担忧因配备多个传感器的物联网设备无处不在而加剧，这些设备可以支持这些监控用例。此处介绍的工作考虑了这样一个用例：使用与智能说话者相关的语音情感识别（SER）分类器。这项工作证明了在不损害智能扬声器的效用的情况下避开与智能扬声器绑定的黑盒SER分类器的能力。这种隐私问题是通过机器学习的对抗性规避来考虑的。我们的解决方案，通过遗传编程击败情感的声音识别（DARE-GP），使用遗传编程来生成非侵入性附加音频扰动（AAP）。通过限制这些AAP的进化，可以保护转录准确性，同时降低SER分类器性能。这些AAP的加性特性，以及以与话语和用户位置无关的方式为固定的用户集合生成这些AAP的方法，支持SER分类器的实时、真实世界规避。DARE-GP使用频谱特征，支持语音的情感内容，允许将AAP转移到以前看不见的黑盒SER分类器。此外，DARE-GP的性能优于最先进的SER规避技术，并对知识渊博的对手所采用的防御具有鲁棒性。本研究中的评估以针对两个现成的商用智能扬声器的声学评估达到高潮，其中单个AAP可以在超过70%的时间内避开黑盒分类器。最终评估将AAP回放部署在与唤醒词系统集成的小型系统（raspberry pi）上，以评估真实世界实时部署的功效，其中DARE-GP通过智能扬声器的唤醒词自动调用。摘要：Emotional Surveillance is an emerging area with wide-reaching privacy concerns. These concerns are exacerbated by ubiquitous IoT devices with multiple sensors that can support these surveillance use cases. The work presented here considers one such use case: the use of a speech emotion recognition (SER) classifier tied to a smart speaker. This work demonstrates the ability to evade black-box SER classifiers tied to a smart speaker without compromising the utility of the smart speaker. This privacy concern is considered through the lens of adversarial evasion of machine learning. Our solution, Defeating Acoustic Recognition of Emotion via Genetic Programming (DARE-GP), uses genetic programming to generate non-invasive additive audio perturbations (AAPs). By constraining the evolution of these AAPs, transcription accuracy can be protected while simultaneously degrading SER classifier performance. The additive nature of these AAPs, along with an approach that generates these AAPs for a fixed set of users in an utterance and user location-independent manner, supports real-time, real-world evasion of SER classifiers. DARE-GP's use of spectral features, which underlay the emotional content of speech, allows the transferability of AAPs to previously unseen black-box SER classifiers. Further, DARE-GP outperforms state-of-the-art SER evasion techniques and is robust against defenses employed by a knowledgeable adversary. The evaluations in this work culminate with acoustic evaluations against two off-the-shelf commercial smart speakers, where a single AAP could evade a black box classifier over 70% of the time. The final evaluation deployed AAP playback on a small-form-factor system (raspberry pi) integrated with a wake-word system to evaluate the efficacy of a real-world, real-time deployment where DARE-GP is automatically invoked with the smart speaker's wake word.
【16】 A Review of Intelligent Music Generation Systems标题：智能音乐生成系统综述链接：https://arxiv.org/abs/2211.09124
作者：Ziyi Zhao,Hanwei Liu,Song Li,Junwei Pang,Maoqing Zhang,Yi Qin,Lei Wang,Qidi Wu机构：College of Electric and Information Engineering, Tongji University, Cao’an Street, Shanghai, China., Department of Music Engineering, Shanghai Conservatory of Music, Fenyang street备注：Overall 24 Pages, 11 Figures, 2 Tables, 96 References items摘要：智能音乐生成是计算机创作领域中最热门的分支之一，它可以降低非专业人员的创作门槛，提高音乐创作的效率。在过去的五年中，基于算法的自动音乐生成的质量显著提高，这是由于使用现代生成算法来学习基于规则约束或音乐语料库的音乐片段中隐含的模式，从而生成各种风格的音乐样本。现有的一些文献综述缺乏生成模型的系统基准，并且在其视角上是传统的和保守的，导致该领域未来发展的愿景与当前快速的科学进步没有深度融合。本文对近年来智能音乐生成技术进行了全面的综述和分析，并进行了批判性的讨论，明确了它们各自的特点，并以一个总表的形式给出了它们。首先介绍了音乐作为信息流的编码方式和相关数据集，然后比较了不同类型的生成算法，总结了它们的优缺点，并讨论了现有的评价方法。最后研究了人工智能在作曲方面的发展，特别是通过比较东西方音乐生成技术的不同特点，分析了该领域的发展前景。摘要：Intelligent music generation, one of the most popular subfields of computer creativity, can lower the creative threshold for non-specialists and increase the efficiency of music creation. In the last five years, the quality of algorithm-based automatic music generation has increased significantly, motivated by the use of modern generative algorithms to learn the patterns implicit within a piece of music based on rule constraints or a musical corpus, thus generating music samples in various styles. Some of the available literature reviews lack a systematic benchmark of generative models and are traditional and conservative in their perspective, resulting in a vision of the future development of the field that is not deeply integrated with the current rapid scientific progress. In this paper, we conduct a comprehensive survey and analysis of recent intelligent music generation techniques,provide a critical discussion, explicitly identify their respective characteristics, and present them in a general table. We first introduce how music as a stream of information is encoded and the relevant datasets, then compare different types of generation algorithms, summarize their strengths and weaknesses, and discuss existing methods for evaluation. Finally, the development of artificial intelligence in composition is studied, especially by comparing the different characteristics of music generation techniques in the East and West and analyzing the development prospects in this field.
【17】 EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance标题：EmoDiff：带软标签引导的强度可控的情感文语转换链接：https://arxiv.org/abs/2211.09496
作者：Yiwei Guo,Chenpeng Du,Xie Chen,Kai Yu机构：MoE Key Lab of Artificial Intelligence, AI Institute, X-LANCE Lab, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China备注：Submitted to ICASSP2023摘要：虽然现有的神经文本语音转换（TTS）模型能够产生高质量的语音，但强度可控的情感TTS仍然是一个具有挑战性的任务。大多数现有方法需要外部优化来计算强度，导致次优结果或质量下降。本文提出了一种基于扩散的TTS模型EmoDiff，该模型中的情感强度可以通过一种基于分类器指导的软标签指导技术来控制.具体而言，EmoDiff不是使用指定情感的单热点向量来引导，而是使用软标签来引导，其中指定情感和\textit{Neutral}的值分别设置为$\alpha$和$1-\alpha$。这里的$\alpha$表示情绪强度，可以从0到1中选择。实验结果表明，EmoDiff能够在保持较高语音质量的同时，精确地控制情感强度。此外，在反向去噪过程中，通过采样可以产生具有特定情感强度的多样性语音。摘要：Although current neural text-to-speech (TTS) models are able to generate high-quality speech, intensity controllable emotional TTS is still a challenging task. Most existing methods need external optimizations for intensity calculation, leading to suboptimal results or degraded quality. In this paper, we propose EmoDiff, a diffusion-based TTS model where emotion intensity can be manipulated by a proposed soft-label guidance technique derived from classifier guidance. Specifically, instead of being guided with a one-hot vector for the specified emotion, EmoDiff is guided with a soft label where the value of the specified emotion and \textit{Neutral} is set to $\alpha$ and $1-\alpha$ respectively. The $\alpha$ here represents the emotion intensity and can be chosen from 0 to 1. Our experiments show that EmoDiff can precisely control the emotion intensity while maintaining high voice quality. Moreover, diverse speech with specified emotion intensity can be generated by sampling in the reverse denoising process.
【18】 Any-speaker Adaptive Text-To-Speech Synthesis with Diffusion Models标题：基于扩散模型的任意说话人自适应文语转换合成链接：https://arxiv.org/abs/2211.09383
作者：Minki Kang,Dongchan Min,Sung Ju Hwang机构：AITRICS, KAIST备注：Under Review摘要：近年来，随着神经生成建模技术的发展，文语转换（TTS）技术取得了长足的进步。然而，现有的任意说话人自适应TTS方法由于在模仿目标说话人风格方面的准确性不够理想，所以其性能并不理想。本文提出了一种基于扩散模型的任意说话人自适应TTS框架Grad-StyleSpeech，该框架能够在给定几秒参考语音的情况下，生成与目标说话人语音具有极高相似度的高度自然的语音。在英语基准测试中，Grad-StyleSpeech显著优于最近的说话者自适应TTS基线。音频示例可在www.example.com上获得https://nardien.github.io/grad-stylespeech-demo。摘要：There has been a significant progress in Text-To-Speech (TTS) synthesis technology in recent years, thanks to the advancement in neural generative modeling. However, existing methods on any-speaker adaptive TTS have achieved unsatisfactory performance, due to their suboptimal accuracy in mimicking the target speakers' styles. In this work, we present Grad-StyleSpeech, which is an any-speaker adaptive TTS framework that is based on a diffusion model that can generate highly natural speech with extremely high similarity to target speakers' voice, given a few seconds of reference speech. Grad-StyleSpeech significantly outperforms recent speaker-adaptive TTS baselines on English benchmarks. Audio samples are available at https://nardien.github.io/grad-stylespeech-demo.
【19】 SpectNet : End-to-End Audio Signal Classification Using Learnable Spectrograms标题：SpectNet：基于可学习频谱图的端到端音频信号分类链接：https://arxiv.org/abs/2211.09352
作者：Md. Istiaq Ansari,Taufiq Hasan机构： Bangladesh University of Engineering and Tech-nology (BUET)摘要：从音频信号进行模式识别是一个活跃的研究课题，包括音频标记、声学场景分类、音乐分类和其他领域。谱图和梅尔倒谱系数（MFCC）是音频信号分析和分类中最常用的特征之一。最近，深度卷积神经网络（CNN）已经成功地用于使用基于谱图的2D特征的音频分类问题。在本文中，我们介绍了SpectNet，这是一个集成的前端层，它在CNN架构中提取频谱图特征，可用于音频模式识别任务。前端层利用可学习伽马通滤波器，其使用mel尺度滤波器初始化。提出的层输出2D谱图图像，该2D谱图图像可以被馈送到2D CNN用于分类。整个网络的参数，包括前端滤波器组，可以通过反向传播来更新。该训练方案允许根据目标音频数据集来微调声谱图图像特征。在两个不同的音频信号分类任务中评估所提出的方法：心音异常检测和声学场景分类。与经典的声谱图图像特征相比，该方法在心音分类任务的MACC和声学场景分类任务的准确率上分别提高了1.02%和2.11%.我们实验的源代码可在以下网址找到：摘要：Pattern recognition from audio signals is an active research topic encompassing audio tagging, acoustic scene classification, music classification, and other areas. Spectrogram and mel-frequency cepstral coefficients (MFCC) are among the most commonly used features for audio signal analysis and classification. Recently, deep convolutional neural networks (CNN) have been successfully used for audio classification problems using spectrogram-based 2D features. In this paper, we present SpectNet, an integrated front-end layer that extracts spectrogram features within a CNN architecture that can be used for audio pattern recognition tasks. The front-end layer utilizes learnable gammatone filters that are initialized using mel-scale filters. The proposed layer outputs a 2D spectrogram image which can be fed into a 2D CNN for classification. The parameters of the entire network, including the front-end filterbank, can be updated via back-propagation. This training scheme allows for fine-tuning the spectrogram-image features according to the target audio dataset. The proposed method is evaluated in two different audio signal classification tasks: heart sound anomaly detection and acoustic scene classification. The proposed method shows a significant 1.02\% improvement in MACC for the heart sound classification task and 2.11\% improvement in accuracy for the acoustic scene classification task compared to the classical spectrogram image features. The source code of our experiments can be found at \url{https://github.com/mHealthBuet/SpectNet}
【20】 Unsupervised Model-based speaker adaptation of end-to-end lattice-free MMI model for speech recognition标题：基于无监督模型的语音识别端到端无格MMI模型说话人自适应链接：https://arxiv.org/abs/2211.09313
作者：Xurong Xie,Xunying Liu,Hui Chen,Hongan Wang机构：Institute of Software, Chinese Academy of Sciences, China, The Chinese University of Hong Kong, Hong Kong SAR, China备注：6 pages, 2 figures, submitted to ICASSP 2022摘要：说话人变异性建模是自动语音识别系统的关键问题。本文采用基于学习隐单元贡献（LHUC）的自适应技术，结合紧凑的说话人相关（SD）参数，对端到端（E2 E）无格MMI（LF-MMI）模型进行说话人自适应训练（SAT）和无监督测试时说话人自适应。提出了一种基于无监督模型的自适应框架，利用LF-MMI和交叉熵（CE）准则估计E2 E范式中的SD参数.标准LHUC自适应的各种正则化方法，贝叶斯LHUC（BLHUC）自适应，以减轻E2 E LF-MMI CNN-TDNN和CNN-TDNN-BLSTM模型上的过拟合风险。采用基于格的置信度估计进行自适应数据选择，降低了监督标签的不确定性。在300小时的Switchboard任务上的实验表明，将BLHUC应用于所提出的无监督E2 E自适应框架中的字节对编码基于BPE的E2 E LF-MMI系统在相对字错误率方面始终优于基线系统（WER）在NIST Hub 5 '00和RT 03评估集上分别降低了10.5%和14.7%，并分别实现了9.0%和9.7%的WER最佳性能。这些结果与现有技术的适应LF-MMI混合系统和适应的基于Conformer的E2 E系统的结果相当。摘要：Modeling the speaker variability is a key challenge for automatic speech recognition (ASR) systems. In this paper, the learning hidden unit contributions (LHUC) based adaptation techniques with compact speaker dependent (SD) parameters are used to facilitate both speaker adaptive training (SAT) and unsupervised test-time speaker adaptation for end-to-end (E2E) lattice-free MMI (LF-MMI) models. An unsupervised model-based adaptation framework is proposed to estimate the SD parameters in E2E paradigm using LF-MMI and cross entropy (CE) criterions. Various regularization methods of the standard LHUC adaptation, e.g., the Bayesian LHUC (BLHUC) adaptation, are systematically investigated to mitigate the risk of overfitting, on E2E LF-MMI CNN-TDNN and CNN-TDNN-BLSTM models. Lattice-based confidence score estimation is used for adaptation data selection to reduce the supervision label uncertainty. Experiments on the 300-hour Switchboard task suggest that applying BLHUC in the proposed unsupervised E2E adaptation framework to byte pair encoding (BPE) based E2E LF-MMI systems consistently outperformed the baseline systems by relative word error rate (WER) reductions up to 10.5% and 14.7% on the NIST Hub5'00 and RT03 evaluation sets, and achieved the best performance in WERs of 9.0% and 9.7%, respectively. These results are comparable to the results of state-of-the-art adapted LF-MMI hybrid systems and adapted Conformer-based E2E systems.
【21】 A Two-Stage Deep Representation Learning-Based Speech Enhancement Method Using Variational Autoencoder and Adversarial Training标题：基于变分自动编码和对抗性训练的两阶段深度表示学习语音增强方法链接：https://arxiv.org/abs/2211.09166
作者：Yang Xiang,Jesper Lisby Højvang,Morten Højfeldt Rasmussen,Mads Græsbøll Christensen备注：Submitted to IEEE/ACM Transactions on Audio, Speech and Language Processing摘要：本文主要研究利用深度表征学习（DRL）进行语音增强（SE）。一般而言，深度神经网络（DNN）的性能严重依赖于数据表示的学习。然而，DRL的重要性在许多基于DNN的SE算法中往往被忽略。为了获得更高质量的增强语音，提出了一种基于DRL的两阶段SE方法。在第一阶段，我们解开不同的潜变量，因为解开的表示可以帮助DNN生成更好的增强语音。具体地，我们使用β-变分自动编码器（VAE）算法来从观测信号获得语音和噪声的后验估计和相关表示。然而，由于后验和表示是难以处理的，并且我们只能应用条件假设来估计它们，因此难以确保这些估计总是相当准确的，这可能潜在地降低信号估计的最终准确性。为了进一步提高增强语音的质量，在第二阶段引入对抗训练，以降低不准确后验估计对信号重构的影响，提高信号估计精度，使算法对潜在的不准确后验估计具有更强的鲁棒性。结果，可以实现更好的SE性能。实验结果表明，该策略可以帮助类似的基于DNN的SE算法获得更高的短时客观可懂度（STOI）、语音质量感知评价（PESQ）和尺度不变信失真比（SI-SDR）分数。此外，该算法的性能也优于现有的SE算法.摘要：This paper focuses on leveraging deep representation learning (DRL) for speech enhancement (SE). In general, the performance of the deep neural network (DNN) is heavily dependent on the learning of data representation. However, the DRL's importance is often ignored in many DNN-based SE algorithms. To obtain a higher quality enhanced speech, we propose a two-stage DRL-based SE method through adversarial training. In the first stage, we disentangle different latent variables because disentangled representations can help DNN generate a better enhanced speech. Specifically, we use the $\beta$-variational autoencoder (VAE) algorithm to obtain the speech and noise posterior estimations and related representations from the observed signal. However, since the posteriors and representations are intractable and we can only apply a conditional assumption to estimate them, it is difficult to ensure that these estimations are always pretty accurate, which may potentially degrade the final accuracy of the signal estimation. To further improve the quality of enhanced speech, in the second stage, we introduce adversarial training to reduce the effect of the inaccurate posterior towards signal reconstruction and improve the signal estimation accuracy, making our algorithm more robust for the potentially inaccurate posterior estimations. As a result, better SE performance can be achieved. The experimental results indicate that the proposed strategy can help similar DNN-based SE algorithms achieve higher short-time objective intelligibility (STOI), perceptual evaluation of speech quality (PESQ), and scale-invariant signal-to-distortion ratio (SI-SDR) scores. Moreover, the proposed algorithm can also outperform recent competitive SE algorithms.
【1】 EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidance标题：EmoDiff：带软标签引导的强度可控的情感文语转换链接：https://arxiv.org/abs/2211.09496
* 与cs.SD语音【17】为同一篇
作者：Yiwei Guo,Chenpeng Du,Xie Chen,Kai Yu机构：MoE Key Lab of Artificial Intelligence, AI Institute, X-LANCE Lab, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China备注：Submitted to ICASSP2023摘要：虽然现有的神经文本语音转换（TTS）模型能够产生高质量的语音，但强度可控的情感TTS仍然是一个具有挑战性的任务。大多数现有方法需要外部优化来计算强度，导致次优结果或质量下降。本文提出了一种基于扩散的TTS模型EmoDiff，该模型中的情感强度可以通过一种基于分类器指导的软标签指导技术来控制.具体而言，EmoDiff不是使用指定情感的单热点向量来引导，而是使用软标签来引导，其中指定情感和\textit{Neutral}的值分别设置为$\alpha$和$1-\alpha$。这里的$\alpha$表示情绪强度，可以从0到1中选择。实验结果表明，EmoDiff能够在保持较高语音质量的同时，精确地控制情感强度。此外，在反向去噪过程中，通过采样可以产生具有特定情感强度的多样性语音。摘要：Although current neural text-to-speech (TTS) models are able to generate high-quality speech, intensity controllable emotional TTS is still a challenging task. Most existing methods need external optimizations for intensity calculation, leading to suboptimal results or degraded quality. In this paper, we propose EmoDiff, a diffusion-based TTS model where emotion intensity can be manipulated by a proposed soft-label guidance technique derived from classifier guidance. Specifically, instead of being guided with a one-hot vector for the specified emotion, EmoDiff is guided with a soft label where the value of the specified emotion and \textit{Neutral} is set to $\alpha$ and $1-\alpha$ respectively. The $\alpha$ here represents the emotion intensity and can be chosen from 0 to 1. Our experiments show that EmoDiff can precisely control the emotion intensity while maintaining high voice quality. Moreover, diverse speech with specified emotion intensity can be generated by sampling in the reverse denoising process.
【2】 Any-speaker Adaptive Text-To-Speech Synthesis with Diffusion Models标题：基于扩散模型的任意说话人自适应文语转换合成链接：https://arxiv.org/abs/2211.09383
* 与cs.SD语音【18】为同一篇
作者：Minki Kang,Dongchan Min,Sung Ju Hwang机构：AITRICS, KAIST备注：Under Review摘要：近年来，随着神经生成建模技术的发展，文语转换（TTS）技术取得了长足的进步。然而，现有的任意说话人自适应TTS方法由于在模仿目标说话人风格方面的准确性不够理想，所以其性能并不理想。本文提出了一种基于扩散模型的任意说话人自适应TTS框架Grad-StyleSpeech，该框架能够在给定几秒参考语音的情况下，生成与目标说话人语音具有极高相似度的高度自然的语音。在英语基准测试中，Grad-StyleSpeech显著优于最近的说话者自适应TTS基线。音频示例可在www.example.com上获得https://nardien.github.io/grad-stylespeech-demo。摘要：There has been a significant progress in Text-To-Speech (TTS) synthesis technology in recent years, thanks to the advancement in neural generative modeling. However, existing methods on any-speaker adaptive TTS have achieved unsatisfactory performance, due to their suboptimal accuracy in mimicking the target speakers' styles. In this work, we present Grad-StyleSpeech, which is an any-speaker adaptive TTS framework that is based on a diffusion model that can generate highly natural speech with extremely high similarity to target speakers' voice, given a few seconds of reference speech. Grad-StyleSpeech significantly outperforms recent speaker-adaptive TTS baselines on English benchmarks. Audio samples are available at https://nardien.github.io/grad-stylespeech-demo.
【3】 SpectNet : End-to-End Audio Signal Classification Using Learnable Spectrograms标题：SpectNet：基于可学习频谱图的端到端音频信号分类链接：https://arxiv.org/abs/2211.09352
* 与cs.SD语音【19】为同一篇
作者：Md. Istiaq Ansari,Taufiq Hasan机构： Bangladesh University of Engineering and Tech-nology (BUET)摘要：从音频信号进行模式识别是一个活跃的研究课题，包括音频标记、声学场景分类、音乐分类和其他领域。谱图和梅尔倒谱系数（MFCC）是音频信号分析和分类中最常用的特征之一。最近，深度卷积神经网络（CNN）已经成功地用于使用基于谱图的2D特征的音频分类问题。在本文中，我们介绍了SpectNet，这是一个集成的前端层，它在CNN架构中提取频谱图特征，可用于音频模式识别任务。前端层利用可学习伽马通滤波器，其使用mel尺度滤波器初始化。提出的层输出2D谱图图像，该2D谱图图像可以被馈送到2D CNN用于分类。整个网络的参数，包括前端滤波器组，可以通过反向传播来更新。该训练方案允许根据目标音频数据集来微调声谱图图像特征。在两个不同的音频信号分类任务中评估所提出的方法：心音异常检测和声学场景分类。与经典的声谱图图像特征相比，该方法在心音分类任务的MACC和声学场景分类任务的准确率上分别提高了1.02%和2.11%.我们实验的源代码可在以下网址找到：摘要：Pattern recognition from audio signals is an active research topic encompassing audio tagging, acoustic scene classification, music classification, and other areas. Spectrogram and mel-frequency cepstral coefficients (MFCC) are among the most commonly used features for audio signal analysis and classification. Recently, deep convolutional neural networks (CNN) have been successfully used for audio classification problems using spectrogram-based 2D features. In this paper, we present SpectNet, an integrated front-end layer that extracts spectrogram features within a CNN architecture that can be used for audio pattern recognition tasks. The front-end layer utilizes learnable gammatone filters that are initialized using mel-scale filters. The proposed layer outputs a 2D spectrogram image which can be fed into a 2D CNN for classification. The parameters of the entire network, including the front-end filterbank, can be updated via back-propagation. This training scheme allows for fine-tuning the spectrogram-image features according to the target audio dataset. The proposed method is evaluated in two different audio signal classification tasks: heart sound anomaly detection and acoustic scene classification. The proposed method shows a significant 1.02\% improvement in MACC for the heart sound classification task and 2.11\% improvement in accuracy for the acoustic scene classification task compared to the classical spectrogram image features. The source code of our experiments can be found at \url{https://github.com/mHealthBuet/SpectNet}
【4】 Unsupervised Model-based speaker adaptation of end-to-end lattice-free MMI model for speech recognition标题：基于无监督模型的语音识别端到端无格MMI模型说话人自适应链接：https://arxiv.org/abs/2211.09313
* 与cs.SD语音【20】为同一篇
作者：Xurong Xie,Xunying Liu,Hui Chen,Hongan Wang机构：Institute of Software, Chinese Academy of Sciences, China, The Chinese University of Hong Kong, Hong Kong SAR, China备注：6 pages, 2 figures, submitted to ICASSP 2022摘要：说话人变异性建模是自动语音识别系统的关键问题。本文采用基于学习隐单元贡献（LHUC）的自适应技术，结合紧凑的说话人相关（SD）参数，对端到端（E2 E）无格MMI（LF-MMI）模型进行说话人自适应训练（SAT）和无监督测试时说话人自适应。提出了一种基于无监督模型的自适应框架，利用LF-MMI和交叉熵（CE）准则估计E2 E范式中的SD参数.标准LHUC自适应的各种正则化方法，贝叶斯LHUC（BLHUC）自适应，以减轻E2 E LF-MMI CNN-TDNN和CNN-TDNN-BLSTM模型上的过拟合风险。采用基于格的置信度估计进行自适应数据选择，降低了监督标签的不确定性。在300小时的Switchboard任务上的实验表明，将BLHUC应用于所提出的无监督E2 E自适应框架中的字节对编码基于BPE的E2 E LF-MMI系统在相对字错误率方面始终优于基线系统（WER）在NIST Hub 5 '00和RT 03评估集上分别降低了10.5%和14.7%，并分别实现了9.0%和9.7%的WER最佳性能。这些结果与现有技术的适应LF-MMI混合系统和适应的基于Conformer的E2 E系统的结果相当。摘要：Modeling the speaker variability is a key challenge for automatic speech recognition (ASR) systems. In this paper, the learning hidden unit contributions (LHUC) based adaptation techniques with compact speaker dependent (SD) parameters are used to facilitate both speaker adaptive training (SAT) and unsupervised test-time speaker adaptation for end-to-end (E2E) lattice-free MMI (LF-MMI) models. An unsupervised model-based adaptation framework is proposed to estimate the SD parameters in E2E paradigm using LF-MMI and cross entropy (CE) criterions. Various regularization methods of the standard LHUC adaptation, e.g., the Bayesian LHUC (BLHUC) adaptation, are systematically investigated to mitigate the risk of overfitting, on E2E LF-MMI CNN-TDNN and CNN-TDNN-BLSTM models. Lattice-based confidence score estimation is used for adaptation data selection to reduce the supervision label uncertainty. Experiments on the 300-hour Switchboard task suggest that applying BLHUC in the proposed unsupervised E2E adaptation framework to byte pair encoding (BPE) based E2E LF-MMI systems consistently outperformed the baseline systems by relative word error rate (WER) reductions up to 10.5% and 14.7% on the NIST Hub5'00 and RT03 evaluation sets, and achieved the best performance in WERs of 9.0% and 9.7%, respectively. These results are comparable to the results of state-of-the-art adapted LF-MMI hybrid systems and adapted Conformer-based E2E systems.
【5】 A Two-Stage Deep Representation Learning-Based Speech Enhancement Method Using Variational Autoencoder and Adversarial Training标题：基于变分自动编码和对抗性训练的两阶段深度表示学习语音增强方法链接：https://arxiv.org/abs/2211.09166
* 与cs.SD语音【21】为同一篇
作者：Yang Xiang,Jesper Lisby Højvang,Morten Højfeldt Rasmussen,Mads Græsbøll Christensen备注：Submitted to IEEE/ACM Transactions on Audio, Speech and Language Processing摘要：本文主要研究利用深度表征学习（DRL）进行语音增强（SE）。一般而言，深度神经网络（DNN）的性能严重依赖于数据表示的学习。然而，DRL的重要性在许多基于DNN的SE算法中往往被忽略。为了获得更高质量的增强语音，提出了一种基于DRL的两阶段SE方法。在第一阶段，我们解开不同的潜变量，因为解开的表示可以帮助DNN生成更好的增强语音。具体地，我们使用β-变分自动编码器（VAE）算法来从观测信号获得语音和噪声的后验估计和相关表示。然而，由于后验和表示是难以处理的，并且我们只能应用条件假设来估计它们，因此难以确保这些估计总是相当准确的，这可能潜在地降低信号估计的最终准确性。为了进一步提高增强语音的质量，在第二阶段引入对抗训练，以降低不准确后验估计对信号重构的影响，提高信号估计精度，使算法对潜在的不准确后验估计具有更强的鲁棒性。结果，可以实现更好的SE性能。实验结果表明，该策略可以帮助类似的基于DNN的SE算法获得更高的短时客观可懂度（STOI）、语音质量感知评价（PESQ）和尺度不变信失真比（SI-SDR）分数。此外，该算法的性能也优于现有的SE算法.摘要：This paper focuses on leveraging deep representation learning (DRL) for speech enhancement (SE). In general, the performance of the deep neural network (DNN) is heavily dependent on the learning of data representation. However, the DRL's importance is often ignored in many DNN-based SE algorithms. To obtain a higher quality enhanced speech, we propose a two-stage DRL-based SE method through adversarial training. In the first stage, we disentangle different latent variables because disentangled representations can help DNN generate a better enhanced speech. Specifically, we use the $\beta$-variational autoencoder (VAE) algorithm to obtain the speech and noise posterior estimations and related representations from the observed signal. However, since the posteriors and representations are intractable and we can only apply a conditional assumption to estimate them, it is difficult to ensure that these estimations are always pretty accurate, which may potentially degrade the final accuracy of the signal estimation. To further improve the quality of enhanced speech, in the second stage, we introduce adversarial training to reduce the effect of the inaccurate posterior towards signal reconstruction and improve the signal estimation accuracy, making our algorithm more robust for the potentially inaccurate posterior estimations. As a result, better SE performance can be achieved. The experimental results indicate that the proposed strategy can help similar DNN-based SE algorithms achieve higher short-time objective intelligibility (STOI), perceptual evaluation of speech quality (PESQ), and scale-invariant signal-to-distortion ratio (SI-SDR) scores. Moreover, the proposed algorithm can also outperform recent competitive SE algorithms.
【6】 Heart Abnormality Detection from Heart Sound Signals using MFCC Feature and Dual Stream Attention Based Network标题：基于MFCC特征和双流注意网络的心音信号心脏异常检测链接：https://arxiv.org/abs/2211.09751
* 与cs.SD语音【1】为同一篇
作者：Nayeeb Rashid,Swapnil Saha,Mohseu Rashid Subah,Rizwan Ahmed Robin,Syed Mortuza Hasan Fahim,Shahed Ahmed,Talha Ibn Mahmud机构：Department of Electrical and Electronic Engineering (EEE), Bangladesh University of Engineering and Technology (BUET), Dhaka - , Bangladesh.摘要：心血管疾病是当今世界死亡的主要原因之一，心脏疾病的早期筛查在预防心血管疾病中发挥着至关重要的作用。心音信号是心脏状况的主要指标之一，并且可以用于检测心脏中的异常。心音信号的采集是无创的、成本有效的并且需要最少的设备。但目前从心音信号中检测心脏异常在很大程度上依赖于医生的专业知识和经验。因此，从心音信号中检测心脏异常的自动检测系统对于生活在不发达地区的人们来说是一笔巨大的财富。本文提出了一种基于深度学习的新型注意机制双流网络，该网络同时使用原始心音信号和MFCC特征来检测患者心脏状况的异常。深度神经网络具有使用原始心音信号的卷积流和使用信号的MFCC特征的循环流。使用新颖的注意力网络将来自这两个流的特征合并在一起，并使其通过分类网络。该模型在最大的公开可用的心音信号数据集上训练，获得了87.11的准确度、82.41的灵敏度、91.8的特异性和87.12的MACC。摘要：Cardiovascular diseases are one of the leading cause of death in today's world and early screening of heart condition plays a crucial role in preventing them. The heart sound signal is one of the primary indicator of heart condition and can be used to detect abnormality in the heart. The acquisition of heart sound signal is non-invasive, cost effective and requires minimum equipment. But currently the detection of heart abnormality from heart sound signal depends largely on the expertise and experience of the physician. As such an automatic detection system for heart abnormality detection from heart sound signal can be a great asset for the people living in underdeveloped areas. In this paper we propose a novel deep learning based dual stream network with attention mechanism that uses both the raw heart sound signal and the MFCC features to detect abnormality in heart condition of a patient. The deep neural network has a convolutional stream that uses the raw heart sound signal and a recurrent stream that uses the MFCC features of the signal. The features from these two streams are merged together using a novel attention network and passed through the classification network. The model is trained on the largest publicly available dataset of PCG signal and achieves an accuracy of 87.11, sensitivity of 82.41, specificty of 91.8 and a MACC of 87.12.
【7】 Stutter-TTS: Controlled Synthesis and Improved Recognition of Stuttered Speech标题：卡顿-TTS：受控合成和改进的卡顿语音识别链接：https://arxiv.org/abs/2211.09731
* 与cs.SD语音【2】为同一篇
作者：Xin Zhang,Iván Vallés-Pérez,Andreas Stolcke,Chengzhu Yu,Jasha Droppo,Olabanji Shonibare,Roberto Barra-Chicote,Venkatesh Ravichandran机构：Amazon Alexa AI备注：None摘要：口吃是一种语言障碍，其中自然的语言流被音节、单词和短语的阻塞、重复或延长所打断。现有的大多数自动语音识别（ASR）接口在处理口吃语音时性能较差，主要是由于缺乏匹配的训练数据。因此，具有口吃的语音的合成提供了改善这种类型的语音的ASR的机会。我们描述了口吃TTS，一种能够合成不同类型口吃话语的端到端神经文本到语音模型。我们开发了一个简单而有效的韵律控制策略，在训练过程中将附加标记引入源文本中以表示特定的口吃特征。通过选择口吃标记的位置，Stutter-TTS允许对合成话语中口吃发生的位置进行单词级控制。我们能够以高准确度合成口吃事件（F1分数在0.63和0.84之间，取决于口吃类型）。通过对合成口吃语音的ASR模型进行微调，我们能够将口吃话语的单词错误相对减少5.7%，而对于流畅话语仅具有较小（〈0.2%相对）的降级。摘要：Stuttering is a speech disorder where the natural flow of speech is interrupted by blocks, repetitions or prolongations of syllables, words and phrases. The majority of existing automatic speech recognition (ASR) interfaces perform poorly on utterances with stutter, mainly due to lack of matched training data. Synthesis of speech with stutter thus presents an opportunity to improve ASR for this type of speech. We describe Stutter-TTS, an end-to-end neural text-to-speech model capable of synthesizing diverse types of stuttering utterances. We develop a simple, yet effective prosody-control strategy whereby additional tokens are introduced into source text during training to represent specific stuttering characteristics. By choosing the position of the stutter tokens, Stutter-TTS allows word-level control of where stuttering occurs in the synthesized utterance. We are able to synthesize stutter events with high accuracy (F1-scores between 0.63 and 0.84, depending on stutter type). By fine-tuning an ASR model on synthetic stuttered speech we are able to reduce word error by 5.7% relative on stuttered utterances, with only minor (<0.2% relative) degradation for fluent utterances.
【8】 Design Considerations For Hypothesis Rejection Modules In Spoken Language Understanding Systems标题：口语理解系统中假设拒绝模块的设计考虑链接：https://arxiv.org/abs/2211.09711
* 与cs.SD语音【3】为同一篇
作者：Aman Alok,Rahul Gupta,Shankar Ananthakrishnan备注：5 pages. ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)摘要：口语理解（SLU）系统通常由一组机器学习模型组成，这些模型联合操作以产生SLU假设。然后，将生成的假设发送到下游组件以进行进一步操作。然而，期望在向下游发送不正确的假设之前丢弃该假设。在本文中，我们提出了两种SLU假设拒绝模块的设计：（i）对域特定SLU假设执行拒绝的方案R1，以及（ii）对从整个SLU系统生成的假设执行拒绝的方案R2。两种方案中的假设拒绝模块基于从指向SLU系统的话语中提取的特征、相关联的SLU假设和SLU置信度分数来拒绝/接受假设。我们的实验表明，两种方案产生类似的结果（方案R1：2.5% FRR@4.5% FAR，方案R2：2.5% FRR@4.6% FAR），性能最佳的系统使用所有可用功能。我们认为，虽然可以选择其中一种抑制方案，但它们存在一些内在差异，在进行选择时需要考虑这些差异。此外，我们还将ASR功能整合到抑制模块中（在3.8% FAR时获得1.9% FRR），并分析改进情况。摘要：Spoken Language Understanding (SLU) systems typically consist of a set of machine learning models that operate in conjunction to produce an SLU hypothesis. The generated hypothesis is then sent to downstream components for further action. However, it is desirable to discard an incorrect hypothesis before sending it downstream. In this work, we present two designs for SLU hypothesis rejection modules: (i) scheme R1 that performs rejection on domain specific SLU hypothesis and, (ii) scheme R2 that performs rejection on hypothesis generated from the overall SLU system. Hypothesis rejection modules in both schemes reject/accept a hypothesis based on features drawn from the utterance directed to the SLU system, the associated SLU hypothesis and SLU confidence score. Our experiments suggest that both the schemes yield similar results (scheme R1: 2.5% FRR @ 4.5% FAR, scheme R2: 2.5% FRR @ 4.6% FAR), with the best performing systems using all the available features. We argue that while either of the rejection schemes can be chosen over the other, they carry some inherent differences which need to be considered while making this choice. Additionally, we incorporate ASR features in the rejection module (obtaining an 1.9% FRR @ 3.8% FAR) and analyze the improvements.
【9】 Listen, denoise, action! Audio-driven motion synthesis with diffusion models标题：听着，丹尼斯，开拍！基于扩散模型的音频驱动运动合成链接：https://arxiv.org/abs/2211.09707
* 与cs.SD语音【4】为同一篇
作者：Simon Alexanderson,Rajmund Nagy,Jonas Beskow,Gustav Eje Henter机构：Division of Speech, Music and Hearing, KTH Royal Institute of Technology, Stockholm, Sweden备注：15 pages, 6 figures摘要：扩散模型作为一种表达能力强、可有效训练的概率模型，受到了广泛的关注。我们表明，这些模型非常适合于合成与音频共同出现的人体运动，例如共同语音手势，因为给定音频，运动是复杂的和高度模糊的，需要概率描述。具体来说，我们采用DiffWave架构来建模3D姿势序列，用Conformers代替膨胀卷积以提高准确性。我们还演示了对运动风格的控制，使用无分类器指导来调整风格表达的强度。在Trinity Speech-Gesture和ZeroEGGS数据集上进行的手势生成实验表明，该方法能够获得一流的运动质量，并具有不同的风格，其表达可以或多或少地变得明显。我们还使用相同的模型架构合成舞蹈运动和路径驱动的移动。最后，我们扩展了指导过程，以一种对合成任务有吸引力的方式执行风格插值，并与专家产品模型有联系，我们认为这是一个独立的贡献。有关视频示例，请访问https://www.speech.kth.se/research/listen-denoise-action/摘要：Diffusion models have experienced a surge of interest as highly expressive yet efficiently trainable probabilistic models. We show that these models are an excellent fit for synthesising human motion that co-occurs with audio, for example co-speech gesticulation, since motion is complex and highly ambiguous given audio, calling for a probabilistic description. Specifically, we adapt the DiffWave architecture to model 3D pose sequences, putting Conformers in place of dilated convolutions for improved accuracy. We also demonstrate control over motion style, using classifier-free guidance to adjust the strength of the stylistic expression. Gesture-generation experiments on the Trinity Speech-Gesture and ZeroEGGS datasets confirm that the proposed method achieves top-of-the-line motion quality, with distinctive styles whose expression can be made more or less pronounced. We also synthesise dance motion and path-driven locomotion using the same model architecture. Finally, we extend the guidance procedure to perform style interpolation in a manner that is appealing for synthesis tasks and has connections to product-of-experts models, a contribution we believe is of independent interest. Video examples are available at https://www.speech.kth.se/research/listen-denoise-action/
【10】 Towards Building Text-To-Speech Systems for the Next Billion Users标题：为下一个十亿用户打造文语转换系统链接：https://arxiv.org/abs/2211.09536
* 与cs.SD语音【5】为同一篇
作者：Gokul Karthik Kumar,Praveen S V,Pratyush Kumar,Mitesh M. Khapra,Karthik Nandakumar机构：AI,Bharat, India, Indian Institute of Technology Madras (IITM), India, Mohamed Bin Zayed University of Artificial Intelligence (MBZUAI), UAE, Microsoft Research, India备注：Under review in ICASSP 2023. First two authors contributed equally摘要：基于深度学习的文本到语音（TTS）系统随着模型架构、训练方法和跨说话者和语言的泛化的进步而迅速发展。然而，这些进展对于印度语言语音合成还没有被彻底研究。考虑到印度语言的数量和多样性，相对较低的资源可用性，以及尚未测试的神经TTS的各种进步，这种研究在计算上是昂贵的。在这篇论文中，我们评估了声学模型、声码器、补充损失函数、训练时间表以及达罗毗荼语和印度-雅利安语的说话人和语言多样性的选择。在此基础上，我们确定了FastPitch和HiFi-GAN V1的单语模型，在男性和女性说话人上联合训练以获得最佳性能。通过这种设置，我们训练和评估了13种语言的TTS模型，发现我们的模型在所有语言中都比现有模型有了显著的改进，如平均意见得分所测量的。我们开放Bhashini平台上的所有模型。摘要：Deep learning based text-to-speech (TTS) systems have been evolving rapidly with advances in model architectures, training methodologies, and generalization across speakers and languages. However, these advances have not been thoroughly investigated for Indian language speech synthesis. Such investigation is computationally expensive given the number and diversity of Indian languages, relatively lower resource availability, and the diverse set of advances in neural TTS that remain untested. In this paper, we evaluate the choice of acoustic models, vocoders, supplementary loss functions, training schedules, and speaker and language diversity for Dravidian and Indo-Aryan languages. Based on this, we identify monolingual models with FastPitch and HiFi-GAN V1, trained jointly on male and female speakers to perform the best. With this setup, we train and evaluate TTS models for 13 languages and find our models to significantly improve upon existing models in all languages as measured by mean opinion scores. We open-source all models on the Bhashini platform.
【11】 Hey ASR System! Why Aren't You More Inclusive? Automatic Speech Recognition Systems' Bias and Proposed Bias Mitigation Techniques. A Literature Review标题：嘿ASR系统！你为什么不更包容一些呢？自动语音识别系统的偏差和提出的偏差缓解技术。文献综述链接：https://arxiv.org/abs/2211.09511
* 与cs.SD语音【6】为同一篇
作者：Mikel K. Ngueajio,Gloria Washington机构：Department of Computer Science, Howard University, Washington DC , USA备注：In press at HCI International 2022 - Late Breaking Papers: Interacting with eXtended Reality and Artificial Intelligence, LNCS 13518摘要：言语是人类之间进行交流的基本手段。人工智能和复杂的语音技术的出现导致了基于人机交互的快速增长，这主要是由自动语音识别（ASR）系统推动的。ASR系统通常采用音频形式的人类语音并将其转换为单词，但对于一些用户，它不能解码语音，并且任何输出文本都充满了人类读者无法理解的错误。这些系统并非对每个人都同样有效，实际上会妨碍某些用户的工作效率。在这篇文章中，我们介绍了针对性别、种族、病人和残疾人的ASR偏见的研究，同时探讨了提出ASR去偏见技术以减轻这些歧视的研究。我们还讨论了设计更易访问和包容性更强的ASR技术的技术。对于调查的每种方法，我们还提供了所用调查和方法、所用ASR系统和语料库以及研究结果的总结，并强调了它们的优势和/或劣势。最后，我们提出未来的机会，为自然语言处理研究人员探索下一个层次的创造ASR技术。摘要：Speech is the fundamental means of communication between humans. The advent of AI and sophisticated speech technologies have led to the rapid proliferation of human-to-computer-based interactions, fueled primarily by Automatic Speech Recognition (ASR) systems. ASR systems normally take human speech in the form of audio and convert it into words, but for some users, it cannot decode the speech, and any output text is filled with errors that are incomprehensible to the human reader. These systems do not work equally for everyone and actually hinder the productivity of some users. In this paper, we present research that addresses ASR biases against gender, race, and the sick and disabled, while exploring studies that propose ASR debiasing techniques for mitigating these discriminations. We also discuss techniques for designing a more accessible and inclusive ASR technology. For each approach surveyed, we also provide a summary of the investigation and methods applied, the ASR systems and corpora used, and the research findings, and highlight their strengths and/or weaknesses. Finally, we propose future opportunities for Natural Language Processing researchers to explore in the next level creation of ASR technologies.
【12】 Adaptive Representations of Sound for Automatic Insect Recognition标题：用于昆虫自动识别的声音自适应表示法链接：https://arxiv.org/abs/2211.09503
* 与cs.SD语音【7】为同一篇
作者：Marius Faiß机构：Evolutionary Biology, Naturalis Biodiversity Center, Evolutionary Ecology, Darwinweg , CR Leiden, The Netherlands, Supervisor: Prof. Dan Stowell, ,, – ,,, EC, Contact: Prof. Dan Stowell, Format: Plos Computational Biology备注：30 pages, 9 figures Dataset: this https URL摘要：昆虫是我们生态系统不可或缺的一部分。这些通常体型小、易躲避的动物对周围环境有很大影响，提供了目前生物多样性和授粉职责的很大一部分，形成了食物链和许多生物和生态过程的基础。由于人类的影响，人口数量和生物多样性随着时间的推移迅速下降。监测这种下降情况对于有效实施保护措施变得越来越重要。但是，监测方法往往是侵入性的，时间和资源密集，并容易产生各种偏见。许多昆虫物种产生特征性的交配声音，这些声音可以容易地被检测和记录，而不需要大的成本或努力。使用深度学习方法，可以自动检测和分类来自现场记录的昆虫声音，以监测生物多样性和物种分布范围。在这个项目中，我使用现有的昆虫声音数据集（直翅目和蝉科）和机器学习方法来实现这一点，并评估它们在昆虫声学监测方面的潜力。我比较了传统的基于频谱图的深度学习方法与新的自适应和基于波形的方法LEAF的性能。通过在训练过程中调整特征提取参数，基于波形的前端获得了比Mel谱图前端更好的分类性能。这一结果对于未来实现用于昆虫声音自动识别的深度学习技术是令人鼓舞的，特别是在更大的数据集可用的情况下。摘要：Insects are an integral part of our ecosystem. These often small and evasive animals have a big impact on their surroundings, providing a large part of the present biodiversity and pollination duties, forming the foundation of the food chain and many biological and ecological processes. Due to factors of human influence, population numbers and biodiversity have been rapidly declining with time. Monitoring this decline has become increasingly important for conservation measures to be effectively implemented. But monitoring methods are often invasive, time and resource intense, and prone to various biases. Many insect species produce characteristic mating sounds that can easily be detected and recorded without large cost or effort. Using deep learning methods, insect sounds from field recordings could be automatically detected and classified to monitor biodiversity and species distribution ranges. In this project, I implement this using existing datasets of insect sounds (Orthoptera and Cicadidae) and machine learning methods and evaluate their potential for acoustic insect monitoring. I compare the performance of the conventional spectrogram-based deep learning method against the new adaptive and waveform-based approach LEAF. The waveform-based frontend achieved significantly better classification performance than the Mel-spectrogram frontend by adapting its feature extraction parameters during training. This result is encouraging for future implementations of deep learning technology for automatic insect sound recognition, especially if larger datasets become available.
【13】 Back-Translation-Style Data Augmentation for Mandarin Chinese Polyphone Disambiguation标题：汉语多音节消解中的回译式数据增强链接：https://arxiv.org/abs/2211.09495
* 与cs.SD语音【8】为同一篇
作者：Chunyu Qiang,Peng Yang,Hao Che,Jinba Xiao,Xiaorui Wang,Zhongyuan Wang机构：Kwai, Beijing, P.R. China备注：Published to APSIPA ASC 2022摘要：汉字形音转换（G2P）在汉语文语转换（TTS）系统中占有重要地位，其中多音字消歧是最大的挑战之一。现有的多音字消歧模型大多是在人工标注的数据集上训练的，而公开的多音字消歧数据集非常缺乏。本文提出了一种简单的反译式数据扩充方法，利用大量未标注文本数据进行汉语多音字消歧。受机器翻译领域提出的回译技术的启发，我们建立了一个字形到音素（G2P）模型来预测多音字的发音，以及一个音素到字形（P2G）模型来预测发音到文本。同时，提出了基于窗口的匹配策略和多模型评分策略来判断伪标签的正确性。针对训练集中典型多音字分布不均衡或数据不足的问题，设计了一种数据均衡策略，以提高多音字的准确率。实验结果验证了所提出的反翻译式数据扩充方法的有效性。摘要：Conversion of Chinese Grapheme-to-Phoneme (G2P) plays an important role in Mandarin Chinese Text-To-Speech (TTS) systems, where one of the biggest challenges is the task of polyphone disambiguation. Most of the previous polyphone disambiguation models are trained on manually annotated datasets, and publicly available datasets for polyphone disambiguation are scarce. In this paper we propose a simple back-translation-style data augmentation method for mandarin Chinese polyphone disambiguation, utilizing a large amount of unlabeled text data. Inspired by the back-translation technique proposed in the field of machine translation, we build a Grapheme-to-Phoneme (G2P) model to predict the pronunciation of polyphonic character, and a Phoneme-to-Grapheme (P2G) model to predict pronunciation into text. Meanwhile, a window-based matching strategy and a multi-model scoring strategy are proposed to judge the correctness of the pseudo-label. We design a data balance strategy to improve the accuracy of some typical polyphonic characters in the training set with imbalanced distribution or data scarcity. The experimental result shows the effectiveness of the proposed back-translation-style data augmentation method.
【14】 LongFNT: Long-form Speech Recognition with Factorized Neural Transducer标题：LongFNT：基于因式分解神经传感器的长格式语音识别链接：https://arxiv.org/abs/2211.09412
* 与cs.SD语音【9】为同一篇
作者：Xun Gong,Yu Wu,Jinyu Li,Shujie Liu,Rui Zhao,Xie Chen,Yanmin Qian机构： MoE Key Lab of Artificial Intelligence, AI Institute, X-LANCE Lab, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Microsoft备注：Submitted to ICASSP2023摘要：传统的自动语音识别（ASR）系统通常只关注单个语音，而没有考虑包含有用历史信息的长语音，这在实际应用中更具有实用性。在我们的初步实验中，简单地关注普通神经转换器模型的较长转录历史并没有显示出太大的收益，因为预测网络不是一个纯粹的语言模型。这促使我们利用因子化的神经转换器结构，其中包含一个真实的语言模型，词汇预测器。我们提出了{LongFNT-Text}结构，该结构将句子级长形式特征直接与词汇预测器的输出融合，然后将单词级长形式特征嵌入到词汇预测器中，并使用预先训练的上下文编码器RoBERTa进一步提高性能。此外，我们提出了{LongFNT}架构，将长格式语音延伸至原始语音输入，并取得最佳的效能。在LibriSpeech和GigaSpeech语料库上的实验结果表明，LongFNT算法的相对误词率分别降低了19%和12%。摘要：Traditional automatic speech recognition~(ASR) systems usually focus on individual utterances, without considering long-form speech with useful historical information, which is more practical in real scenarios. Simply attending longer transcription history for a vanilla neural transducer model shows no much gain in our preliminary experiments, since the prediction network is not a pure language model. This motivates us to leverage the factorized neural transducer structure, containing a real language model, the vocabulary predictor. We propose the {LongFNT-Text} architecture, which fuses the sentence-level long-form features directly with the output of the vocabulary predictor and then embeds token-level long-form features inside the vocabulary predictor, with a pre-trained contextual encoder RoBERTa to further boost the performance. Moreover, we propose the {LongFNT} architecture by extending the long-form speech to the original speech input and achieve the best performance. The effectiveness of our LongFNT approach is validated on LibriSpeech and GigaSpeech corpora with 19% and 12% relative word error rate~(WER) reduction, respectively.
【15】 NANSY++: Unified Voice Synthesis with Neural Analysis and Synthesis标题：Nansy++：具有神经分析和合成功能的统一语音合成链接：https://arxiv.org/abs/2211.09407
* 与cs.SD语音【10】为同一篇
作者：Hyeong-Seok Choi,Jinhyeok Yang,Juheon Lee,Hyeongju Kim机构：Seoul National University, Supertone, Inc.备注：Submitted to ICLR 2023摘要：尽管语音合成的各种应用共同产生“语音”作为输出，但是它们已经被独立地开发。此外，大多数语音合成模型仍然需要大量与注释标签配对的音频数据（例如，文本转录和乐谱）用于训练。为此，我们提出了一个统一的框架，合成和操纵语音信号的分析特征，称为NANSY++。NANSY++的主干网络以自我监督的方式进行训练，不需要任何与音频配对的注释。在训练主干网络之后，我们通过对每个任务所需的分析功能进行部分建模，有效地处理了四种语音应用-即语音转换、文本到语音、歌声合成和语音设计。大量实验表明，该框架具有可控性强、数据效率高、训练收敛快等优点，同时能够提供高质量的合成结果.音频样本：tinyurl.com/8tnsy3uc。摘要：Various applications of voice synthesis have been developed independently despite the fact that they generate "voice" as output in common. In addition, most of the voice synthesis models still require a large number of audio data paired with annotated labels (e.g., text transcription and music score) for training. To this end, we propose a unified framework of synthesizing and manipulating voice signals from analysis features, dubbed NANSY++. The backbone network of NANSY++ is trained in a self-supervised manner that does not require any annotations paired with audio. After training the backbone network, we efficiently tackle four voice applications - i.e. voice conversion, text-to-speech, singing voice synthesis, and voice designing - by partially modeling the analysis features required for each task. Extensive experiments show that the proposed framework offers competitive advantages such as controllability, data efficiency, and fast training convergence, while providing high quality synthesis. Audio samples: tinyurl.com/8tnsy3uc.
【16】 ComMU: Dataset for Combinatorial Music Generation标题：COMU：用于组合音乐生成的数据集链接：https://arxiv.org/abs/2211.09385
* 与cs.SD语音【11】为同一篇
作者：Lee Hyun,Taehyun Kim,Hyolim Kang,Minjoo Ki,Hyeonchan Hwang,Kwanho Park,Sharang Han,Seon Joo Kim机构：Pozalabs, Yonsei Univ.备注：19 pages, 12 figures摘要：自动音乐创作的商业采用需要生成适合于期望的环境（例如，用于浪漫电影、动作游戏、餐馆等的音乐）。本文介绍了组合音乐生成技术，它是一种根据给定的条件生成变化的背景音乐的新技术。组合音乐生成创建具有丰富音乐元数据的短音乐样本，并将它们组合以产生完整的音乐。此外，我们还引入了第一个由短音乐样本及其对应的12个音乐元数据组成的符号音乐数据集ComMU，用于组合音乐生成。ComMU的显著特性是（1）数据集由专业作曲家手动构建，具有诱导规律性的客观准则，以及（2）它具有包含作曲家意图的12个音乐元数据。实验结果表明，仅使用元数据就可以生成多样的高质量音乐，而我们独特的元数据，如音轨角色和扩展和弦质量，提高了自动作曲的能力。我们强烈建议在阅读本文之前先观看我们的视频（https：//pozalabs.github.io/ComMU）。摘要：Commercial adoption of automatic music composition requires the capability of generating diverse and high-quality music suitable for the desired context (e.g., music for romantic movies, action games, restaurants, etc.). In this paper, we introduce combinatorial music generation, a new task to create varying background music based on given conditions. Combinatorial music generation creates short samples of music with rich musical metadata, and combines them to produce a complete music. In addition, we introduce ComMU, the first symbolic music dataset consisting of short music samples and their corresponding 12 musical metadata for combinatorial music generation. Notable properties of ComMU are that (1) dataset is manually constructed by professional composers with an objective guideline that induces regularity, and (2) it has 12 musical metadata that embraces composers' intentions. Our results show that we can generate diverse high-quality music only with metadata, and that our unique metadata such as track-role and extended chord quality improves the capacity of the automatic composition. We highly recommend watching our video before reading the paper (https://pozalabs.github.io/ComMU).
【17】 Token-level Speaker Change Detection Using Speaker Difference and Speech Content via Continuous Integrate-and-fire标题：基于说话人差异和语音内容的连续积分点火令牌级说话人变化检测链接：https://arxiv.org/abs/2211.09381
* 与cs.SD语音【12】为同一篇
作者：Zhiyun Fan,Zhenlin Liang,Linhao Dong,Yi Liu,Shiyu Zhou,Meng Cai,Jun Zhang,Zejun Ma,Bo Xu机构：Institute of Automation, Chinese Academy of Sciences, China, School of Artificial Intelligence, University of Chinese Academy of Sciences, China, Bytedance AI LAB摘要：在诸如会议和会话的多讲话者场景中，通常需要语音处理系统来分割音频，然后转录每个分割。这两个阶段分别由说话人变化检测（SCD）和自动语音识别（ASR）解决。大多数现有的SCD系统仅依赖于说话人信息而忽略了语音内容的重要性。本文提出了一种同时考虑说话人差异和语音内容的SCD系统。通过连续积分触发机制将这两个线索转换为表征级表示，然后将其组合以检测表征声学边界上的说话人变化。我们在公开的真实会议记录数据集AISHELL-4上评估了我们的方法的性能。实验结果表明，该方法比竞争性的帧级基线系统的等覆盖率-纯度（ECP）提高了2.45%。此外，我们还证明了语音内容和说话人差异对SCD任务的重要性，以及在令牌声学边界上进行SCD与逐帧进行SCD相比的优势。摘要：In multi-talker scenarios such as meetings and conversations, speech processing systems are usually required to segment the audio and then transcribe each segmentation. These two stages are addressed separately by speaker change detection (SCD) and automatic speech recognition (ASR). Most previous SCD systems rely solely on speaker information and ignore the importance of speech content. In this paper, we propose a novel SCD system that considers both cues of speaker difference and speech content. These two cues are converted into token-level representations by the continuous integrate-and-fire (CIF) mechanism and then combined for detecting speaker changes on the token acoustic boundaries. We evaluate the performance of our approach on a public real-recorded meeting dataset, AISHELL-4. The experiment results show that our method outperforms a competitive frame-level baseline system by 2.45% equal coverage-purity (ECP). In addition, we demonstrate the importance of speech content and speaker difference to the SCD task, and the advantages of conducting SCD on the token acoustic boundaries compared with conducting SCD frame by frame.
【18】 Balanced Deep CCA for Bird Vocalization Detection标题：用于鸟类发声检测的平衡深度CCA链接：https://arxiv.org/abs/2211.09376
* 与cs.SD语音【13】为同一篇
作者：Sumit Kumar,B. Anshuman,Linus Ruettimann,Richard H. R. Hahnloser,Vipul Arora机构：Department of Electrical Engineering, Indian Institute of Technology, Kanpur, India., Institute of Neuroinformatics, University of Z¨urich and ETH Z¨urich, Z¨urich, Switzerland.摘要：当事件由两个不同的模态而不是仅一个模态捕获时，事件检测得到改善。但是，在多个模态上训练检测系统是具有挑战性的，特别是当存在大量未标记数据但有限量的标记数据时。我们开发了一种新颖的用于多模态数据的自监督学习技术，该技术学习同时记录的麦克风（声音）信号和加速度计（身体振动）信号之间的（隐藏）相关性。本研究的主要目标是在标记数据稀少且感兴趣的音频事件（鸣禽发声）稀疏时，学习与下游事件检测任务中的高性能相关的有用嵌入。我们的方法基于深度典型相关分析（DCCA），它遭受事件稀疏性。我们首先从标记数据中学习数据采样模型，并对它产生的输出应用DCCA，从而克服了正标记的稀疏性。与经典DCCA相比，该方法提高了无监督嵌入在下游有监督音频检测任务中的性能。由于数据标签经常是不平衡的，我们的方法可能在低资源场景中具有广泛的实用性。摘要：Event detection improves when events are captured by two different modalities rather than just one. But to train detection systems on multiple modalities is challenging, in particular when there is abundance of unlabelled data but limited amounts of labeled data. We develop a novel self-supervised learning technique for multi-modal data that learns (hidden) correlations between simultaneously recorded microphone (sound) signals and accelerometer (body vibration) signals. The key objective of this work is to learn useful embeddings associated with high performance in downstream event detection tasks when labeled data is scarce and the audio events of interest (songbird vocalizations) are sparse. We base our approach on deep canonical correlation analysis (DCCA) that suffers from event sparseness. We overcome the sparseness of positive labels by first learning a data sampling model from the labelled data and by applying DCCA on the output it produces. This method that we term balanced DCCA (b-DCCA) improves the performance of the unsupervised embeddings on the downstream supervised audio detection task compared to classsical DCCA. Because data labels are frequently imbalanced, our method might be of broad utility in low-resource scenarios.
【19】 Low-Resource Mongolian Speech Synthesis Based on Automatic Prosody Annotation标题：基于韵律自动标注的低资源蒙古语语音合成链接：https://arxiv.org/abs/2211.09365
* 与cs.SD语音【14】为同一篇
作者：Xin Yuan,Robin Feng,Mingming Ye机构：Ye[,−,−,−,], VXI, China备注：submitted to NCMMSC2022摘要：虽然基于深度学习的文本到语音（TTS）模型（如VTS）已经显示出了出色的效果，但它们通常需要大量高质量的〈text，audio〉对来训练，而收集这些数据的成本很高。迄今为止，世界上大多数语言仍然缺乏开发TTS系统所需的训练数据。针对低资源蒙古语语音合成面临的两个问题，提出了两种改进方法：a）鉴于缺乏高质量的数据对，难以对从语言特征到声学特征的映射问题进行建模。利用预先训练好的VITS模型和迁移学习方法进行了改进。2）针对标注信息量少的问题，提出了一种自动韵律标注方法，对文本及其对应语音的韵律信息进行标注，从而提高低资源蒙古语的自然度和可懂度。通过实证研究，本文提出的方法的N-MOS为4.195，I-MOS为4.228。摘要：While deep learning-based text-to-speech (TTS) models such as VITS have shown excellent results, they typically require a sizable set of high-qualitypairs to train, which is expensive to collect. So far, most languages in the world still lack the training data needed to develop TTS systems. This paper proposes two improvement methods for the two problems faced by low-resource Mongolian speech synthesis: a) In view of the lack of high-qualitypairs of data, it is difficult to model the mapping problem from linguistic features to acoustic features. Improvements are made using pre-trained VITS model and transfer learning methods. b) In view of the problem of less labeled information, this paper proposes to use an automatic prosodic annotation method to label the prosodic information of text and corresponding speech, thereby improving the naturalness and intelligibility of low-resource Mongolian language. Through empirical research, the N-MOS of the method proposed in this paper is 4.195, and the I-MOS is 4.228.
【20】 Privacy against Real-Time Speech Emotion Detection via Acoustic Adversarial Evasion of Machine Learning标题：基于机器学习的声学对抗性回避的隐私对抗实时语音情感检测链接：https://arxiv.org/abs/2211.09273
* 与cs.SD语音【15】为同一篇
作者：Brian Testa,Yi Xiao,Avery Gump,Asif Salekin机构： Syracuse University摘要：情绪监控是一个新兴的领域，具有广泛的隐私问题。这些担忧因配备多个传感器的物联网设备无处不在而加剧，这些设备可以支持这些监控用例。此处介绍的工作考虑了这样一个用例：使用与智能说话者相关的语音情感识别（SER）分类器。这项工作证明了在不损害智能扬声器的效用的情况下避开与智能扬声器绑定的黑盒SER分类器的能力。这种隐私问题是通过机器学习的对抗性规避来考虑的。我们的解决方案，通过遗传编程击败情感的声音识别（DARE-GP），使用遗传编程来生成非侵入性附加音频扰动（AAP）。通过限制这些AAP的进化，可以保护转录准确性，同时降低SER分类器性能。这些AAP的加性特性，以及以与话语和用户位置无关的方式为固定的用户集合生成这些AAP的方法，支持SER分类器的实时、真实世界规避。DARE-GP使用频谱特征，支持语音的情感内容，允许将AAP转移到以前看不见的黑盒SER分类器。此外，DARE-GP的性能优于最先进的SER规避技术，并对知识渊博的对手所采用的防御具有鲁棒性。本研究中的评估以针对两个现成的商用智能扬声器的声学评估达到高潮，其中单个AAP可以在超过70%的时间内避开黑盒分类器。最终评估将AAP回放部署在与唤醒词系统集成的小型系统（raspberry pi）上，以评估真实世界实时部署的功效，其中DARE-GP通过智能扬声器的唤醒词自动调用。摘要：Emotional Surveillance is an emerging area with wide-reaching privacy concerns. These concerns are exacerbated by ubiquitous IoT devices with multiple sensors that can support these surveillance use cases. The work presented here considers one such use case: the use of a speech emotion recognition (SER) classifier tied to a smart speaker. This work demonstrates the ability to evade black-box SER classifiers tied to a smart speaker without compromising the utility of the smart speaker. This privacy concern is considered through the lens of adversarial evasion of machine learning. Our solution, Defeating Acoustic Recognition of Emotion via Genetic Programming (DARE-GP), uses genetic programming to generate non-invasive additive audio perturbations (AAPs). By constraining the evolution of these AAPs, transcription accuracy can be protected while simultaneously degrading SER classifier performance. The additive nature of these AAPs, along with an approach that generates these AAPs for a fixed set of users in an utterance and user location-independent manner, supports real-time, real-world evasion of SER classifiers. DARE-GP's use of spectral features, which underlay the emotional content of speech, allows the transferability of AAPs to previously unseen black-box SER classifiers. Further, DARE-GP outperforms state-of-the-art SER evasion techniques and is robust against defenses employed by a knowledgeable adversary. The evaluations in this work culminate with acoustic evaluations against two off-the-shelf commercial smart speakers, where a single AAP could evade a black box classifier over 70% of the time. The final evaluation deployed AAP playback on a small-form-factor system (raspberry pi) integrated with a wake-word system to evaluate the efficacy of a real-world, real-time deployment where DARE-GP is automatically invoked with the smart speaker's wake word.
【21】 A Review of Intelligent Music Generation Systems标题：智能音乐生成系统综述链接：https://arxiv.org/abs/2211.09124
* 与cs.SD语音【16】为同一篇
作者：Ziyi Zhao,Hanwei Liu,Song Li,Junwei Pang,Maoqing Zhang,Yi Qin,Lei Wang,Qidi Wu机构：College of Electric and Information Engineering, Tongji University, Cao’an Street, Shanghai, China., Department of Music Engineering, Shanghai Conservatory of Music, Fenyang street备注：Overall 24 Pages, 11 Figures, 2 Tables, 96 References items摘要：智能音乐生成是计算机创作领域中最热门的分支之一，它可以降低非专业人员的创作门槛，提高音乐创作的效率。在过去的五年中，基于算法的自动音乐生成的质量显著提高，这是由于使用现代生成算法来学习基于规则约束或音乐语料库的音乐片段中隐含的模式，从而生成各种风格的音乐样本。现有的一些文献综述缺乏生成模型的系统基准，并且在其视角上是传统的和保守的，导致该领域未来发展的愿景与当前快速的科学进步没有深度融合。本文对近年来智能音乐生成技术进行了全面的综述和分析，并进行了批判性的讨论，明确了它们各自的特点，并以一个总表的形式给出了它们。首先介绍了音乐作为信息流的编码方式和相关数据集，然后比较了不同类型的生成算法，总结了它们的优缺点，并讨论了现有的评价方法。最后研究了人工智能在作曲方面的发展，特别是通过比较东西方音乐生成技术的不同特点，分析了该领域的发展前景。摘要：Intelligent music generation, one of the most popular subfields of computer creativity, can lower the creative threshold for non-specialists and increase the efficiency of music creation. In the last five years, the quality of algorithm-based automatic music generation has increased significantly, motivated by the use of modern generative algorithms to learn the patterns implicit within a piece of music based on rule constraints or a musical corpus, thus generating music samples in various styles. Some of the available literature reviews lack a systematic benchmark of generative models and are traditional and conservative in their perspective, resulting in a vision of the future development of the field that is not deeply integrated with the current rapid scientific progress. In this paper, we conduct a comprehensive survey and analysis of recent intelligent music generation techniques,provide a critical discussion, explicitly identify their respective characteristics, and present them in a general table. We first introduce how music as a stream of information is encoded and the relevant datasets, then compare different types of generation algorithms, summarize their strengths and weaknesses, and discuss existing methods for evaluation. Finally, the development of artificial intelligence in composition is studied, especially by comparing the different characteristics of music generation techniques in the East and West and analyzing the development prospects in this field.
机器翻译，仅供参考
每日语音速递