今日论文合集：cs.SD语音9篇，eess.AS音频处理11篇。【1】SpokesBiz -- an Open Corpus of Conversational Polish
标题：SpokeBiz--一个开放的波兰对话语料库链接：https://arxiv.org/abs/2312.12364作者：Piotr Pęzik,Sylwia Karasińska,Anna Cichosz,Łukasz Jałowiecki,Konrad Kaczyński,Małgorzata Krawentek,Karolina Walkusz,Paweł Wilk,Mariusz Kleć,Krzysztof Szklanny,Szymon Marszałkowski摘要：本文宣布提前发布的SpokesBiz，一个免费提供的语料库会话波兰语内开发的ESPIN-BIZ项目，包括超过650小时的录音。转录的录音已作了日记，并对标点符号和大小写作了人工注释。我们概述了语料库的一般结构和内容，展示了在语言学研究，评估和改进自动语音识别（ASR）系统中的选定应用摘要：This paper announces the early release of SpokesBiz, a freely available corpus of conversational Polish developed within the CLARIN-BIZ project and comprising over 650 hours of recordings. The transcribed recordings have been diarized and manually annotated for punctuation and casing. We outline the general structure and content of the corpus, showcasing selected applications in linguistic research, evaluation and improvement of automatic speech recognition (ASR) systems【2】 Automated speech audiometry: Can it work using open-source pre-trained Kaldi-NL automatic speech recognition?标题：自动语音测听：它可以使用开源的预训练Kaldi-NL自动语音识别吗？链接：https://arxiv.org/abs/2312.12269作者：Gloria Araiza-Illan,Luke Meyer,Khiet P. Truong,Deniz Baskent备注：25 pages (double spaced), 5 figures, 3 tables, 54 references摘要：一种实用的言语测听工具是噪声数字（DIN）测试，用于不同年龄和听力状态人群的听力筛查。测试通常由人类监督者（例如，临床医生），其对收听者说出的响应进行评分，或者在线，其中软件对收听者输入的响应进行评分。该测试有24个数字三联体呈现在自适应阶梯程序中，从而产生语音接收阈值（SRT）。我们提出了一种替代的自动DIN测试设置，可以评估口头响应，同时进行没有人类监督员，使用开源的自动语音识别工具包，Kaldi-NL。30名自报听力正常的荷兰成年人（19-64岁）完成了一项DIN+Kaldi-NL测试。记录他们的口头回答，并用于评估Kaldi-NL解码的回答的转录本。研究1通过其单词错误率（WER）评估了Kaldi-NL的表现，与口头回答中存在的数字总数相比，转录本中仅发现数字的总解码错误百分比。参与者的平均WER为5.0%（范围0 - 48%，SD = 8.8%），平均解码错误为三个三联体。研究2分析了Kaldi-NL解码错误的三胞胎对DIN测试输出（SRT）的影响，使用自举模拟。先前的研究表明，0.70 dB是听力正常成年人的典型受试者内SRT变异性。研究2表明，多达四个解码错误的三联体在此范围内产生SRT变化，这表明我们提出的设置对于临床应用是可行的。摘要：A practical speech audiometry tool is the digits-in-noise (DIN) test for hearing screening of populations of varying ages and hearing status. The test is usually conducted by a human supervisor (e.g., clinician), who scores the responses spoken by the listener, or online, where a software scores the responses entered by the listener. The test has 24 digit-triplets presented in an adaptive staircase procedure, resulting in a speech reception threshold (SRT). We propose an alternative automated DIN test setup that can evaluate spoken responses whilst conducted without a human supervisor, using the open-source automatic speech recognition toolkit, Kaldi-NL. Thirty self-reported normal-hearing Dutch adults (19-64 years) completed one DIN+Kaldi-NL test. Their spoken responses were recorded, and used for evaluating the transcript of decoded responses by Kaldi-NL. Study 1 evaluated the Kaldi-NL performance through its word error rate (WER), percentage of summed decoding errors regarding only digits found in the transcript compared to the total number of digits present in the spoken responses. Average WER across participants was 5.0% (range 0 - 48%, SD = 8.8%), with average decoding errors in three triplets per participant. Study 2 analysed the effect that triplets with decoding errors from Kaldi-NL had on the DIN test output (SRT), using bootstrapping simulations. Previous research indicated 0.70 dB as the typical within-subject SRT variability for normal-hearing adults. Study 2 showed that up to four triplets with decoding errors produce SRT variations within this range, suggesting that our proposed setup could be feasible for clinical applications.【3】 StyleSpeech: Self-supervised Style Enhancing with VQ-VAE-based Pre-training for Expressive Audiobook Speech Synthesis标题：StyleSpeech：基于VQ-VAE预训练的表达性有声读物语音合成的自监督风格增强链接：https://arxiv.org/abs/2312.12181作者：Xueyuan Chen,Xi Wang,Shaofei Zhang,Lei He,Zhiyong Wu,Xixin Wu,Helen Meng备注：Accepted to ICASSP 2024摘要：有声读物合成语音的表达质量受到训练数据中普遍的模型结构和不平衡的风格分布的限制。为了解决这些问题，在本文中，我们提出了一种自监督的风格增强方法与VQ-VAE的基础上的预训练表达有声读物语音合成。首先，使用大量未标记的纯文本数据预训练文本样式编码器。其次，基于VQ-VAE的声谱图风格提取器以自监督的方式进行预训练，具有大量的音频数据，覆盖复杂的风格变化。在风格抽取器的指导下，设计了一种具有两条编码-解码路径的结构，分别对语音和高层风格表达进行建模。客观和主观的评价表明，我们提出的方法可以有效地提高在有声读物合成的合成语音的自然度和表现力，特别是对于角色和域外的情况。摘要：The expressive quality of synthesized speech for audiobooks is limited by generalized model architecture and unbalanced style distribution in the training data. To address these issues, in this paper, we propose a self-supervised style enhancing method with VQ-VAE-based pre-training for expressive audiobook speech synthesis. Firstly, a text style encoder is pre-trained with a large amount of unlabeled text-only data. Secondly, a spectrogram style extractor based on VQ-VAE is pre-trained in a self-supervised manner, with plenty of audio data that covers complex style variations. Then a novel architecture with two encoder-decoder paths is specially designed to model the pronunciation and high-level style expressiveness respectively, with the guidance of the style extractor. Both objective and subjective evaluations demonstrate that our proposed method can effectively improve the naturalness and expressiveness of the synthesized speech in audiobook synthesis especially for the role and out-of-domain scenarios.【4】 Noise robust distillation of self-supervised speech models via correlation metrics标题：基于相关度量的自监督语音模型的噪声稳健提取链接：https://arxiv.org/abs/2312.12153作者：Fabian Ritter-Gutierrez,Kuan-Po Huang,Dianwen Ng,Jeremy H. M. Wong,Hung-yi Lee,Eng Siong Chng,Nancy F. Chen备注：6 pages摘要：与大的语音基础模型相比，小的蒸馏模型表现出降低的噪声鲁棒性。学生的鲁棒性可以通过在预训练期间在输入处引入噪声来提高。尽管如此，使用标准蒸馏损失仍然会导致学生的表现下降。因此，本文提出了通过相关度量的蒸馏来提高学生的鲁棒性。教师行为是通过最大化教师和学生对同一性的表征之间的互相关矩阵来学习的。噪声鲁棒性通过学生的自相关最小化来鼓励。所提出的方法是不可知的教师模型，并始终优于以前的方法。这项工作还提出了一种启发式的自动权衡两个相关项的重要性。实验表明，在SUPERB Challenge上，意图分类、关键词识别和自动语音识别任务的干净和噪声泛化能力始终更好。摘要：Compared to large speech foundation models, small distilled models exhibit degraded noise robustness. The student's robustness can be improved by introducing noise at the inputs during pre-training. Despite this, using the standard distillation loss still yields a student with degraded performance. Thus, this paper proposes improving student robustness via distillation with correlation metrics. Teacher behavior is learned by maximizing the teacher and student cross-correlation matrix between their representations towards identity. Noise robustness is encouraged via the student's self-correlation minimization. The proposed method is agnostic of the teacher model and consistently outperforms the previous approach. This work also proposes an heuristic to weigh the importance of the two correlation terms automatically. Experiments show consistently better clean and noise generalization on Intent Classification, Keyword Spotting, and Automatic Speech Recognition tasks on SUPERB Challenge.【5】 Ms-senet: Enhancing Speech Emotion Recognition Through Multi-scale Feature Fusion With Squeeze-and-excitation Blocks标题：MS-SENET：基于压缩激励块的多尺度特征融合增强语音情感识别链接：https://arxiv.org/abs/2312.11974作者：Mengbo Li,Yulun Wu,Dichucheng Li,Yuanzhong Zheng,Yaoxuan Wang,Haojun Fei摘要：语音情感识别（SER）已成为人机交互领域的研究热点。时空特征在SER中起着至关重要的作用，但目前的研究缺乏全面的时空特征学习。本文的重点是通过提出一种新的方法来解决这一差距。在本文中，我们采用卷积神经网络（CNN）与不同的内核大小的空间和时间特征提取。此外，我们引入挤压和激励（SE）模块来捕获和融合多尺度特征，促进有效的信息融合，以提高情感识别和更深入地了解语音情感的时间演变。此外，我们采用跳跃连接和空间丢弃（SD）层来防止过拟合并增加模型的深度。我们的方法优于以前的最先进的方法，在六个基准SER数据集上，平均UAR和WAR分别提高了1.62%和1.32%。进一步的实验表明，我们的方法可以充分提取时空特征在低资源条件下。摘要：Speech Emotion Recognition (SER) has become a growing focus of research in human-computer interaction. Spatiotemporal features play a crucial role in SER, yet current research lacks comprehensive spatiotemporal feature learning. This paper focuses on addressing this gap by proposing a novel approach. In this paper, we employ Convolutional Neural Network (CNN) with varying kernel sizes for spatial and temporal feature extraction. Additionally, we introduce Squeeze-and-Excitation (SE) modules to capture and fuse multi-scale features, facilitating effective information fusion for improved emotion recognition and a deeper understanding of the temporal evolution of speech emotion. Moreover, we employ skip connections and Spatial Dropout (SD) layers to prevent overfitting and increase the model's depth. Our method outperforms the previous state-of-the-art method, achieving an average UAR and WAR improvement of 1.62% and 1.32%, respectively, across six benchmark SER datasets. Further experiments demonstrated that our method can fully extract spatiotemporal features in low-resource conditions.【6】 Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling标题：基于异构图的上下文建模用于会话语音合成的情感渲染链接：https://arxiv.org/abs/2312.11947作者：Rui Liu,Yifan Hu,Yi Ren,Xiang Yin,Haizhou Li备注：9 pages, 4 figures, Accepted by AAAI'2024, Code and audio samples: this https URL摘要：会话语音合成（CSS）的目标是在会话环境中准确地表达具有适当韵律和情感变化的话语。虽然认识到CSS任务的重要性，但由于情感会话数据集的稀缺和有状态情感建模的困难，先前的研究没有深入研究情感表达问题。本文提出了一种新的情感CSS模型ECSS，它包括两个主要部分：1）为了增强情感理解，我们引入了一种基于异构图的情感上下文建模机制，以多源对话历史为输入，对对话上下文进行建模，并从上下文中学习情感线索; 2）在情感渲染方面，采用了基于对比学习的情感渲染模块来推断目标话语的准确情感风格。为了解决数据稀缺的问题，我们精心创建了类别和强度方面的情感标签，并在现有的会话数据集（DailyTalk）上注释了额外的情感信息。客观和主观的评估表明，我们的模型优于基线模型在理解和渲染情绪。这些评估也强调了全面的情感注释的重要性。代码和音频示例可以在https://github.com/walker-hyf/ECSS上找到。摘要：Conversational Speech Synthesis (CSS) aims to accurately express an utterance with the appropriate prosody and emotional inflection within a conversational setting. While recognising the significance of CSS task, the prior studies have not thoroughly investigated the emotional expressiveness problems due to the scarcity of emotional conversational datasets and the difficulty of stateful emotion modeling. In this paper, we propose a novel emotional CSS model, termed ECSS, that includes two main components: 1) to enhance emotion understanding, we introduce a heterogeneous graph-based emotional context modeling mechanism, which takes the multi-source dialogue history as input to model the dialogue context and learn the emotion cues from the context; 2) to achieve emotion rendering, we employ a contrastive learning-based emotion renderer module to infer the accurate emotion style for the target utterance. To address the issue of data scarcity, we meticulously create emotional labels in terms of category and intensity, and annotate additional emotional information on the existing conversational dataset (DailyTalk). Both objective and subjective evaluations suggest that our model outperforms the baseline models in understanding and rendering emotions. These evaluations also underscore the importance of comprehensive emotional annotations. Code and audio samples can be found at: https://github.com/walker-hyf/ECSS.【7】 MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separation标题：MOSSFormer2：结合Transformer和无RNN递归网络的增强时域单声道语音分离链接：https://arxiv.org/abs/2312.11825作者：Shengkui Zhao,Yukun Ma,Chongjia Ni,Chong Zhang,Hao Wang,Trung Hieu Nguyen,Kun Zhou,Jiaqi Yip,Dianwen Ng,Bin Ma备注：5 pages, 3 figures, accepted by ICASSP 2024摘要：我们先前提出的MossFormer在单声道语音分离中取得了令人满意的性能。然而，它主要采用基于自我注意力的MossFormer模块，该模块倾向于强调更长范围、更粗尺度的依赖关系，在有效建模更细尺度的循环模式方面存在不足。在本文中，我们介绍了一种新的混合模型，它提供了建模的能力，通过将一个经常性的模块集成到MossFormer框架的长距离，粗规模的依赖关系和细规模的经常性模式。我们提出了一个基于前馈顺序记忆网络（FSMN）的递归模块，而不是应用使用传统递归连接的递归神经网络（RNN），由于能够在不使用递归连接的情况下捕获递归模式，因此被认为是“无RNN”递归网络。我们的递归模块主要包括通过使用门控卷积单元（GCU）和密集连接增强的膨胀FSMN块。此外，还增加了瓶颈层和输出层，用于控制信息流。循环模块依赖于线性投影和卷积，以实现整个序列的无缝并行处理。集成的MossFormer 2混合模型展示了对MossFormer的显着增强，并超越了WSJ 0 -2/3 mix，Libri 2 Mix和WHAM中的其他最先进的方法！/哇！基准。摘要：Our previously proposed MossFormer has achieved promising performance in monaural speech separation. However, it predominantly adopts a self-attention-based MossFormer module, which tends to emphasize longer-range, coarser-scale dependencies, with a deficiency in effectively modelling finer-scale recurrent patterns. In this paper, we introduce a novel hybrid model that provides the capabilities to model both long-range, coarse-scale dependencies and fine-scale recurrent patterns by integrating a recurrent module into the MossFormer framework. Instead of applying the recurrent neural networks (RNNs) that use traditional recurrent connections, we present a recurrent module based on a feedforward sequential memory network (FSMN), which is considered "RNN-free" recurrent network due to the ability to capture recurrent patterns without using recurrent connections. Our recurrent module mainly comprises an enhanced dilated FSMN block by using gated convolutional units (GCU) and dense connections. In addition, a bottleneck layer and an output layer are also added for controlling information flow. The recurrent module relies on linear projections and convolutions for seamless, parallel processing of the entire sequence. The integrated MossFormer2 hybrid model demonstrates remarkable enhancements over MossFormer and surpasses other state-of-the-art methods in WSJ0-2/3mix, Libri2Mix, and WHAM!/WHAMR! benchmarks.【8】 A review-based study on different Text-to-Speech technologies标题：不同文语转换技术的综述研究链接：https://arxiv.org/abs/2312.11563作者：Md. Jalal Uddin Chowdhury,Ashab Hussan备注：4 pages摘要：本文对各种文本语音转换（TTS）技术进行了全面的综述性研究。TTS技术是人机交互的一个重要方面，使机器能够将书面文本转换为可听语音。本文探讨了不同的TTS技术，包括拼接TTS，共振峰合成TTS，统计参数TTS。该研究的重点是比较这些技术的优点和局限性，在他们的自然语音，系统的复杂程度，以及他们对不同应用程序的适用性。此外，本文还探讨了语音合成技术的最新进展，包括神经语音合成和混合语音合成。这项研究的结果将为研究人员，开发人员和希望了解不同TTS技术及其适用于特定应用的用户提供有价值的见解。摘要：This research paper presents a comprehensive review-based study on various Text-to-Speech (TTS) technologies. TTS technology is an important aspect of human-computer interaction, enabling machines to convert written text into audible speech. The paper examines the different TTS technologies available, including concatenative TTS, formant synthesis TTS, and statistical parametric TTS. The study focuses on comparing the advantages and limitations of these technologies in terms of their naturalness of voice, the level of complexity of the system, and their suitability for different applications. In addition, the paper explores the latest advancements in TTS technology, including neural TTS and hybrid TTS. The findings of this research will provide valuable insights for researchers, developers, and users who want to understand the different TTS technologies and their suitability for specific applications.【9】 Evaluating Speech-in-Speech Perception via a Humanoid Robot标题：通过类人机器人评估言语中的言语感知链接：https://arxiv.org/abs/2312.12262作者：Luke Meyer,Gloria Araiza-Illan,Laura Rachman,Etienne Gaudrain,Deniz Baskent备注：25 pages (single spaced), 6 figures (at the end of the manuscript), 88 references, under revision with Frontiers摘要：被背景说话者掩盖的言语感知的潜在机制是一种常见的日常听力条件，通常使用各种冗长的心理物理测试来研究。社交代理的存在，例如交互式人形NAO机器人，可以帮助保持参与和注意力。然而，这种机器人可能具有有限的声音质量或处理速度。作为使用NAO进行言语感知的心理物理测试的第一步，我们比较了听力正常的年轻人在使用标准计算机界面时的表现，以及使用NAO机器人引入测试并呈现所有相应刺激时的表现。目标句子呈现颜色和数字关键字在竞争的掩蔽语音存在下，在不同的目标到掩蔽比例。句子是由同一个说话人产生的，但是目标和掩蔽者之间的声音差异是使用语音合成方法引入的。为了评估测试性能，比较了计算机和NAO设置之间的语音清晰度和数据收集持续时间。使用对机器人的消极态度量表（NARS）和行为线索（反向通道）的量化来评估人机交互。语音清晰度的结果表明，计算机和NAO设置之间的功能相似。使用NAO时，数据收集持续时间较长。NARS的结果显示，参与者在与NAO互动之前对机器人有更积极的态度。当使用NAO时，存在更多的正反向通道，这表明与计算机相比，机器人的参与度更高。总体而言，这项研究提出了潜在的NAO presentingspeech材料和收集心理物理测量的语音在语音感知。摘要：Underlying mechanisms of speech perception masked by background speakers, a common daily listening condition, are often investigated using various and lengthy psychophysical tests. The presence of a social agent, such as an interactive humanoid NAO robot, may help maintain engagement and attention. However, such robots potentially have limited sound quality or processing speed. As a first step towards the use of NAO in psychophysical testing of speech-in-speech perception, we compared normal-hearing young adults' performance when using the standard computer interface to that when using a NAO robot to introduce the test and present all corresponding stimuli. Target sentences were presented with colour and number keywords in the presence of competing masker speech at varying target-to-masker ratios. Sentences were produced by the same speaker, but voice differences between the target and masker were introduced using speech synthesis methods. To assess test performance, speech intelligibility and data collection duration were compared between the computer and NAO setups. Human-robot interaction was assessed using the Negative Attitude Towards Robot Scale (NARS) and quantification of behavioural cues (backchannels). Speech intelligibility results showed functional similarity between the computer and NAO setups. Data collection durations were longer when using NAO. NARS results showed participants had a more positive attitude toward robots prior to their interaction with NAO. The presence of more positive backchannels when using NAO suggest higher engagement with the robot in comparison to the computer. Overall, the study presents the potential of the NAO for presentingspeech materials and collecting psychophysical measurements for speech-in-speech perception.【1】 On real-time multi-stage speech enhancement systems标题：实时多级语音增强系统研究链接：https://arxiv.org/abs/2312.12415作者：Lingjun Meng,Jozef Coldenhoff,Paul Kendrick,Tijana Stojkovic,Andrew Harper,Kiril Ratmanski,Milos Cernak备注：To appear at ICASSP 2024摘要：最近，多级系统在基于深度学习的语音增强方法中脱颖而出。然而，这些系统总是在复杂性高，需要数以百万计的参数和强大的计算资源，这限制了它们的应用在低功耗设备的实时处理。此外，各种影响因素对多级系统成功的贡献仍然不清楚，这对减小这些系统的尺寸提出了挑战。在本文中，我们广泛研究了一个轻量级的两阶段网络，只有560k的总参数。该方法由第一阶段的Mel尺度震级掩蔽模型和第二阶段的复谱映射模型组成。我们首先提供了一个关于Mel尺度掩蔽模型的增益功率因子、后置滤波器和训练标签的角色的综合视图。然后，我们探讨了两阶段网络的几种训练方案，并对两阶段网络的优越性提供了一些见解。我们表明，由最佳方案训练的两阶段网络的性能与大四倍的开源模型DeepFilterNet2相似。摘要：Recently, multi-stage systems have stood out among deep learning-based speech enhancement methods. However, these systems are always high in complexity, requiring millions of parameters and powerful computational resources, which limits their application for real-time processing in low-power devices. Besides, the contribution of various influencing factors to the success of multi-stage systems remains unclear, which presents challenges to reduce the size of these systems. In this paper, we extensively investigate a lightweight two-stage network with only 560k total parameters. It consists of a Mel-scale magnitude masking model in the first stage and a complex spectrum mapping model in the second stage. We first provide a consolidated view of the roles of gain power factor, post-filter, and training labels for the Mel-scale masking model. Then, we explore several training schemes for the two-stage network and provide some insights into the superiority of the two-stage network. We show that the proposed two-stage network trained by an optimal scheme achieves a performance similar to a four times larger open source model DeepFilterNet2.【2】 Evaluating Speech-in-Speech Perception via a Humanoid Robot标题：通过类人机器人评估言语中的言语感知链接：https://arxiv.org/abs/2312.12262作者：Luke Meyer,Gloria Araiza-Illan,Laura Rachman,Etienne Gaudrain,Deniz Baskent备注：25 pages (single spaced), 6 figures (at the end of the manuscript), 88 references, under revision with Frontiers摘要：被背景说话者掩盖的言语感知的潜在机制是一种常见的日常听力条件，通常使用各种冗长的心理物理测试来研究。社交代理的存在，例如交互式人形NAO机器人，可以帮助保持参与和注意力。然而，这种机器人可能具有有限的声音质量或处理速度。作为使用NAO进行言语感知的心理物理测试的第一步，我们比较了听力正常的年轻人在使用标准计算机界面时的表现，以及使用NAO机器人引入测试并呈现所有相应刺激时的表现。目标句子呈现颜色和数字关键字在竞争的掩蔽语音存在下，在不同的目标到掩蔽比例。句子是由同一个说话人产生的，但是目标和掩蔽者之间的声音差异是使用语音合成方法引入的。为了评估测试性能，比较了计算机和NAO设置之间的语音清晰度和数据收集持续时间。使用对机器人的消极态度量表（NARS）和行为线索（反向通道）的量化来评估人机交互。语音清晰度的结果表明，计算机和NAO设置之间的功能相似。使用NAO时，数据收集持续时间较长。NARS的结果显示，参与者在与NAO互动之前对机器人有更积极的态度。当使用NAO时，存在更多的正反向通道，这表明与计算机相比，机器人的参与度更高。总体而言，这项研究提出了潜在的NAO presentingspeech材料和收集心理物理测量的语音在语音感知。摘要：Underlying mechanisms of speech perception masked by background speakers, a common daily listening condition, are often investigated using various and lengthy psychophysical tests. The presence of a social agent, such as an interactive humanoid NAO robot, may help maintain engagement and attention. However, such robots potentially have limited sound quality or processing speed. As a first step towards the use of NAO in psychophysical testing of speech-in-speech perception, we compared normal-hearing young adults' performance when using the standard computer interface to that when using a NAO robot to introduce the test and present all corresponding stimuli. Target sentences were presented with colour and number keywords in the presence of competing masker speech at varying target-to-masker ratios. Sentences were produced by the same speaker, but voice differences between the target and masker were introduced using speech synthesis methods. To assess test performance, speech intelligibility and data collection duration were compared between the computer and NAO setups. Human-robot interaction was assessed using the Negative Attitude Towards Robot Scale (NARS) and quantification of behavioural cues (backchannels). Speech intelligibility results showed functional similarity between the computer and NAO setups. Data collection durations were longer when using NAO. NARS results showed participants had a more positive attitude toward robots prior to their interaction with NAO. The presence of more positive backchannels when using NAO suggest higher engagement with the robot in comparison to the computer. Overall, the study presents the potential of the NAO for presentingspeech materials and collecting psychophysical measurements for speech-in-speech perception.【3】 SpokesBiz -- an Open Corpus of Conversational Polish标题：SpokeBiz--一个开放的波兰对话语料库链接：https://arxiv.org/abs/2312.12364作者：Piotr Pęzik,Sylwia Karasińska,Anna Cichosz,Łukasz Jałowiecki,Konrad Kaczyński,Małgorzata Krawentek,Karolina Walkusz,Paweł Wilk,Mariusz Kleć,Krzysztof Szklanny,Szymon Marszałkowski摘要：本文宣布提前发布的SpokesBiz，一个免费提供的语料库会话波兰语内开发的ESPIN-BIZ项目，包括超过650小时的录音。转录的录音已作了日记，并对标点符号和大小写作了人工注释。我们概述了语料库的一般结构和内容，展示了在语言学研究，评估和改进自动语音识别（ASR）系统中的选定应用摘要：This paper announces the early release of SpokesBiz, a freely available corpus of conversational Polish developed within the CLARIN-BIZ project and comprising over 650 hours of recordings. The transcribed recordings have been diarized and manually annotated for punctuation and casing. We outline the general structure and content of the corpus, showcasing selected applications in linguistic research, evaluation and improvement of automatic speech recognition (ASR) systems【4】 Automated speech audiometry: Can it work using open-source pre-trained Kaldi-NL automatic speech recognition?标题：自动语音测听：它可以使用开源的预先训练的Kaldi-NL自动语音识别吗？链接：https://arxiv.org/abs/2312.12269作者：Gloria Araiza-Illan,Luke Meyer,Khiet P. Truong,Deniz Baskent备注：25 pages (double spaced), 5 figures, 3 tables, 54 references摘要：一种实用的言语测听工具是噪声数字（DIN）测试，用于不同年龄和听力状态人群的听力筛查。测试通常由人类监督者（例如，临床医生），其对收听者说出的响应进行评分，或者在线，其中软件对收听者输入的响应进行评分。该测试有24个数字三联体呈现在自适应阶梯程序中，从而产生语音接收阈值（SRT）。我们提出了一种替代的自动DIN测试设置，可以评估口头响应，同时进行没有人类监督员，使用开源的自动语音识别工具包，Kaldi-NL。30名自报听力正常的荷兰成年人（19-64岁）完成了一项DIN+Kaldi-NL测试。记录他们的口头回答，并用于评估Kaldi-NL解码的回答的转录本。研究1通过其单词错误率（WER）评估了Kaldi-NL的表现，与口头回答中存在的数字总数相比，转录本中仅发现数字的总解码错误百分比。参与者的平均WER为5.0%（范围0 - 48%，SD = 8.8%），平均解码错误为三个三联体。研究2分析了Kaldi-NL解码错误的三胞胎对DIN测试输出（SRT）的影响，使用自举模拟。先前的研究表明，0.70 dB是听力正常成年人的典型受试者内SRT变异性。研究2表明，多达四个解码错误的三联体在此范围内产生SRT变化，这表明我们提出的设置对于临床应用是可行的。摘要：A practical speech audiometry tool is the digits-in-noise (DIN) test for hearing screening of populations of varying ages and hearing status. The test is usually conducted by a human supervisor (e.g., clinician), who scores the responses spoken by the listener, or online, where a software scores the responses entered by the listener. The test has 24 digit-triplets presented in an adaptive staircase procedure, resulting in a speech reception threshold (SRT). We propose an alternative automated DIN test setup that can evaluate spoken responses whilst conducted without a human supervisor, using the open-source automatic speech recognition toolkit, Kaldi-NL. Thirty self-reported normal-hearing Dutch adults (19-64 years) completed one DIN+Kaldi-NL test. Their spoken responses were recorded, and used for evaluating the transcript of decoded responses by Kaldi-NL. Study 1 evaluated the Kaldi-NL performance through its word error rate (WER), percentage of summed decoding errors regarding only digits found in the transcript compared to the total number of digits present in the spoken responses. Average WER across participants was 5.0% (range 0 - 48%, SD = 8.8%), with average decoding errors in three triplets per participant. Study 2 analysed the effect that triplets with decoding errors from Kaldi-NL had on the DIN test output (SRT), using bootstrapping simulations. Previous research indicated 0.70 dB as the typical within-subject SRT variability for normal-hearing adults. Study 2 showed that up to four triplets with decoding errors produce SRT variations within this range, suggesting that our proposed setup could be feasible for clinical applications.【5】 StyleSpeech: Self-supervised Style Enhancing with VQ-VAE-based Pre-training for Expressive Audiobook Speech Synthesis标题：StyleSpeech：基于VQ-VAE预训练的表达性有声读物语音合成的自监督风格增强链接：https://arxiv.org/abs/2312.12181作者：Xueyuan Chen,Xi Wang,Shaofei Zhang,Lei He,Zhiyong Wu,Xixin Wu,Helen Meng备注：Accepted to ICASSP 2024摘要：有声读物合成语音的表达质量受到训练数据中普遍的模型结构和不平衡的风格分布的限制。为了解决这些问题，在本文中，我们提出了一种自监督的风格增强方法与VQ-VAE的基础上的预训练表达有声读物语音合成。首先，使用大量未标记的纯文本数据预训练文本样式编码器。其次，基于VQ-VAE的声谱图风格提取器以自监督的方式进行预训练，具有大量的音频数据，覆盖复杂的风格变化。在风格抽取器的指导下，设计了一种具有两条编码-解码路径的结构，分别对语音和高层风格表达进行建模。客观和主观的评价表明，我们提出的方法可以有效地提高在有声读物合成的合成语音的自然度和表现力，特别是对于角色和域外的情况。摘要：The expressive quality of synthesized speech for audiobooks is limited by generalized model architecture and unbalanced style distribution in the training data. To address these issues, in this paper, we propose a self-supervised style enhancing method with VQ-VAE-based pre-training for expressive audiobook speech synthesis. Firstly, a text style encoder is pre-trained with a large amount of unlabeled text-only data. Secondly, a spectrogram style extractor based on VQ-VAE is pre-trained in a self-supervised manner, with plenty of audio data that covers complex style variations. Then a novel architecture with two encoder-decoder paths is specially designed to model the pronunciation and high-level style expressiveness respectively, with the guidance of the style extractor. Both objective and subjective evaluations demonstrate that our proposed method can effectively improve the naturalness and expressiveness of the synthesized speech in audiobook synthesis especially for the role and out-of-domain scenarios.【6】 Noise robust distillation of self-supervised speech models via correlation metrics标题：基于相关度量的自监督语音模型的噪声稳健提取链接：https://arxiv.org/abs/2312.12153作者：Fabian Ritter-Gutierrez,Kuan-Po Huang,Dianwen Ng,Jeremy H. M. Wong,Hung-yi Lee,Eng Siong Chng,Nancy F. Chen备注：6 pages摘要：与大的语音基础模型相比，小的蒸馏模型表现出降低的噪声鲁棒性。学生的鲁棒性可以通过在预训练期间在输入处引入噪声来提高。尽管如此，使用标准蒸馏损失仍然会导致学生的表现下降。因此，本文提出了通过相关度量的蒸馏来提高学生的鲁棒性。教师行为是通过最大化教师和学生对同一性的表征之间的互相关矩阵来学习的。噪声鲁棒性通过学生的自相关最小化来鼓励。所提出的方法是不可知的教师模型，并始终优于以前的方法。这项工作还提出了一种启发式的自动权衡两个相关项的重要性。实验表明，在SUPERB Challenge上，意图分类、关键词识别和自动语音识别任务的干净和噪声泛化能力始终更好。摘要：Compared to large speech foundation models, small distilled models exhibit degraded noise robustness. The student's robustness can be improved by introducing noise at the inputs during pre-training. Despite this, using the standard distillation loss still yields a student with degraded performance. Thus, this paper proposes improving student robustness via distillation with correlation metrics. Teacher behavior is learned by maximizing the teacher and student cross-correlation matrix between their representations towards identity. Noise robustness is encouraged via the student's self-correlation minimization. The proposed method is agnostic of the teacher model and consistently outperforms the previous approach. This work also proposes an heuristic to weigh the importance of the two correlation terms automatically. Experiments show consistently better clean and noise generalization on Intent Classification, Keyword Spotting, and Automatic Speech Recognition tasks on SUPERB Challenge.【7】 Ms-senet: Enhancing Speech Emotion Recognition Through Multi-scale Feature Fusion With Squeeze-and-excitation Blocks标题：MS-SENET：基于压缩激励块的多尺度特征融合增强语音情感识别链接：https://arxiv.org/abs/2312.11974作者：Mengbo Li,Yulun Wu,Dichucheng Li,Yuanzhong Zheng,Yaoxuan Wang,Haojun Fei摘要：语音情感识别（SER）已成为人机交互领域的研究热点。时空特征在SER中起着至关重要的作用，但目前的研究缺乏全面的时空特征学习。本文的重点是通过提出一种新的方法来解决这一差距。在本文中，我们采用卷积神经网络（CNN）与不同的内核大小的空间和时间特征提取。此外，我们引入挤压和激励（SE）模块来捕获和融合多尺度特征，促进有效的信息融合，以提高情感识别和更深入地了解语音情感的时间演变。此外，我们采用跳跃连接和空间丢弃（SD）层来防止过拟合并增加模型的深度。我们的方法优于以前的最先进的方法，在六个基准SER数据集上，平均UAR和WAR分别提高了1.62%和1.32%。进一步的实验表明，我们的方法可以充分提取时空特征在低资源条件下。摘要：Speech Emotion Recognition (SER) has become a growing focus of research in human-computer interaction. Spatiotemporal features play a crucial role in SER, yet current research lacks comprehensive spatiotemporal feature learning. This paper focuses on addressing this gap by proposing a novel approach. In this paper, we employ Convolutional Neural Network (CNN) with varying kernel sizes for spatial and temporal feature extraction. Additionally, we introduce Squeeze-and-Excitation (SE) modules to capture and fuse multi-scale features, facilitating effective information fusion for improved emotion recognition and a deeper understanding of the temporal evolution of speech emotion. Moreover, we employ skip connections and Spatial Dropout (SD) layers to prevent overfitting and increase the model's depth. Our method outperforms the previous state-of-the-art method, achieving an average UAR and WAR improvement of 1.62% and 1.32%, respectively, across six benchmark SER datasets. Further experiments demonstrated that our method can fully extract spatiotemporal features in low-resource conditions.【8】 Emotion Rendering for Conversational Speech Synthesis with Heterogeneous Graph-Based Context Modeling标题：基于异构图上下文建模的会话语音合成情感渲染链接：https://arxiv.org/abs/2312.11947作者：Rui Liu,Yifan Hu,Yi Ren,Xiang Yin,Haizhou Li备注：9 pages, 4 figures, Accepted by AAAI'2024, Code and audio samples: this https URL摘要：会话语音合成（CSS）的目标是在会话环境中准确地表达具有适当韵律和情感变化的话语。虽然认识到CSS任务的重要性，但由于情感会话数据集的稀缺和有状态情感建模的困难，先前的研究没有深入研究情感表达问题。本文提出了一种新的情感CSS模型ECSS，它包括两个主要部分：1）为了增强情感理解，我们引入了一种基于异构图的情感上下文建模机制，以多源对话历史为输入，对对话上下文进行建模，并从上下文中学习情感线索; 2）在情感渲染方面，采用了基于对比学习的情感渲染模块来推断目标话语的准确情感风格。为了解决数据稀缺的问题，我们精心创建了类别和强度方面的情感标签，并在现有的会话数据集（DailyTalk）上注释了额外的情感信息。客观和主观的评估表明，我们的模型优于基线模型在理解和渲染情绪。这些评估也强调了全面的情感注释的重要性。代码和音频示例可以在https://github.com/walker-hyf/ECSS上找到。摘要：Conversational Speech Synthesis (CSS) aims to accurately express an utterance with the appropriate prosody and emotional inflection within a conversational setting. While recognising the significance of CSS task, the prior studies have not thoroughly investigated the emotional expressiveness problems due to the scarcity of emotional conversational datasets and the difficulty of stateful emotion modeling. In this paper, we propose a novel emotional CSS model, termed ECSS, that includes two main components: 1) to enhance emotion understanding, we introduce a heterogeneous graph-based emotional context modeling mechanism, which takes the multi-source dialogue history as input to model the dialogue context and learn the emotion cues from the context; 2) to achieve emotion rendering, we employ a contrastive learning-based emotion renderer module to infer the accurate emotion style for the target utterance. To address the issue of data scarcity, we meticulously create emotional labels in terms of category and intensity, and annotate additional emotional information on the existing conversational dataset (DailyTalk). Both objective and subjective evaluations suggest that our model outperforms the baseline models in understanding and rendering emotions. These evaluations also underscore the importance of comprehensive emotional annotations. Code and audio samples can be found at: https://github.com/walker-hyf/ECSS.【9】 MossFormer2: Combining Transformer and RNN-Free Recurrent Network for Enhanced Time-Domain Monaural Speech Separation标题：MossFormer2：结合Transformer和RNN自由递归网络的增强时域单声道语音分离链接：https://arxiv.org/abs/2312.11825作者：Shengkui Zhao,Yukun Ma,Chongjia Ni,Chong Zhang,Hao Wang,Trung Hieu Nguyen,Kun Zhou,Jiaqi Yip,Dianwen Ng,Bin Ma备注：5 pages, 3 figures, accepted by ICASSP 2024摘要：我们先前提出的MossFormer在单声道语音分离中取得了令人满意的性能。然而，它主要采用基于自我注意力的MossFormer模块，该模块倾向于强调更长范围、更粗尺度的依赖关系，在有效建模更细尺度的循环模式方面存在不足。在本文中，我们介绍了一种新的混合模型，它提供了建模的能力，通过将一个经常性的模块集成到MossFormer框架的长距离，粗规模的依赖关系和细规模的经常性模式。我们提出了一个基于前馈顺序记忆网络（FSMN）的递归模块，而不是应用使用传统递归连接的递归神经网络（RNN），由于能够在不使用递归连接的情况下捕获递归模式，因此被认为是“无RNN”递归网络。我们的递归模块主要包括通过使用门控卷积单元（GCU）和密集连接增强的膨胀FSMN块。此外，还增加了瓶颈层和输出层，用于控制信息流。循环模块依赖于线性投影和卷积，以实现整个序列的无缝并行处理。集成的MossFormer 2混合模型展示了对MossFormer的显着增强，并超越了WSJ 0 -2/3 mix，Libri 2 Mix和WHAM中的其他最先进的方法！/哇！基准。摘要：Our previously proposed MossFormer has achieved promising performance in monaural speech separation. However, it predominantly adopts a self-attention-based MossFormer module, which tends to emphasize longer-range, coarser-scale dependencies, with a deficiency in effectively modelling finer-scale recurrent patterns. In this paper, we introduce a novel hybrid model that provides the capabilities to model both long-range, coarse-scale dependencies and fine-scale recurrent patterns by integrating a recurrent module into the MossFormer framework. Instead of applying the recurrent neural networks (RNNs) that use traditional recurrent connections, we present a recurrent module based on a feedforward sequential memory network (FSMN), which is considered "RNN-free" recurrent network due to the ability to capture recurrent patterns without using recurrent connections. Our recurrent module mainly comprises an enhanced dilated FSMN block by using gated convolutional units (GCU) and dense connections. In addition, a bottleneck layer and an output layer are also added for controlling information flow. The recurrent module relies on linear projections and convolutions for seamless, parallel processing of the entire sequence. The integrated MossFormer2 hybrid model demonstrates remarkable enhancements over MossFormer and surpasses other state-of-the-art methods in WSJ0-2/3mix, Libri2Mix, and WHAM!/WHAMR! benchmarks.【10】 A review-based study on different Text-to-Speech technologies标题：不同文语转换技术的综述研究链接：https://arxiv.org/abs/2312.11563作者：Md. Jalal Uddin Chowdhury,Ashab Hussan备注：4 pages摘要：本文对各种文本语音转换（TTS）技术进行了全面的综述性研究。TTS技术是人机交互的一个重要方面，使机器能够将书面文本转换为可听语音。本文探讨了不同的TTS技术，包括拼接TTS，共振峰合成TTS，统计参数TTS。该研究的重点是比较这些技术的优点和局限性，在他们的自然语音，系统的复杂程度，以及他们对不同应用程序的适用性。此外，本文还探讨了语音合成技术的最新进展，包括神经语音合成和混合语音合成。这项研究的结果将为研究人员，开发人员和希望了解不同TTS技术及其适用于特定应用的用户提供有价值的见解。摘要：This research paper presents a comprehensive review-based study on various Text-to-Speech (TTS) technologies. TTS technology is an important aspect of human-computer interaction, enabling machines to convert written text into audible speech. The paper examines the different TTS technologies available, including concatenative TTS, formant synthesis TTS, and statistical parametric TTS. The study focuses on comparing the advantages and limitations of these technologies in terms of their naturalness of voice, the level of complexity of the system, and their suitability for different applications. In addition, the paper explores the latest advancements in TTS technology, including neural TTS and hybrid TTS. The findings of this research will provide valuable insights for researchers, developers, and users who want to understand the different TTS technologies and their suitability for specific applications.【11】 Toward A Reinforcement-Learning-Based System for Adjusting Medication to Minimize Speech Disfluency标题：一种基于强化学习的药物调整系统链接：https://arxiv.org/abs/2312.11509作者：Pavlos Constas,Vikram Rawal,Matthew Honorio Oliveira,Andreas Constas,Aditya Khan,Kaison Cheung,Najma Sultani,Carrie Chen,Micol Altomare,Michael Akzam,Jiacheng Chen,Vhea He,Lauren Altomare,Heraa Murqi,Asad Khan,Nimit Amikumar Bhanshali,Youssef Rachad,Michael Guerzhoy备注：In Proc. Machine Learning for Cognitive and Mental Health Workshop (ML4CMH) at AAAI 2024摘要：我们提出了一种基于强化学习的系统，该系统将自动为假设的患者开药，这些药物可能有助于患者治疗与心理健康相关的言语不流利，并根据患者的数据调整药物和剂量。我们展示了该系统的组成部分：一个模块，用于检测和评估我们构建的大型数据集上的语音不流利，以及一个强化学习算法，可以自动找到良好的药物组合。为了支持这两个模块，我们从文献中收集了关于精神病药物对言语不流利的影响的数据，并建立了一个合理的患者模拟系统。我们证明了强化学习系统在某些情况下能够收敛到一个良好的药物治疗方案。我们收集并标记了一个可能存在言语不流利的人的数据集，并使用该数据集演示了我们的方法。我们的工作是一个概念的证明：我们表明，使用自动数据收集来解决不流利的想法是有希望的。摘要：We propose a Reinforcement-Learning-based system that would automatically prescribe a hypothetical patient medications that may help the patient with their mental-health-related speech disfluency, and adjust the medication and the dosages in response to data from the patient. We demonstrate the components of the system: a module that detects and evaluates speech disfluency on a large dataset we built, and a Reinforcement Learning algorithm that automatically finds good combinations of medications. To support the two modules, we collect data on the effect of psychiatric medications for speech disfluency from the literature, and build a plausible patient simulation system. We demonstrate that the Reinforcement Learning system is, under some circumstances, able to converge to a good medication regime. We collect and label a dataset of people with possible speech disfluency and demonstrate our methods using that dataset. Our work is a proof of concept: we show that there is promise in the idea of using automatic data collection to address disfluency. 
每日语音速递
