今日论文合集：cs.SD语音7篇，eess.AS音频处理11篇。【1】 Synthesizing Cough Audio with GAN for COVID-19 Detection标题：新冠肺炎检测用GaN合成咳嗽音链接：https://arxiv.org/abs/2305.04810
作者：Yahya Saleh机构：B.Sc. (Hons) in Intelligent Systems备注：Bachelor's thesis摘要：对于这个最后一年的项目，目标是添加到已发表的工作中的数据合成卫生保健。该项目的最终产品是一个经过训练的模型，该模型生成可用于扩展医学数据集的合成图像（Pierre，2021）。本项目选择的领域是Covid-19咳嗽记录，已被证明是检测Covid的可行数据源。这是一个探索不足的领域，尽管它的巨大的重要性，因为有限的数据集可用于任务。一旦这个模型被开发出来，它的影响将通过训练有和没有扩展数据集的最先进的模型来说明，并测量性能的差异。最后，将通过将模型嵌入到Web应用程序中来将所有内容放在一起，以说明其功能。为了实现上述目标，广泛的文献综述将进行到最近的创新，使用生成模型的图像合成。摘要：For this final year project, the goal is to add to the published works within data synthesis for health care. The end product of this project is a trained model that generates synthesized images that can be used to expand a medical dataset (Pierre, 2021). The chosen domain for this project is the Covid-19 cough recording which is have been proven to be a viable data source for detecting Covid. This is an under-explored domain despite its huge importance because of the limited dataset available for the task. Once this model is developed its impact will be illustrated by training state-of-the-art models with and without the expanded dataset and measuring the difference in performance. Lastly, everything will be put together by embedding the model within a web application to illustrate its power. To achieve the said goals, an extensive literature review will be conducted into the recent innovations for image synthesis using generative models.【2】 A method for analyzing sampling jitter in audio equipment标题：一种音频设备中采样抖动的分析方法链接：https://arxiv.org/abs/2305.04531作者：Makoto Takeuchi,Haruo Saito机构：Graduate School of Arts and Sciences, The University of Tokyo, Tokyo,-, JAPAN备注：15 pages, 12 figures摘要：提出了一种音频设备采样抖动的分析方法。该方法是基于时域分析的时间波动的零交叉点记录的正弦波被用来表征抖动。当同一重放信号被同时馈送到两个音频记录器中时，该方法使得能够将音频播放器中的抖动评估与音频记录器中的抖动评估分开。实验使用市售的便携式设备进行，最大采样率为192~000个样本 秒。结果表明，在音频播放器中可以识别几十皮秒的抖动值。此外，所提出的方法使得能够利用音频设备的左声道和右声道将抖动与相位无关的噪声分离。因此，该方法适用于音频设备、信号发生器和时钟源的性能评估。摘要：A method for analyzing sampling jitter in audio equipment is proposed. The method is based on the time-domain analysis where the time fluctuations of zero-crossing points in recorded sinusoidal waves are employed to characterize jitter. This method enables the separate evaluation of jitter in an audio player from those in audio recorders when the same playback signal is simultaneously fed into two audio recorders. Experiments are conducted using commercially available portable devices with a maximum sampling rate of 192~000 samples per second. The results show jitter values of a few tens of picoseconds can be identified in an audio player. Moreover, the proposed method enables the separation of jitter from phase-independent noise utilizing the left and right channels of the audio equipment. As such, this method is applicable for performance evaluation of audio equipment, signal generators, and clock sources.【3】 Pitch Estimation by Denoising Preprocessor and Hybrid Estimation Model标题：基于去噪预处理器和混合估计模型的基音估计链接：https://arxiv.org/abs/2305.03982作者：Yu Cheng Hung,Ping Hung Chen,Jian Jiun Ding机构：Communication Engineering, National Taiwan University, Taipei, Taiwan, Ping-Hung Chen, Jian-Jiun Ding备注：From ICCE-Taiwan摘要：基音周期估计是对基音频率和midi数的估计，在音乐信号分析和语音信号处理中起着至关重要的作用。在这项工作中，我们提出了一种基于学习的增强预处理器的新架构，并结合了几种传统和深度学习的基音估计方法，以在嘈杂和干净的场景中实现更好的基音估计性能。我们测试了17种不同类型的噪声和4种SNRdb噪声水平。实验结果表明，该算法在噪声环境和噪声环境下均能获得较好的基音估计性能，且响应时间短。摘要：Pitch estimation is to estimate the fundamental frequency and the midi number and plays a critical role in music signal analysis and vocal signal processing. In this work, we proposed a new architecture based on a learning-based enhancement preprocessor and a combination of several traditional and deep learning pitch estimation methods to achieve better pitch estimation performance in both noisy and clean scenarios. We test 17 different types of noise and 4 SNRdb noise levels. The results show that the proposed pitch estimation can perform better in both noisy and clean scenarios with short response time.【4】 Accented Text-to-Speech Synthesis with Limited Data标题：有限数据下的重音文本到语音合成链接：https://arxiv.org/abs/2305.04816作者：Xuehao Zhou,Mingyang Zhang,Yi Zhou,Zhizheng Wu,Haizhou Li摘要：本文提出了一种带口音的文本到语音（TTS）合成框架与有限的训练数据。我们研究了关于口音渲染的两个方面：语音（音素差异）和韵律（音高模式和音素持续时间）变化。所提出的重音TTS框架由两个模型组成：用于字素到音素（G2P）转换的重音前端和具有用于音素到Mel声谱图预测的集成音高和持续时间预测器的重音声学模型。重音前端直接模拟语音变化，而重音声学模型显式地控制韵律变化。具体来说，两个模型首先在大量数据上进行预训练，然后仅在目标口音的有限数据量上对口音相关层进行微调。在实验中，三种英语口音的语音数据，即，一般美式英语、爱尔兰英语和英式英语标准发音用于预培训。然后，预训练的模型分别用苏格兰和普通澳大利亚英语口音进行微调。客观和主观评价结果均表明，用小的重音语音词典（5k个单词）微调的重音TTS前端有效地处理了重音的语音变化，而用有限量的重音语音数据（大约3分钟）微调的重音TTS声学模型有效地改善了包括音高和持续时间的韵律渲染。整体口音建模有助于改善语音质量和口音相似性。摘要：This paper presents an accented text-to-speech (TTS) synthesis framework with limited training data. We study two aspects concerning accent rendering: phonetic (phoneme difference) and prosodic (pitch pattern and phoneme duration) variations. The proposed accented TTS framework consists of two models: an accented front-end for grapheme-to-phoneme (G2P) conversion and an accented acoustic model with integrated pitch and duration predictors for phoneme-to-Mel-spectrogram prediction. The accented front-end directly models the phonetic variation, while the accented acoustic model explicitly controls the prosodic variation. Specifically, both models are first pre-trained on a large amount of data, then only the accent-related layers are fine-tuned on a limited amount of data for the target accent. In the experiments, speech data of three English accents, i.e., General American English, Irish English, and British English Received Pronunciation, are used for pre-training. The pre-trained models are then fine-tuned with Scottish and General Australian English accents, respectively. Both objective and subjective evaluation results show that the accented TTS front-end fine-tuned with a small accented phonetic lexicon (5k words) effectively handles the phonetic variation of accents, while the accented TTS acoustic model fine-tuned with a limited amount of accented speech data (approximately 3 minutes) effectively improves the prosodic rendering including pitch and duration. The overall accent modeling contributes to improved speech quality and accent similarity.【5】 AlignSTS: Speech-to-Singing Conversion via Cross-Modal Alignment标题：AlignSTS：跨通道对齐的语音到歌唱转换链接：https://arxiv.org/abs/2305.04476作者：Ruiqi Li,Rongjie Huang,Lichao Zhang,Jinglin Liu,Zhou Zhao机构：Zhejiang University备注：Accepted by Association for Computational Linguistics 2023摘要：语音到歌唱（STS）语音转换任务旨在生成与语音记录相对应的歌唱样本，同时面临以下主要挑战：目标（歌唱）音高轮廓和源（语音）内容之间的对准在无文本的情况下难以学习。本文提出了AlignSTS，STS模型的基础上明确的跨模态对齐，语音方差，如音高和内容作为不同的模态。受人类如何将歌词唱到旋律的机制的启发，AlignSTS：1）采用新颖的节奏适配器来预测目标节奏表示以桥接内容和音高之间的模态间隙，其中节奏表示以简单而有效的方式计算并且被量化到离散空间中;以及2）使用预测的节奏表示来基于交叉注意重新对准内容，并且进行交叉模态融合以用于重新合成。大量的实验表明，AlignSTS在客观和主观指标方面都取得了优异的性能。音频样本可在www.example.com上获得https: alignsts.github.io。摘要：The speech-to-singing (STS) voice conversion task aims to generate singing samples corresponding to speech recordings while facing a major challenge: the alignment between the target (singing) pitch contour and the source (speech) content is difficult to learn in a text-free situation. This paper proposes AlignSTS, an STS model based on explicit cross-modal alignment, which views speech variance such as pitch and content as different modalities. Inspired by the mechanism of how humans will sing the lyrics to the melody, AlignSTS: 1) adopts a novel rhythm adaptor to predict the target rhythm representation to bridge the modality gap between content and pitch, where the rhythm representation is computed in a simple yet effective way and is quantized into a discrete space; and 2) uses the predicted rhythm representation to re-align the content based on cross-attention and conducts a cross-modal fusion for re-synthesize. Extensive experiments show that AlignSTS achieves superior performance in terms of both objective and subjective metrics. Audio samples are available at https: alignsts.github.io.【6】 Neural Steerer: Novel Steering Vector Synthesis with a Causal Neural Field over Frequency and Source Positions标题：神经舵机：一种新的频率和声源位置因果神经场导向矢量综合方法链接：https://arxiv.org/abs/2305.04447作者：Diego Di Carlo,Aditya Arie Nugraha,Mathieu Fontaine,Mathieu Fontaine,Kazuyoshi Yoshii机构：Center for Advanced Intelligence Project (AIP), RIKEN, Tokyo, Japan., LTCI, Télécom Paris, Institut Polytechnique de Paris, Paris, France., Yoshiaki Bando, National Institute of Advanced Industrial Science and Technology (AIST)备注：Submitted to EUSIPCO23 conference摘要：神经场已经成功地应用于许多研究领域，因为它们具有从有限数量的观测值估计连续函数的能力。在音频处理中，该技术已被应用于声学和头部相关的传递函数插值。然而，大多数现有的方法估计的实值的幅度函数在一个预定义的离散的频率集。在这项研究中，我们提出了一种新的方法转向矢量插值，把频率作为连续的输入变量。此外，我们提出了一种新的无监督正则化项执行估计的过滤器是因果关系。使用实际的导向矢量的实验表明，所提出的频率分辨率的方法优于现有的方法操作在离散的频率集。摘要：Neural fields have successfully been used in many research fields for their native ability to estimate a continuous function from a finite number of observations. In audio processing, this technique has been applied to acoustic and head-related transfer function interpolation. However, most of the existing methods estimate the real-valued magnitude function over a predefined discrete set of frequencies. In this study, we propose a novel approach for steering vector interpolation that regards frequencies as continuous input variables. Moreover, we propose a novel unsupervised regularization term enforcing the estimated filters to be causal. The experiment using real steering vectors show that the proposed frequency resolution-free method outperformed the existing methods operating over discrete set of frequencies.【7】 Mask The Bias: Improving Domain-Adaptive Generalization of CTC-based ASR with Internal Language Model Estimation标题：掩盖偏见：用内部语言模型估计改进基于CTC的ASR的领域自适应泛化链接：https://arxiv.org/abs/2305.03837作者：Nilaksh Das,Monica Sunkara,Sravan Bodapati,Jinglun Cai,Devang Kulshreshtha,Jeff Farris,Katrin Kirchhoff机构：AWS AI Labs, USA备注：Accepted to ICASSP 2023摘要：在大量数据上训练的端到端ASR模型倾向于隐含地偏向于训练数据的语言语义。内部语言模型估计（ILME）已经被提出来减轻自回归模型（诸如基于注意力的编码器-解码器和RNN-T）的这种偏差。通常，ILME是通过模块化模型架构的声学和语言组件，并消除声学输入执行对数线性插值与纯文本后验。然而，对于基于CTC的ASR，将模型解耦成这样的声学和语言分量并不那么简单，因为CTC对数后验是以非自回归方式计算的。在这项工作中，我们提出了一种新的ILME技术的CTC为基础的ASR模型。我们的方法迭代地掩蔽音频时间步，以估计一个伪对数似然的内部LM通过积累对数后验仅掩蔽的时间步。在多个域外数据集的广泛评估表明，所提出的方法提高了WER高达9.8%和OOV F1分数高达24.6%，相对于浅融合，只有来自目标域的文本数据是可用的。在zero-shot域自适应的情况下，没有访问任何目标域数据，我们证明，去除源域的偏见与ILME仍然可以优于浅融合，以提高WER高达9.3%的相对。摘要：End-to-end ASR models trained on large amount of data tend to be implicitly biased towards language semantics of the training data. Internal language model estimation (ILME) has been proposed to mitigate this bias for autoregressive models such as attention-based encoder-decoder and RNN-T. Typically, ILME is performed by modularizing the acoustic and language components of the model architecture, and eliminating the acoustic input to perform log-linear interpolation with the text-only posterior. However, for CTC-based ASR, it is not as straightforward to decouple the model into such acoustic and language components, as CTC log-posteriors are computed in a non-autoregressive manner. In this work, we propose a novel ILME technique for CTC-based ASR models. Our method iteratively masks the audio timesteps to estimate a pseudo log-likelihood of the internal LM by accumulating log-posteriors for only the masked timesteps. Extensive evaluation across multiple out-of-domain datasets reveals that the proposed approach improves WER by up to 9.8% and OOV F1-score by up to 24.6% relative to Shallow Fusion, when only text data from target domain is available. In the case of zero-shot domain adaptation, with no access to any target domain data, we demonstrate that removing the source domain bias with ILME can still outperform Shallow Fusion to improve WER by up to 9.3% relative.【1】 Accented Text-to-Speech Synthesis with Limited Data标题：有限数据下的重音文本到语音合成链接：https://arxiv.org/abs/2305.04816作者：Xuehao Zhou,Mingyang Zhang,Yi Zhou,Zhizheng Wu,Haizhou Li摘要：本文提出了一种带口音的文本到语音（TTS）合成框架与有限的训练数据。我们研究了关于口音渲染的两个方面：语音（音素差异）和韵律（音高模式和音素持续时间）变化。所提出的重音TTS框架由两个模型组成：用于字素到音素（G2P）转换的重音前端和具有用于音素到Mel声谱图预测的集成音高和持续时间预测器的重音声学模型。重音前端直接模拟语音变化，而重音声学模型显式地控制韵律变化。具体来说，两个模型首先在大量数据上进行预训练，然后仅在目标口音的有限数据量上对口音相关层进行微调。在实验中，三种英语口音的语音数据，即，一般美式英语、爱尔兰英语和英式英语标准发音用于预培训。然后，预训练的模型分别用苏格兰和普通澳大利亚英语口音进行微调。客观和主观评价结果均表明，用小的重音语音词典（5k个单词）微调的重音TTS前端有效地处理了重音的语音变化，而用有限量的重音语音数据（大约3分钟）微调的重音TTS声学模型有效地改善了包括音高和持续时间的韵律渲染。整体口音建模有助于改善语音质量和口音相似性。摘要：This paper presents an accented text-to-speech (TTS) synthesis framework with limited training data. We study two aspects concerning accent rendering: phonetic (phoneme difference) and prosodic (pitch pattern and phoneme duration) variations. The proposed accented TTS framework consists of two models: an accented front-end for grapheme-to-phoneme (G2P) conversion and an accented acoustic model with integrated pitch and duration predictors for phoneme-to-Mel-spectrogram prediction. The accented front-end directly models the phonetic variation, while the accented acoustic model explicitly controls the prosodic variation. Specifically, both models are first pre-trained on a large amount of data, then only the accent-related layers are fine-tuned on a limited amount of data for the target accent. In the experiments, speech data of three English accents, i.e., General American English, Irish English, and British English Received Pronunciation, are used for pre-training. The pre-trained models are then fine-tuned with Scottish and General Australian English accents, respectively. Both objective and subjective evaluation results show that the accented TTS front-end fine-tuned with a small accented phonetic lexicon (5k words) effectively handles the phonetic variation of accents, while the accented TTS acoustic model fine-tuned with a limited amount of accented speech data (approximately 3 minutes) effectively improves the prosodic rendering including pitch and duration. The overall accent modeling contributes to improved speech quality and accent similarity.【2】 AlignSTS: Speech-to-Singing Conversion via Cross-Modal Alignment标题：AlignSTS：跨通道对齐的语音到歌唱转换链接：https://arxiv.org/abs/2305.04476*与cs.SD语音【5】为同一篇作者：Ruiqi Li,Rongjie Huang,Lichao Zhang,Jinglin Liu,Zhou Zhao机构：Zhejiang University备注：Accepted by Association for Computational Linguistics 2023摘要：语音到歌唱（STS）语音转换任务旨在生成与语音记录相对应的歌唱样本，同时面临以下主要挑战：目标（歌唱）音高轮廓和源（语音）内容之间的对准在无文本的情况下难以学习。本文提出了AlignSTS，STS模型的基础上明确的跨模态对齐，语音方差，如音高和内容作为不同的模态。受人类如何将歌词唱到旋律的机制的启发，AlignSTS：1）采用新颖的节奏适配器来预测目标节奏表示以桥接内容和音高之间的模态间隙，其中节奏表示以简单而有效的方式计算并且被量化到离散空间中;以及2）使用预测的节奏表示来基于交叉注意重新对准内容，并且进行交叉模态融合以用于重新合成。大量的实验表明，AlignSTS在客观和主观指标方面都取得了优异的性能。音频样本可在www.example.com上获得https: alignsts.github.io。摘要：The speech-to-singing (STS) voice conversion task aims to generate singing samples corresponding to speech recordings while facing a major challenge: the alignment between the target (singing) pitch contour and the source (speech) content is difficult to learn in a text-free situation. This paper proposes AlignSTS, an STS model based on explicit cross-modal alignment, which views speech variance such as pitch and content as different modalities. Inspired by the mechanism of how humans will sing the lyrics to the melody, AlignSTS: 1) adopts a novel rhythm adaptor to predict the target rhythm representation to bridge the modality gap between content and pitch, where the rhythm representation is computed in a simple yet effective way and is quantized into a discrete space; and 2) uses the predicted rhythm representation to re-align the content based on cross-attention and conducts a cross-modal fusion for re-synthesize. Extensive experiments show that AlignSTS achieves superior performance in terms of both objective and subjective metrics. Audio samples are available at https: alignsts.github.io.【3】 Neural Steerer: Novel Steering Vector Synthesis with a Causal Neural Field over Frequency and Source Positions标题：神经舵机：一种新的频率和声源位置因果神经场导向矢量综合方法链接：https://arxiv.org/abs/2305.04447*与cs.SD语音【6】为同一篇作者：Diego Di Carlo,Aditya Arie Nugraha,Mathieu Fontaine,Mathieu Fontaine,Kazuyoshi Yoshii机构：Center for Advanced Intelligence Project (AIP), RIKEN, Tokyo, Japan., LTCI, Télécom Paris, Institut Polytechnique de Paris, Paris, France., Yoshiaki Bando, National Institute of Advanced Industrial Science and Technology (AIST)备注：Submitted to EUSIPCO23 conference摘要：神经场已经成功地应用于许多研究领域，因为它们具有从有限数量的观测值估计连续函数的能力。在音频处理中，该技术已被应用于声学和头部相关的传递函数插值。然而，大多数现有的方法估计的实值的幅度函数在一个预定义的离散的频率集。在这项研究中，我们提出了一种新的方法转向矢量插值，把频率作为连续的输入变量。此外，我们提出了一种新的无监督正则化项执行估计的过滤器是因果关系。使用实际的导向矢量的实验表明，所提出的频率分辨率的方法优于现有的方法操作在离散的频率集。摘要：Neural fields have successfully been used in many research fields for their native ability to estimate a continuous function from a finite number of observations. In audio processing, this technique has been applied to acoustic and head-related transfer function interpolation. However, most of the existing methods estimate the real-valued magnitude function over a predefined discrete set of frequencies. In this study, we propose a novel approach for steering vector interpolation that regards frequencies as continuous input variables. Moreover, we propose a novel unsupervised regularization term enforcing the estimated filters to be causal. The experiment using real steering vectors show that the proposed frequency resolution-free method outperformed the existing methods operating over discrete set of frequencies.【4】 Lookahead When It Matters: Adaptive Non-causal Transformers for Streaming Neural Transducers标题：事关重大时先行一步：用于分流神经传感器的自适应非因果转换器链接：https://arxiv.org/abs/2305.04159作者：Grant P. Strimel,Yi Xie,Brian King,Martin Radfar,Ariya Rastrow,Athanasios Mouchtaris摘要：流式语音识别架构用于低延迟、实时应用。这样的架构通常以它们的因果关系为特征。因果架构在每个帧发出令牌，仅依赖于当前和过去的信号，而非因果模型在每个步骤暴露于未来帧的窗口，以提高预测准确性。这种二分法相当于实时自动语音识别（ASR）系统设计的折衷：从严格因果架构的低延迟优势中获益，同时接受预测性能限制，或者实现未来上下文模型的建模优势，伴随着更高的延迟惩罚。在这项工作中，我们放松了这种选择的限制，并提出了自适应非因果性注意力传感器（ANCAT）。我们的架构在传统意义上是非因果关系的，但通过动态选择何时依赖于未来的上下文以及在音频流中的程度来以低延迟、流式传输的方式执行。当与我们的新型正则化算法相结合时，所产生的机制提供了与非因果配置相当的准确性，同时显着改善了延迟，缩小了与因果配置的差距。我们展示了我们的设计实验报告的准确性和延迟的措施公开访问和生产规模，语音助手数据集的ASR任务结果进行比较。摘要：Streaming speech recognition architectures are employed for low-latency, real-time applications. Such architectures are often characterized by their causality. Causal architectures emit tokens at each frame, relying only on current and past signal, while non-causal models are exposed to a window of future frames at each step to increase predictive accuracy. This dichotomy amounts to a trade-off for real-time Automatic Speech Recognition (ASR) system design: profit from the low-latency benefit of strictly-causal architectures while accepting predictive performance limitations, or realize the modeling benefits of future-context models accompanied by their higher latency penalty. In this work, we relax the constraints of this choice and present the Adaptive Non-Causal Attention Transducer (ANCAT). Our architecture is non-causal in the traditional sense, but executes in a low-latency, streaming manner by dynamically choosing when to rely on future context and to what degree within the audio stream. The resulting mechanism, when coupled with our novel regularization algorithms, delivers comparable accuracy to non-causal configurations while improving significantly upon latency, closing the gap with their causal counterparts. We showcase our design experimentally by reporting comparative ASR task results with measures of accuracy and latency on both publicly accessible and production-scale, voice-assistant datasets.【5】 Mask The Bias: Improving Domain-Adaptive Generalization of CTC-based ASR with Internal Language Model Estimation标题：掩盖偏见：用内部语言模型估计改进基于CTC的ASR的领域自适应泛化链接：https://arxiv.org/abs/2305.03837*与cs.SD语音【7】为同一篇作者：Nilaksh Das,Monica Sunkara,Sravan Bodapati,Jinglun Cai,Devang Kulshreshtha,Jeff Farris,Katrin Kirchhoff机构：AWS AI Labs, USA备注：Accepted to ICASSP 2023摘要：在大量数据上训练的端到端ASR模型倾向于隐含地偏向于训练数据的语言语义。内部语言模型估计（ILME）已经被提出来减轻自回归模型（诸如基于注意力的编码器-解码器和RNN-T）的这种偏差。通常，ILME是通过模块化模型架构的声学和语言组件，并消除声学输入执行对数线性插值与纯文本后验。然而，对于基于CTC的ASR，将模型解耦成这样的声学和语言分量并不那么简单，因为CTC对数后验是以非自回归方式计算的。在这项工作中，我们提出了一种新的ILME技术的CTC为基础的ASR模型。我们的方法迭代地掩蔽音频时间步，以估计一个伪对数似然的内部LM通过积累对数后验仅掩蔽的时间步。在多个域外数据集的广泛评估表明，所提出的方法提高了WER高达9.8%和OOV F1分数高达24.6%，相对于浅融合，只有来自目标域的文本数据是可用的。在zero-shot域自适应的情况下，没有访问任何目标域数据，我们证明，去除源域的偏见与ILME仍然可以优于浅融合，以提高WER高达9.3%的相对。摘要：End-to-end ASR models trained on large amount of data tend to be implicitly biased towards language semantics of the training data. Internal language model estimation (ILME) has been proposed to mitigate this bias for autoregressive models such as attention-based encoder-decoder and RNN-T. Typically, ILME is performed by modularizing the acoustic and language components of the model architecture, and eliminating the acoustic input to perform log-linear interpolation with the text-only posterior. However, for CTC-based ASR, it is not as straightforward to decouple the model into such acoustic and language components, as CTC log-posteriors are computed in a non-autoregressive manner. In this work, we propose a novel ILME technique for CTC-based ASR models. Our method iteratively masks the audio timesteps to estimate a pseudo log-likelihood of the internal LM by accumulating log-posteriors for only the masked timesteps. Extensive evaluation across multiple out-of-domain datasets reveals that the proposed approach improves WER by up to 9.8% and OOV F1-score by up to 24.6% relative to Shallow Fusion, when only text data from target domain is available. In the case of zero-shot domain adaptation, with no access to any target domain data, we demonstrate that removing the source domain bias with ILME can still outperform Shallow Fusion to improve WER by up to 9.3% relative.【6】 What Do Patients Say About Their Disease Symptoms? Deep Multilabel Text Classification With Human-in-the-Loop Curation for Automatic Labeling of Patient Self Reports of Problems标题：患者对他们的疾病症状有什么看法？深度多标签文本分类--用于自动标记患者自我报告问题的人在回路中的过程链接：https://arxiv.org/abs/2305.04905作者：Lakshmi Arbatti,Abhishek Hosamath,Vikram Ramanarayanan,Ira Shoulson机构：Modality.AI, Inc.摘要：美国食品和药物管理局已给予越来越多的重视，病人报告的问题，在临床和研究环境。在本文中，我们探索了最大的在线数据集之一，其中包括170，141个开放式的自我报告响应（称为“verba”），这些响应来自帕金森病患者（PwPs），以回答关于帕金森病困扰他们的问题以及它如何影响他们的日常功能，也称为帕金森病患者问题报告。将这种语言分类为多个临床相关症状类别是一个重要的问题，并且需要多个步骤-专家策展、多标签文本分类（MLTC）方法和大量标记的训练数据。此外，这样的大型数据集的人工注释是繁琐且昂贵的。我们提出了一种新的解决方案，其中我们使用由包括临床专家和PwPs在内的9位策展人注释的2，341（170，141）verbabetry构建了一个基线数据集。我们使用NLP技术和基于图数据库的专家短语查询系统开发了一个基于规则的语言词典，以将注释扩展到生成机器注释数据集的剩余队列，并最终为这两个数据集构建基于Keras-Tensorflow的MLTC模型。机器注释模型显著优于基线模型，在65个症状类别中，在保留的测试集上的F1得分为95%。摘要：The USA Food and Drug Administration has accorded increasing importance to patient-reported problems in clinical and research settings. In this paper, we explore one of the largest online datasets comprising 170,141 open-ended self-reported responses (called "verbatims") from patients with Parkinson's (PwPs) to questions about what bothers them about their Parkinson's Disease and how it affects their daily functioning, also known as the Parkinson's Disease Patient Report of Problems. Classifying such verbatims into multiple clinically relevant symptom categories is an important problem and requires multiple steps - expert curation, a multi-label text classification (MLTC) approach and large amounts of labelled training data. Further, human annotation of such large datasets is tedious and expensive. We present a novel solution to this problem where we build a baseline dataset using 2,341 (of the 170,141) verbatims annotated by nine curators including clinical experts and PwPs. We develop a rules based linguistic-dictionary using NLP techniques and graph database-based expert phrase-query system to scale the annotation to the remaining cohort generating the machine annotated dataset, and finally build a Keras-Tensorflow based MLTC model for both datasets. The machine annotated model significantly outperforms the baseline model with a F1-score of 95% across 65 symptom categories on a held-out test set.【7】 Synthesizing Cough Audio with GAN for COVID-19 Detection标题：新冠肺炎检测用GaN合成咳嗽音链接：https://arxiv.org/abs/2305.04810*与cs.SD语音【1】为同一篇作者：Yahya Saleh机构：B.Sc. (Hons) in Intelligent Systems备注：Bachelor's thesis摘要：对于这个最后一年的项目，目标是添加到已发表的工作中的数据合成卫生保健。该项目的最终产品是一个经过训练的模型，该模型生成可用于扩展医学数据集的合成图像（Pierre，2021）。本项目选择的领域是Covid-19咳嗽记录，已被证明是检测Covid的可行数据源。这是一个探索不足的领域，尽管它的巨大的重要性，因为有限的数据集可用于任务。一旦这个模型被开发出来，它的影响将通过训练有和没有扩展数据集的最先进的模型来说明，并测量性能的差异。最后，将通过将模型嵌入到Web应用程序中来将所有内容放在一起，以说明其功能。为了实现上述目标，广泛的文献综述将进行到最近的创新，使用生成模型的图像合成。摘要：For this final year project, the goal is to add to the published works within data synthesis for health care. The end product of this project is a trained model that generates synthesized images that can be used to expand a medical dataset (Pierre, 2021). The chosen domain for this project is the Covid-19 cough recording which is have been proven to be a viable data source for detecting Covid. This is an under-explored domain despite its huge importance because of the limited dataset available for the task. Once this model is developed its impact will be illustrated by training state-of-the-art models with and without the expanded dataset and measuring the difference in performance. Lastly, everything will be put together by embedding the model within a web application to illustrate its power. To achieve the said goals, an extensive literature review will be conducted into the recent innovations for image synthesis using generative models.【8】 Putting Natural in Natural Language Processing标题：自然语言处理中的自然语言处理链接：https://arxiv.org/abs/2305.04572作者：Grzegorz Chrupała机构：Department of Cognitive Science and Artificial Intelligence, Tilburg University备注：Findings of the ACL 2023摘要：人类的语言首先是口语，其次才是书面语。然而，文本是一种非常方便和高效的方法 语言的代表性，现代文明已经使它 无处不在因此，NLP领域具有 绝大多数人专注于处理书面而不是口语 语言。另一方面，口语研究一直是 在很大程度上独立的语音处理社区中被孤立 它一直过度专注于将语音转录成 短信。深度学习的最新进展导致了一个偶然的 语音处理和主流NLP之间的方法融合。可以说，统一这两个领域的时机已经成熟， 并开始认真对待口语 人类沟通的方式。真正的自然语言处理 可以更好地与其他语言科学相结合 并可能导致更有效的数据和更多 像人类一样，可以超越文本进行交流 模态摘要：Human language is firstly spoken and only secondarily written. Text, however, is a very convenient and efficient representation of language, and modern civilization has made it ubiquitous. Thus the field of NLP has overwhelmingly focused on processing written rather than spoken language. Work on spoken language, on the other hand, has been siloed off within the largely separate speech processing community which has been inordinately preoccupied with transcribing speech into text. Recent advances in deep learning have led to a fortuitous convergence in methods between speech processing and mainstream NLP. Arguably, the time is ripe for a unification of these two fields, and for starting to take spoken language seriously as the primary mode of human communication. Truly natural language processing could lead to better integration with the rest of language science and could lead to systems which are more data-efficient and more human-like, and which can communicate beyond the textual modality.【9】 Multi-Temporal Lip-Audio Memory for Visual Speech Recognition标题：用于视觉语音识别的多时相唇音频记忆链接：https://arxiv.org/abs/2305.04542作者：Jeong Hun Yeo,Minsu Kim,Yong Man Ro机构：School of Electrical Engineering, KAIST, South Korea备注：Presented at ICASSP 2023摘要：视觉语音识别（VSR）是一个从嘴唇运动预测句子或单词的任务。最近已经提出了一些使用音频信号来补充视觉信息的作品。然而，现有的方法仅利用有限的信息，例如音素级特征和自动语音识别（ASR）网络的软标签。在本文中，我们提出了一个多时间唇音频记忆（MTLAM），使最好的利用音频信号，以补充不足的信息嘴唇的运动。所提出的方法主要由两部分组成：1）MTLAM保存从短期和长期音频信号产生的多时间音频特征，并且MTLAM存储视觉到音频映射以在推断阶段从视觉特征加载存储的多时间音频特征。2)我们设计了一个音频时间模型，以产生多时间的音频功能捕获的上下文相邻的话。此外，为了构建有效的视觉到音频映射，音频时间模型可以生成与视觉特征时间对齐的音频特征。通过大量的实验，我们验证的MTLAM实现国家的最先进的性能在两个公共VSR数据集的有效性。摘要：Visual Speech Recognition (VSR) is a task to predict a sentence or word from lip movements. Some works have been recently presented which use audio signals to supplement visual information. However, existing methods utilize only limited information such as phoneme-level features and soft labels of Automatic Speech Recognition (ASR) networks. In this paper, we present a Multi-Temporal Lip-Audio Memory (MTLAM) that makes the best use of audio signals to complement insufficient information of lip movements. The proposed method is mainly composed of two parts: 1) MTLAM saves multi-temporal audio features produced from short- and long-term audio signals, and the MTLAM memorizes a visual-to-audio mapping to load stored multi-temporal audio features from visual features at the inference phase. 2) We design an audio temporal model to produce multi-temporal audio features capturing the context of neighboring words. In addition, to construct effective visual-to-audio mapping, the audio temporal models can generate audio features time-aligned with visual features. Through extensive experiments, we validate the effectiveness of the MTLAM achieving state-of-the-art performances on two public VSR datasets.【10】 A method for analyzing sampling jitter in audio equipment标题：一种音频设备中采样抖动的分析方法链接：https://arxiv.org/abs/2305.04531*与cs.SD语音【2】为同一篇作者：Makoto Takeuchi,Haruo Saito机构：Graduate School of Arts and Sciences, The University of Tokyo, Tokyo,-, JAPAN备注：15 pages, 12 figures摘要：提出了一种音频设备采样抖动的分析方法。该方法是基于时域分析的时间波动的零交叉点记录的正弦波被用来表征抖动。当同一重放信号被同时馈送到两个音频记录器中时，该方法使得能够将音频播放器中的抖动评估与音频记录器中的抖动评估分开。实验使用市售的便携式设备进行，最大采样率为192~000个样本 秒。结果表明，在音频播放器中可以识别几十皮秒的抖动值。此外，所提出的方法使得能够利用音频设备的左声道和右声道将抖动与相位无关的噪声分离。因此，该方法适用于音频设备、信号发生器和时钟源的性能评估。摘要：A method for analyzing sampling jitter in audio equipment is proposed. The method is based on the time-domain analysis where the time fluctuations of zero-crossing points in recorded sinusoidal waves are employed to characterize jitter. This method enables the separate evaluation of jitter in an audio player from those in audio recorders when the same playback signal is simultaneously fed into two audio recorders. Experiments are conducted using commercially available portable devices with a maximum sampling rate of 192~000 samples per second. The results show jitter values of a few tens of picoseconds can be identified in an audio player. Moreover, the proposed method enables the separation of jitter from phase-independent noise utilizing the left and right channels of the audio equipment. As such, this method is applicable for performance evaluation of audio equipment, signal generators, and clock sources.【11】 Pitch Estimation by Denoising Preprocessor and Hybrid Estimation Model标题：基于去噪预处理器和混合估计模型的基音估计链接：https://arxiv.org/abs/2305.03982*与cs.SD语音【3】为同一篇作者：Yu Cheng Hung,Ping Hung Chen,Jian Jiun Ding机构：Communication Engineering, National Taiwan University, Taipei, Taiwan, Ping-Hung Chen, Jian-Jiun Ding备注：From ICCE-Taiwan摘要：基音周期估计是对基音频率和midi数的估计，在音乐信号分析和语音信号处理中起着至关重要的作用。在这项工作中，我们提出了一种基于学习的增强预处理器的新架构，并结合了几种传统和深度学习的基音估计方法，以在嘈杂和干净的场景中实现更好的基音估计性能。我们测试了17种不同类型的噪声和4种SNRdb噪声水平。实验结果表明，该算法在噪声环境和噪声环境下均能获得较好的基音估计性能，且响应时间短。摘要：Pitch estimation is to estimate the fundamental frequency and the midi number and plays a critical role in music signal analysis and vocal signal processing. In this work, we proposed a new architecture based on a learning-based enhancement preprocessor and a combination of several traditional and deep learning pitch estimation methods to achieve better pitch estimation performance in both noisy and clean scenarios. We test 17 different types of noise and 4 SNRdb noise levels. The results show that the proposed pitch estimation can perform better in both noisy and clean scenarios with short response time. 
每日语音速递
