语音/音频处理学术速递[10.26]

今日论文合集：cs.SD语音9篇，eess.AS音频处理11篇。

cs.SD语音

【1】 ArTST: Arabic Text and Speech Transformer标题：Artst：阿拉伯语文本和语音转换器链接：https://arxiv.org/abs/2310.16621

作者：Hawau Olamide Toyin,Amirbek Djanibekov,Ajinkya Kulkarni,Hanan Aldarmaki备注：11 pages, 1 figure, SIGARAB ArabicNLP 2023摘要：我们提出了ArTST，一个预先训练的阿拉伯语文本和语音Transformer，用于支持阿拉伯语的开源语音技术。该模型架构遵循最近为英语发布的统一模态框架SpeechT5，并专注于现代标准阿拉伯语（MSA），计划在未来版本中扩展方言和代码转换阿拉伯语的模型。我们在MSA语音和文本数据上从头开始对模型进行预训练，并针对以下任务对其进行微调：自动语音识别（ASR），文本到语音合成（TTS）和口语方言识别。在我们的实验中，将ArTST与SpeechT5以及先前报道的这些任务的结果进行了比较，ArTST在所有三项任务中的表现与当前最先进的技术相当或超过了当前最先进的技术。此外，我们发现我们的预训练有利于泛化，这在低资源TTS任务中尤其明显。预训练的模型以及微调的ASR和TTS模型被发布用于研究。摘要：We present ArTST, a pre-trained Arabic text and speech transformer for supporting open-source speech technologies for the Arabic language. The model architecture follows the unified-modal framework, SpeechT5, that was recently released for English, and is focused on Modern Standard Arabic (MSA), with plans to extend the model for dialectal and code-switched Arabic in future editions. We pre-trained the model from scratch on MSA speech and text data, and fine-tuned it for the following tasks: Automatic Speech Recognition (ASR), Text-To-Speech synthesis (TTS), and spoken dialect identification. In our experiments comparing ArTST with SpeechT5, as well as with previously reported results in these tasks, ArTST performs on a par with or exceeding the current state-of-the-art in all three tasks. Moreover, we find that our pre-training is conducive for generalization, which is particularly evident in the low-resource TTS task. The pre-trained model as well as the fine-tuned ASR and TTS models are released for research use.

【2】 Back Transcription as a Method for Evaluating Robustness of Natural Language Understanding Models to Speech Recognition Errors标题：反转录作为一种评估自然语言理解模型对语音识别错误稳健性的方法链接：https://arxiv.org/abs/2310.16609

作者：Marek Kubis,Paweł Skórzewski,Marcin Sowański,Tomasz Ziętkiewicz备注：Accepted to EMNLP 2023 main conference摘要：在口语对话系统中，NLU模型之前是语音识别系统，这可能会降低自然语言理解的性能。本文提出了一种研究语音识别错误对自然语言理解模型性能影响的方法。所提出的方法结合了回转录过程与细粒度的技术，用于分类的错误，影响NLU模型的性能。该方法依赖于使用合成语音进行NLU评估。我们表明，使用合成语音代替录音不会改变所提出的技术的结果在一个显着的方式。摘要：In a spoken dialogue system, an NLU model is preceded by a speech recognition system that can deteriorate the performance of natural language understanding. This paper proposes a method for investigating the impact of speech recognition errors on the performance of natural language understanding models. The proposed method combines the back transcription procedure with a fine-grained technique for categorizing the errors that affect the performance of NLU models. The method relies on the usage of synthesized speech for NLU evaluation. We show that the use of synthesized speech in place of audio recording does not change the outcomes of the presented technique in a significant way.

【3】 Dynamic Processing Neural Network Architecture For Hearing Loss Compensation标题：听力损失补偿的动态处理神经网络结构链接：https://arxiv.org/abs/2310.16550

作者：Szymon Drgas,Lars Bramsløw,Archontis Politis,Gaurav Naithani,Tuomas Virtanen摘要：本文提出了神经网络补偿感音神经性听力损失。听力损失补偿任务的目的是转换语音信号，以在听力受损的人进一步处理后提高语音清晰度，听力受损的人通过听力损失模型建模。我们提出了一个可解释的模型，称为动态处理网络，它具有类似于带式动态压缩器的结构。该网络是可微分的，因此允许学习其参数以最大化语音清晰度。还测试了基于卷积层的更通用的模型。测试架构的性能进行了评估，使用频谱-时间客观指数（STOI）与听力阈值噪声和助听器语音清晰度（HASPI）指标。与流行的压缩增益规定规则Camfit相比，动态处理网络给出了STOI和HASPI的显著改进。足够大的卷积网络可以以更大的计算负载为代价胜过可解释模型。最后，将动态处理网络与卷积神经网络相结合，在STOI和HASPI方面给出了最佳结果。摘要：This paper proposes neural networks for compensating sensorineural hearing loss. The aim of the hearing loss compensation task is to transform a speech signal to increase speech intelligibility after further processing by a person with a hearing impairment, which is modeled by a hearing loss model. We propose an interpretable model called dynamic processing network, which has a structure similar to band-wise dynamic compressor. The network is differentiable, and therefore allows to learn its parameters to maximize speech intelligibility. More generic models based on convolutional layers were tested as well. The performance of the tested architectures was assessed using spectro-temporal objective index (STOI) with hearing-threshold noise and hearing aid speech intelligibility (HASPI) metrics. The dynamic processing network gave a significant improvement of STOI and HASPI in comparison to popular compressive gain prescription rule Camfit. A large enough convolutional network could outperform the interpretable model with the cost of larger computational load. Finally, a combination of the dynamic processing network with convolutional neural network gave the best results in terms of STOI and HASPI.

【4】 AccoMontage-3: Full-Band Accompaniment Arrangement via Sequential Style Transfer and Multi-Track Function Prior标题：Acco蒙太奇-3：全频段伴奏编排通过顺序风格转换和多声道功能优先链接：https://arxiv.org/abs/2310.16334

作者：Jingwei Zhao,Gus Xia,Ye Wang摘要：我们提出了AccoMontage-3，一个符号音乐自动化系统，能够基于具有和弦的主旋律的输入来生成多音轨、全波段伴奏（即，铅板）。该系统包含三个模块化组件，每个组件都模拟了全波段组成的一个重要方面。第一个组件是一个钢琴伴奏器，它通过使用潜在的和弦纹理解缠和纹理供体的启发式检索将纹理风格转移到和弦来生成铅片的钢琴伴奏。第二部分根据各个音轨函数编码的配器风格将钢琴伴奏乐谱配器成全波段编曲。第三个部分是连接前两个部分的先验模型，它表征了整首音乐作品中配器风格的整体结构。从端到端，系统学习以自我监督的方式生成全波段伴奏，在复调作曲的两个层面上应用风格转移：织体和配器。实验表明，我们的系统优于基线显着，和模块化设计提供了有效的控制，在音乐上有意义的方式。摘要：We propose AccoMontage-3, a symbolic music automation system capable of generating multi-track, full-band accompaniment based on the input of a lead melody with chords (i.e., a lead sheet). The system contains three modular components, each modelling a vital aspect of full-band composition. The first component is a piano arranger that generates piano accompaniment for the lead sheet by transferring texture styles to the chords using latent chord-texture disentanglement and heuristic retrieval of texture donors. The second component orchestrates the piano accompaniment score into full-band arrangement according to the orchestration style encoded by individual track functions. The third component, which connects the previous two, is a prior model characterizing the global structure of orchestration style over the whole piece of music. From end to end, the system learns to generate full-band accompaniment in a self-supervised fashion, applying style transfer at two levels of polyphonic composition: texture and orchestration. Experiments show that our system outperforms the baselines significantly, and the modular design offers effective controls in a musically meaningful way.

【5】 Towards Streaming Speech-to-Avatar Synthesis标题：走向流媒体语音到头像的合成链接：https://arxiv.org/abs/2310.16287

作者：Tejas S. Prabhune,Peter Wu,Bohan Yu,Gopala K. Anumanchipalli备注：Submitted to ICASSP 2024摘要：流式语音到化身合成从音频数据为虚拟角色创建实时动画。语音的准确化身表示对于语言学、语音学和音系学中的声音的可视化、辅助第二语言习得的视觉反馈以及瘫痪患者的虚拟体现是重要的。以前的作品已经强调了使用电磁关节成像（EMA）功能执行高质量化身动画的深度发音反转的能力。然而，这些模型集中于具有记录而不是实时音频的离线化身合成，这对于实况化身可视化或实施例是必要的。为了解决这个问题，我们提出了一种使用发音反转的方法，用于从实时音频流传输高质量的面部和嘴内化身动画。我们的方法实现了每0.1秒音频的130 ms平均流延迟，与地面真实表达的相关性为0.792。最后，我们显示生成的嘴和舌头的动画，以证明我们的方法的有效性。摘要：Streaming speech-to-avatar synthesis creates real-time animations for a virtual character from audio data. Accurate avatar representations of speech are important for the visualization of sound in linguistics, phonetics, and phonology, visual feedback to assist second language acquisition, and virtual embodiment for paralyzed patients. Previous works have highlighted the capability of deep articulatory inversion to perform high-quality avatar animation using electromagnetic articulography (EMA) features. However, these models focus on offline avatar synthesis with recordings rather than real-time audio, which is necessary for live avatar visualization or embodiment. To address this issue, we propose a method using articulatory inversion for streaming high quality facial and inner-mouth avatar animation from real-time audio. Our approach achieves 130ms average streaming latency for every 0.1 seconds of audio with a 0.792 correlation with ground truth articulations. Finally, we show generated mouth and tongue animations to demonstrate the efficacy of our methodology.

【6】 Complex Image Generation SwinTransformer Network for Audio Denoising标题：用于音频去噪的复数图像生成SwinTransformer网络链接：https://arxiv.org/abs/2310.16109

作者：Youshan Zhang,Jialu Li摘要：实现高性能的音频去噪在实际应用中仍然是一项具有挑战性的任务。现有的时频方法往往忽略了生成的频域图像的质量。本文将音频去噪问题转化为一个图像生成任务。我们首先开发了一个复杂的图像生成SwinTransformer网络，从复杂的傅立叶域捕获更多的信息。然后，我们施加结构相似性和详细的损失函数来生成高质量的图像，并开发SDR损失，以最大限度地减少去噪和干净音频之间的差异。在两个基准数据集上的大量实验表明，我们提出的模型优于最先进的方法。摘要：Achieving high-performance audio denoising is still a challenging task in real-world applications. Existing time-frequency methods often ignore the quality of generated frequency domain images. This paper converts the audio denoising problem into an image generation task. We first develop a complex image generation SwinTransformer network to capture more information from the complex Fourier domain. We then impose structure similarity and detailed loss functions to generate high-quality images and develop an SDR loss to minimize the difference between denoised and clean audios. Extensive experiments on two benchmark datasets demonstrate that our proposed model is better than state-of-the-art methods.

【7】 A Novel Approach for Object Based Audio Broadcasting标题：一种新的基于对象的音频广播方法链接：https://arxiv.org/abs/2310.16481

作者：Mohammad Reza Hasanabadi备注：Accepted in ABU Technical Review Journal 2020/9摘要：基于对象的音频（OBA）提供了一种新的音频体验，交付给观众个性化和定制他们的收听体验，并让他们选择听什么和如何听他们的音频内容。OBA可以应用于不同的平台，如广播，流媒体和电影声音。本文提出了一种新的方法，用于创建基于对象的音频生产端。这里的方法提出了基于音频（SSOBA）的逐样本对象嵌入。SSOBA以这样一种方式放置音频对象样本，允许观众根据他们的兴趣和需求轻松地个性化他们选择的音频源。SSOBA是一项额外的服务，而不是替代品，因此它也符合传统的音频播放器。SSOBA的最大优点是它不需要在广播链中添加任何特殊的额外硬件，因此很容易实现并为传统播放器和解码器配备增强功能。输入音频对象、输出通道数和采样率是影响SSOBA性能并指定其为无损或有损的三个重要因素。SSOBA在解码器端采用插值来补偿被消除的样本。主观和客观的实验进行评估的输出结果在每一步。编码步骤后进行的MUSHRA主观实验显示，SSOBA具有高达5个对象的良好质量性能。信噪比测量和客观的实验，解码和插值后进行，显示出显着的成功恢复和分离的音频对象。实验结果表明，在一个立体声通道中，最低采样率为96 kHz时，最多可编码5个目标，同时获得良好的主观和客观效果。摘要：Object Based Audio (OBA) provides a new kind of audio experience, delivered to the audience to personalize and customize their experience of listening and to give them choice of what and how to hear their audio content. OBA can be applied to different platforms such as broadcasting, streaming and cinema sound. This paper presents a novel approach for creating object-based audio on the production side. The approach here presents Sample-by-Sample Object Based Audio (SSOBA) embedding. SSOBA places audio object samples in such a way that allows audiences to easily individualize their chosen audio sources according to their interests and needs. SSOBA is an extra service and not an alternative, so it is also compliant with legacy audio players. The biggest advantage of SSOBA is that it does not require any special additional hardware in the broadcasting chain and it is therefore easy to implement and equip legacy players and decoders with enhanced ability. Input audio objects, number of output channels and sampling rates are three important factors affecting SSOBA performance and specifying it to be lossless or lossy. SSOBA adopts interpolation at the decoder side to compensate for eliminated samples. Both subjective and objective experiments are carried out to evaluate the output results at each step. MUSHRA subjective experiments conducted after the encoding step shows good-quality performance of SSOBA with up to five objects. SNR measurements and objective experiments, performed after decoding and interpolation, show significant successful recovery and separation of audio objects. Experimental results show that a minimum sampling rate of 96 kHz is indicated to encode up to five objects in a Stereo-mode channel to acquire good subjective and objective results simultaneously.

【8】 Generative Pre-training for Speech with Flow Matching标题：基于流匹配的语音生成性预训练链接：https://arxiv.org/abs/2310.16338

作者：Alexander H. Liu,Matt Le,Apoorv Vyas,Bowen Shi,Andros Tjandra,Wei-Ning Hsu备注：Preprint, under review摘要：近年来，生成式模型因其在需要估计和采样数据分布以生成高保真合成数据的任务中的显着成功而受到越来越多的关注。在语音方面，文本到语音合成和神经声码器是生成模型大放异彩的好例子。虽然生成模型已被应用于语音中的不同应用，但不存在直接对语音进行建模的通用生成模型。在这项工作中，我们朝着这个方向迈出了一步，展示了一个预先训练的生成模型可以适应不同的下游任务，并具有强大的性能。具体来说，我们在60k小时的未转录语音上预训练了一个名为SpeechFlow的生成模型，其中包含Flow Matching和掩蔽条件。实验结果表明，预训练的生成模型可以与特定任务的数据进行微调，以匹配或超过现有的专家模型的语音增强，分离和合成。我们的工作表明，可以通过生成预训练建立语音生成任务的基础模型。摘要：Generative models have gained more and more attention in recent years for their remarkable success in tasks that required estimating and sampling data distribution to generate high-fidelity synthetic data. In speech, text-to-speech synthesis and neural vocoder are good examples where generative models have shined. While generative models have been applied to different applications in speech, there exists no general-purpose generative model that models speech directly. In this work, we take a step toward this direction by showing a single pre-trained generative model can be adapted to different downstream tasks with strong performance. Specifically, we pre-trained a generative model, named SpeechFlow, on 60k hours of untranscribed speech with Flow Matching and masked conditions. Experiment results show the pre-trained generative model can be fine-tuned with task-specific data to match or surpass existing expert models on speech enhancement, separation, and synthesis. Our work suggested a foundational model for generation tasks in speech can be built with generative pre-training.

【9】 IA Para el Mantenimiento Predictivo en Canteras: Modelado标题：A Para el Mantenimiento Predictivo en Canteras：Modelado链接：https://arxiv.org/abs/2310.16140

作者：Fernando Marcos,Rodrigo Tamaki,Mateo Cámara,Virginia Yagüe,José Luis Blanco备注：10 pages, in Spanish language, 5 figures. Presented in Tecniacustica 2023 conference (Cuenca, Spain)摘要：对原材料的依赖，特别是在采矿业，是当今经济的一个关键部分。骨料至关重要，是仅次于水的第二大使用原材料。数字化转型是优化运营的关键。然而，由于该行业、机械和环境条件的特殊性，监督和维护（预测性和纠正性）是该行业很少探讨的挑战。所有这一切，尽管在其他情况下取得了成功，在监测与声学和接触传感器。我们提出了一个无监督的学习计划，训练一组录音的变分自动编码器模型。这是在加工厂操作期间收集的第一个此类数据集，包含来自加工线不同点的信息。我们的研究结果表明，该模型的能力，重建和代表在潜在的空间记录的声音，在操作条件和不同的设备之间的差异。今后，这将有助于对声音进行分类，并有助于检测机器运行中的异常情况和退化模式。摘要：Dependence on raw materials, especially in the mining sector, is a key part of today's economy. Aggregates are vital, being the second most used raw material after water. Digitally transforming this sector is key to optimizing operations. However, supervision and maintenance (predictive and corrective) are challenges little explored in this sector, due to the particularities of the sector, machinery and environmental conditions. All this, despite the successes achieved in other scenarios in monitoring with acoustic and contact sensors. We present an unsupervised learning scheme that trains a variational autoencoder model on a set of sound records. This is the first such dataset collected during processing plant operations, containing information from different points of the processing line. Our results demonstrate the model's ability to reconstruct and represent in latent space the recorded sounds, the differences in operating conditions and between different equipment. In the future, this should facilitate the classification of sounds, as well as the detection of anomalies and degradation patterns in the operation of the machinery.

eess.AS音频处理

【1】 A Novel Approach for Object Based Audio Broadcasting标题：一种新的基于对象的音频广播方法链接：https://arxiv.org/abs/2310.16481

【2】 UniX-Encoder: A Universal $X$-Channel Speech Encoder for Ad-Hoc Microphone Array Speech Processing标题：Unix编码器：一种用于自组织麦克风阵列语音处理的通用$X$通道语音编码器链接：https://arxiv.org/abs/2310.16367

作者：Zili Huang,Yiwen Shao,Shi-Xiong Zhang,Dong Yu备注：Submitted to ICASSP 2024摘要：None摘要：The speech field is evolving to solve more challenging scenarios, such as multi-channel recordings with multiple simultaneous talkers. Given the many types of microphone setups out there, we present the UniX-Encoder. It's a universal encoder designed for multiple tasks, and worked with any microphone array, in both solo and multi-talker environments. Our research enhances previous multi-channel speech processing efforts in four key areas: 1) Adaptability: Contrasting traditional models constrained to certain microphone array configurations, our encoder is universally compatible. 2) Multi-Task Capability: Beyond the single-task focus of previous systems, UniX-Encoder acts as a robust upstream model, adeptly extracting features for diverse tasks including ASR and speaker recognition. 3) Self-Supervised Training: The encoder is trained without requiring labeled multi-channel data. 4) End-to-End Integration: In contrast to models that first beamform then process single-channels, our encoder offers an end-to-end solution, bypassing explicit beamforming or separation. To validate its effectiveness, we tested the UniX-Encoder on a synthetic multi-channel dataset from the LibriSpeech corpus. Across tasks like speech recognition and speaker diarization, our encoder consistently outperformed combinations like the WavLM model with the BeamformIt frontend.

【3】 Generative Pre-training for Speech with Flow Matching标题：基于流匹配的语音生成性预训练链接：https://arxiv.org/abs/2310.16338

作者：Alexander H. Liu,Matt Le,Apoorv Vyas,Bowen Shi,Andros Tjandra,Wei-Ning Hsu备注：Preprint, under review摘要：None摘要：Generative models have gained more and more attention in recent years for their remarkable success in tasks that required estimating and sampling data distribution to generate high-fidelity synthetic data. In speech, text-to-speech synthesis and neural vocoder are good examples where generative models have shined. While generative models have been applied to different applications in speech, there exists no general-purpose generative model that models speech directly. In this work, we take a step toward this direction by showing a single pre-trained generative model can be adapted to different downstream tasks with strong performance. Specifically, we pre-trained a generative model, named SpeechFlow, on 60k hours of untranscribed speech with Flow Matching and masked conditions. Experiment results show the pre-trained generative model can be fine-tuned with task-specific data to match or surpass existing expert models on speech enhancement, separation, and synthesis. Our work suggested a foundational model for generation tasks in speech can be built with generative pre-training.

【4】 Covariance Blocking and Whitening Method for Successive Relative Transfer Function Vector Estimation in Multi-Speaker Scenarios标题：多说话人场景下逐次相对传递函数向量估计的协方差分块白化方法链接：https://arxiv.org/abs/2310.16327

作者：Henri Gode,Simon Doclo备注：IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), New Paltz, NY, USA, Oct 22-25, 2023摘要：本文讨论了在嘈杂和混响环境中估计多个扬声器的相对传递函数（RTF）向量的挑战。更具体地说，我们考虑两个扬声器连续激活的情况。在这种情况下，可以以直接的方式估计第一说话者的RTF向量，并且主要挑战在于在两个说话者同时活跃的片段期间估计第二说话者的RTF向量。为了估计第二说话者的RTF向量，所谓的盲斜投影（BOP）方法确定最佳地阻挡第二说话者的斜投影算子。在本文中，我们提出了一种协方差阻塞和白化（CBW）的方法，而不是阻止第二扬声器，它首先块第一扬声器和应用白化使用估计的噪声协方差矩阵，然后估计的RTF向量的第二扬声器的基础上的奇异值分解。当在线性约束最小方差波束形成器中使用两个扬声器的估计RTF向量时，使用多个扬声器位置的真实世界记录的仿真结果表明，所提出的CBW方法在信噪比改善方面优于传统的BOP和协方差白化方法。摘要：This paper addresses the challenge of estimating the relative transfer function (RTF) vectors of multiple speakers in a noisy and reverberant environment. More specifically, we consider a scenario where two speakers activate successively. In this scenario, the RTF vector of the first speaker can be estimated in a straightforward way and the main challenge lies in estimating the RTF vector of the second speaker during segments where both speakers are simultaneously active. To estimate the RTF vector of the second speaker the so-called blind oblique projection (BOP) method determines the oblique projection operator that optimally blocks the second speaker. Instead of blocking the second speaker, in this paper we propose a covariance blocking and whitening (CBW) method, which first blocks the first speaker and applies whitening using the estimated noise covariance matrix and then estimates the RTF vector of the second speaker based on a singular value decomposition. When using the estimated RTF vectors of both speakers in a linearly constrained minimum variance beamformer, simulation results using real-world recordings for multiple speaker positions demonstrate that the proposed CBW method outperforms the conventional BOP and covariance whitening methods in terms of signal-to-interferer-and-noise ratio improvement.

【5】 IA Para el Mantenimiento Predictivo en Canteras: Modelado标题：A Para el Mantenimiento Predictivo en Canteras：Modelado链接：https://arxiv.org/abs/2310.16140

作者：Fernando Marcos,Rodrigo Tamaki,Mateo Cámara,Virginia Yagüe,José Luis Blanco备注：10 pages, in Spanish language, 5 figures. Presented in Tecniacustica 2023 conference (Cuenca, Spain)摘要：None摘要：Dependence on raw materials, especially in the mining sector, is a key part of today's economy. Aggregates are vital, being the second most used raw material after water. Digitally transforming this sector is key to optimizing operations. However, supervision and maintenance (predictive and corrective) are challenges little explored in this sector, due to the particularities of the sector, machinery and environmental conditions. All this, despite the successes achieved in other scenarios in monitoring with acoustic and contact sensors. We present an unsupervised learning scheme that trains a variational autoencoder model on a set of sound records. This is the first such dataset collected during processing plant operations, containing information from different points of the processing line. Our results demonstrate the model's ability to reconstruct and represent in latent space the recorded sounds, the differences in operating conditions and between different equipment. In the future, this should facilitate the classification of sounds, as well as the detection of anomalies and degradation patterns in the operation of the machinery.

【6】 ArTST: Arabic Text and Speech Transformer标题：Artst：阿拉伯语文本和语音转换器链接：https://arxiv.org/abs/2310.16621

【7】 Back Transcription as a Method for Evaluating Robustness of Natural Language Understanding Models to Speech Recognition Errors标题：反转录作为一种评估自然语言理解模型对语音识别错误稳健性的方法链接：https://arxiv.org/abs/2310.16609

【8】 Dynamic Processing Neural Network Architecture For Hearing Loss Compensation标题：听力损失补偿的动态处理神经网络结构链接：https://arxiv.org/abs/2310.16550

作者：Szymon Drgas,Lars Bramsløw,Archontis Politis,Gaurav Naithani,Tuomas Virtanen摘要：None摘要：This paper proposes neural networks for compensating sensorineural hearing loss. The aim of the hearing loss compensation task is to transform a speech signal to increase speech intelligibility after further processing by a person with a hearing impairment, which is modeled by a hearing loss model. We propose an interpretable model called dynamic processing network, which has a structure similar to band-wise dynamic compressor. The network is differentiable, and therefore allows to learn its parameters to maximize speech intelligibility. More generic models based on convolutional layers were tested as well. The performance of the tested architectures was assessed using spectro-temporal objective index (STOI) with hearing-threshold noise and hearing aid speech intelligibility (HASPI) metrics. The dynamic processing network gave a significant improvement of STOI and HASPI in comparison to popular compressive gain prescription rule Camfit. A large enough convolutional network could outperform the interpretable model with the cost of larger computational load. Finally, a combination of the dynamic processing network with convolutional neural network gave the best results in terms of STOI and HASPI.

【9】 AccoMontage-3: Full-Band Accompaniment Arrangement via Sequential Style Transfer and Multi-Track Function Prior标题：Acco蒙太奇-3：全频段伴奏编排通过顺序风格转换和多声道功能优先链接：https://arxiv.org/abs/2310.16334

【10】 Towards Streaming Speech-to-Avatar Synthesis标题：走向流媒体语音到头像的合成链接：https://arxiv.org/abs/2310.16287

【11】 Complex Image Generation SwinTransformer Network for Audio Denoising标题：用于音频去噪的复数图像生成SwinTransformer网络链接：https://arxiv.org/abs/2310.16109

作者：Youshan Zhang,Jialu Li摘要：None摘要：Achieving high-performance audio denoising is still a challenging task in real-world applications. Existing time-frequency methods often ignore the quality of generated frequency domain images. This paper converts the audio denoising problem into an image generation task. We first develop a complex image generation SwinTransformer network to capture more information from the complex Fourier domain. We then impose structure similarity and detailed loss functions to generate high-quality images and develop an SDR loss to minimize the difference between denoised and clean audios. Extensive experiments on two benchmark datasets demonstrate that our proposed model is better than state-of-the-art methods.

机器翻译，仅供参考

语音/音频处理学术速递[10.26]

评论 0