今日论文合集：cs.SD语音3篇，eess.AS音频处理3篇。
cs.SD语音
【1】 Music De-limiter Networks via Sample-wise Gain Inversion标题：基于采样增益倒置的音乐去限器网络链接：https://arxiv.org/abs/2308.01187
作者：Chang-Bin Jeon,Kyogu Lee备注：Accepted to IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) 2023摘要：响度之战是音乐产业中的一种持续现象，其特征在于增加音乐的最终响度同时减小其动态范围，几十年来一直是一个有争议的话题。音乐控制工程师使用限制器来严重压缩音乐并使其更响亮，这可能会导致耳朵疲劳和听力损失。在本文中，我们介绍了音乐去限幅器网络，估计未经压缩的音乐，从严重压缩的信号。受限幅器原理的启发，我们提出了一个采样增益反转（SGI）的框架，该限幅器对给定信号进行采样增益降低。我们还介绍了musdb-XL-train数据集，由300 k段组成，这些段是通过应用商业限制器插件来训练真实世界友好的去限制器网络而创建的。我们提出的去限幅器网络实现了出色的性能与规模不变的源失真比（SI-SDR）为23.8 dB的musdb-HQ重建musdb-XL数据，musdb-HQ的限制器应用版本。训练数据、代码和模型权重可以在我们的存储库中找到（https：github.com jeonchangbin49 De-limiter）。摘要：The loudness war, an ongoing phenomenon in the music industry characterized by the increasing final loudness of music while reducing its dynamic range, has been a controversial topic for decades. Music mastering engineers have used limiters to heavily compress and make music louder, which can induce ear fatigue and hearing loss in listeners. In this paper, we introduce music de-limiter networks that estimate uncompressed music from heavily compressed signals. Inspired by the principle of a limiter, which performs sample-wise gain reduction of a given signal, we propose the framework of sample-wise gain inversion (SGI). We also present the musdb-XL-train dataset, consisting of 300k segments created by applying a commercial limiter plug-in for training real-world friendly de-limiter networks. Our proposed de-limiter network achieves excellent performance with a scale-invariant source-to-distortion ratio (SI-SDR) of 23.8 dB in reconstructing musdb-HQ from musdb- XL data, a limiter-applied version of musdb-HQ. The training data, codes, and model weights are available in our repository (https: github.com jeonchangbin49 De-limiter).
【2】 Inaudible Adversarial Perturbation: Manipulating the Recognition of User  Speech in Real Tim标题：听不见的对抗性扰动：实时操纵用户语音识别链接：https://arxiv.org/abs/2308.01040
作者：Xinfeng Li,Chen Yan,Xuancun Lu,Zihan Zeng,Xiaoyu Ji,Wenyuan Xu备注：Accepted by NDSS Symposium 2024摘要：自动语音识别（ASR）系统已被证明易受对抗性示例（AE）的影响。最近的成功都假设用户不会注意到或破坏攻击过程，尽管存在音乐 类似噪音的声音和来自语音助手的自发响应。尽管如此，在实际的用户存在的场景中，用户意识可以使发起意外声音或ASR使用的现有攻击尝试无效。在本文中，我们寻求弥合现有研究的差距，并将攻击扩展到用户存在的情况下。我们提出了VRIFLE，一种听不见的对抗性扰动（IAP）攻击，通过超声递送，可以操纵ASR作为用户说话。可听声音和超声波之间的固有差异使得IAP递送面临前所未有的挑战，例如失真、噪声和不稳定性。在这方面，我们设计了一个新的超声转换模型，以增强精心制作的扰动是物理有效的，甚至生存的长距离递送。我们进一步使VRIFLE的鲁棒性，通过在生成过程中采用一系列增强用户和现实世界的变化。通过这种方式，VRIFLE具有从不同距离和用户的任何语音下的ASR输出的有效实时操纵，具有抑制用户中断影响的交替和静音策略。我们在数字和物理世界中的广泛实验验证了VRIFLE在各种配置下的有效性，对六种防御的鲁棒性，以及有针对性的通用性。我们还表明，VRIFLE可以提供便携式攻击设备，甚至日常生活扬声器。摘要：Automatic speech recognition (ASR) systems have been shown to be vulnerable to adversarial examples (AEs). Recent success all assumes that users will not notice or disrupt the attack process despite the existence of music noise-like sounds and spontaneous responses from voice assistants. Nonetheless, in practical user-present scenarios, user awareness may nullify existing attack attempts that launch unexpected sounds or ASR usage. In this paper, we seek to bridge the gap in existing research and extend the attack to user-present scenarios. We propose VRIFLE, an inaudible adversarial perturbation (IAP) attack via ultrasound delivery that can manipulate ASRs as a user speaks. The inherent differences between audible sounds and ultrasounds make IAP delivery face unprecedented challenges such as distortion, noise, and instability. In this regard, we design a novel ultrasonic transformation model to enhance the crafted perturbation to be physically effective and even survive long-distance delivery. We further enable VRIFLE's robustness by adopting a series of augmentation on user and real-world variations during the generation process. In this way, VRIFLE features an effective real-time manipulation of the ASR output from different distances and under any speech of users, with an alter-and-mute strategy that suppresses the impact of user disruption. Our extensive experiments in both digital and physical worlds verify VRIFLE's effectiveness under various configurations, robustness against six kinds of defenses, and universality in a targeted manner. We also show that VRIFLE can be delivered with a portable attack device and even everyday-life loudspeakers.
【3】 SALTTS: Leveraging Self-Supervised Speech Representations for improved  Text-to-Speech Synthesis标题：SALTTS：利用自监督语音表示法改进文本到语音合成链接：https://arxiv.org/abs/2308.01018
作者：Ramanan Sivaguru,Vasista Sai Lodagala,S Umesh备注：Accepted for publication at Interspeech 2023摘要：虽然FastSpeech2的目标是将语音的各个方面，如音高，能量和持续时间作为条件输入，但它仍然为更丰富的表示留下了空间。作为这项工作的一部分，我们利用各种自我监督学习（SSL）模型的表示，以提高合成语音的质量。特别是，我们通过一系列的编码器层的FastSpeech2编码器的长度调节的输出与重建的SSL表示的目标。在SALTTS并行实现中，来自该第二编码器的表示用于具有SSL特征的辅助重构损失。然而，SALTTS级联实现除了具有重构损失之外还使这些表示通过解码器。从SSL功能的语音特征的丰富性反映在输出语音质量，与所提出的方法的客观和主观评价措施优于基线FastSpeech2。摘要：While FastSpeech2 aims to integrate aspects of speech such as pitch, energy, and duration as conditional inputs, it still leaves scope for richer representations. As a part of this work, we leverage representations from various Self-Supervised Learning (SSL) models to enhance the quality of the synthesized speech. In particular, we pass the FastSpeech2 encoder's length-regulated outputs through a series of encoder layers with the objective of reconstructing the SSL representations. In the SALTTS-parallel implementation, the representations from this second encoder are used for an auxiliary reconstruction loss with the SSL features. The SALTTS-cascade implementation, however, passes these representations through the decoder in addition to having the reconstruction loss. The richness of speech characteristics from the SSL features reflects in the output speech quality, with the objective and subjective evaluation measures of the proposed approach outperforming the baseline FastSpeech2.
eess.AS音频处理
【1】 Music De-limiter Networks via Sample-wise Gain Inversion标题：基于采样增益倒置的音乐去限器网络链接：https://arxiv.org/abs/2308.01187
作者：Chang-Bin Jeon,Kyogu Lee备注：Accepted to IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) 2023摘要：响度之战是音乐产业中的一种持续现象，其特征在于增加音乐的最终响度同时减小其动态范围，几十年来一直是一个有争议的话题。音乐控制工程师使用限制器来严重压缩音乐并使其更响亮，这可能会导致耳朵疲劳和听力损失。在本文中，我们介绍了音乐去限幅器网络，估计未经压缩的音乐，从严重压缩的信号。受限幅器原理的启发，我们提出了一个采样增益反转（SGI）的框架，该限幅器对给定信号进行采样增益降低。我们还介绍了musdb-XL-train数据集，由300 k段组成，这些段是通过应用商业限制器插件来训练真实世界友好的去限制器网络而创建的。我们提出的去限幅器网络实现了出色的性能与规模不变的源失真比（SI-SDR）为23.8 dB的musdb-HQ重建musdb-XL数据，musdb-HQ的限制器应用版本。训练数据、代码和模型权重可以在我们的存储库中找到（https：github.com jeonchangbin49 De-limiter）。摘要：The loudness war, an ongoing phenomenon in the music industry characterized by the increasing final loudness of music while reducing its dynamic range, has been a controversial topic for decades. Music mastering engineers have used limiters to heavily compress and make music louder, which can induce ear fatigue and hearing loss in listeners. In this paper, we introduce music de-limiter networks that estimate uncompressed music from heavily compressed signals. Inspired by the principle of a limiter, which performs sample-wise gain reduction of a given signal, we propose the framework of sample-wise gain inversion (SGI). We also present the musdb-XL-train dataset, consisting of 300k segments created by applying a commercial limiter plug-in for training real-world friendly de-limiter networks. Our proposed de-limiter network achieves excellent performance with a scale-invariant source-to-distortion ratio (SI-SDR) of 23.8 dB in reconstructing musdb-HQ from musdb- XL data, a limiter-applied version of musdb-HQ. The training data, codes, and model weights are available in our repository (https: github.com jeonchangbin49 De-limiter).
【2】 Inaudible Adversarial Perturbation: Manipulating the Recognition of User  Speech in Real Tim标题：听不见的对抗性扰动：实时操纵用户语音识别链接：https://arxiv.org/abs/2308.01040
作者：Xinfeng Li,Chen Yan,Xuancun Lu,Zihan Zeng,Xiaoyu Ji,Wenyuan Xu备注：Accepted by NDSS Symposium 2024摘要：自动语音识别（ASR）系统已被证明易受对抗性示例（AE）的影响。最近的成功都假设用户不会注意到或破坏攻击过程，尽管存在音乐 类似噪音的声音和来自语音助手的自发响应。尽管如此，在实际的用户存在的场景中，用户意识可以使发起意外声音或ASR使用的现有攻击尝试无效。在本文中，我们寻求弥合现有研究的差距，并将攻击扩展到用户存在的情况下。我们提出了VRIFLE，一种听不见的对抗性扰动（IAP）攻击，通过超声递送，可以操纵ASR作为用户说话。可听声音和超声波之间的固有差异使得IAP递送面临前所未有的挑战，例如失真、噪声和不稳定性。在这方面，我们设计了一个新的超声转换模型，以增强精心制作的扰动是物理有效的，甚至生存的长距离递送。我们进一步使VRIFLE的鲁棒性，通过在生成过程中采用一系列增强用户和现实世界的变化。通过这种方式，VRIFLE具有从不同距离和用户的任何语音下的ASR输出的有效实时操纵，具有抑制用户中断影响的交替和静音策略。我们在数字和物理世界中的广泛实验验证了VRIFLE在各种配置下的有效性，对六种防御的鲁棒性，以及有针对性的通用性。我们还表明，VRIFLE可以提供便携式攻击设备，甚至日常生活扬声器。摘要：Automatic speech recognition (ASR) systems have been shown to be vulnerable to adversarial examples (AEs). Recent success all assumes that users will not notice or disrupt the attack process despite the existence of music noise-like sounds and spontaneous responses from voice assistants. Nonetheless, in practical user-present scenarios, user awareness may nullify existing attack attempts that launch unexpected sounds or ASR usage. In this paper, we seek to bridge the gap in existing research and extend the attack to user-present scenarios. We propose VRIFLE, an inaudible adversarial perturbation (IAP) attack via ultrasound delivery that can manipulate ASRs as a user speaks. The inherent differences between audible sounds and ultrasounds make IAP delivery face unprecedented challenges such as distortion, noise, and instability. In this regard, we design a novel ultrasonic transformation model to enhance the crafted perturbation to be physically effective and even survive long-distance delivery. We further enable VRIFLE's robustness by adopting a series of augmentation on user and real-world variations during the generation process. In this way, VRIFLE features an effective real-time manipulation of the ASR output from different distances and under any speech of users, with an alter-and-mute strategy that suppresses the impact of user disruption. Our extensive experiments in both digital and physical worlds verify VRIFLE's effectiveness under various configurations, robustness against six kinds of defenses, and universality in a targeted manner. We also show that VRIFLE can be delivered with a portable attack device and even everyday-life loudspeakers.
【3】 SALTTS: Leveraging Self-Supervised Speech Representations for improved  Text-to-Speech Synthesis标题：SALTTS：利用自监督语音表示法改进文本到语音合成链接：https://arxiv.org/abs/2308.01018
作者：Ramanan Sivaguru,Vasista Sai Lodagala,S Umesh备注：Accepted for publication at Interspeech 2023摘要：虽然FastSpeech2的目标是将语音的各个方面，如音高，能量和持续时间作为条件输入，但它仍然为更丰富的表示留下了空间。作为这项工作的一部分，我们利用各种自我监督学习（SSL）模型的表示，以提高合成语音的质量。特别是，我们通过一系列的编码器层的FastSpeech2编码器的长度调节的输出与重建的SSL表示的目标。在SALTTS并行实现中，来自该第二编码器的表示用于具有SSL特征的辅助重构损失。然而，SALTTS级联实现除了具有重构损失之外还使这些表示通过解码器。从SSL功能的语音特征的丰富性反映在输出语音质量，与所提出的方法的客观和主观评价措施优于基线FastSpeech2。摘要：While FastSpeech2 aims to integrate aspects of speech such as pitch, energy, and duration as conditional inputs, it still leaves scope for richer representations. As a part of this work, we leverage representations from various Self-Supervised Learning (SSL) models to enhance the quality of the synthesized speech. In particular, we pass the FastSpeech2 encoder's length-regulated outputs through a series of encoder layers with the objective of reconstructing the SSL representations. In the SALTTS-parallel implementation, the representations from this second encoder are used for an auxiliary reconstruction loss with the SSL features. The SALTTS-cascade implementation, however, passes these representations through the decoder in addition to having the reconstruction loss. The richness of speech characteristics from the SSL features reflects in the output speech quality, with the objective and subjective evaluation measures of the proposed approach outperforming the baseline FastSpeech2.
机器翻译，仅供参考