今日论文合集：cs.SD语音2篇，eess.AS音频处理2篇。
cs.SD语音

【1】Decouple Non-parametric Knowledge Distillation For End-to-end Speech Translation标题：端到端语音翻译中的解耦非参数知识提取链接：http://arxiv.org/pdf/2304.10295v1

作者：Hao Zhang,Nianwen Si,Yaqi Chen,Wenlin Zhang,Xukui Yang,Dan Qu,Zhen Li机构：University of Information Engineering, Zhengzhou, China, Department of Electronic Engineering, Tsinghua University, Beijing, China备注：Accepted by ICASSP 2023摘要：现有技术通常尝试使用一些精细的技术将知识从强大的机器翻译（MT）转移到语音翻译（ST）模型，这通常需要在训练期间将转录作为额外的输入。然而，转录并不总是可用的，并且如何在没有转录的情况下改善ST模型性能，即，数据效率，在文献中很少被研究。在本文中，我们提出了解耦的非参数知识蒸馏（DNKD）从数据的角度来提高数据的效率。我们的方法遵循知识蒸馏范式。然而，我们不是从复杂的MT模型中获得教师分布，而是通过k最近邻（kNN）检索从非参数数据库中构建它，这消除了对转录和MT模型的依赖。然后将经典知识蒸馏损失解耦为目标蒸馏和非目标蒸馏，以增强非目标logits中知识的作用，即突出的“暗知识”。在MuST-C语料库上的实验表明，该方法在不需要任何转录的情况下，能够在强基线上实现一致的改进。摘要：Existing techniques often attempt to make knowledge transfer from a powerful machine translation (MT) to speech translation (ST) model with some elaborate techniques, which often requires transcription as extra input during training. However, transcriptions are not always available, and how to improve the ST model performance without transcription, i.e., data efficiency, has rarely been studied in the literature. In this paper, we propose Decoupled Non-parametric Knowledge Distillation (DNKD) from data perspective to improve the data efficiency. Our method follows the knowledge distillation paradigm. However, instead of obtaining the teacher distribution from a sophisticated MT model, we construct it from a non-parametric datastore via k-Nearest-Neighbor (kNN) retrieval, which removes the dependence on transcription and MT model. Then we decouple the classic knowledge distillation loss into target and non-target distillation to enhance the effect of the knowledge among non-target logits, which is the prominent "dark knowledge". Experiments on MuST-C corpus show that, the proposed method can achieve consistent improvement over the strong baseline without requiring any transcription.

【2】Towards the Universal Defense for Query-Based Audio Adversarial Attacks标题：面向基于查询的音频攻击的通用防御链接：http://arxiv.org/pdf/2304.10088v1

作者：Feng Guo,Zheng Sun,Yuxuan Chen,Lei Ju机构：cn 1School of Cyber Science and Technology, Shandong University备注：Submitted to Cybersecurity journal摘要：最近，研究表明，基于深度学习的自动语音识别（ASR）系统容易受到对抗性示例（AE）的影响，这些示例会在原始音频示例中添加少量噪声。这些AE攻击给深度学习安全带来了新的挑战，并引发了对部署ASR系统和设备的重大担忧。现有的抗辩方法要么适用范围有限，要么只从结果上抗辩，而不从过程上抗辩。在这项工作中，我们提出了一种新的方法来推断对手的意图和发现音频对抗的例子的基础上的AE生成过程。该方法的见解基于以下观察：许多现有的音频AE攻击利用基于查询的方法，这意味着对手必须在音频AE生成过程期间向目标ASR模型发送连续且类似的查询。受此启发，本文提出了一种记忆机制，采用音频指纹技术分析当前查询与一定长度的记忆查询的相似度。因此，我们可以识别何时查询序列看起来可怀疑生成音频AE。通过对四种最先进的音频AE攻击的广泛评估，我们证明，平均而言，我们的防御识别对手意图的准确率超过90%。仔细考虑的鲁棒性评估，我们还分析了我们提出的防御和它的实力，以抵御两个自适应攻击。最后，我们的计划是现成的，并直接兼容任何合奏的ASR防御模型，以发现音频AE攻击有效地没有模型再训练。摘要：Recently, studies show that deep learning-based automatic speech recognition (ASR) systems are vulnerable to adversarial examples (AEs), which add a small amount of noise to the original audio examples. These AE attacks pose new challenges to deep learning security and have raised significant concerns about deploying ASR systems and devices. The existing defense methods are either limited in application or only defend on results, but not on process. In this work, we propose a novel method to infer the adversary intent and discover audio adversarial examples based on the AEs generation process. The insight of this method is based on the observation: many existing audio AE attacks utilize query-based methods, which means the adversary must send continuous and similar queries to target ASR models during the audio AE generation process. Inspired by this observation, We propose a memory mechanism by adopting audio fingerprint technology to analyze the similarity of the current query with a certain length of memory query. Thus, we can identify when a sequence of queries appears to be suspectable to generate audio AEs. Through extensive evaluation on four state-of-the-art audio AE attacks, we demonstrate that on average our defense identify the adversary intent with over 90% accuracy. With careful regard for robustness evaluations, we also analyze our proposed defense and its strength to withstand two adaptive attacks. Finally, our scheme is available out-of-the-box and directly compatible with any ensemble of ASR defense models to uncover audio AE attacks effectively without model retraining.
eess.AS音频处理

【1】Decouple Non-parametric Knowledge Distillation For End-to-end Speech Translation标题：端到端语音翻译中的解耦非参数知识提取链接：http://arxiv.org/pdf/2304.10295v1

*与cs.SD语音【1】为同一篇

作者：Hao Zhang,Nianwen Si,Yaqi Chen,Wenlin Zhang,Xukui Yang,Dan Qu,Zhen Li机构：University of Information Engineering, Zhengzhou, China, Department of Electronic Engineering, Tsinghua University, Beijing, China备注：Accepted by ICASSP 2023摘要：现有技术通常尝试使用一些精细的技术将知识从强大的机器翻译（MT）转移到语音翻译（ST）模型，这通常需要在训练期间将转录作为额外的输入。然而，转录并不总是可用的，并且如何在没有转录的情况下改善ST模型性能，即，数据效率，在文献中很少被研究。在本文中，我们提出了解耦的非参数知识蒸馏（DNKD）从数据的角度来提高数据的效率。我们的方法遵循知识蒸馏范式。然而，我们不是从复杂的MT模型中获得教师分布，而是通过k最近邻（kNN）检索从非参数数据库中构建它，这消除了对转录和MT模型的依赖。然后将经典知识蒸馏损失解耦为目标蒸馏和非目标蒸馏，以增强非目标logits中知识的作用，即突出的“暗知识”。在MuST-C语料库上的实验表明，该方法在不需要任何转录的情况下，能够在强基线上实现一致的改进。摘要：Existing techniques often attempt to make knowledge transfer from a powerful machine translation (MT) to speech translation (ST) model with some elaborate techniques, which often requires transcription as extra input during training. However, transcriptions are not always available, and how to improve the ST model performance without transcription, i.e., data efficiency, has rarely been studied in the literature. In this paper, we propose Decoupled Non-parametric Knowledge Distillation (DNKD) from data perspective to improve the data efficiency. Our method follows the knowledge distillation paradigm. However, instead of obtaining the teacher distribution from a sophisticated MT model, we construct it from a non-parametric datastore via k-Nearest-Neighbor (kNN) retrieval, which removes the dependence on transcription and MT model. Then we decouple the classic knowledge distillation loss into target and non-target distillation to enhance the effect of the knowledge among non-target logits, which is the prominent "dark knowledge". Experiments on MuST-C corpus show that, the proposed method can achieve consistent improvement over the strong baseline without requiring any transcription.

【2】Towards the Universal Defense for Query-Based Audio Adversarial Attacks标题：面向基于查询的音频攻击的通用防御链接：http://arxiv.org/pdf/2304.10088v1

*与cs.SD语音【2】为同一篇

作者：Feng Guo,Zheng Sun,Yuxuan Chen,Lei Ju机构：cn 1School of Cyber Science and Technology, Shandong University备注：Submitted to Cybersecurity journal摘要：最近，研究表明，基于深度学习的自动语音识别（ASR）系统容易受到对抗性示例（AE）的影响，这些示例会在原始音频示例中添加少量噪声。这些AE攻击给深度学习安全带来了新的挑战，并引发了对部署ASR系统和设备的重大担忧。现有的抗辩方法要么适用范围有限，要么只从结果上抗辩，而不从过程上抗辩。在这项工作中，我们提出了一种新的方法来推断对手的意图和发现音频对抗的例子的基础上的AE生成过程。该方法的见解基于以下观察：许多现有的音频AE攻击利用基于查询的方法，这意味着对手必须在音频AE生成过程期间向目标ASR模型发送连续且类似的查询。受此启发，本文提出了一种记忆机制，采用音频指纹技术分析当前查询与一定长度的记忆查询的相似度。因此，我们可以识别何时查询序列看起来可怀疑生成音频AE。通过对四种最先进的音频AE攻击的广泛评估，我们证明，平均而言，我们的防御识别对手意图的准确率超过90%。仔细考虑的鲁棒性评估，我们还分析了我们提出的防御和它的实力，以抵御两个自适应攻击。最后，我们的计划是现成的，并直接兼容任何合奏的ASR防御模型，以发现音频AE攻击有效地没有模型再训练。摘要：Recently, studies show that deep learning-based automatic speech recognition (ASR) systems are vulnerable to adversarial examples (AEs), which add a small amount of noise to the original audio examples. These AE attacks pose new challenges to deep learning security and have raised significant concerns about deploying ASR systems and devices. The existing defense methods are either limited in application or only defend on results, but not on process. In this work, we propose a novel method to infer the adversary intent and discover audio adversarial examples based on the AEs generation process. The insight of this method is based on the observation: many existing audio AE attacks utilize query-based methods, which means the adversary must send continuous and similar queries to target ASR models during the audio AE generation process. Inspired by this observation, We propose a memory mechanism by adopting audio fingerprint technology to analyze the similarity of the current query with a certain length of memory query. Thus, we can identify when a sequence of queries appears to be suspectable to generate audio AEs. Through extensive evaluation on four state-of-the-art audio AE attacks, we demonstrate that on average our defense identify the adversary intent with over 90% accuracy. With careful regard for robustness evaluations, we also analyze our proposed defense and its strength to withstand two adaptive attacks. Finally, our scheme is available out-of-the-box and directly compatible with any ensemble of ASR defense models to uncover audio AE attacks effectively without model retraining.

机器翻译，仅供参考