今日论文合集：cs.SD语音2篇，eess.AS音频处理2篇。
cs.SD语音

【1】 Selective Data Augmentation for Robust Speech Translation标题：用于稳健语音翻译的选择性数据增强链接：http://arxiv.org/pdf/2304.03169v1

作者：Rajul Acharya,Ashish Panda,Sunil Kumar Kopparapu机构：TCS Research, Tata Consultancy Services Limited, India备注：4 pages摘要：语音翻译（ST）系统将一种语言的语音翻译成另一种语言的文本。端到端ST系统（e2 e-ST）由于其由于减少的延迟和计算成本而增强的性能而比级联系统更受欢迎。虽然资源密集型，e2 e-ST系统具有保留与级联系统不同的语音的对位和非语言特征的固有能力。在本文中，我们建议使用E2 E架构的英语印地语（EN-HI）ST。我们使用两个不完善的机器翻译（MT）服务，翻译成喜文本Libri-trans EN文本。虽然每个服务给MT数据单独生成并行ST数据，我们提出了一个数据增强策略的噪声MT数据，以帮助强大的ST。本文的主要贡献是数据增强策略的建议。我们表明，这导致更好的ST（BLEU评分）相比，蛮力增强MT数据。我们观察到我们的方法的1.59 BLEU评分的绝对改善。摘要：Speech translation (ST) systems translate speech in one language to text in another language. End-to-end ST systems (e2e-ST) have gained popularity over cascade systems because of their enhanced performance due to reduced latency and computational cost. Though resource intensive, e2e-ST systems have the inherent ability to retain para and non-linguistic characteristics of the speech unlike cascade systems. In this paper, we propose to use an e2e architecture for English-Hindi (en-hi) ST. We use two imperfect machine translation (MT) services to translate Libri-trans en text into hi text. While each service gives MT data individually to generate parallel ST data, we propose a data augmentation strategy of noisy MT data to aid robust ST. The main contribution of this paper is the proposal of a data augmentation strategy. We show that this results in better ST (BLEU score) compared to brute force augmentation of MT data. We observed an absolute improvement of 1.59 BLEU score with our approach.

【2】 Efficient Audio Captioning Transformer with Patchout and Text Guidance标题：具有Patchout和文本引导功能的高效音频字幕转换器链接：http://arxiv.org/pdf/2304.02916v1

作者：Thodoris Kouzelis,Grigoris Bastas,Athanasios Katsamanis,Alexandros Potamianos机构：Institute for Language and Speech Processing, Athena Research Center, Athens, Greece, School of ECE, National Technical University of Athens, Athens, Greece备注：5 pages, 1 figure摘要：自动音频字幕是一种多模态翻译任务，旨在为给定的音频片段生成文本描述。在本文中，我们提出了一个完整的Transformer架构，该架构利用了[1]中提出的Patchout，显著降低了计算复杂度并避免了过拟合。字幕生成部分地取决于由预训练的分类模型提取的文本AudioSet标签，该分类模型被微调以最大化AudioSet标签和地面实况字幕之间的语义相似性。为了缓解自动音频字幕的数据稀缺问题，我们从上游音频相关任务和扩大的域内数据集引入了迁移学习。此外，我们提出了一种方法来应用混合增强AAC。进行消融研究，以调查Patchout和文本指导对最终性能的影响。结果表明，所提出的技术提高了系统的性能，同时降低了计算复杂度。我们提出的方法在2022年DCASE挑战赛的Task6A中获得了评委奖。摘要：Automated audio captioning is multi-modal translation task that aim to generate textual descriptions for a given audio clip. In this paper we propose a full Transformer architecture that utilizes Patchout as proposed in [1], significantly reducing the computational complexity and avoiding overfitting. The caption generation is partly conditioned on textual AudioSet tags extracted by a pre-trained classification model which is fine-tuned to maximize the semantic similarity between AudioSet labels and ground truth captions. To mitigate the data scarcity problem of Automated Audio Captioning we introduce transfer learning from an upstream audio-related task and an enlarged in-domain dataset. Moreover, we propose a method to apply Mixup augmentation for AAC. Ablation studies are carried out to investigate how Patchout and text guidance contribute to the final performance. The results show that the proposed techniques improve the performance of our system and while reducing the computational complexity. Our proposed method received the Judges Award at the Task6A of DCASE Challenge 2022.
eess.AS音频处理

【1】 Selective Data Augmentation for Robust Speech Translation标题：用于稳健语音翻译的选择性数据增强链接：http://arxiv.org/pdf/2304.03169v1

*与cs.SD语音【1】为同一篇

作者：Rajul Acharya,Ashish Panda,Sunil Kumar Kopparapu机构：TCS Research, Tata Consultancy Services Limited, India备注：4 pages摘要：语音翻译（ST）系统将一种语言的语音翻译成另一种语言的文本。端到端ST系统（e2 e-ST）由于其由于减少的延迟和计算成本而增强的性能而比级联系统更受欢迎。虽然资源密集型，e2 e-ST系统具有保留与级联系统不同的语音的对位和非语言特征的固有能力。在本文中，我们建议使用E2 E架构的英语印地语（EN-HI）ST。我们使用两个不完善的机器翻译（MT）服务，翻译成喜文本Libri-trans EN文本。虽然每个服务给MT数据单独生成并行ST数据，我们提出了一个数据增强策略的噪声MT数据，以帮助强大的ST。本文的主要贡献是数据增强策略的建议。我们表明，这导致更好的ST（BLEU评分）相比，蛮力增强MT数据。我们观察到我们的方法的1.59 BLEU评分的绝对改善。摘要：Speech translation (ST) systems translate speech in one language to text in another language. End-to-end ST systems (e2e-ST) have gained popularity over cascade systems because of their enhanced performance due to reduced latency and computational cost. Though resource intensive, e2e-ST systems have the inherent ability to retain para and non-linguistic characteristics of the speech unlike cascade systems. In this paper, we propose to use an e2e architecture for English-Hindi (en-hi) ST. We use two imperfect machine translation (MT) services to translate Libri-trans en text into hi text. While each service gives MT data individually to generate parallel ST data, we propose a data augmentation strategy of noisy MT data to aid robust ST. The main contribution of this paper is the proposal of a data augmentation strategy. We show that this results in better ST (BLEU score) compared to brute force augmentation of MT data. We observed an absolute improvement of 1.59 BLEU score with our approach.

【2】 Efficient Audio Captioning Transformer with Patchout and Text Guidance标题：具有Patchout和文本引导功能的高效音频字幕转换器链接：http://arxiv.org/pdf/2304.02916v1

*与cs.SD语音【2】为同一篇

作者：Thodoris Kouzelis,Grigoris Bastas,Athanasios Katsamanis,Alexandros Potamianos机构：Institute for Language and Speech Processing, Athena Research Center, Athens, Greece, School of ECE, National Technical University of Athens, Athens, Greece备注：5 pages, 1 figure摘要：自动音频字幕是一种多模态翻译任务，旨在为给定的音频片段生成文本描述。在本文中，我们提出了一个完整的Transformer架构，该架构利用了[1]中提出的Patchout，显著降低了计算复杂度并避免了过拟合。字幕生成部分地取决于由预训练的分类模型提取的文本AudioSet标签，该分类模型被微调以最大化AudioSet标签和地面实况字幕之间的语义相似性。为了缓解自动音频字幕的数据稀缺问题，我们从上游音频相关任务和扩大的域内数据集引入了迁移学习。此外，我们提出了一种方法来应用混合增强AAC。进行消融研究，以调查Patchout和文本指导对最终性能的影响。结果表明，所提出的技术提高了系统的性能，同时降低了计算复杂度。我们提出的方法在2022年DCASE挑战赛的Task6A中获得了评委奖。摘要：Automated audio captioning is multi-modal translation task that aim to generate textual descriptions for a given audio clip. In this paper we propose a full Transformer architecture that utilizes Patchout as proposed in [1], significantly reducing the computational complexity and avoiding overfitting. The caption generation is partly conditioned on textual AudioSet tags extracted by a pre-trained classification model which is fine-tuned to maximize the semantic similarity between AudioSet labels and ground truth captions. To mitigate the data scarcity problem of Automated Audio Captioning we introduce transfer learning from an upstream audio-related task and an enlarged in-domain dataset. Moreover, we propose a method to apply Mixup augmentation for AAC. Ablation studies are carried out to investigate how Patchout and text guidance contribute to the final performance. The results show that the proposed techniques improve the performance of our system and while reducing the computational complexity. Our proposed method received the Judges Award at the Task6A of DCASE Challenge 2022.

机器翻译，仅供参考