今日论文合集：cs.SD语音5篇，eess.AS音频处理5篇。
cs.SD语音

【1】 Soft Dynamic Time Warping for Multi-Pitch Estimation and Beyond标题：用于多基音估计及超越的软动态时间规整链接：http://arxiv.org/pdf/2304.05032v1

作者：Michael Krause,Christof Weiß,Meinard Müller机构：International Audio Laboratories Erlangen备注：Accepted at ICASSP 2023摘要：音乐信息检索（MIR）中的许多任务涉及弱对齐数据，其中确切的时间对应关系是未知的。连接主义时间分类（CTC）损失是基于弱对齐训练数据学习特征表示的标准技术。然而，CTC仅限于离散值的靶序列，并且可能难以扩展到多标记问题。在这篇文章中，我们展示了软动态时间规整（SoftDTW），经典DTW的一个可微变体，如何可以用作CTC的替代方案。使用多音高估计作为一个例子的情况下，我们表明，SoftDTW产生的结果与国家的最先进的多标签扩展CTC。除了在算法公式方面更优雅之外，SoftDTW自然扩展到实值目标序列。摘要：Many tasks in music information retrieval (MIR) involve weakly aligned data, where exact temporal correspondences are unknown. The connectionist temporal classification (CTC) loss is a standard technique to learn feature representations based on weakly aligned training data. However, CTC is limited to discrete-valued target sequences and can be difficult to extend to multi-label problems. In this article, we show how soft dynamic time warping (SoftDTW), a differentiable variant of classical DTW, can be used as an alternative to CTC. Using multi-pitch estimation as an example scenario, we show that SoftDTW yields results on par with a state-of-the-art multi-label extension of CTC. In addition to being more elegant in terms of its algorithmic formulation, SoftDTW naturally extends to real-valued target sequences.

【2】 Sim-T: Simplify the Transformer Network by Multiplexing Technique for Speech Recognition标题：SIM-T：利用语音识别的多路复用技术简化Transformer网络链接：http://arxiv.org/pdf/2304.04991v1

作者：Guangyong Wei,Zhikui Duan,Shiren Li,Guangguang Yang,Xinmei Yu,Junhua Li摘要：近年来，Transformer网络由于其优良的模型性能，在语音识别任务中得到了广泛的关注。然而，Transformer网络总是涉及繁重的计算和大量的参数，导致严重的部署问题，在有限的计算源或存储内存的设备。在本文中，一个新的轻量级模型，称为Sim-T已被提出来扩大Transformer模型的通用性。在新开发的多路复用技术的帮助下，Sim-T可以有效地压缩模型，其性能上的牺牲可以忽略不计。更准确地说，所提出的技术包括两个部分，即，模块权重复用和注意力分数复用。此外，已经提出了一种新的解码器结构，以促进注意力分数复用。大量的实验已经进行了验证Sim-T的有效性。在Aishell-1数据集中，当提出的Sim-T比基线Transformer小48%的参数时，可以获得0.4%的CER改善。或者，如果Sim-T提供与基线Transformer相同的性能，则可以实现69%的参数减少。对于HKUST和WSJ eval 92数据集，当Sim-T中的参数比基线Transformer小40%时，CER和WER将分别提高0.3%和0.2%。摘要：In recent years, a great deal of attention has been paid to the Transformer network for speech recognition tasks due to its excellent model performance. However, the Transformer network always involves heavy computation and large number of parameters, causing serious deployment problems in devices with limited computation sources or storage memory. In this paper, a new lightweight model called Sim-T has been proposed to expand the generality of the Transformer model. Under the help of the newly developed multiplexing technique, the Sim-T can efficiently compress the model with negligible sacrifice on its performance. To be more precise, the proposed technique includes two parts, that are, module weight multiplexing and attention score multiplexing. Moreover, a novel decoder structure has been proposed to facilitate the attention score multiplexing. Extensive experiments have been conducted to validate the effectiveness of Sim-T. In Aishell-1 dataset, when the proposed Sim-T is 48% parameter less than the baseline Transformer, 0.4% CER improvement can be obtained. Alternatively, 69% parameter reduction can be achieved if the Sim-T gives the same performance as the baseline Transformer. With regard to the HKUST and WSJ eval92 datasets, CER and WER will be improved by 0.3% and 0.2%, respectively, when parameters in Sim-T are 40% less than the baseline Transformer.

【3】 AffectMachine-Classical: A novel system for generating affective classical music标题：AffectMachine-Classical：一种产生情感古典音乐的新系统链接：http://arxiv.org/pdf/2304.04915v1

作者：Kat R. Agres,Adyasha Dash,Phoebe Chua机构：Yong Siew Toh Conservatory of Music, National University of Singapore, Singapore, Centre for Music and Health, National University of Singapore, Singapore, Augmented Human Lab, DISA, National University of Singapore, Singapore备注：K. Agres and A. Dash share first authorship摘要：这项工作介绍了一种新的音乐生成系统，称为AffectMachine-Classical，能够实时生成情感经典音乐。AffectMachine被设计成被纳入生物反馈系统（如脑机接口），以帮助用户意识到并最终调解他们自己的动态情感状态。也就是说，该系统是为基于音乐的医疗技术开发的，以支持用户的实时情绪自我调节。我们提供了一个基于规则的，概率系统架构的概述，描述了系统的主要方面，以及它们是如何新颖的。然后，我们提出了一个听众的研究，进行了验证系统的能力，可靠地传达目标的情感给听众的结果。研究结果表明，AffectMachine-Classical在向听众传达各种水平的唤醒（$R^2 = 0.96 $）方面非常有效，并且在效价方面也相当令人信服（R^2 = 0.90）。未来的工作将嵌入AffectMachine-Classical到生物反馈系统中，以利用情感音乐对听众情绪健康的功效。摘要：This work introduces a new music generation system, called AffectMachine-Classical, that is capable of generating affective Classic music in real-time. AffectMachine was designed to be incorporated into biofeedback systems (such as brain-computer-interfaces) to help users become aware of, and ultimately mediate, their own dynamic affective states. That is, this system was developed for music-based MedTech to support real-time emotion self-regulation in users. We provide an overview of the rule-based, probabilistic system architecture, describing the main aspects of the system and how they are novel. We then present the results of a listener study that was conducted to validate the ability of the system to reliably convey target emotions to listeners. The findings indicate that AffectMachine-Classical is very effective in communicating various levels of Arousal ($R^2 = .96$) to listeners, and is also quite convincing in terms of Valence (R^2 = .90). Future work will embed AffectMachine-Classical into biofeedback systems, to leverage the efficacy of the affective music for emotional well-being in listeners.

【4】 Audio Bank: A High-Level Acoustic Signal Representation for Audio Event Recognition标题：Audio Bank：一种用于音频事件识别的高级声学信号表示链接：http://arxiv.org/pdf/2304.05067v1

作者：Tushar Sandhan,Sukanya Sonowal,Jin Young Choi机构：Department of Electrical and Computer Engineering, Seoul National University, Seoul, Republic of Korea ,-备注：6 pages, 9 figures, published in IEEE International Conf ICCAS 2014 (Best paper award)摘要：自动音频事件识别在使人机交互更加紧密方面起着关键作用，并且在工业自动化、控制和监视系统中具有广泛的适用性。音频事件是由复杂的声音模式，这些模式是和谐纠缠。音频识别主要是由低级和中级特征主导的，这些特征已经证明了它们的识别能力，但它们具有高计算成本和低语义含义。在本文中，我们提出了一个新的计算效率的音频识别框架。音频库是一种新的高层次音频表示方法，它由不同的音频检测器组成，在频率-时间空间中表示每个音频类。使用非负矩阵分解降低了所得到的特征向量的维数，保留了其可辨别性和丰富的语义信息。使用几种分类器（支持向量机，神经网络，高斯过程分类和k-最近邻）的高音频识别性能表明所提出的方法的有效性。摘要：Automatic audio event recognition plays a pivotal role in making human robot interaction more closer and has a wide applicability in industrial automation, control and surveillance systems. Audio event is composed of intricate phonic patterns which are harmonically entangled. Audio recognition is dominated by low and mid-level features, which have demonstrated their recognition capability but they have high computational cost and low semantic meaning. In this paper, we propose a new computationally efficient framework for audio recognition. Audio Bank, a new high-level representation of audio, is comprised of distinctive audio detectors representing each audio class in frequency-temporal space. Dimensionality of the resulting feature vector is reduced using non-negative matrix factorization preserving its discriminability and rich semantic information. The high audio recognition performance using several classifiers (SVM, neural network, Gaussian process classification and k-nearest neighbors) shows the effectiveness of the proposed method.

【5】 Wav2code: Restore Clean Speech Representations via Codebook Lookup for Noise-Robust ASR标题：Wav2code：通过码本查找恢复干净的语音表示，以实现抗噪声ASR链接：http://arxiv.org/pdf/2304.04974v1

作者：Yuchen Hu,Chen Chen,Qiushi Zhu,Eng Siong Chng备注：12 pages, 7 figures, Submitted to IEEEACM TASLP摘要：由于深度学习的最新进展，自动语音识别（ASR）取得了显着的成功，但在现实世界的嘈杂条件下，它通常会显着下降。近年来的研究工作将语音增强技术引入到语音质量改善的前端，这种方法被证明是有效的，但由于语音失真问题，对于下行ASR来说可能不是最优的。在此基础上，最新的作品结合SE和目前流行的自监督学习（SSL），以减轻失真和提高噪声鲁棒性。尽管有效，由常规SE引起的语音失真仍然不能完全消除。在本文中，我们提出了一个名为Wav2code的自监督框架，以实现无失真的广义SE，用于噪声鲁棒ASR。首先，在预训练阶段，利用最近邻特征匹配的方法，将SSL模型中的纯净语音表示发送到离散码本中，然后利用得到的码序列重构原始的纯净语音表示，并将其存储在码本中作为先验知识。其次，在微调过程中，我们提出了一个基于Transformer的代码预测器，通过对输入噪声表示的全局依赖关系进行建模来准确预测干净的代码，从而能够发现和恢复高质量的干净表示而不会失真。此外，我们提出了一个交互式特征融合网络，结合原始的嘈杂和恢复干净的表示，考虑保真度和质量，从而为下游ASR提供更多的信息功能。最后，在合成和真实噪声数据集上的实验结果表明，Wav2code算法能够有效地解决语音失真问题，提高语音识别性能，具有较强的鲁棒性。摘要：Automatic speech recognition (ASR) has gained a remarkable success thanks to recent advances of deep learning, but it usually degrades significantly under real-world noisy conditions. Recent works introduce speech enhancement (SE) as front-end to improve speech quality, which is proved effective but may not be optimal for downstream ASR due to speech distortion problem. Based on that, latest works combine SE and currently popular self-supervised learning (SSL) to alleviate distortion and improve noise robustness. Despite the effectiveness, the speech distortion caused by conventional SE still cannot be completely eliminated. In this paper, we propose a self-supervised framework named Wav2code to implement a generalized SE without distortions for noise-robust ASR. First, in pre-training stage the clean speech representations from SSL model are sent to lookup a discrete codebook via nearest-neighbor feature matching, the resulted code sequence are then exploited to reconstruct the original clean representations, in order to store them in codebook as prior. Second, during finetuning we propose a Transformer-based code predictor to accurately predict clean codes by modeling the global dependency of input noisy representations, which enables discovery and restoration of high-quality clean representations without distortions. Furthermore, we propose an interactive feature fusion network to combine original noisy and the restored clean representations to consider both fidelity and quality, resulting in even more informative features for downstream ASR. Finally, experiments on both synthetic and real noisy datasets demonstrate that Wav2code can solve the speech distortion and improve ASR performance under various noisy conditions, resulting in stronger robustness.
eess.AS音频处理

【1】 Audio Bank: A High-Level Acoustic Signal Representation for Audio Event Recognition标题：Audio Bank：一种用于音频事件识别的高级声学信号表示链接：http://arxiv.org/pdf/2304.05067v1

*与cs.SD语音【4】为同一篇

作者：Tushar Sandhan,Sukanya Sonowal,Jin Young Choi机构：Department of Electrical and Computer Engineering, Seoul National University, Seoul, Republic of Korea ,-备注：6 pages, 9 figures, published in IEEE International Conf ICCAS 2014 (Best paper award)摘要：自动音频事件识别在使人机交互更加紧密方面起着关键作用，并且在工业自动化、控制和监视系统中具有广泛的适用性。音频事件是由复杂的声音模式，这些模式是和谐纠缠。音频识别主要是由低级和中级特征主导的，这些特征已经证明了它们的识别能力，但它们具有高计算成本和低语义含义。在本文中，我们提出了一个新的计算效率的音频识别框架。音频库是一种新的高层次音频表示方法，它由不同的音频检测器组成，在频率-时间空间中表示每个音频类。使用非负矩阵分解降低了所得到的特征向量的维数，保留了其可辨别性和丰富的语义信息。使用几种分类器（支持向量机，神经网络，高斯过程分类和k-最近邻）的高音频识别性能表明所提出的方法的有效性。摘要：Automatic audio event recognition plays a pivotal role in making human robot interaction more closer and has a wide applicability in industrial automation, control and surveillance systems. Audio event is composed of intricate phonic patterns which are harmonically entangled. Audio recognition is dominated by low and mid-level features, which have demonstrated their recognition capability but they have high computational cost and low semantic meaning. In this paper, we propose a new computationally efficient framework for audio recognition. Audio Bank, a new high-level representation of audio, is comprised of distinctive audio detectors representing each audio class in frequency-temporal space. Dimensionality of the resulting feature vector is reduced using non-negative matrix factorization preserving its discriminability and rich semantic information. The high audio recognition performance using several classifiers (SVM, neural network, Gaussian process classification and k-nearest neighbors) shows the effectiveness of the proposed method.

【2】 Wav2code: Restore Clean Speech Representations via Codebook Lookup for Noise-Robust ASR标题：Wav2code：通过码本查找恢复干净的语音表示，以实现抗噪声ASR链接：http://arxiv.org/pdf/2304.04974v1

*与cs.SD语音【5】为同一篇

作者：Yuchen Hu,Chen Chen,Qiushi Zhu,Eng Siong Chng备注：12 pages, 7 figures, Submitted to IEEEACM TASLP摘要：由于深度学习的最新进展，自动语音识别（ASR）取得了显着的成功，但在现实世界的嘈杂条件下，它通常会显着下降。近年来的研究工作将语音增强技术引入到语音质量改善的前端，这种方法被证明是有效的，但由于语音失真问题，对于下行ASR来说可能不是最优的。在此基础上，最新的作品结合SE和目前流行的自监督学习（SSL），以减轻失真和提高噪声鲁棒性。尽管有效，由常规SE引起的语音失真仍然不能完全消除。在本文中，我们提出了一个名为Wav2code的自监督框架，以实现无失真的广义SE，用于噪声鲁棒ASR。首先，在预训练阶段，利用最近邻特征匹配的方法，将SSL模型中的纯净语音表示发送到离散码本中，然后利用得到的码序列重构原始的纯净语音表示，并将其存储在码本中作为先验知识。其次，在微调过程中，我们提出了一个基于Transformer的代码预测器，通过对输入噪声表示的全局依赖关系进行建模来准确预测干净的代码，从而能够发现和恢复高质量的干净表示而不会失真。此外，我们提出了一个交互式特征融合网络，结合原始的嘈杂和恢复干净的表示，考虑保真度和质量，从而为下游ASR提供更多的信息功能。最后，在合成和真实噪声数据集上的实验结果表明，Wav2code算法能够有效地解决语音失真问题，提高语音识别性能，具有较强的鲁棒性。摘要：Automatic speech recognition (ASR) has gained a remarkable success thanks to recent advances of deep learning, but it usually degrades significantly under real-world noisy conditions. Recent works introduce speech enhancement (SE) as front-end to improve speech quality, which is proved effective but may not be optimal for downstream ASR due to speech distortion problem. Based on that, latest works combine SE and currently popular self-supervised learning (SSL) to alleviate distortion and improve noise robustness. Despite the effectiveness, the speech distortion caused by conventional SE still cannot be completely eliminated. In this paper, we propose a self-supervised framework named Wav2code to implement a generalized SE without distortions for noise-robust ASR. First, in pre-training stage the clean speech representations from SSL model are sent to lookup a discrete codebook via nearest-neighbor feature matching, the resulted code sequence are then exploited to reconstruct the original clean representations, in order to store them in codebook as prior. Second, during finetuning we propose a Transformer-based code predictor to accurately predict clean codes by modeling the global dependency of input noisy representations, which enables discovery and restoration of high-quality clean representations without distortions. Furthermore, we propose an interactive feature fusion network to combine original noisy and the restored clean representations to consider both fidelity and quality, resulting in even more informative features for downstream ASR. Finally, experiments on both synthetic and real noisy datasets demonstrate that Wav2code can solve the speech distortion and improve ASR performance under various noisy conditions, resulting in stronger robustness.

【3】 Soft Dynamic Time Warping for Multi-Pitch Estimation and Beyond标题：用于多基音估计及超越的软动态时间规整链接：http://arxiv.org/pdf/2304.05032v1

*与cs.SD语音【1】为同一篇

作者：Michael Krause,Christof Weiß,Meinard Müller机构：International Audio Laboratories Erlangen备注：Accepted at ICASSP 2023摘要：音乐信息检索（MIR）中的许多任务涉及弱对齐数据，其中确切的时间对应关系是未知的。连接主义时间分类（CTC）损失是基于弱对齐训练数据学习特征表示的标准技术。然而，CTC仅限于离散值的靶序列，并且可能难以扩展到多标记问题。在这篇文章中，我们展示了软动态时间规整（SoftDTW），经典DTW的一个可微变体，如何可以用作CTC的替代方案。使用多音高估计作为一个例子的情况下，我们表明，SoftDTW产生的结果与国家的最先进的多标签扩展CTC。除了在算法公式方面更优雅之外，SoftDTW自然扩展到实值目标序列。摘要：Many tasks in music information retrieval (MIR) involve weakly aligned data, where exact temporal correspondences are unknown. The connectionist temporal classification (CTC) loss is a standard technique to learn feature representations based on weakly aligned training data. However, CTC is limited to discrete-valued target sequences and can be difficult to extend to multi-label problems. In this article, we show how soft dynamic time warping (SoftDTW), a differentiable variant of classical DTW, can be used as an alternative to CTC. Using multi-pitch estimation as an example scenario, we show that SoftDTW yields results on par with a state-of-the-art multi-label extension of CTC. In addition to being more elegant in terms of its algorithmic formulation, SoftDTW naturally extends to real-valued target sequences.

【4】 Sim-T: Simplify the Transformer Network by Multiplexing Technique for Speech Recognition标题：SIM-T：利用语音识别的多路复用技术简化Transformer网络链接：http://arxiv.org/pdf/2304.04991v1

*与cs.SD语音【2】为同一篇

作者：Guangyong Wei,Zhikui Duan,Shiren Li,Guangguang Yang,Xinmei Yu,Junhua Li摘要：近年来，Transformer网络由于其优良的模型性能，在语音识别任务中得到了广泛的关注。然而，Transformer网络总是涉及繁重的计算和大量的参数，导致严重的部署问题，在有限的计算源或存储内存的设备。在本文中，一个新的轻量级模型，称为Sim-T已被提出来扩大Transformer模型的通用性。在新开发的多路复用技术的帮助下，Sim-T可以有效地压缩模型，其性能上的牺牲可以忽略不计。更准确地说，所提出的技术包括两个部分，即，模块权重复用和注意力分数复用。此外，已经提出了一种新的解码器结构，以促进注意力分数复用。大量的实验已经进行了验证Sim-T的有效性。在Aishell-1数据集中，当提出的Sim-T比基线Transformer小48%的参数时，可以获得0.4%的CER改善。或者，如果Sim-T提供与基线Transformer相同的性能，则可以实现69%的参数减少。对于HKUST和WSJ eval 92数据集，当Sim-T中的参数比基线Transformer小40%时，CER和WER将分别提高0.3%和0.2%。摘要：In recent years, a great deal of attention has been paid to the Transformer network for speech recognition tasks due to its excellent model performance. However, the Transformer network always involves heavy computation and large number of parameters, causing serious deployment problems in devices with limited computation sources or storage memory. In this paper, a new lightweight model called Sim-T has been proposed to expand the generality of the Transformer model. Under the help of the newly developed multiplexing technique, the Sim-T can efficiently compress the model with negligible sacrifice on its performance. To be more precise, the proposed technique includes two parts, that are, module weight multiplexing and attention score multiplexing. Moreover, a novel decoder structure has been proposed to facilitate the attention score multiplexing. Extensive experiments have been conducted to validate the effectiveness of Sim-T. In Aishell-1 dataset, when the proposed Sim-T is 48% parameter less than the baseline Transformer, 0.4% CER improvement can be obtained. Alternatively, 69% parameter reduction can be achieved if the Sim-T gives the same performance as the baseline Transformer. With regard to the HKUST and WSJ eval92 datasets, CER and WER will be improved by 0.3% and 0.2%, respectively, when parameters in Sim-T are 40% less than the baseline Transformer.

【5】 AffectMachine-Classical: A novel system for generating affective classical music标题：AffectMachine-Classical：一种产生情感古典音乐的新系统链接：http://arxiv.org/pdf/2304.04915v1

*与cs.SD语音【3】为同一篇

作者：Kat R. Agres,Adyasha Dash,Phoebe Chua机构：Yong Siew Toh Conservatory of Music, National University of Singapore, Singapore, Centre for Music and Health, National University of Singapore, Singapore, Augmented Human Lab, DISA, National University of Singapore, Singapore备注：K. Agres and A. Dash share first authorship摘要：这项工作介绍了一种新的音乐生成系统，称为AffectMachine-Classical，能够实时生成情感经典音乐。AffectMachine被设计成被纳入生物反馈系统（如脑机接口），以帮助用户意识到并最终调解他们自己的动态情感状态。也就是说，该系统是为基于音乐的医疗技术开发的，以支持用户的实时情绪自我调节。我们提供了一个基于规则的，概率系统架构的概述，描述了系统的主要方面，以及它们是如何新颖的。然后，我们提出了一个听众的研究，进行了验证系统的能力，可靠地传达目标的情感给听众的结果。研究结果表明，AffectMachine-Classical在向听众传达各种水平的唤醒（$R^2 = 0.96 $）方面非常有效，并且在效价方面也相当令人信服（R^2 = 0.90）。未来的工作将嵌入AffectMachine-Classical到生物反馈系统中，以利用情感音乐对听众情绪健康的功效。摘要：This work introduces a new music generation system, called AffectMachine-Classical, that is capable of generating affective Classic music in real-time. AffectMachine was designed to be incorporated into biofeedback systems (such as brain-computer-interfaces) to help users become aware of, and ultimately mediate, their own dynamic affective states. That is, this system was developed for music-based MedTech to support real-time emotion self-regulation in users. We provide an overview of the rule-based, probabilistic system architecture, describing the main aspects of the system and how they are novel. We then present the results of a listener study that was conducted to validate the ability of the system to reliably convey target emotions to listeners. The findings indicate that AffectMachine-Classical is very effective in communicating various levels of Arousal ($R^2 = .96$) to listeners, and is also quite convincing in terms of Valence (R^2 = .90). Future work will embed AffectMachine-Classical into biofeedback systems, to leverage the efficacy of the affective music for emotional well-being in listeners.

机器翻译，仅供参考