今天跟大家分享一篇语音相关的论文合集：cs.SD语音4篇，eess.AS音频处理4篇。
【1】 Improving Post-Processing of Audio Event Detectors Using Reinforcement Learning
标题：利用强化学习改进音频事件检测器的后处理
链接：https://arxiv.org/abs/2208.09201
作者：Petros Giannakopoulos,Aggelos Pikrakis,Yannis Cotronis机构：COTRONIS, Research in this paper was supported by the Hellenic Foundation for Research and Innovation (H.F.R.I.) under the “,st Call for H.F.R.I.备注：Published on IEEE Access journal, Volume 10, 2022摘要：We apply post-processing to the class probability distribution outputs of audio event classification models and employ reinforcement learning to jointly discover the optimal parameters for various stages of a post-processing stack, such as the classification thresholds and the kernel sizes of median filtering algorithms used to smooth out model predictions. To achieve this we define a reinforcement learning environment where: 1) a state is the class probability distribution provided by the model for a given audio sample, 2) an action is the choice of a candidate optimal value for each parameter of the post-processing stack, 3) the reward is based on the classification accuracy metric we aim to optimize, which is the audio event-based macro F1-score in our case. We apply our post-processing to the class probability distribution outputs of two audio event classification models submitted to the DCASE Task4 2020 challenge. We find that by using reinforcement learning to discover the optimal per-class parameters for the post-processing stack that is applied to the outputs of audio event classification models, we can improve the audio event-based macro F1-score (the main metric used in the DCASE challenge to compare audio event classification accuracy) by 4-5% compared to using the same post-processing stack with manually tuned parameters.
【2】 3M: An Effective Multi-view, Multi-granularity, and Multi-aspect Modeling Approach to English Pronunciation Assessment
标题：3M：一种有效的多视角、多粒度、多角度的英语语音评估建模方法
链接：https://arxiv.org/abs/2208.09110
作者：Fu-An Chao,Tien-Hong Lo,Tzu-I Wu,Yao-Ting Sung,Berlin Chen机构：Research Center for Psychological and Educational Testing, National Taiwan Normal University, Taiwan, Department of Computer Science and Information Engineering, National Taiwan Normal University, Taiwan备注：7 pages, 2 figures, submitted to APSIPA ASC 2022摘要：As an indispensable ingredient of computer-assisted pronunciation training (CAPT), automatic pronunciation assessment (APA) plays a pivotal role in aiding self-directed language learners by providing multi-aspect and timely feedback. However, there are at least two potential obstacles that might hinder its performance for practical use. On one hand, most of the studies focus exclusively on leveraging segmental (phonetic)-level features such as goodness of pronunciation (GOP); this, however, may cause a discrepancy of feature granularity when performing suprasegmental (prosodic)-level pronunciation assessment. On the other hand, automatic pronunciation assessments still suffer from the lack of large-scale labeled speech data of non-native speakers, which inevitably limits the performance of pronunciation assessment. In this paper, we tackle these problems by integrating multiple prosodic and phonological features to provide a multi-view, multi-granularity, and multi-aspect (3M) pronunciation modeling. Specifically, we augment GOP with prosodic and self-supervised learning (SSL) features, and meanwhile develop a vowel/consonant positional embedding for a more phonology-aware automatic pronunciation assessment. A series of experiments conducted on the publicly-available speechocean762 dataset show that our approach can obtain significant improvements on several assessment granularities in comparison with previous work, especially on the assessment of speaking fluency and speech prosody.
【3】 Representation Learning for the Automatic Indexing of Sound Effects Libraries
标题：基于表示学习的音效库自动标引
链接：https://arxiv.org/abs/2208.09096
作者：Alison B. Ma,Alexander Lerch机构：Music Informatics Group, Georgia Institute of Technology备注：Accepted at the 23rd International Society for Music Information Retrieval Conference (ISMIR 2022), 10 pages, 7 figures摘要：Labeling and maintaining a commercial sound effects library is a time-consuming task exacerbated by databases that continually grow in size and undergo taxonomy updates. Moreover, sound search and taxonomy creation are complicated by non-uniform metadata, an unrelenting problem even with the introduction of a new industry standard, the Universal Category System. To address these problems and overcome dataset-dependent limitations that inhibit the successful training of deep learning models, we pursue representation learning to train generalized embeddings that can be used for a wide variety of sound effects libraries and are a taxonomy-agnostic representation of sound. We show that a task-specific but dataset-independent representation can successfully address data issues such as class imbalance, inconsistent class labels, and insufficient dataset size, outperforming established representations such as OpenL3. Detailed experimental results show the impact of metric learning approaches and different cross-dataset training methods on representational effectiveness.
【4】 Feature Selection Enhancement and Feature Space Visualization for Speech-Based Emotion Recognition
标题：语音情感识别中的特征选择增强与特征空间可视化
链接：https://arxiv.org/abs/2208.09269
作者：Sofia Kanwal,Sohail Asghar,Hazrat Ali机构：Department of Computer Science Islamabad, Comsats University Islamabad, Pakistan, Department of Computer Science, University of Poonch Rawalakot, Azad Kashmir, Collage of Science and Engineering, Hamad Bin Khalifa Univeristy, Qatar Foundation, Doha, Qatar备注：Accepted at PeerJ Computer Science摘要：Robust speech emotion recognition relies on the quality of the speech features. We present speech features enhancement strategy that improves speech emotion recognition. We used the INTERSPEECH 2010 challenge feature-set. We identified subsets from the features set and applied Principle Component Analysis to the subsets. Finally, the features are fused horizontally. The resulting feature set is analyzed using t-distributed neighbour embeddings (t-SNE) before the application of features for emotion recognition. The method is compared with the state-of-the-art methods used in the literature. The empirical evidence is drawn using two well-known datasets: Emotional Speech Dataset (EMO-DB) and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) for two languages, German and English, respectively. Our method achieved an average recognition gain of 11.5\% for six out of seven emotions for the EMO-DB dataset, and 13.8\% for seven out of eight emotions for the RAVDESS dataset as compared to the baseline study.
eess.AS音频处理
【1】 Feature Selection Enhancement and Feature Space Visualization for Speech-Based Emotion Recognition
标题：语音情感识别中的特征选择增强与特征空间可视化
链接：https://arxiv.org/abs/2208.09269
* 与cs.SD语音【4】为同一篇
作者：Sofia Kanwal,Sohail Asghar,Hazrat Ali机构：Department of Computer Science Islamabad, Comsats University Islamabad, Pakistan, Department of Computer Science, University of Poonch Rawalakot, Azad Kashmir, Collage of Science and Engineering, Hamad Bin Khalifa Univeristy, Qatar Foundation, Doha, Qatar备注：Accepted at PeerJ Computer Science摘要：Robust speech emotion recognition relies on the quality of the speech features. We present speech features enhancement strategy that improves speech emotion recognition. We used the INTERSPEECH 2010 challenge feature-set. We identified subsets from the features set and applied Principle Component Analysis to the subsets. Finally, the features are fused horizontally. The resulting feature set is analyzed using t-distributed neighbour embeddings (t-SNE) before the application of features for emotion recognition. The method is compared with the state-of-the-art methods used in the literature. The empirical evidence is drawn using two well-known datasets: Emotional Speech Dataset (EMO-DB) and Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS) for two languages, German and English, respectively. Our method achieved an average recognition gain of 11.5\% for six out of seven emotions for the EMO-DB dataset, and 13.8\% for seven out of eight emotions for the RAVDESS dataset as compared to the baseline study.
【2】 Improving Post-Processing of Audio Event Detectors Using Reinforcement Learning
标题：利用强化学习改进音频事件检测器的后处理
链接：https://arxiv.org/abs/2208.09201
* 与cs.SD语音【1】为同一篇
作者：Petros Giannakopoulos,Aggelos Pikrakis,Yannis Cotronis机构：COTRONIS, Research in this paper was supported by the Hellenic Foundation for Research and Innovation (H.F.R.I.) under the “,st Call for H.F.R.I.备注：Published on IEEE Access journal, Volume 10, 2022摘要：We apply post-processing to the class probability distribution outputs of audio event classification models and employ reinforcement learning to jointly discover the optimal parameters for various stages of a post-processing stack, such as the classification thresholds and the kernel sizes of median filtering algorithms used to smooth out model predictions. To achieve this we define a reinforcement learning environment where: 1) a state is the class probability distribution provided by the model for a given audio sample, 2) an action is the choice of a candidate optimal value for each parameter of the post-processing stack, 3) the reward is based on the classification accuracy metric we aim to optimize, which is the audio event-based macro F1-score in our case. We apply our post-processing to the class probability distribution outputs of two audio event classification models submitted to the DCASE Task4 2020 challenge. We find that by using reinforcement learning to discover the optimal per-class parameters for the post-processing stack that is applied to the outputs of audio event classification models, we can improve the audio event-based macro F1-score (the main metric used in the DCASE challenge to compare audio event classification accuracy) by 4-5% compared to using the same post-processing stack with manually tuned parameters.
【3】 3M: An Effective Multi-view, Multi-granularity, and Multi-aspect Modeling Approach to English Pronunciation Assessment
标题：3M：一种有效的多视角、多粒度、多角度的英语语音评估建模方法
链接：https://arxiv.org/abs/2208.09110
* 与cs.SD语音【2】为同一篇
作者：Fu-An Chao,Tien-Hong Lo,Tzu-I Wu,Yao-Ting Sung,Berlin Chen机构：Research Center for Psychological and Educational Testing, National Taiwan Normal University, Taiwan, Department of Computer Science and Information Engineering, National Taiwan Normal University, Taiwan备注：7 pages, 2 figures, submitted to APSIPA ASC 2022摘要：As an indispensable ingredient of computer-assisted pronunciation training (CAPT), automatic pronunciation assessment (APA) plays a pivotal role in aiding self-directed language learners by providing multi-aspect and timely feedback. However, there are at least two potential obstacles that might hinder its performance for practical use. On one hand, most of the studies focus exclusively on leveraging segmental (phonetic)-level features such as goodness of pronunciation (GOP); this, however, may cause a discrepancy of feature granularity when performing suprasegmental (prosodic)-level pronunciation assessment. On the other hand, automatic pronunciation assessments still suffer from the lack of large-scale labeled speech data of non-native speakers, which inevitably limits the performance of pronunciation assessment. In this paper, we tackle these problems by integrating multiple prosodic and phonological features to provide a multi-view, multi-granularity, and multi-aspect (3M) pronunciation modeling. Specifically, we augment GOP with prosodic and self-supervised learning (SSL) features, and meanwhile develop a vowel/consonant positional embedding for a more phonology-aware automatic pronunciation assessment. A series of experiments conducted on the publicly-available speechocean762 dataset show that our approach can obtain significant improvements on several assessment granularities in comparison with previous work, especially on the assessment of speaking fluency and speech prosody.
【4】 Representation Learning for the Automatic Indexing of Sound Effects Libraries
标题：基于表示学习的音效库自动标引
链接：https://arxiv.org/abs/2208.09096
* 与cs.SD语音【3】为同一篇
作者：Alison B. Ma,Alexander Lerch机构：Music Informatics Group, Georgia Institute of Technology备注：Accepted at the 23rd International Society for Music Information Retrieval Conference (ISMIR 2022), 10 pages, 7 figures摘要：Labeling and maintaining a commercial sound effects library is a time-consuming task exacerbated by databases that continually grow in size and undergo taxonomy updates. Moreover, sound search and taxonomy creation are complicated by non-uniform metadata, an unrelenting problem even with the introduction of a new industry standard, the Universal Category System. To address these problems and overcome dataset-dependent limitations that inhibit the successful training of deep learning models, we pursue representation learning to train generalized embeddings that can be used for a wide variety of sound effects libraries and are a taxonomy-agnostic representation of sound. We show that a task-specific but dataset-independent representation can successfully address data issues such as class imbalance, inconsistent class labels, and insufficient dataset size, outperforming established representations such as OpenL3. Detailed experimental results show the impact of metric learning approaches and different cross-dataset training methods on representational effectiveness.
机器翻译，仅供参考
每日语音速递