【1】 Examining Audio Communication Mechanisms for Supervising Fleets of Agricultural Robots
标题：农业机器人机群监控的音频通信机制研究
链接：https://arxiv.org/abs/2208.10455
作者：Abhi Kamboj,Tianchen Ji,Katie Driggs-Campbell机构：Driggs-Campbell are with the department ofElectrical and Computer Engineering at the University of Illinois, eduThis work was supported by the USDA National Institute of Foodand Agriculture (USDANIFA)备注：Camera ready version for IEEE RO-MAN 2022摘要：Agriculture is facing a labor crisis, leading to increased interest in fleets of small, under-canopy robots (agbots) that can perform precise, targeted actions (e.g., crop scouting, weeding, fertilization), while being supervised by human operators remotely. However, farmers are not necessarily experts in robotics technology and will not adopt technologies that add to their workload or do not provide an immediate payoff. In this work, we explore methods for communication between a remote human operator and multiple agbots and examine the impact of audio communication on the operator's preferences and productivity. We develop a simulation platform where agbots are deployed across a field, randomly encounter failures, and call for help from the operator. As the agbots report errors, various audio communication mechanisms are tested to convey which robot failed and what type of failure occurs. The human is tasked with verbally diagnosing the failure while completing a secondary task. A user study was conducted to test three audio communication methods: earcons, single-phrase commands, and full sentence communication. Each participant completed a survey to determine their preferences and each method's overall effectiveness. Our results suggest that the system using single phrases is the most positively perceived by participants and may allow for the human to complete the secondary task more efficiently. The code is available at: https://github.com/akamboj2/Agbot-Sim.
【2】 The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation
标题：2022年GENEA挑战赛：基于数据驱动的共语手势生成方法的大规模评测
链接：https://arxiv.org/abs/2208.10441
作者：Youngwoo Yoon,Pieter Wolfert,Taras Kucherenko,Carla Viegas,Teodor Nikolov,Mihail Tsakov,Gustav Eje Henter机构：ETRI, Daejeon, Republic of Korea, IDLab, Ghent University – imec, Ghent, Belgium, SEED – Electronic Arts (EA), Stockholm, Sweden, Carnegie Mellon University, Pittsburgh, USA, NOVA University Lisbon, Lisbon, Portugal, Umeå University, Umeå, Sweden备注：12 pages, 5 figures; final version for ACM ICMI 2022摘要：This paper reports on the second GENEA Challenge to benchmark data-driven automatic co-speech gesture generation. Participating teams used the same speech and motion dataset to build gesture-generation systems. Motion generated by all these systems was rendered to video using a standardised visualisation pipeline and evaluated in several large, crowdsourced user studies. Unlike when comparing different research papers, differences in results are here only due to differences between methods, enabling direct comparison between systems. This year's dataset was based on 18 hours of full-body motion capture, including fingers, of different persons engaging in dyadic conversation. Ten teams participated in the challenge across two tiers: full-body and upper-body gesticulation. For each tier we evaluated both the human-likeness of the gesture motion and its appropriateness for the specific speech signal. Our evaluations decouple human-likeness from gesture appropriateness, which previously was a major challenge in the field. The evaluation results are a revolution, and a revelation. Some synthetic conditions are rated as significantly more human-like than human motion capture. To the best of our knowledge, this has never been shown before on a high-fidelity avatar. On the other hand, all synthetic motion is found to be vastly less appropriate for the speech than the original motion-capture recordings. Additional material is available via the project website at https://youngwoo-yoon.github.io/GENEAchallenge2022/
【3】 Multi-View Attention Transfer for Efficient Speech Enhancement
标题：基于多视点注意力转移的语音增强算法
链接：https://arxiv.org/abs/2208.10367
作者：Wooseok Shin,Hyun Joon Park,Jin Sob Kim,Byung Hoon Lee,Sung Won Han机构：School of Industrial and Management Engineering, Korea University, Seoul, Republic of Korea备注：Accepted by Interspeech 2022摘要：Recent deep learning models have achieved high performance in speech enhancement; however, it is still challenging to obtain a fast and low-complexity model without significant performance degradation. Previous knowledge distillation studies on speech enhancement could not solve this problem because their output distillation methods do not fit the speech enhancement task in some aspects. In this study, we propose multi-view attention transfer (MV-AT), a feature-based distillation, to obtain efficient speech enhancement models in the time domain. Based on the multi-view features extraction model, MV-AT transfers multi-view knowledge of the teacher network to the student network without additional parameters. The experimental results show that the proposed method consistently improved the performance of student models of various sizes on the Valentini and deep noise suppression (DNS) datasets. MANNER-S-8.1GF with our proposed method, a lightweight model for efficient deployment, achieved 15.4x and 4.71x fewer parameters and floating-point operations (FLOPs), respectively, compared to the baseline model with similar performance.
【4】 Representation Learning with Graph Neural Networks for Speech Emotion Recognition
标题：基于图神经网络表征学习的语音情感识别
链接：https://arxiv.org/abs/2208.09830
机构：Department of Artificial Intelligence, Dongguk University, Seoul, Korea备注：AAAI 2022 Workshop on Graphs and More Complex Structures for Learning and Reasoning (GCLR)摘要：Learning expressive representation is crucial in deep learning. In speech emotion recognition (SER), vacuum regions or noises in the speech interfere with expressive representation learning. However, traditional RNN-based models are susceptible to such noise. Recently, Graph Neural Network (GNN) has demonstrated its effectiveness for representation learning, and we adopt this framework for SER. In particular, we propose a cosine similarity-based graph as an ideal graph structure for representation learning in SER. We present a Cosine similarity-based Graph Convolutional Network (CoGCN) that is robust to perturbation and noise. Experimental results show that our method outperforms state-of-the-art methods or provides competitive results with a significant model size reduction with only 1/30 parameters.
【5】 An Initial Investigation for Detecting Vocoder Fingerprints of Fake Audio
标题：伪音频声码器指纹检测方法初探
链接：https://arxiv.org/abs/2208.09646
作者：Xinrui Yan,Jiangyan Yi,Jianhua Tao,Chenglong Wang,Haoxin Ma,Tao Wang,Shiming Wang,Ruibo Fu机构：Institute of Automation, Chinese, Academy of Sciences, University of Chinese Academy of, Beijing, China, University of Science and Technology, of China备注：Accepted by ACM Multimedia 2022 Workshop: First International Workshop on Deepfake Detection for Audio Multimedia摘要：Many effective attempts have been made for fake audio detection. However, they can only provide detection results but no countermeasures to curb this harm. For many related practical applications, what model or algorithm generated the fake audio also is needed. Therefore, We propose a new problem for detecting vocoder fingerprints of fake audio. Experiments are conducted on the datasets synthesized by eight state-of-the-art vocoders. We have preliminarily explored the features and model architectures. The t-SNE visualization shows that different vocoders generate distinct vocoder fingerprints.
【6】 Fully Automated End-to-End Fake Audio Detection
标题：全自动端到端虚假音频检测
链接：https://arxiv.org/abs/2208.09618
作者：Chenglong Wang,Jiangyan Yi,Jianhua Tao,Haiyang Sun,Xun Chen,Zhengkun Tian,Haoxin Ma,Cunhang Fan,Ruibo Fu机构：University of Science and Technology, of China, Hefei, China, NLPR, Institute of Automation, Beijing, China, Chinese Academy of Sciences, China, School of Artificial Intelligence, University of Chinese Academy of, CAS Center for Excellence in Brain摘要：The existing fake audio detection systems often rely on expert experience to design the acoustic features or manually design the hyperparameters of the network structure. However, artificial adjustment of the parameters can have a relatively obvious influence on the results. It is almost impossible to manually set the best set of parameters. Therefore this paper proposes a fully automated end-toend fake audio detection method. We first use wav2vec pre-trained model to obtain a high-level representation of the speech. Furthermore, for the network structure, we use a modified version of the differentiable architecture search (DARTS) named light-DARTS. It learns deep speech representations while automatically learning and optimizing complex neural structures consisting of convolutional operations and residual blocks. The experimental results on the ASVspoof 2019 LA dataset show that our proposed system achieves an equal error rate (EER) of 1.08%, which outperforms the state-of-the-art single system.
【7】 Visualising Model Training via Vowel Space for Text-To-Speech Systems
标题：文语转换系统中基于元音空间的可视化模型训练
链接：https://arxiv.org/abs/2208.09775
作者：Binu Abeysinghe,Jesin James,Catherine I. Watson,Felix Marattukalam机构：Dept. of Electrical, Computer, and Software Engineering, The University of Auckland, New Zealand备注：Accepted to Interspeech 2022摘要：With the recent developments in speech synthesis via machine learning, this study explores incorporating linguistics knowledge to visualise and evaluate synthetic speech model training. If changes to the first and second formant (in turn, the vowel space) can be seen and heard in synthetic speech, this knowledge can inform speech synthesis technology developers. A speech synthesis model trained on a large General American English database was fine-tuned into a New Zealand English voice to identify if the changes in the vowel space of synthetic speech could be seen and heard. The vowel spaces at different intervals during the fine-tuning were analysed to determine if the model learned the New Zealand English vowel space. Our findings based on vowel space analysis show that we can visualise how a speech synthesis model learns the vowel space of the database it is trained on. Perception tests confirmed that humans could perceive when a speech synthesis model has learned characteristics of the speech database it is training on. Using the vowel space as an intermediary evaluation helps understand what sounds are to be added to the training database and build speech synthesis models based on linguistics knowledge.
【1】 Spatially Selective Active Noise Control Systems
标题：空间选择性有源噪声控制系统
链接：https://arxiv.org/abs/2208.09997
作者：Tong Xiao,Buye Xu,Chuming Zhao机构：UniversityofTechnologySydney摘要：Wearable active noise control (ANC) systems are commonly designed to achieve maximal sound reduction regardless of the incident direction of the sound. When desired sound is present, the state-of-the-art methods add a separate system to reconstruct it, often with distortion and latency. In this work, we propose a multi-channel ANC system that only controls the sound from the undesired directions. The system truly preserves the desired sound instead of reproducing it. The proposed algorithm imposes a spatial constraint on the hybrid ANC cost function to achieve spatial selectivity. We simulated the proposed algorithm based on a microphone array on a pair of augmented eye-glasses and compared it with the existing methods in the literature. Not only did the proposed system provide better noise reduction while preserving the physical sound wave from the desired source, but it also consumed much less energy, which is critical for lightweight wearable devices. Overall, the proposed method has demonstrated significant advantages over the state-of-the-art methods.
【2】 Visualising Model Training via Vowel Space for Text-To-Speech Systems
标题：文语转换系统中基于元音空间的可视化模型训练
链接：https://arxiv.org/abs/2208.09775
* 与cs.SD语音【7】为同一篇
作者：Binu Abeysinghe,Jesin James,Catherine I. Watson,Felix Marattukalam机构：Dept. of Electrical, Computer, and Software Engineering, The University of Auckland, New Zealand备注：Accepted to Interspeech 2022摘要：With the recent developments in speech synthesis via machine learning, this study explores incorporating linguistics knowledge to visualise and evaluate synthetic speech model training. If changes to the first and second formant (in turn, the vowel space) can be seen and heard in synthetic speech, this knowledge can inform speech synthesis technology developers. A speech synthesis model trained on a large General American English database was fine-tuned into a New Zealand English voice to identify if the changes in the vowel space of synthetic speech could be seen and heard. The vowel spaces at different intervals during the fine-tuning were analysed to determine if the model learned the New Zealand English vowel space. Our findings based on vowel space analysis show that we can visualise how a speech synthesis model learns the vowel space of the database it is trained on. Perception tests confirmed that humans could perceive when a speech synthesis model has learned characteristics of the speech database it is training on. Using the vowel space as an intermediary evaluation helps understand what sounds are to be added to the training database and build speech synthesis models based on linguistics knowledge.
【3】 Examining Audio Communication Mechanisms for Supervising Fleets of Agricultural Robots
标题：农业机器人机群监控的音频通信机制研究
链接：https://arxiv.org/abs/2208.10455
* 与cs.SD语音【1】为同一篇
作者：Abhi Kamboj,Tianchen Ji,Katie Driggs-Campbell机构：Driggs-Campbell are with the department ofElectrical and Computer Engineering at the University of Illinois, eduThis work was supported by the USDA National Institute of Foodand Agriculture (USDANIFA)备注：Camera ready version for IEEE RO-MAN 2022摘要：Agriculture is facing a labor crisis, leading to increased interest in fleets of small, under-canopy robots (agbots) that can perform precise, targeted actions (e.g., crop scouting, weeding, fertilization), while being supervised by human operators remotely. However, farmers are not necessarily experts in robotics technology and will not adopt technologies that add to their workload or do not provide an immediate payoff. In this work, we explore methods for communication between a remote human operator and multiple agbots and examine the impact of audio communication on the operator's preferences and productivity. We develop a simulation platform where agbots are deployed across a field, randomly encounter failures, and call for help from the operator. As the agbots report errors, various audio communication mechanisms are tested to convey which robot failed and what type of failure occurs. The human is tasked with verbally diagnosing the failure while completing a secondary task. A user study was conducted to test three audio communication methods: earcons, single-phrase commands, and full sentence communication. Each participant completed a survey to determine their preferences and each method's overall effectiveness. Our results suggest that the system using single phrases is the most positively perceived by participants and may allow for the human to complete the secondary task more efficiently. The code is available at: https://github.com/akamboj2/Agbot-Sim.
【4】 The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation
标题：2022年GENEA挑战赛：基于数据驱动的共语手势生成方法的大规模评测
链接：https://arxiv.org/abs/2208.10441
* 与cs.SD语音【2】为同一篇
作者：Youngwoo Yoon,Pieter Wolfert,Taras Kucherenko,Carla Viegas,Teodor Nikolov,Mihail Tsakov,Gustav Eje Henter机构：ETRI, Daejeon, Republic of Korea, IDLab, Ghent University – imec, Ghent, Belgium, SEED – Electronic Arts (EA), Stockholm, Sweden, Carnegie Mellon University, Pittsburgh, USA, NOVA University Lisbon, Lisbon, Portugal, Umeå University, Umeå, Sweden备注：12 pages, 5 figures; final version for ACM ICMI 2022摘要：This paper reports on the second GENEA Challenge to benchmark data-driven automatic co-speech gesture generation. Participating teams used the same speech and motion dataset to build gesture-generation systems. Motion generated by all these systems was rendered to video using a standardised visualisation pipeline and evaluated in several large, crowdsourced user studies. Unlike when comparing different research papers, differences in results are here only due to differences between methods, enabling direct comparison between systems. This year's dataset was based on 18 hours of full-body motion capture, including fingers, of different persons engaging in dyadic conversation. Ten teams participated in the challenge across two tiers: full-body and upper-body gesticulation. For each tier we evaluated both the human-likeness of the gesture motion and its appropriateness for the specific speech signal. Our evaluations decouple human-likeness from gesture appropriateness, which previously was a major challenge in the field. The evaluation results are a revolution, and a revelation. Some synthetic conditions are rated as significantly more human-like than human motion capture. To the best of our knowledge, this has never been shown before on a high-fidelity avatar. On the other hand, all synthetic motion is found to be vastly less appropriate for the speech than the original motion-capture recordings. Additional material is available via the project website at https://youngwoo-yoon.github.io/GENEAchallenge2022/
【5】 Multi-View Attention Transfer for Efficient Speech Enhancement
标题：基于多视点注意力转移的语音增强算法
链接：https://arxiv.org/abs/2208.10367
* 与cs.SD语音【3】为同一篇
作者：Wooseok Shin,Hyun Joon Park,Jin Sob Kim,Byung Hoon Lee,Sung Won Han机构：School of Industrial and Management Engineering, Korea University, Seoul, Republic of Korea备注：Accepted by Interspeech 2022摘要：Recent deep learning models have achieved high performance in speech enhancement; however, it is still challenging to obtain a fast and low-complexity model without significant performance degradation. Previous knowledge distillation studies on speech enhancement could not solve this problem because their output distillation methods do not fit the speech enhancement task in some aspects. In this study, we propose multi-view attention transfer (MV-AT), a feature-based distillation, to obtain efficient speech enhancement models in the time domain. Based on the multi-view features extraction model, MV-AT transfers multi-view knowledge of the teacher network to the student network without additional parameters. The experimental results show that the proposed method consistently improved the performance of student models of various sizes on the Valentini and deep noise suppression (DNS) datasets. MANNER-S-8.1GF with our proposed method, a lightweight model for efficient deployment, achieved 15.4x and 4.71x fewer parameters and floating-point operations (FLOPs), respectively, compared to the baseline model with similar performance.
【6】 Representation Learning with Graph Neural Networks for Speech Emotion Recognition
标题：基于图神经网络表征学习的语音情感识别
链接：https://arxiv.org/abs/2208.09830
* 与cs.SD语音【4】为同一篇
机构：Department of Artificial Intelligence, Dongguk University, Seoul, Korea备注：AAAI 2022 Workshop on Graphs and More Complex Structures for Learning and Reasoning (GCLR)摘要：Learning expressive representation is crucial in deep learning. In speech emotion recognition (SER), vacuum regions or noises in the speech interfere with expressive representation learning. However, traditional RNN-based models are susceptible to such noise. Recently, Graph Neural Network (GNN) has demonstrated its effectiveness for representation learning, and we adopt this framework for SER. In particular, we propose a cosine similarity-based graph as an ideal graph structure for representation learning in SER. We present a Cosine similarity-based Graph Convolutional Network (CoGCN) that is robust to perturbation and noise. Experimental results show that our method outperforms state-of-the-art methods or provides competitive results with a significant model size reduction with only 1/30 parameters.
【7】 An Initial Investigation for Detecting Vocoder Fingerprints of Fake Audio
标题：伪音频声码器指纹检测方法初探
链接：https://arxiv.org/abs/2208.09646
* 与cs.SD语音【5】为同一篇
作者：Xinrui Yan,Jiangyan Yi,Jianhua Tao,Chenglong Wang,Haoxin Ma,Tao Wang,Shiming Wang,Ruibo Fu机构：Institute of Automation, Chinese, Academy of Sciences, University of Chinese Academy of, Beijing, China, University of Science and Technology, of China备注：Accepted by ACM Multimedia 2022 Workshop: First International Workshop on Deepfake Detection for Audio Multimedia摘要：Many effective attempts have been made for fake audio detection. However, they can only provide detection results but no countermeasures to curb this harm. For many related practical applications, what model or algorithm generated the fake audio also is needed. Therefore, We propose a new problem for detecting vocoder fingerprints of fake audio. Experiments are conducted on the datasets synthesized by eight state-of-the-art vocoders. We have preliminarily explored the features and model architectures. The t-SNE visualization shows that different vocoders generate distinct vocoder fingerprints.
【8】 Fully Automated End-to-End Fake Audio Detection
标题：全自动端到端虚假音频检测
链接：https://arxiv.org/abs/2208.09618
* 与cs.SD语音【6】为同一篇
作者：Chenglong Wang,Jiangyan Yi,Jianhua Tao,Haiyang Sun,Xun Chen,Zhengkun Tian,Haoxin Ma,Cunhang Fan,Ruibo Fu机构：University of Science and Technology, of China, Hefei, China, NLPR, Institute of Automation, Beijing, China, Chinese Academy of Sciences, China, School of Artificial Intelligence, University of Chinese Academy of, CAS Center for Excellence in Brain摘要：The existing fake audio detection systems often rely on expert experience to design the acoustic features or manually design the hyperparameters of the network structure. However, artificial adjustment of the parameters can have a relatively obvious influence on the results. It is almost impossible to manually set the best set of parameters. Therefore this paper proposes a fully automated end-toend fake audio detection method. We first use wav2vec pre-trained model to obtain a high-level representation of the speech. Furthermore, for the network structure, we use a modified version of the differentiable architecture search (DARTS) named light-DARTS. It learns deep speech representations while automatically learning and optimizing complex neural structures consisting of convolutional operations and residual blocks. The experimental results on the ASVspoof 2019 LA dataset show that our proposed system achieves an equal error rate (EER) of 1.08%, which outperforms the state-of-the-art single system.
每日语音速递