今日论文合集：cs.SD语音7篇，eess.AS音频处理19篇。
cs.SD语音

【1】 A two-stage speaker extraction algorithm under adverse acoustic conditions using a single-microphone

标题：不利声学条件下的单麦克风两级说话人提取算法链接：http://arxiv.org/pdf/2303.07072v1

作者：Aviad Eisenberg,Sharon Gannot,Shlomo E. Chazan摘要：在这项工作中，我们提出了一个两阶段的方法，说话人提取混响和噪声条件下。给定期望说话者的参考信号，首先从噪声混合信号中提取干净但仍有混响的期望说话者。在第二阶段中，通过联合去混响和残留噪声及干扰减少来进一步增强所提取的信号。所提出的架构包括两个子网络，一个用于提取任务，第二个用于去混响任务。我们提出了一种针对该结构的训练策略，并表明当应用于WHAMR时，所提出的方法的性能与其他最先进的（SOTA）方法不相上下！数据集。此外，我们提出了一个新的数据集与更现实的不利声学条件，并表明我们的方法优于竞争方法时，应用于这个数据集以及。摘要：In this work, we present a two-stage method for speaker extraction under reverberant and noisy conditions. Given a reference signal of the desired speaker, the clean, but the still reverberant, desired speaker is first extracted from the noisy-mixed signal. In the second stage, the extracted signal is further enhanced by joint dereverberation and residual noise and interference reduction. The proposed architecture comprises two sub-networks, one for the extraction task and the second for the dereverberation task. We present a training strategy for this architecture and show that the performance of the proposed method is on par with other state-of-the-art (SOTA) methods when applied to the WHAMR! dataset. Furthermore, we present a new dataset with more realistic adverse acoustic conditions and show that our method outperforms the competing methods when applied to this dataset as well.

【2】 Cross-device Federated Learning for Mobile Health Diagnostics: A First Study on COVID-19 Detection标题：移动健康诊断的跨设备联合学习：新冠肺炎检测的初步研究链接：http://arxiv.org/pdf/2303.07067v1

作者：Tong Xia,Jing Han,Abhirup Ghosh,Cecilia Mascolo备注：This paper has been accepted by IEEE ICASSP 2023摘要：联合学习（FL）辅助的健康诊断模型可以合并来自大量个人边缘设备（例如，移动电话），同时将数据保持在始发设备本地，很大程度上确保了隐私。然而，这种用于健康诊断的跨器械FL方法仍然由于局部数据不平衡（局部数据由单个疾病类别组成的极端情况）和全局数据不平衡（疾病流行率在人群中通常较低）而带来许多挑战。由于联邦服务器无法访问数据分布信息，因此解决无偏模型的不平衡问题并非易事。在这篇文章中，我们提出了一个新的跨设备的健康诊断FL框架FedLoss。在这里，联邦服务器根据本地数据的预测损失对在边缘设备上训练的模型求平均值，而不是仅使用样本数作为权重。由于预测丢失可以更好地量化设备上的数据分布，因此FedLoss可以减轻数据不平衡的影响。通过呼吸音和基于症状的COVID-$19$检测任务的真实世界数据集，验证了FedLoss的优越性。与AUC-ROC为$79 %$的集中式模型相比，它实现了具有竞争力的COVID-$19$检测性能。它在灵敏度和收敛速度方面也优于最先进的FL基线。我们的工作不仅展示了联合COVID-19 $检测的前景，而且为以保护隐私的方式开发大量移动健康模型铺平了道路。摘要：Federated learning (FL) aided health diagnostic models can incorporate data from a large number of personal edge devices (e.g., mobile phones) while keeping the data local to the originating devices, largely ensuring privacy. However, such a cross-device FL approach for health diagnostics still imposes many challenges due to both local data imbalance (as extreme as local data consists of a single disease class) and global data imbalance (the disease prevalence is generally low in a population). Since the federated server has no access to data distribution information, it is not trivial to solve the imbalance issue towards an unbiased model. In this paper, we propose FedLoss, a novel cross-device FL framework for health diagnostics. Here the federated server averages the models trained on edge devices according to the predictive loss on the local data, rather than using only the number of samples as weights. As the predictive loss better quantifies the data distribution at a device, FedLoss alleviates the impact of data imbalance. Through a real-world dataset on respiratory sound and symptom-based COVID-$19$ detection task, we validate the superiority of FedLoss. It achieves competitive COVID-$19$ detection performance compared to a centralised model with an AUC-ROC of $79 %$. It also outperforms the state-of-the-art FL baselines in sensitivity and convergence speed. Our work not only demonstrates the promise of federated COVID-$19$ detection but also paves the way to a plethora of mobile health model development in a privacy-preserving fashion.

【3】 Analysing the Masked predictive coding training criterion for pre-training a Speech Representation Model标题：分析用于预训练语音表示模型的掩蔽预测编码训练准则链接：http://arxiv.org/pdf/2303.06982v1

作者：Hemant Yadav,Sunayana Sitaram,Rajiv Ratn Shah摘要：利用自监督学习（SSL）的预训练语音表示的最新发展已经在各种下游任务上产生了异常的结果。已知为掩蔽预测编码（MPC）的一种这样的技术已经被一些最高性能的模型采用。在本研究中，我们使用九个探测任务，调查了MPC丢失对HuBERT模型中不同层学习的信息类型的影响。我们的研究结果表明，在HuBERT模型的各个层学习到的内容信息量与MPC丢失呈正相关。此外，还观察到，在模型的中间层学习的任何说话者相关信息是学习过程的间接结果，因此不能使用MPC损失来控制。这些发现可以用作对语音社区中的进一步研究的启发，特别是在开发新的预训练任务或探索新的预训练准则中，所述新的预训练准则直接保留学习模型的各个层处的说话者和内容信息。摘要：Recent developments in pre-trained speech representation utilizing self-supervised learning (SSL) have yielded exceptional results on a variety of downstream tasks. One such technique, known as masked predictive coding (MPC), has been employed by some of the most high-performing models. In this study, we investigate the impact of MPC loss on the type of information learnt at various layers in the HuBERT model, using nine probing tasks. Our findings indicate that the amount of content information learned at various layers of the HuBERT model has a positive correlation to the MPC loss. Additionally, it is also observed that any speaker-related information learned at intermediate layers of the model, is an indirect consequence of the learning process, and therefore cannot be controlled using the MPC loss. These findings may serve as inspiration for further research in the speech community, specifically in the development of new pre-training tasks or the exploration of new pre-training criterion's that directly preserves both speaker and content information at various layers of a learnt model.

【4】 Improving the Intent Classification accuracy in Noisy Environment标题：提高噪声环境下意图分类的准确率链接：http://arxiv.org/pdf/2303.06585v1

作者：Mohamed Nabih Ali,Alessio Brutti,Daniele Falavigna摘要：意图分类是口语理解领域中的一个基本任务，最近引起了科学界的注意，主要是因为用端到端神经模型来处理它的可行性。以这种方式，避免使用中间步骤（即，自动语音识别）是可能的，因此避免了由于背景噪声、自发语音、用户的说话风格等引起的错误传播。为了开发可应用于真实场景的解决方案，研究环境噪声和相关的降噪技术如何利用端到端神经模型来解决意图分类任务是令人感兴趣的。本文在一个含噪的流利语音命令数据集上进行实验，将意图分类器与基于Wave-U-Net的时域语音增强方案相结合，并考虑不同的训练策略。实验结果表明，对于该任务，语音增强的使用极大地提高了噪声条件下的分类准确度，特别是当分类模型在增强信号上训练时。摘要：Intent classification is a fundamental task in the spoken language understanding field that has recently gained the attention of the scientific community, mainly because of the feasibility of approaching it with end-to-end neural models. In this way, avoiding using intermediate steps, i.e. automatic speech recognition, is possible, thus the propagation of errors due to background noise, spontaneous speech, speaking styles of users, etc. Towards the development of solutions applicable in real scenarios, it is interesting to investigate how environmental noise and related noise reduction techniques to address the intent classification task with end-to-end neural models. In this paper, we experiment with a noisy version of the fluent speech command data set, combining the intent classifier with a time-domain speech enhancement solution based on Wave-U-Net and considering different training strategies. Experimental results reveal that, for this task, the use of speech enhancement greatly improves the classification accuracy in noisy conditions, in particular when the classification model is trained on enhanced signals.

【5】 TaylorAECNet: A Taylor Style Neural Network for Full-Band Echo Cancellation标题：Taylor AECNet：一种泰勒式全频带回波抵消神经网络链接：http://arxiv.org/pdf/2303.06379v1

作者：Weiming Xu,Zhihao Guo摘要：本文介绍了aecX团队参加ICASSP 2023声学回声消除（AEC）挑战赛的情况。该系统由一个自适应滤波器和作为后置滤波器的全频带泰勒式声学回声消除神经网络（TaylorAECNet）组成。具体而言，我们利用了基于泰勒展开的解耦式可解释语音增强的最新进展，并探讨了其在AEC任务中的可行性。我们基于TaylorAECNet的方法实现了4.241的总体平均意见得分（MOS），0.767的单词准确率（WAcc），在非个性化跟踪（跟踪1）中排名第五。摘要：This paper describes aecX team's entry to the ICASSP 2023 acoustic echo cancellation (AEC) challenge. Our system consists of an adaptive filter and a proposed full-band Taylor-style acoustic echo cancellation neural network (TaylorAECNet) as a post-filter. Specifically, we leverage the recent advances in Taylor expansion based decoupling-style interpretable speech enhancement and explore its feasibility in the AEC task. Our TaylorAECNet based approach achieves an overall mean opinion score (MOS) of 4.241, a word accuracy (WAcc) ratio of 0.767, and ranks 5th in the non-personalized track (track 1).

【6】Multi-Microphone Speaker Separation by Spatial Regions标题：基于空间区域的多麦克风说话人分离链接：http://arxiv.org/pdf/2303.07143v1

作者：Julian Wechsler,Srikanth Raj Chetupalli,Wolfgang Mack,Emanuël A. P. Habets备注：Submitted to the 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing摘要：我们考虑基于区域的混响多麦克风录音源分离的任务。我们假设预定义的空间区域，每个区域具有单个活动源。目的是估计来自由参考麦克风捕获的各个空间区域的信号，同时保持信号和空间区域之间的对应性。我们提出了一种数据驱动的方法，使用一个最先进的网络的修改版本，其中不同的层模型的空间和光谱-时间信息。网络被训练以强制区域到网络输出的固定映射。使用来自LibriMix的语音，我们构建了一个专门设计用于包含区域信息的数据集。此外，我们使用置换不变量训练来训练网络。我们证明了两种训练方法都能产生区域到网络输出的固定映射，达到相当的性能，并且网络利用了空间信息。该网络在尺度不变信号失真比方面比基线网络高出1.5dB。摘要：We consider the task of region-based source separation of reverberant multi-microphone recordings. We assume pre-defined spatial regions with a single active source per region. The objective is to estimate the signals from the individual spatial regions as captured by a reference microphone while retaining a correspondence between signals and spatial regions. We propose a data-driven approach using a modified version of a state-of-the-art network, where different layers model spatial and spectro-temporal information. The network is trained to enforce a fixed mapping of regions to network outputs. Using speech from LibriMix, we construct a data set specifically designed to contain the region information. Additionally, we train the network with permutation invariant training. We show that both training methods result in a fixed mapping of regions to network outputs, achieve comparable performance, and that the networks exploit spatial information. The proposed network outperforms a baseline network by 1.5 dB in scale-invariant signal-to-distortion ratio.

【7】 Neural Diarization with Non-autoregressive Intermediate Attractors标题：具有非自回归中间吸引子的神经二元化链接：http://arxiv.org/pdf/2303.06806v1

作者：Yusuke Fujita,Tatsuya Komatsu,Robin Scheibler,Yusuke Kida,Tetsuji Ogawa备注：ICASSP 2023摘要：基于编解码器吸引子（EDA）的端到端神经网络二值化（EEND）是一种用单个神经网络同时处理整个说话人二值化问题的有效方法。虽然EEND模型可以同时产生所有帧级说话人标签，但它忽略了输出标签依赖性。在这篇论文中，我们提出了一个新的EEND模型，它在帧之间引入了标签依赖性。该方法在低层产生非自回归中间吸引子以产生说话人标记，并利用这些标记对后续层进行条件化。虽然所提出的模型以非自回归的方式工作，但是说话人标签通过参考整个中间标签序列来精炼。在双说话人CALLHOME数据集上的实验表明，具有非自回归中间吸引子的中间标签提高了Diarization的性能.与EEND-EDA相比，所提的具有更深网络的方法从中间标签中获益更多，从而导致更好的性能和训练吞吐量。摘要：End-to-end neural diarization (EEND) with encoder-decoder-based attractors (EDA) is a promising method to handle the whole speaker diarization problem simultaneously with a single neural network. While the EEND model can produce all frame-level speaker labels simultaneously, it disregards output label dependency. In this work, we propose a novel EEND model that introduces the label dependency between frames. The proposed method generates non-autoregressive intermediate attractors to produce speaker labels at the lower layers and conditions the subsequent layers with these labels. While the proposed model works in a non-autoregressive manner, the speaker labels are refined by referring to the whole sequence of intermediate labels. The experiments with the two-speaker CALLHOME dataset show that the intermediate labels with the proposed non-autoregressive intermediate attractors boost the diarization performance. The proposed method with the deeper network benefits more from the intermediate labels, resulting in better performance and training throughput than EEND-EDA.
eess.AS音频处理

【1】 Multi-Microphone Speaker Separation by Spatial Regions标题：基于空间区域的多麦克风说话人分离链接：http://arxiv.org/pdf/2303.07143v1

*与cs.SD语音【6】为同一篇

作者：Julian Wechsler,Srikanth Raj Chetupalli,Wolfgang Mack,Emanuël A. P. Habets备注：Submitted to the 2023 IEEE International Conference on Acoustics, Speech, and Signal Processing摘要：我们考虑基于区域的混响多麦克风录音源分离的任务。我们假设预定义的空间区域，每个区域具有单个活动源。目的是估计来自由参考麦克风捕获的各个空间区域的信号，同时保持信号和空间区域之间的对应性。我们提出了一种数据驱动的方法，使用一个最先进的网络的修改版本，其中不同的层模型的空间和光谱-时间信息。网络被训练以强制区域到网络输出的固定映射。使用来自LibriMix的语音，我们构建了一个专门设计用于包含区域信息的数据集。此外，我们使用置换不变量训练来训练网络。我们证明了两种训练方法都能产生区域到网络输出的固定映射，达到相当的性能，并且网络利用了空间信息。该网络在尺度不变信号失真比方面比基线网络高出1.5dB。摘要：We consider the task of region-based source separation of reverberant multi-microphone recordings. We assume pre-defined spatial regions with a single active source per region. The objective is to estimate the signals from the individual spatial regions as captured by a reference microphone while retaining a correspondence between signals and spatial regions. We propose a data-driven approach using a modified version of a state-of-the-art network, where different layers model spatial and spectro-temporal information. The network is trained to enforce a fixed mapping of regions to network outputs. Using speech from LibriMix, we construct a data set specifically designed to contain the region information. Additionally, we train the network with permutation invariant training. We show that both training methods result in a fixed mapping of regions to network outputs, achieve comparable performance, and that the networks exploit spatial information. The proposed network outperforms a baseline network by 1.5 dB in scale-invariant signal-to-distortion ratio.

【2】 Can spoofing countermeasure and speaker verification systems be jointly optimised?标题：欺骗对抗和说话人验证系统能否联合优化？链接：http://arxiv.org/pdf/2303.07073v1

作者：Wanying Ge,Hemlata Tak,Massimiliano Todisco,Nicholas Evans备注：Accepted to ICASSP 2023. Code will be available soon摘要：电子欺骗对策（CM）和自动说话人验证（ASV）子系统可以与后端分类器串联使用，作为对电子欺骗感知说话人验证（SASV）任务的解决方案。这两个子系统通常被独立地训练以解决不同的任务。虽然我们以前的工作证明了联合优化的潜力，但它也显示出过度适合扬声器和缺乏子系统互补性的趋势。仅使用从新说话人收集的适量辅助数据，我们表明联合优化降低了单独CM和ASV子系统的性能，但它仍然改善了互补性，从而提供了优越的SASV性能。使用标准的SASV评估数据和协议，联合优化相对于使用固定的、独立优化的子系统在相似训练条件下获得的性能，将等误差率降低了27%。摘要：Spoofing countermeasure (CM) and automatic speaker verification (ASV) sub-systems can be used in tandem with a backend classifier as a solution to the spoofing aware speaker verification (SASV) task. The two sub-systems are typically trained independently to solve different tasks. While our previous work demonstrated the potential of joint optimisation, it also showed a tendency to over-fit to speakers and a lack of sub-system complementarity. Using only a modest quantity of auxiliary data collected from new speakers, we show that joint optimisation degrades the performance of separate CM and ASV sub-systems, but that it nonetheless improves complementarity, thereby delivering superior SASV performance. Using standard SASV evaluation data and protocols, joint optimisation reduces the equal error rate by 27 % relative to performance obtained using fixed, independently-optimised sub-systems under like-for-like training conditions.

【3】 Adaptive Dereverberation, Noise and Interferer Reduction Using Sparse Weighted Linearly Constrained Minimum Power Beamforming标题：基于稀疏加权线性约束最小功率波束形成的自适应混响、噪声和干扰抑制链接：http://arxiv.org/pdf/2303.07027v1

作者：Henri Gode,Simon Doclo备注：30th European Signal Processing Conference (EUSIPCO 2022)摘要：干扰源、背景噪声和混响降低了助听器应用中的语音质量和可懂度。本文提出了一种基于wBLCMP波束形成器的自适应算法，旨在实现去混响、噪声和干扰抑制以及双耳线索的保留。wBLCMP波束形成器将执行去混响的多信道加权预测误差方法和执行噪声和干扰减少的线性约束最小功率波束形成器统一到单个卷积波束形成器中。我们提出在稀疏提升的lp范数代价函数中加入一个指数窗来自适应地计算最优滤波器，从而实现对运动目标说话人的跟踪。对不同位置的连续目标说话人的仿真结果表明，所提出的自适应wBLCMP波束形成器在客观语音增强性能指标方面优于非自适应wBLCMP波束形成器。摘要：Interfering sources, background noise and reverberation degrade speech quality and intelligibility in hearing aid applications. In this paper, we present an adaptive algorithm aiming at dereverberation, noise and interferer reduction and preservation of binaural cues based on the wBLCMP beamformer. The wBLCMP beamformer unifies the multi-channel weighted prediction error method performing dereverberation and the linearly constrained minimum power beamformer performing noise and interferer reduction into a single convolutional beamformer. We propose to adaptively compute the optimal filter by incorporating an exponential window into a sparsity-promoting lp-norm cost function, which enables to track a moving target speaker. Simulation results with successive target speakers at different positions show that the proposed adaptive version of the wBLCMP beamformer outperforms a non-adaptive version in terms of objective speech enhancement performance measures.

【4】 Real-Time Audio-Visual End-to-End Speech Enhancement标题：实时视听端到端语音增强链接：http://arxiv.org/pdf/2303.07005v1

作者：Zirun Zhu,Hemin Yang,Min Tang,Ziyi Yang,Sefik Emre Eskimez,Huaming Wang备注：Accepted by ICASSP 2023摘要：视听语音增强（AV-SE）方法利用辅助视觉线索来增强说话人的语音。因此，从技术上讲，它们应该能够胜过仅音频语音增强（SE）方法。然而，在文献中很少有关于能够在CPU上实时工作的AV-SE系统的著作。本文基于最近提出的端到端增强网络（E3 Net），提出了一种低时延实时音视频端到端增强（AV-E3 Net）模型。我们的主要贡献包括两个方面：1）采用密集连接模块解决了深度模型结构带来的性能下降问题。该模块显著提高了模型在AV-SE任务上的性能。2)提出了一种多级选通求和（GS）融合模块来融合音频和视觉线索。实验结果表明，与E3 net模型相比，提出的模型具有更好的感知质量和可理解性，且计算成本增加可忽略不计。摘要：Audio-visual speech enhancement (AV-SE) methods utilize auxiliary visual cues to enhance speakers' voices. Therefore, technically they should be able to outperform the audio-only speech enhancement (SE) methods. However, there are few works in the literature on an AV-SE system that can work in real time on a CPU. In this paper, we propose a low-latency real-time audio-visual end-to-end enhancement (AV-E3Net) model based on the recently proposed end-to-end enhancement network (E3Net). Our main contribution includes two aspects: 1) We employ a dense connection module to solve the performance degradation caused by the deep model structure. This module significantly improves the model's performance on the AV-SE task. 2) We propose a multi-stage gating-and-summation (GS) fusion module to merge audio and visual cues. Our results show that the proposed model provides better perceptual quality and intelligibility than the baseline E3net model with a negligible computational cost increase.

【5】 Learning-based Robust Speaker Counting and Separation with the Aid of Spatial Coherence标题：基于学习的空间相干性稳健说话人计数与分离链接：http://arxiv.org/pdf/2303.06867v1

作者：Yicheng Hsu,Mingsian Bai备注：17 pages, 14 figures摘要：提出了一种两阶段的方法，用于噪声和混响环境下的说话人计数和语音分离。使用跨时间帧的白化相对传递函数（wRTF）来计算空间相干矩阵（SCM）。基于使用SCM的特征向量构造的单纯形来估计每个说话人的全局活动函数，而根据时频仓的wRTF与目标说话人的全局活动函数加权RTF之间的相干性来计算局部相干函数。在说话人计数中，我们使用SCM的特征值和两个说话人之间帧间全局活动分布的最大相似度作为说话人计数网络（SCnet）的输入特征。在说话人分离中，利用全局和局部活动驱动网络（GLADnet）来估计说话人掩模，这对于高度重叠的语音信号特别有用。真实会议录音的实验结果表明，该系统在不需要阵列结构先验知识的情况下，具有良好的说话人计数和说话人分离性能。摘要：A two-stage approach is proposed for speaker counting and speech separation in noisy and reverberant environments. A spatial coherence matrix (SCM) is computed using whitened relative transfer functions (wRTFs) across time frames. The global activity functions of each speaker are estimated on the basis of a simplex constructed using the eigenvectors of the SCM, while the local coherence functions are computed from the coherence between the wRTFs of a time-frequency bin and the global activity function-weighted RTF of the target speaker. In speaker counting, we use the eigenvalues of the SCM and the maximum similarity of the interframe global activity distributions between two speakers as the input features to the speaker counting network (SCnet). In speaker separation, a global and local activity-driven network (GLADnet) is utilized to estimate a speaker mask, which is particularly useful for highly overlapping speech signals. Experimental results obtained from the real meeting recordings demonstrated the superior speaker counting and speaker separation performance achieved by the proposed learning-based system without prior knowledge of the array configurations.

【6】 Two-step Band-split Neural Network Approach for Full-band Residual Echo Suppression标题：全频带残余回波抑制的两步带分裂神经网络方法链接：http://arxiv.org/pdf/2303.06828v1

作者：Zihan Zhang,Shimin Zhang,Mingshuai Liu,Yanhong Leng,Zhe Han,Li Chen,Lei Xie备注：Accepted by ICASSP 2023摘要：本文提出了一种两步频带分割神经网络（TBNN）的全频带回声消除方法。具体来说，在线性滤波之后，我们将全频带信号分成宽带（16 KHz）和高带（16- 48 KHz），以便以较低的建模难度去除残留回声。宽带信号由U2编码器的更新选通卷积递归网络（GCRN）处理，而高带信号由复杂度较低的高带后置滤波器网络处理。我们提交给ICASSP 2023 AEC Challenge的方法获得了4.344的总体平均意见得分（MOS）和0.795的单词准确率（WAcc），在非个性化赛道排名中获得了2$^{nd}$（并列）。摘要：This paper describes a Two-step Band-split Neural Network (TBNN) approach for full-band acoustic echo cancellation. Specifically, after linear filtering, we split the full-band signal into wide-band (16KHz) and high-band (16-48KHz) for residual echo removal with lower modeling difficulty. The wide-band signal is processed by an updated gated convolutional recurrent network (GCRN) with U$^2$ encoder while the high-band signal is processed by a high-band post-filter net with lower complexity. Our approach submitted to ICASSP 2023 AEC Challenge has achieved an overall mean opinion score (MOS) of 4.344 and a word accuracy (WAcc) ratio of 0.795, leading to the 2$^{nd}$ (tied) in the ranking of the non-personalized track.

【7】 The NPU-Elevoc Personalized Speech Enhancement System for ICASSP2023 DNS Challenge标题：面向ICASSP2023域名系统挑战的NPU-Elevoc个性化语音增强系统链接：http://arxiv.org/pdf/2303.06811v1

作者：Xiaopeng Yan,Yindi Yang,Zhihao Guo,Liangliang Peng,Lei Xie摘要：本文介绍了我们在ICASSP 2023第五届深度噪声抑制挑战赛中使用的NPU-Elevoc个性化语音增强系统（NAPSE）。该系统在两阶段模型TEA-PSE 2. 0的基础上，重点研究了说话人嵌入融合策略，优化了模型训练流水线，充分利用了对抗性训练和多尺度损失。根据结果，我们的系统在头戴式耳机领域（领域1）并列第一，在免提领域（领域2）排名第二。摘要：This paper describes our NPU-Elevoc personalized speech enhancement system (NAPSE) for the 5th Deep Noise Suppression Challenge at ICASSP 2023. Based on the superior two-stage model TEA-PSE 2.0, our system particularly explores better strategy for speaker embedding fusion, optimizes the model training pipeline, and leverages adversarial training and multi-scale loss. According to the results, our system is tied for the 1st place in the headset track (track 1) and ranked 2nd in the speakerphone track (track 2).

【8】 Neural Diarization with Non-autoregressive Intermediate Attractors标题：具有非自回归中间吸引子的神经二元化链接：http://arxiv.org/pdf/2303.06806v1

*与cs.SD语音【7】为同一篇

作者：Yusuke Fujita,Tatsuya Komatsu,Robin Scheibler,Yusuke Kida,Tetsuji Ogawa备注：ICASSP 2023摘要：基于编解码器吸引子（EDA）的端到端神经网络二值化（EEND）是一种用单个神经网络同时处理整个说话人二值化问题的有效方法。虽然EEND模型可以同时产生所有帧级说话人标签，但它忽略了输出标签依赖性。在这篇论文中，我们提出了一个新的EEND模型，它在帧之间引入了标签依赖性。该方法在低层产生非自回归中间吸引子以产生说话人标记，并利用这些标记对后续层进行条件化。虽然所提出的模型以非自回归的方式工作，但是说话人标签通过参考整个中间标签序列来精炼。在双说话人CALLHOME数据集上的实验表明，具有非自回归中间吸引子的中间标签提高了Diarization的性能.与EEND-EDA相比，所提的具有更深网络的方法从中间标签中获益更多，从而导致更好的性能和训练吞吐量。摘要：End-to-end neural diarization (EEND) with encoder-decoder-based attractors (EDA) is a promising method to handle the whole speaker diarization problem simultaneously with a single neural network. While the EEND model can produce all frame-level speaker labels simultaneously, it disregards output label dependency. In this work, we propose a novel EEND model that introduces the label dependency between frames. The proposed method generates non-autoregressive intermediate attractors to produce speaker labels at the lower layers and conditions the subsequent layers with these labels. While the proposed model works in a non-autoregressive manner, the speaker labels are refined by referring to the whole sequence of intermediate labels. The experiments with the two-speaker CALLHOME dataset show that the intermediate labels with the proposed non-autoregressive intermediate attractors boost the diarization performance. The proposed method with the deeper network benefits more from the intermediate labels, resulting in better performance and training throughput than EEND-EDA.

【9】 Fine-tuning Strategies for Faster Inference using Speech Self-Supervised Models: A Comparative Study标题：使用语音自监督模型进行更快推理的微调策略的比较研究链接：http://arxiv.org/pdf/2303.06740v1

作者：Salah Zaiem,Robin Algayres,Titouan Parcollet,Slim Essid,Mirco Ravanelli备注：Submitted to ICASSP "Self-supervision in Audio, Speech and Beyond" workshop摘要：自监督学习（SSL）技术使得低资源环境下的自动语音识别（ASR）性能有了实质性的提高。在此背景下，已经证明较大的自监督特征提取器对于实现较低的下游ASR错误率是至关重要的。因此，更好的表现可能会被更长的推论所认可。本文探讨了在微调过程中可以部署的不同方法，以减少SSL编码器中所需的计算，从而加快推理速度。我们将大量现有技术应用于常见的ASR设置，并对它们进行基准测试，显示推理时间方面的性能下降和提高。有趣的是，我们发现在给定足够的下游数据的情况下，输入序列的简单下采样优于其他方法，具有低性能下降和高计算节省，减少了61.3%的计算，WER仅增加0.81。最后，我们分析了数据集条件变化时比较的鲁棒性，揭示了对数据集大小的敏感性。摘要：Self-supervised learning (SSL) has allowed substantial progress in Automatic Speech Recognition (ASR) performance in low-resource settings. In this context, it has been demonstrated that larger self-supervised feature extractors are crucial for achieving lower downstream ASR error rates. Thus, better performance might be sanctioned with longer inferences. This article explores different approaches that may be deployed during the fine-tuning to reduce the computations needed in the SSL encoder, leading to faster inferences. We adapt a number of existing techniques to common ASR settings and benchmark them, displaying performance drops and gains in inference times. Interestingly, we found that given enough downstream data, a simple downsampling of the input sequences outperforms the other methods with both low performance drops and high computational savings, reducing computations by 61.3% with an WER increase of only 0.81. Finally, we analyze the robustness of the comparison to changes in dataset conditions, revealing sensitivity to dataset size.

【10】 ICASSP 2023 Speech Signal Improvement Challenge标题：ICASSP 2023语音信号改善挑战赛链接：http://arxiv.org/pdf/2303.06566v1

作者：Ross Cutler,Ando Saabas,Babak Naderi,Nicolae-Cătălin Ristea,Sebastian Braun,Solomiya Branets摘要：ICASSP 2023语音信号改进挑战赛旨在促进通信系统中语音信号质量改进领域的研究。语音信号质量可以用ITU-T P.835中的SIG来测量，并且仍然是音频通信和会议系统中的首要问题。例如，在ICASSP 2022深度噪声抑制挑战赛中，背景（BAK）和整体（OVRL）质量的改善令人印象深刻，但语音信号（SIG）的改善在统计上为零。为了改善SIG，必须解决以下言语障碍领域：色彩、不连续性、响度和混响。为挑战提供了数据集和测试集，并使用扩展的ITU-T P.804众包实现确定了获胜者。结果表明，在语音质量的所有测量维度上都取得了显著改善。摘要：The ICASSP 2023 Speech Signal Improvement Challenge is intended to stimulate research in the area of improving the speech signal quality in communication systems. The speech signal quality can be measured with SIG in ITU-T P.835 and is still a top issue in audio communication and conferencing systems. For example, in the ICASSP 2022 Deep Noise Suppression challenge, the improvement in the background (BAK) and overall (OVRL) quality is impressive, but the improvement in the speech signal (SIG) is statistically zero. To improve SIG the following speech impairment areas must be addressed: coloration, discontinuity, loudness, and reverberation. A dataset and test set were provided for the challenge, and the winners were determined using an extended crowdsourced implementation of ITU-T P.804. The results show significant improvement was made across all measured dimensions of speech quality.

【11】 Transcription free filler word detection with Neural semi-CRFs标题：基于神经网络的无转录填充词检测链接：http://arxiv.org/pdf/2303.06475v1

作者：Ge Zhu,Yujia Yan,Juan-Pablo Caceres,Zhiyao Duan备注：Accepted by ICASSP 2023摘要：None摘要：Non-linguistic filler words, such as "uh" or "um", are prevalent in spontaneous speech and serve as indicators for expressing hesitation or uncertainty. Previous works for detecting certain non-linguistic filler words are highly dependent on transcriptions from a well-established commercial automatic speech recognition (ASR) system. However, certain ASR systems are not universally accessible from many aspects, e.g., budget, target languages, and computational power. In this work, we investigate filler word detection system that does not depend on ASR systems. We show that, by using the structured state space sequence model (S4) and neural semi-Markov conditional random fields (semi-CRFs), we achieve an absolute F1 improvement of 6.4% (segment level) and 3.1% (event level) on the PodcastFillers dataset. We also conduct a qualitative analysis on the detected results to analyze the limitations of our proposed system.

【12】 Relating EEG recordings to speech using envelope tracking and the speech-FFR标题：使用包络跟踪和语音FFR将脑电记录与语音相关联链接：http://arxiv.org/pdf/2303.06435v1

作者：Mike Thornton,Danilo Mandic,Tobias Reichenbach备注：2 pages, 3 figures. Accepted for ICASSP 2023 (challenge track)摘要：在言语感知过程中，听者的脑电图（EEG）反映了声学层面的处理以及更高层面的认知因素，如言语理解和注意力。然而，由于EEG信号的低信噪比，从EEG记录解码语音是具有挑战性的。我们报告了为ICASSP 2023“听觉EEG解码”信号处理大挑战赛开发的方法。一种简单的集成方法被证明可以显著地提高基线解码器的性能。通过对语音诱发的频率跟随响应和对语音的时间包络的响应进行联合解码，以及通过对各个对象微调解码器，可以实现甚至更高的分类率。我们的研究结果可以应用于听力障碍的诊断或认知导向助听器。摘要：During speech perception, a listener's electroencephalogram (EEG) reflects acoustic-level processing as well as higher-level cognitive factors such as speech comprehension and attention. However, decoding speech from EEG recordings is challenging due to the low signal-to-noise ratios of EEG signals. We report on an approach developed for the ICASSP 2023 'Auditory EEG Decoding' Signal Processing Grand Challenge. A simple ensembling method is shown to considerably improve upon the baseline decoder performance. Even higher classification rates are achieved by jointly decoding the speech-evoked frequency-following response and responses to the temporal envelope of speech, as well as by fine-tuning the decoders to individual subjects. Our results could have applications in the diagnosis of hearing disorders or in cognitively steered hearing aids.

【13】 Multi-Task Sub-Band Network For Deep Residual Echo Suppression标题：用于深层残留回波抑制的多任务子带网络链接：http://arxiv.org/pdf/2303.06404v1

作者：Jiayao Sun,Dawei Luo,Zhaoxia Li,Jindong Li,Yukai Ju,Yang Li摘要：本文介绍了SWANT团队参加ICASSP 2023 AEC挑战赛的情况。我们提出一个系统，级联一个线性滤波器与神经后滤波器。特别地，我们采用子带处理来处理全频带信号，并通过多任务学习来形成网络，其中双信号语音活动检测（DSVAD）和回声估计作为辅助任务。此外，我们还特别改进了时频卷积模块（TFCM），使用小卷积核来增加感受野。最终，我们的系统在ICASSP 2023 AEC Challenge非个性化赛道排名第4。摘要：This paper introduces the SWANT team entry to the ICASSP 2023 AEC Challenge. We submit a system that cascades a linear filter with a neural post-filter. Particularly, we adopt sub-band processing to handle full-band signals and shape the network with multi-task learning, where dual signal voice activity detection (DSVAD) and echo estimation are adopted as auxiliary tasks. Moreover, we particularly improve the time frequency convolution module (TFCM) to increase the receptive field using small convolution kernels. Finally, our system has ranked 4th in ICASSP 2023 AEC Challenge Non-personalized track.

【14】 The NPU-ASLP System for Audio-Visual Speech Recognition in MISP 2022 Challenge标题：MPU-ASLP视听语音识别系统在MISP2022挑战赛中的应用链接：http://arxiv.org/pdf/2303.06341v1

作者：Pengcheng Guo,He Wang,Bingshen Mu,Ao Zhang,Peikun Chen备注：2 pages, accepted by ICASSP 2023摘要：本文描述了我们的NPU-ASLP系统在基于多模态信息的语音处理（MISP）2022挑战赛中的音视频分析和识别（AVDR）任务。具体地，首先使用加权预测误差（WPE）和引导源分离（GSS）技术来减少混响并为每个单个扬声器生成干净的信号。然后，我们探讨了基于Branchformer和E-Branchformer的ASR系统的有效性。为了更好地利用视觉通道，提出了一种基于交叉注意的多通道融合模型，该模型显式地学习不同通道之间的上下文关系。实验结果表明，该系统在Dev和Eval集上的级联最小置换字符错误率（cpCER）分别为28.13%和31.21%，在挑战赛中获得第二名.摘要：This paper describes our NPU-ASLP system for the Audio-Visual Diarization and Recognition (AVDR) task in the Multi-modal Information based Speech Processing (MISP) 2022 Challenge. Specifically, the weighted prediction error (WPE) and guided source separation (GSS) techniques are used to reduce reverberation and generate clean signals for each single speaker first. Then, we explore the effectiveness of Branchformer and E-Branchformer based ASR systems. To better make use of the visual modality, a cross-attention based multi-modal fusion module is proposed, which explicitly learns the contextual relationship between different modalities. Experiments show that our system achieves a concatenated minimum-permutation character error rate (cpCER) of 28.13 % and 31.21 % on the Dev and Eval set, and obtains second place in the challenge.

【15】 A two-stage speaker extraction algorithm under adverse acoustic conditions using a single-microphone

标题：不利声学条件下的单麦克风两级说话人提取算法链接：http://arxiv.org/pdf/2303.07072v1

*与cs.SD语音【1】为同一篇

作者：Aviad Eisenberg,Sharon Gannot,Shlomo E. Chazan摘要：在这项工作中，我们提出了一个两阶段的方法，说话人提取混响和噪声条件下。给定期望说话者的参考信号，首先从噪声混合信号中提取干净但仍有混响的期望说话者。在第二阶段中，通过联合去混响和残留噪声及干扰减少来进一步增强所提取的信号。所提出的架构包括两个子网络，一个用于提取任务，第二个用于去混响任务。我们提出了一种针对该结构的训练策略，并表明当应用于WHAMR时，所提出的方法的性能与其他最先进的（SOTA）方法不相上下！数据集。此外，我们提出了一个新的数据集与更现实的不利声学条件，并表明我们的方法优于竞争方法时，应用于这个数据集以及。摘要：In this work, we present a two-stage method for speaker extraction under reverberant and noisy conditions. Given a reference signal of the desired speaker, the clean, but the still reverberant, desired speaker is first extracted from the noisy-mixed signal. In the second stage, the extracted signal is further enhanced by joint dereverberation and residual noise and interference reduction. The proposed architecture comprises two sub-networks, one for the extraction task and the second for the dereverberation task. We present a training strategy for this architecture and show that the performance of the proposed method is on par with other state-of-the-art (SOTA) methods when applied to the WHAMR! dataset. Furthermore, we present a new dataset with more realistic adverse acoustic conditions and show that our method outperforms the competing methods when applied to this dataset as well.

【16】 Cross-device Federated Learning for Mobile Health Diagnostics: A First Study on COVID-19 Detection标题：移动健康诊断的跨设备联合学习：新冠肺炎检测的初步研究链接：http://arxiv.org/pdf/2303.07067v1

*与cs.SD语音【2】为同一篇

作者：Tong Xia,Jing Han,Abhirup Ghosh,Cecilia Mascolo备注：This paper has been accepted by IEEE ICASSP 2023摘要：联合学习（FL）辅助的健康诊断模型可以合并来自大量个人边缘设备（例如，移动电话），同时将数据保持在始发设备本地，很大程度上确保了隐私。然而，这种用于健康诊断的跨器械FL方法仍然由于局部数据不平衡（局部数据由单个疾病类别组成的极端情况）和全局数据不平衡（疾病流行率在人群中通常较低）而带来许多挑战。由于联邦服务器无法访问数据分布信息，因此解决无偏模型的不平衡问题并非易事。在这篇文章中，我们提出了一个新的跨设备的健康诊断FL框架FedLoss。在这里，联邦服务器根据本地数据的预测损失对在边缘设备上训练的模型求平均值，而不是仅使用样本数作为权重。由于预测丢失可以更好地量化设备上的数据分布，因此FedLoss可以减轻数据不平衡的影响。通过呼吸音和基于症状的COVID-$19$检测任务的真实世界数据集，验证了FedLoss的优越性。与AUC-ROC为$79 %$的集中式模型相比，它实现了具有竞争力的COVID-$19$检测性能。它在灵敏度和收敛速度方面也优于最先进的FL基线。我们的工作不仅展示了联合COVID-19 $检测的前景，而且为以保护隐私的方式开发大量移动健康模型铺平了道路。摘要：Federated learning (FL) aided health diagnostic models can incorporate data from a large number of personal edge devices (e.g., mobile phones) while keeping the data local to the originating devices, largely ensuring privacy. However, such a cross-device FL approach for health diagnostics still imposes many challenges due to both local data imbalance (as extreme as local data consists of a single disease class) and global data imbalance (the disease prevalence is generally low in a population). Since the federated server has no access to data distribution information, it is not trivial to solve the imbalance issue towards an unbiased model. In this paper, we propose FedLoss, a novel cross-device FL framework for health diagnostics. Here the federated server averages the models trained on edge devices according to the predictive loss on the local data, rather than using only the number of samples as weights. As the predictive loss better quantifies the data distribution at a device, FedLoss alleviates the impact of data imbalance. Through a real-world dataset on respiratory sound and symptom-based COVID-$19$ detection task, we validate the superiority of FedLoss. It achieves competitive COVID-$19$ detection performance compared to a centralised model with an AUC-ROC of $79 %$. It also outperforms the state-of-the-art FL baselines in sensitivity and convergence speed. Our work not only demonstrates the promise of federated COVID-$19$ detection but also paves the way to a plethora of mobile health model development in a privacy-preserving fashion.

【17】 Analysing the Masked predictive coding training criterion for pre-training a Speech Representation Model标题：分析用于预训练语音表示模型的掩蔽预测编码训练准则链接：http://arxiv.org/pdf/2303.06982v1

*与cs.SD语音【3】为同一篇

作者：Hemant Yadav,Sunayana Sitaram,Rajiv Ratn Shah摘要：利用自监督学习（SSL）的预训练语音表示的最新发展已经在各种下游任务上产生了异常的结果。已知为掩蔽预测编码（MPC）的一种这样的技术已经被一些最高性能的模型采用。在本研究中，我们使用九个探测任务，调查了MPC丢失对HuBERT模型中不同层学习的信息类型的影响。我们的研究结果表明，在HuBERT模型的各个层学习到的内容信息量与MPC丢失呈正相关。此外，还观察到，在模型的中间层学习的任何说话者相关信息是学习过程的间接结果，因此不能使用MPC损失来控制。这些发现可以用作对语音社区中的进一步研究的启发，特别是在开发新的预训练任务或探索新的预训练准则中，所述新的预训练准则直接保留学习模型的各个层处的说话者和内容信息。摘要：Recent developments in pre-trained speech representation utilizing self-supervised learning (SSL) have yielded exceptional results on a variety of downstream tasks. One such technique, known as masked predictive coding (MPC), has been employed by some of the most high-performing models. In this study, we investigate the impact of MPC loss on the type of information learnt at various layers in the HuBERT model, using nine probing tasks. Our findings indicate that the amount of content information learned at various layers of the HuBERT model has a positive correlation to the MPC loss. Additionally, it is also observed that any speaker-related information learned at intermediate layers of the model, is an indirect consequence of the learning process, and therefore cannot be controlled using the MPC loss. These findings may serve as inspiration for further research in the speech community, specifically in the development of new pre-training tasks or the exploration of new pre-training criterion's that directly preserves both speaker and content information at various layers of a learnt model.

【18】 Improving the Intent Classification accuracy in Noisy Environment标题：提高噪声环境下意图分类的准确率链接：http://arxiv.org/pdf/2303.06585v1

*与cs.SD语音【4】为同一篇

作者：Mohamed Nabih Ali,Alessio Brutti,Daniele Falavigna摘要：意图分类是口语理解领域中的一个基本任务，最近引起了科学界的注意，主要是因为用端到端神经模型来处理它的可行性。以这种方式，避免使用中间步骤（即，自动语音识别）是可能的，因此避免了由于背景噪声、自发语音、用户的说话风格等引起的错误传播。为了开发可应用于真实场景的解决方案，研究环境噪声和相关的降噪技术如何利用端到端神经模型来解决意图分类任务是令人感兴趣的。本文在一个含噪的流利语音命令数据集上进行实验，将意图分类器与基于Wave-U-Net的时域语音增强方案相结合，并考虑不同的训练策略。实验结果表明，对于该任务，语音增强的使用极大地提高了噪声条件下的分类准确度，特别是当分类模型在增强信号上训练时。摘要：Intent classification is a fundamental task in the spoken language understanding field that has recently gained the attention of the scientific community, mainly because of the feasibility of approaching it with end-to-end neural models. In this way, avoiding using intermediate steps, i.e. automatic speech recognition, is possible, thus the propagation of errors due to background noise, spontaneous speech, speaking styles of users, etc. Towards the development of solutions applicable in real scenarios, it is interesting to investigate how environmental noise and related noise reduction techniques to address the intent classification task with end-to-end neural models. In this paper, we experiment with a noisy version of the fluent speech command data set, combining the intent classifier with a time-domain speech enhancement solution based on Wave-U-Net and considering different training strategies. Experimental results reveal that, for this task, the use of speech enhancement greatly improves the classification accuracy in noisy conditions, in particular when the classification model is trained on enhanced signals.

【19】 TaylorAECNet: A Taylor Style Neural Network for Full-Band Echo Cancellation标题：Taylor AECNet：一种泰勒式全频带回波抵消神经网络链接：http://arxiv.org/pdf/2303.06379v1

*与cs.SD语音【5】为同一篇

作者：Weiming Xu,Zhihao Guo摘要：本文介绍了aecX团队参加ICASSP 2023声学回声消除（AEC）挑战赛的情况。该系统由一个自适应滤波器和作为后置滤波器的全频带泰勒式声学回声消除神经网络（TaylorAECNet）组成。具体而言，我们利用了基于泰勒展开的解耦式可解释语音增强的最新进展，并探讨了其在AEC任务中的可行性。我们基于TaylorAECNet的方法实现了4.241的总体平均意见得分（MOS），0.767的单词准确率（WAcc），在非个性化跟踪（跟踪1）中排名第五。摘要：This paper describes aecX team's entry to the ICASSP 2023 acoustic echo cancellation (AEC) challenge. Our system consists of an adaptive filter and a proposed full-band Taylor-style acoustic echo cancellation neural network (TaylorAECNet) as a post-filter. Specifically, we leverage the recent advances in Taylor expansion based decoupling-style interpretable speech enhancement and explore its feasibility in the AEC task. Our TaylorAECNet based approach achieves an overall mean opinion score (MOS) of 4.241, a word accuracy (WAcc) ratio of 0.767, and ranks 5th in the non-personalized track (track 1).

机器翻译，仅供参考