今日论文合集：CS.SD语音与音频 共 10 篇。

本文经arXiv每日学术速递授权转载，微信公众号：arXiv_Daily

[机构]信息由AI分析生成，可能存在错误，仅供参考，以论文实际显示为准


快速导航

1. 语音合成与声音生成 1 篇

2. 音频事件检测与场景理解 3 篇

3. 音乐信息检索与音乐生成 1 篇

4. 语音翻译与语音语言模型 2 篇

5. 数据集、基准与评测 1 篇

6. 其他/综合语音音频 2 篇


1. 语音合成与声音生成 | 1 篇

1. Real-Time Short-Response Selection for Spoken Dialogue Systems

口语对话系统的实时短响应选择

AI 总结：针对口语对话系统的自然轮次切换需求，提出基于对比学习的实时短响应选择方法，可掩盖生成延迟，性能优于随机及LLM基线，且仅增加极小开销。

链接：https://arxiv.org/abs/2611.09057

机构：The University of Osaka（大阪大学）

作者：Hien Ohnaka, Kazuyo Onishi, Koichiro Yoshino

英文摘要：To realize spoken dialogue systems that can interact naturally with users, it is important for the system to respond to user utterances at appropriate timings. To achieve such natural turn-taking, the system needs not only to predict appropriate response timings but also to mask delays caused by language generation and speech synthesis so that they are not perceived by the user. We propose a real-time short-response selection task that can be integrated with existing turn-taking models to mask generation delay. To address this task, the proposed model learns the mapping between dialogue context and short responses using contrastive learning. When a turn-taking event is detected, the system selects a response from a set of pre-generated candidates. The selected response is played while the main response is being generated. Experimental results show that the proposed method learns to select short responses based on dialogue context, outperforming random and LLM-based baselines. Moreover, integrating response selection maintained turn-taking prediction performance and added minimal processing overhead.


2. 音频事件检测与场景理解 | 3 篇

2. Fuzzy-FedAvg: Server-Aware Client Selection for Federated Speech Emotion Recognition in Heterogeneous Environments

Fuzzy-FedAvg：异构环境下联邦语音情感识别的服务器感知客户端选择方法

AI 总结：本文提出Fuzzy-FedAvg框架，通过模糊Q学习学习客户端选择权重，结合PCAS节点管理掉队者，在IEMOCAP等数据集上较FedAvg等方法提升SER性能并减少训练时间。

链接：https://arxiv.org/abs/2611.08818

机构：College of Aeronautical Engineering, National University of Sciences & Technology (NUST)（巴基斯坦国立科技大学航空工程学院）； Queensland University of Technology (QUT)（昆士兰科技大学）； Commonwealth Scientific and Industrial Research Organisation (CSIRO)（联邦科学与工业研究组织）

作者：Farrukh Pervez, Siddique Latif, Moid Sandhu, Raja Jurdak

英文摘要：Federated speech emotion recognition (SER) is challenged by speaker-dependent, class-imbalanced data and by heterogeneous client devices. This paper proposes \textit{Fuzzy-FedAvg}, a server-aware framework in which the server learns Client Selection Weights (CSWs) through fuzzy Q-learning from global-model feedback. The same CSWs weight client accuracy, training time, and \divkpi{} in both client selection and aggregation, while a dedicated PCAS node manages stragglers through staleness-discounted deferred updates. On IEMOCAP and CREMA-D, \textit{Fuzzy-FedAvg} achieves higher unweighted average recall (UAR) than FedAvg in all configurations under severe non-IID conditions, and in 22 of 24 participation-level and 10 of 12 training-horizon configurations overall. It also outperforms FedProx and Oort, while SCAFFOLD and Power-of-Choice remain competitive, and it reduces cumulative training time by 81.2\% in a FedScale-based evaluation. The benefit is greatest under strong statistical heterogeneity and narrows as data become more balanced.


3. SAS: Scene-Annotated AudioSet Strong for Joint Scene-Event Analysis

SAS：用于联合场景-事件分析的带场景标注的AudioSet Strong

AI 总结：该研究提出带场景标注的AudioSet Strong数据集，用于联合场景-事件分析，发现场景特有的事件更能表征场景，重新加权事件共现可提升表示效果，场景标注能为机器听觉系统提供补充信息。

链接：https://arxiv.org/abs/2611.09199

机构：LTCI； Télécom Paris（巴黎电信学院）； Institut Polytechnique de Paris（巴黎综合理工学院）； Université de Lorraine（洛林大学）； NVIDIA（英伟达）

作者：Michel Olvera (S2A, LTCI, IDS), Paraskevas Stamatiadis (S2A, LTCI, IDS), Romain Serizel (MULTISPEECH), Slim Essid (IDS, S2A, LTCI), Gaël Richard (S2A, IDS)

英文摘要：Context is central to human auditory perception, as listeners interpret sounds within their acoustic environment to resolve ambiguity. However, large-scale datasets that jointly capture sound events and acoustic scenes remain limited. We address this by augmenting AudioSet Strong with clip-level scene annotations under two taxonomies for joint sceneevent modeling. A corpus-scale analysis shows that scene identity is better captured by events distinctive to a scene than by those that merely occur frequently. Reweighting event co-occurrences toward scene-specific associations produces representations that better align with human-derived taxonomies, while also exposing scenes where the acoustic and semantic views diverge. This geometric divergence predicts classification errors, while supervised validation confirms that the proposed scene annotations provide meaningful complementary information for developing scalable context-aware machine listening systems.


4. AudioSphere: Towards Self-Supervised Spatial Audio Representation Models

AudioSphere：面向自监督空间音频表示模型

AI 总结：本研究提出AudioSphere多通道自监督空间音频模型，通过联合时空掩码策略学习声学与空间特征，实现基于冻结骨干网络的声音事件检测与定位，性能优于现有模型并发布了RealSELD基准。

链接：https://arxiv.org/abs/2611.09275

机构：Donders Institute, Radboud University（拉德堡德大学唐德斯研究所）； Mortimer B Zuckerman Institute, Columbia University（哥伦比亚大学莫蒂默·B·扎克曼研究所）

作者：Goksenin Yuksel, Marcel van Gerven, Kiki van der Heijden

英文摘要：Recent audio embedding models learn representations that transfer across clip- and frame-level tasks, yet they operate almost exclusively on single-channel signals and are blind to spatial structure. Spatial audio models capture that structure but rely on clip-level supervision or contrastive objectives, yielding embeddings unsuitable for frame-level tasks like sound event detection and localization (SELD). We argue this dichotomy is not fundamental: patch-level spatial self-supervision can jointly learn acoustic and spatial features at both temporal granularities. We introduce AudioSphere, a multi-channel masked autoencoder trained on Ambisonics recordings that reconstructs both spectral content and directional cues via a joint spatiotemporal masking strategy that masks both time-frequency patches and intensity vectors. AudioSphere supports frame-level predictions while producing strong clip-level representations via aggregation. Alongside the model, we release RealSELD, a standardized benchmark for evaluating frozen backbones on real-world multi-channel SELD. AudioSphere enables, for the first time, SELD with a frozen self-supervised backbone, while matching or exceeding prior self-supervised audio embedding models on clip-level classification tasks spanning environmental sound, speech, and music, and on frame-level sound event detection. All code and data are available at this https URL and this https URL.


3. 音乐信息检索与音乐生成 | 1 篇

5. SPDMX: Large-Scale Open Stem Audio From Symbolic Music

SPDMX：来自符号音乐的大规模公开 Stem 音频

AI 总结：该研究提出基于聆听指导的公开领域 PDMX 乐谱渲染方法 SPDMX，构建大规模公开许可合成多轨 Stem 语料库，可支撑多种音乐相关机器学习任务。

链接：https://arxiv.org/abs/2611.09140

机构：University of California, San Diego（加利福尼亚大学圣地亚哥分校）

作者：Phillip Long, Eduardo Escoto, Julian McAuley, Zachary Novack

英文摘要：Large public datasets have driven recent progress in machine learning, from text and vision to speech and general audio. Music remains an outlier, as copyright blocks redistribution of training audio, while many modern tasks need individual stems rather than mixtures. Existing multitrack resources trade off realism, scale, and licensing, while synthetic alternatives rely on proprietary VSTs or naive open rendering that yields weak stems. We introduce SPDMX, a listening-guided rendering of public-domain PDMX scores into an openly-licensed stem corpus with aligned scores and an open render pipeline: $253{,}982$ songs, $490{,}159$ mono stems, and $5{,}469$ song-hours at $44.1$ kHz under CC BY 4.0, with paired MIDI and identifiers linking to PDMX metadata. $88{,}403$ songs ($34.8\%$; $2{,}734$ song-hours) provide multitrack supervision. To our knowledge, this is the largest openly-licensed synthetic multitrack subset by song-hours, enabling source separation, stem-conditioned synthesis, MIDI-to-audio, transcription, and joint symbolic-audio modeling. We release the corpus, metadata, and open pipeline for reuse on other symbolic collections. Demos and experimental notes are on the project page.


4. 语音翻译与语音语言模型 | 2 篇

6. FoleyForge: Free-form, Controllable, Rectified-flow Editing of Sound Effects

FoleyForge：自由形式、可控制的整流流音效编辑

AI 总结：FoleyForge是一种生成模型，通过引入数据管道解决音频编辑数据缺乏问题，微调整流流模型并扩展属性控制，实现了指令遵循、音频质量与源保真度的良好平衡。

链接：https://arxiv.org/abs/2611.08695

机构：Adobe（奥多比公司）； New York University（纽约大学）

作者：Giovana Morais, Hugo Flores García, Prem Seetharaman, Johannes Imort, Justin Salamon, Magdalena Fuentes, Oriol Nieto

英文摘要：We present FoleyForge, a generative model for instruction-based editing of sound effects. Given an input clip and a free-form instruction describing the desired change, it applies the edit while preserving the input's identity. A single model handles semantic, stylistic, and DSP-like changes. We introduce a data pipeline that guides an audio language model to author both DSP-based acoustic edits and similarity-preserving semantic edits, thus addressing the lack of publicly available audio editing data. Such data is then used to fine-tune a rectified-flow audio generation model, which we extend to expose attribute controls for loudness change, edit strength, and an input-derived contour envelope, each tunable via classifier-free guidance, yielding a more expressive editing model. FoleyForge achieves state-of-the-art instruction adherence at competitive audio quality and source faithfulness.


7. Improving Large Audio-Language Model Embedding Baselines for Instruction-Aware Speech Retrieval

改进用于指令感知语音检索的大型音频-语言模型嵌入基线

AI 总结：本研究针对指令感知语音检索问题，提出IRIS-Embed模型，通过对比训练与指令感知音频层加权适配大型音频-语言模型为嵌入模型，在INSPIRE和MAEB基准上均实现显著性能提升。

链接：https://arxiv.org/abs/2611.09122

机构：National Taiwan University（台湾大学）； NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)（台湾大学人工智能卓越研究中心）

作者：Chen-An Li, Cheng-Han Chiang, Hung-yi Lee

英文摘要：Speech retrieval finds relevant spoken documents given a spoken query. The spoken query alone may not reveal the retrieval goal, such as finding an answer or matching speaker identity. Instruction-aware speech retrieval expresses this goal through a natural-language text instruction. Large audio-language models (LALMs) effectively process speech and text, and our analysis shows that different retrieval requirements favor hidden states from different audio-encoder layers. We introduce IRIS-Embed, which adapts LALMs into embedding models through contrastive training and instruction-aware audio layer weighting. We construct about 900K training samples from real data and synthetic data, with instructions expressing requirements for one or more speech attributes. On INSPIRE, an instruction-aware speech retrieval benchmark, IRIS-Embed consistently outperforms the corresponding LALM baselines. The best model achieves 62.92% macro-average Recall@50, surpassing the strongest baseline at 45.15%. Controlled comparisons demonstrate instruction awareness and the benefits of audio layer weighting. On MAEB, a massive audio embedding benchmark, macro-average gains of 9.4-19.8 points over the corresponding baselines show generalization beyond speech retrieval.


5. 数据集、基准与评测 | 1 篇

8. AircraftWitness: How Often Aircraft Can Locate Recordings

AircraftWitness：飞机能定位录音位置的频率

AI 总结：本研究提出AircraftWitness基准，利用飞机音频的多普勒曲线和ADS-B数据，测试了三种定位器对户外录音位置的定位频率，发现其定位效果优于随机猜测但整体准确率较低。

链接：https://arxiv.org/abs/2611.08791

机构：New York University（纽约大学）

作者：Kris Atallah (New York University, New York, USA)

英文摘要：Aircraft overhead can reveal where an outdoor recording was made. Propellers and rotors leave Doppler curves in audio, and public ADS-B archives log equipped aircraft. We measure how often, blind and against chance. Our main test was registered and timestamped before any audio, and its two amendments came before any answer was read. It covers 24 new places, with the recorders' own clocks, a 300 s slack and a 20 km square. The best of three locators places clear-pass clips within 1 km at 4 places, where a uniform guess predicts 0.57 (p = 0.0065 after correcting for the pick). With the wrong time or swapped audio it places none. The other two hit 1 and 2. On 360 clips from 7 places of a GPS-timed network, two locators place 6% and 8% within 1 km. By place, a unit chosen after that run, they hit 4 and 5 of 7 where a uniform guess predicts 2.3. On 1,600 random public recordings, picked and cut without the truth in a time-bounded run, the three locators answer at most 20 times and place at most 2. We release AircraftWitness, a benchmark, and pass labels for 11.7 million recordings.


6. 其他/综合语音音频 | 2 篇

9. Concordance-Optimal Continuous Music Emotion Estimation: Variance Decomposition, Dual-Branch Temporal Fusion, and Conformal Prediction Intervals

一致性最优连续音乐情感估计：方差分解、双分支时间融合与共形预测区间

AI 总结：本研究针对连续音乐情感识别的标准指标缺陷，通过方差分解优化目标函数，提出双分支时间融合的MMAS模型，在DEAM和PMEmo数据集上取得优于基线的性能，还发布了带共形预测区间的开源模型。

链接：https://arxiv.org/abs/2611.08966

机构：Birla Institute of Technology, Mesra（贝拉理工学院梅斯拉校区）

作者：Mohammad Faraz Akhtar, Aditya Parashar, Ayush Jain, Madhur Gupta, Ritesh Kumar, Soubhik Chakraborty

英文摘要：Continuous music emotion recognition (MER) estimates how Arousal and Valence evolve within a song, yet the field's standard metric, pooled frame-level correlation, is dominated by between-song differences and cannot distinguish a model that tracks emotional dynamics from one that only ranks songs by their average level. We decompose annotation variance into between- and within-song components and show that mean squared error (MSE) targets the former, while windowed Lin's concordance correlation coefficient (CCC = $r \cdot C_b$) targets the latter by penalising the variance compression MSE induces. A single weight $\lambda$ controls the trade-off: on 1,764 DEAM songs, raising $\lambda$ from 0 to 1 lowers pooled Arousal $r$ from 0.843 to 0.810 while raising mean within-song $r$ from 0.204 to 0.368. Building on this objective, we present MMAS (Multimodal Music Analysis System), an open-source model that predicts Arousal and Valence every 0.5 s from raw audio. It fuses frozen MERT-v1-95M embeddings with ten symbolic features from automatic transcription through a dual-branch temporal convolutional network with cross-modal attention. Because both branches are audio-derived, one model trains jointly on DEAM and PMEmo without domain conditioning. Under song-grouped 5-fold cross-validation (2,531 songs), pooled $r$ reaches 0.801/0.685 (Arousal/Valence) on DEAM and 0.742/0.746 on PMEmo. Against a ridge baseline on identical features, within-song Arousal $r$ roughly triples (0.344 vs. 0.125; $n = 353$ songs, Wilcoxon $p = 1.7 \times 10^{-21}$, $d = 0.56$). Complete transcription failure costs at most 0.04 pooled $r$. Split-conformal intervals attain 93.7% (Arousal) and 90.0% (Valence) marginal coverage at a 95% target, with the shortfall traced to within-song dependence. MMAS is released as a pip-installable package.


10. AnomSep: Bridging Optimization-Based Separation and Deep Neural Refinement for Anomaly Detection with Listenable Explanations

AnomSep：结合基于优化的分离与深度神经网络优化以实现带可听解释的异常检测


AI 总结：AnomSep结合优化分离与深度神经网络优化，在实现与端到端基线相当的异常检测性能的同时，还能提供与检测决策直接关联的可听解释，且分离质量优于基线分离器。

链接：https://arxiv.org/abs/2611.09279

机构：The University of Tokyo（东京大学）

作者：Koki Shoda, Jun Younes Louhi Kasahara, Qi An, Atsushi Yamashita

英文摘要：We propose AnomSep, which combines optimiza-tion-based separation with deep neural refinement to achieve accurate anomaly detection with listenable explanations. Anomaly detection requires no predefined anomalous categories, and understanding the basis for each detection helps security personnel assess the validity of each detection. End-to-end anomaly detectors can achieve high detection accuracy but output only scalar scores, and existing anomalous sound separation