今日论文合集：CS.SD语音与音频 共 10 篇。

本文经arXiv每日学术速递授权转载，微信公众号：arXiv_Daily

[机构]信息由AI分析生成，可能存在错误，仅供参考，以论文实际显示为准


快速导航

1. 语音识别与关键词检测 2 篇

2. 音乐信息检索与音乐生成 1 篇

3. 语音翻译与语音语言模型 1 篇

4. 数据集、基准与评测 2 篇

5. 安全、隐私与深度伪造音频 1 篇

6. 其他/综合语音音频 3 篇


1. 语音识别与关键词检测 | 2 篇

1. Swanada: Split-Encoder Capability-Routed Conformer CTC and Sub-30 MB On-Device Speech Recognition Across Twelve Scripts

Swanada：拆分编码器能力路由的Conformer CTC与跨12种文字的30MB以下端侧语音识别

AI 总结：针对端侧ASR的存储与性能瓶颈，提出Swanada系统，采用能力路由的Conformer编码器与热插拔CTC头，实现跨12种文字的30MB以下端侧语音识别，在多数据集上取得低字符错误率与低延迟表现。

链接：https://arxiv.org/abs/2611.13814

作者：Hemanth HM

英文摘要：Client-side automatic speech recognition (ASR) across Indic and Classical scripts eliminates cloud transcription costs, network tail latency, and server-side audio retention, yet neural exports face four systems bottlenecks in browsers and mobile runtimes: (1) monolithic per-language Conformer checkpoints require ~177 MiB per script (> 2.1 GiB across 12 scripts), exhausting CacheStorage quotas; (2) static relative positional encoding tables embed 10.2-20.5 MB of FP32 constants while 1x1 Conv1d layers trigger slow or unsupported ConvInteger kernels in WebAssembly SIMD; (3) sliding-window CTC flickers on boundary tokens; and (4) mobile WebAudio pipelines suffer 44.1/48 kHz sample-rate mismatches and silent AudioWorklet stalls under iOS WKWebView. We present Swanada, an open-source 16 kHz streaming ASR system spanning Classical Sanskrit, Indian English, and ten Indian languages (sa, en, hi, kn, mr, bn, ta, te, ml, gu, pa, or) across browser WebAssembly, this http URL, and Python. Swanada pairs capability-routed shared INT8 Conformer encoders (a 17-layer 177.38 MiB Studio tier at $d = 512$ for desktop/servers and a distilled 14-layer 25.35 MiB Mobile tier at $d = 256$ for phones) with twelve hot-swappable 515.0 KiB native-script linear CTC heads ($257 \times 512$), cutting 12-language storage by $11.6\times$ (183.4 MiB) and $67.8\times$ (31.39 MiB; 25.85 MiB first load). Runtime protobuf tail surgery exposes the shared representation in $< 2$ ms, while dynamic positional synthesis, MatMul pointwise convolutions, and post-quantization QInt8 head calibration preserve sub-30 MB accuracy. Across $N = 480$ held-out and $N = 1,280$ open public test clips (FLEURS, LibriSpeech, GLOBE, AMI, wild Sanskrit), Swanada achieves 5.43% held-out and 8.87% 12-language open CER on Studio (18.56% and 19.23% on Mobile) at $0.0097\times$-$0.0264\times$ RTF, with 30.6 ms median TTFT and 0.00% committed-prefix churn.


2. Earlybird: Predictive Streaming Speech Recognition in 29M Parameters

Earlybird：参数规模为29M的预测式流式语音识别模型

AI 总结：本研究提出参数29M的Earlybird流式语音识别模型，通过随机截断音频学习，在多个数据集上实现单词发音前更早预测，准确率优于多数同类模型。

链接：https://arxiv.org/abs/2611.14421

作者：Sahil Mahendrakar

英文摘要：Streaming speech recognizers output a word only after the speaker has finished it, typically 90-400 ms later. For a voice agent every millisecond counts: the sooner the words are outputted, the sooner the agent can start a search or plan its reply. We present Earlybird, a 29M-parameter streaming recognizer (16.8 MB at 4 bits). Earlybird learns from audio cut off at random points, including partway through a word, and is asked to predict every word that is due by then in full, plus a guess of the next two words. Its raw token probabilities give a confidence for every word including the one still being spoken. On clean audiobook speech (LibriSpeech test-clean), the median word is predicted correctly 90 ms before the speaker finishes it, while released streaming models get it 90 to 400 ms after it ends. Earlybird is the earliest on all six test sets we time, covering audiobooks, TED talks, public speech, earnings calls, and synthetic speech. On the same audiobooks, 69% of words are predicted correctly before they end, against 3-25% for the other models. For words still being spoken, confidence is calibrated, with an expected calibration error (ECE) of 3.7 to 5.5. On the eight English Open ASR Leaderboard sets, it is more accurate than Moonshine Tiny on every set and roughly ties Whistle, a model of the same size.


2. 音乐信息检索与音乐生成 | 1 篇

3. MusicFlux: Flexible Conditioning for Text-Guided Music Generation and Editing

MusicFlux：用于文本引导的音乐生成与编辑的灵活条件控制

AI 总结：本研究开发了基于流匹配文本到音频模型的MusicFlux，提出两种权衡编辑性能与计算效率的条件控制方法，用于文本引导的音乐生成与编辑，相关方法性能与强基线相当且效率有提升。

链接：https://arxiv.org/abs/2611.13834

作者：Teysir Baoueb (IP Paris, LTCI, IDS, S2A), Leonardo Boulitreau de Menezes Martins Marques (IP Paris, LTCI, IDS, S2A), Paraskevas Stamatiadis (S2A, LTCI, IDS), Ga{ë}l Richard (S2A, IDS, IP Paris, LTCI)

英文摘要：Music editing transforms existing recordings while preserving their original attributes. Recent approaches adapt pretrained text-to-music models for tasks like inpainting, event insertion, or restoration by conditioning them with context audio. It was recently proposed for image editing to unify generation and editing by treating the source image as in-context information. However, applying this directly to music is computationally expensive due to long audio sequences. In this work, we develop MusicFlux, a text-guided music editor built on an efficient pretrained flow-matching text-to-audio model. We introduce two conditioning approaches that trade off editing performance and computational efficiency. Experiments show that both are competitive with strong baselines, with one achieving better editing quality and the other preserving the original sequence length and substantially improving computational efficiency.


3. 语音翻译与语音语言模型 | 1 篇

4. AURA: Auditory Spatial Intelligence via a Unified Representation from a General-Audio Encoder through Complete Alignment

AURA：通过通用音频编码器的完整对齐实现统一表示的听觉空间智能

AI 总结：AURA研究将未空间监督的通用音频编解码器经完整对齐后，在SpatialSoundQA数据集上，以约40%可训练参数实现比最强专用系统高10.31个百分点的双声源方向准确率。

链接：https://arxiv.org/abs/2611.13687

机构：Southeast University（东南大学）； Rutgers University（罗格斯大学）； University of California, Merced（加州大学默塞德分校）

作者：Boyan Han, Xiaoyan Wang, Zhe Sun, Ziqi Wang, Yujun Cai, Hongli Cao, Yiwei Wang, Ming-Hsuan Yang, Liang An

英文摘要：Answering spatial questions about multi-source scenes requires a model both to identify each sound and to locate it. Existing spatial audio-language models obtain spatial information from a dedicated encoder, which separates spatial and semantic processing into distinct feature spaces. We study whether a general-audio codec trained without spatial supervision can serve both purposes. A controlled layer-wise analysis shows that the frozen codec retains spatial information, with lateral cues carried by the difference between the two ear channels. This information, however, is not readily accessible: in dual-source scenes, a model trained on these features predicts left/right at chance and, on the remaining axes, assigns the detected direction to the queried and the non-queried source equally often. We attribute this failure to an audio--text information asymmetry, in which the audio contains both sources while the answer describes only the queried one. A controlled $2\times2$ experiment shows that either completing the scene description or exposing the inter-channel difference lifts left/right accuracy above chance, whereas binding directions to the queried source requires both. Building on these findings, AURA keeps the codec and the language model frozen and exceeds the strongest dedicated system on SpatialSoundQA by 10.31 points in dual-source direction accuracy, with about 40\% of its trainable parameters.


4. 数据集、基准与评测 | 2 篇

5. FaiSonic MMD: A popular music dataset with temporally aligned multidimensional annotations for multitask music analysis

FaiSonic MMD：用于多任务音乐分析的带有时间对齐多维标注的流行音乐数据集

AI 总结：该研究推出含500首歌曲的FaiSonic MMD数据集，为多任务音乐分析提供时间对齐的多维标注与基准，支持音乐维度间关系研究。

链接：https://arxiv.org/abs/2611.13503


作者：Tieyao Zhang, Yuke Liu, Jiaxing Yu, Xinda Wu, Genfang Chen

英文摘要：Temporal analysis in music information retrieval concerns the positions of musical events and structures within recordings. Joint modeling has demonstrated the value of combining musical descriptions, but existing resources differ in joint coverage and expert verification across dimensions, constraining detailed joint supervision and the analysis of relationships on the same recordings. To support multitask modeling and the study of relationships among musical dimensions in popular music, we introduce FaiSonic Multidimensional Music Dataset (FaiSonic MMD), containing 500 songs totaling approximately 32.86 hours, comprising 250 Chinese-language and 250 English-language songs. Professional musicians annotate phrases, sections, chords, and rhythm in real recordings, followed by independent review. Events and intervals across dimensions share a recording-based time reference while retaining their respective musical granularity and boundaries, allowing direct comparison across musical descriptions. We describe the annotations, collection procedure, and statistics, and establish a comprehensive multitask baseline for phrase boundaries, section boundaries, section functions, chords, beats, and downbeats. We report reference performance for all prediction targets. FaiSonic MMD provides shared expert annotations and accompanying benchmarks for temporal music analysis.


6. Assessing Alignment in Automatic Piano Transcription Datasets: A Benchmark for Scoring Methods

自动钢琴转录数据集中的对齐评估：评分方法基准

AI 总结：该研究针对自动钢琴转录数据集的对齐误差问题，提出评估音频-MIDI对齐评分方法的框架，引入STI评分方法并完成基准测试，发现STI等方法性能最优，为数据集整理等提供支持。

链接：https://arxiv.org/abs/2611.14443

作者：Lucas de Souza Brandão, Lucas Gris, Joaquim Cavalcante, Chukwuemeka Nkama, Andrea Poltronieri

英文摘要：Automatic Music Transcription (AMT) relies on paired audio--MIDI datasets, implicitly assuming near-perfect temporal alignment. However, widely used piano datasets have been shown to contain alignment errors that may bias model evaluation and training. We propose a framework for evaluating audio--MIDI alignment scoring methods through controlled temporal, pitch, and structural MIDI corruptions. We also introduce STI, a scoring method built on AMT models, and benchmark it against existing alignment scoring approaches. Our results show that STI, together with methods based on CENS and spectrograms, achieves the strongest overall performance, with STI exhibiting the highest sensitivity to fine-grained temporal misalignments, the main alignment error in AMT datasets. We then apply the best-performing methods to widely used piano AMT datasets and identify trends in their relative alignment quality. These results support more informed dataset curation, model training, and AMT evaluation.


5. 安全、隐私与深度伪造音频 | 1 篇

7. Which Acoustic Features Survive Compression? Tracker-Estimated First-Formant Statistics for Codec-Robust Audio DeepfakeDetection

哪些声学特征能在压缩后保留？用于编解码器鲁棒音频深度伪造检测的跟踪器估计第一共振峰统计量

AI 总结：该研究针对音频深度伪造检测中编解码器压缩导致性能下降的问题，筛选出4个具有高区分度的第一共振峰统计量，构建的4特征RBF-SVM相比全特征模型，在编解码器不匹配下性能下降幅度更小，为音频深度伪造检测提供了稳定的补充诊断信号。

链接：https://arxiv.org/abs/2611.13258

机构：Clarkson University（克拉克森大学）

作者：Yash Sukhdeve (1), Aaron Storey (2), Masudul H. Imtiaz (1) ((1) Department of Electrical and Computer Engineering, Clarkson University, Potsdam, NY 13699, USA, (2) Department of Computer Science, Clarkson University, Potsdam, NY 13699, USA)

英文摘要：Audio deepfake detectors that work well on clean lab recordings often degrade after telephone or platform compression. This paper asks a specific question: which low-dimensional acoustic descriptors stay informative after codec degradation? We screen 36 utterance-level features(formants, PNCCs, LFCCs, prosody) on ASVspoof 2019 data and test on the official ASVspoof 2021 LA set across seven codec conditions. Of 33 statistically significant descriptors, only four show large practical separation (|r| >0.5, bootstrap 95% CIs), all first-formant (F1) measures, highly redundant (PC1: 79% variance). The payoff is stability: the 36-feature SVM degrades 14 pp under codec mismatch (9.5%-23.5% EER), while the four-feature RBF-SVM degrades only 5.9 pp (17.0%-22.9%). The F1 subset varies 3.4 pp across conditions versus 13.1 pp for the full set. F1 differences persist after controlling for speaker identity (p greater than 10^10, mixed-effects model). The results suggest that physically interpretable F1 statistics offer a compact, codec-stable diagnostic signal that may complement higher accuracy neural detectors.


6. 其他/综合语音音频 | 3 篇

8. Navigating relational agency in AI feedback musicianship: a diffractive dialogue

在AI反馈音乐技艺中导航关系性能动性：一种衍射式对话

AI 总结：本研究以能动性实在论与衍射式对话方法论，通过五次使用AI乐器Sù和Agentier的表演，揭示人机创意协作的关系性能动性，打破主导-响应二元对立，贡献相关词汇与协作模型。

链接：https://arxiv.org/abs/2611.13555

机构：The Australian National University（澳大利亚国立大学）； Dolby Laboratories（杜比实验室）

作者：Yichen Wang, Charles Patrick Martin, David Gunawan, David Cooper

英文摘要：Musical AI systems behave in non-linear, dynamic ways that exceed the tool-oriented framings inherited from earlier instrument design. Existing frames of control and ensemble interaction do not articulate the entangled, relational agency that emerges when performers work with AI feedback instruments over time. We examine a duet practice with two AI-enabled feedback instruments, Sù and Agentier, in which feedback operates at the inner level of the AI's recurrent processes and the outer loop of musical interaction between performer, instrument, and co-performer. Our work draws on agential realism and a diffractive dialogue methodology to read our independent reflections through one another across five performances, examining how agency is enacted between performers and AI instruments. We find that control and agency are relational and shift across performances; musical textures enact feedback moments of control with performers; reaching a productive feedback environment is itself a shared exploratory act; and performers maintain their instruments through funnelling and injection, developing the skill of navigating agency along a continuous spectrum. Feedback musicianship with AI instruments demands navigation of this relational agency, displacing the leading-versus-responding binary that initially framed our reflections. This work contributes a materially-grounded vocabulary for AI feedback musicianship and a model of human-AI creative collaboration in which mutual agency is worked for through practice rather than designed in.


9. Spatio-Temporal Calibration of a Vehicle-Mounted Microphone Array for Autonomous Driving

自动驾驶车载麦克风阵列的时空校准

AI 总结：针对自动驾驶车载麦克风阵列校准的空白，提出以气球爆裂为跨模态目标的加权校准流程，在全尺寸车辆上实现高精度声源跟踪，布置12次爆裂即可满足需求。

链接：https://arxiv.org/abs/2611.14003

机构：Technical University of Munich（慕尼黑工业大学）； TUM School of Engineering and Design（慕尼黑工业大学工程与设计学院）； Munich Institute of Robotics and Machine Intelligence (MIRMI)（慕尼黑机器人与机器智能研究所）

作者：Marvin Seegert, Richard Taupitz, Korbinian Moller, Johannes Betz

英文摘要：Microphones let an autonomous vehicle sense into areas its cameras and LiDAR cannot see, by picking up the sound that other traffic participants emit. Locating those sources and fusing them with the visual perception requires the microphone array to be calibrated and registered relative to the vehicle's other sensors. Existing calibration methods assume small, unobstructed indoor rigs, whereas a vehicle body casts an acoustic shadow that straight-line propagation cannot represent, and to our knowledge no calibration of an array distributed around a vehicle has been reported. We propose a pipeline that uses a balloon burst as a single cross-modal target, which the LiDAR locates in metric coordinates, every microphone hears as a sharp onset and a camera times. Each observation is weighted by its own timing variance, which discounts shadowed arrivals instead of discarding them and needs no model of the vehicle. On a full-size vehicle with eight microphones over a 4.9 m aperture, the held-out range residual of 6.37 cm approaches the 6.07 cm floor set by the onset timing noise, and the calibrated array tracks a moving source to within a few degrees of the vehicle's own LiDAR. A dozen bursts spread around the vehicle suffice, and spreading them matters more than adding more.


10. Exploring Post-Hoc Disentanglement of Audio Foundation Models for Music Similarity

探索音频基础模型的事后解纠缠以用于音乐相似度计算

AI 总结：本文研究将条件相似度网络（CSN）适配于冻结的预训练音频基础模型，实现事后解纠缠，以提升音乐相似度任务的属性解纠缠和检索性能。

链接：https://arxiv.org/abs/2611.14415

作者：Lucas de Souza Brandão, Gabriel da Costa Merlin, Diego Furtado Silva

英文摘要：Audio foundation models like CLAP, MuQ, and MERT excel at capturing general musical similarity. On the other hand, their entangled latent spaces limit interpretability and fine-grained attribute control. Conditional Similarity Networks (CSN) address this by pairing a backbone representation model with learned masks that gate the embedding space into attribute-specific subspaces. However, existing methods train the backbone from scratch, rather than leveraging pretrained foundation models. We instead present a systematic study of post-hoc disentanglement, adapting CSN to gate frozen pretrained embeddings. We evaluate adding trainable parameters to adapt the embedding under different masking and pooling regimes across two datasets. Our findings show that minimal adaptation capacity is sufficient for post-hoc gating to improve both attribute-specific disentanglement and overall retrieval performance, while pooling, masking, and depth choices yield limited additional gains.