今日论文合集：CS.SD语音与音频 共 5 篇。

本文经arXiv每日学术速递授权转载，微信公众号：arXiv_Daily

[机构]信息由AI分析生成，可能存在错误，仅供参考，以论文实际显示为准


快速导航

1. 语音合成与声音生成 3 篇

2. 安全、隐私与深度伪造音频 1 篇

3. 其他/综合语音音频 2 篇


1. 语音合成与声音生成 | 3 篇

1. CAST-Bench: Tracing Every Sound to Its Source in Multi-Shot Audio-Video Generation

CAST-Bench：在多镜头音视频生成中追踪每一个声音的来源

AI 总结：针对现有多镜头音视频生成基准无法追踪声音来源的问题，提出CAST-BENCH基准，通过18项指标评估12个生成器，发现视觉质量接近饱和但音频语义分数差异大，多镜头提示表现更差，多人语音等仍为挑战。

链接：https://arxiv.org/abs/2611.05422

机构：Northwestern Polytechnical University（西北工业大学）； Fudan University（复旦大学）； Zhejiang University（浙江大学）； Peking University（北京大学）

作者：Yucong Liu, Qi Tian, Shuyuan Tu, Kaihang Pan, Weijie Kong, Chao Pan

英文摘要：Existing benchmarks for joint text-to-audio-video (T2AV) generation evaluate perceptual quality and audiovisual alignment, but balanced sound-source sam?pling and separate reporting by shot structure remain limited. This makes it dif?ficult to determine whether a specific sound is missing, attributed to the wrong source, or lost across shot transitions. To address this, we introduce CAST?BENCH, a benchmark designed to trace every generated sound back to its in?tended source in multi-shot audio-video generation. The core idea is to formu?late T2AV correctness as an audiovisual contract in which each requested sound must be emitted by the right source, synchronized with the corresponding visual action, and carry the right semantic content. Concretely, the benchmark orga?nizes 900 prompts into 9 sound-source categories, with 100 prompts per cate?gory, spanning speech, animals, instruments, ambient sounds, and other emitter types. The prompts include 439 single-shot and 461 multi-shot narratives, each derived from a structured specification that links every audio event to its emit?ter, subject, and temporal slot. The benchmark provides an 18-metric evaluation suite covering three dimensions: Visual Performance, Content Consistency, and Audio & Semantic Alignment. We further benchmark 12 representative gen?erators and find that visual quality is nearly saturated across leading systems, yet audio semantic scores still differ markedly. Across the nine shared mod?els, multi-shot prompts receive lower aggregate scores under the subset-specific scoring rules and yield different model rankings. Multi-person speech, con?sistent speaker identity across shots, and cross-shot event retention remain the main unresolved challenges. All prompts and evaluation code are available at this https URL.


2. VIPBench: A Human-Aligned Benchmark for Voice Identity Perception in the Age of Voice Cloning

VIPBench：语音克隆时代的人类对齐语音身份感知基准

AI 总结：该研究推出VIPBench基准，含1290名听众对9800组含真实、克隆、融合语音的录音对的判断，评估11款语音模型与人类身份感知的对齐度，发现经说话人标签训练的模型对齐度更优，推动基于听众判断训练说话人模型。

链接：https://arxiv.org/abs/2611.05638

机构：Northwestern University（西北大学）； Kellogg School of Management, Northwestern University（西北大学凯洛格管理学院）

作者：Xudong Tang, Matthew Groh

英文摘要：Speaker-recognition benchmarks test how accurately a model recovers the speaker label of a recording, treating voice identity as a property of the person who produced it. However, these benchmarks do not evaluate how humans perceive voice identity. Whether a listener hears two voices as the same person has consequences for intellectual property rights, cybersecurity, and everyday life. We introduce the Voice Identity Perception benchmark (VIPBench), which contains 124,876 same-or-different judgments from 1,290 listeners on 9,800 recording pairs from 100 public figures. The pairs include real recordings, voice clones generated by a text-to-speech system, and morphs between two speakers' voices. We also define four tasks that evaluate the alignment between speaker-representation models and human identity perception. The first two tasks measure how well a model's similarity score predicts the proportion of listeners who respond "same" and the majority response. The other two measure whether a model and listeners find the same speaker pairs similar, and whether a model's relationship to listener judgments holds equally for real and synthetic recordings. We evaluate eleven publicly available speech models. Listener judgments diverge from speaker labels most often for voice clones: in 41% of responses, listeners judged a clone and a real recording of the same speaker to be different people. No model reaches the noise ceiling; the best model explains 61% of the explainable variance. Models trained with speaker labels align with listeners better than self-supervised models do, yet a model with a 12% error rate on speaker labels aligns with listeners as well as models with error rates below 2%. VIPBench enables the evaluation of speaker models against human perception on real, cloned, and morphed speech, and our results motivate training speaker models directly on listener judgments.


3. LiveFoley: Dynamic Real-Time Video-to-Audio Generation for Immersive Interactive Worlds

LiveFoley：面向沉浸式交互世界的动态实时视频转音频生成

AI 总结：提出LiveFoley流式因果自回归扩散架构，通过尾随滑动窗口、统一Varhop框架及改进LocDiT头，解决流式V2A的视听对齐与音频质量问题，性能优于现有流式系统且接近离线模型。

链接：https://arxiv.org/abs/2611.05988

机构：Monash University（莫纳什大学）； Nanjing University（南京大学）； ShengShu Technology（声数科技）； Tampere University（坦佩雷大学）； Alibaba Group（阿里巴巴集团）； Tsinghua University（清华大学）

作者：Yusheng Dai, Vladimir Iashin, Kangdi Wang, Yuxuan Jiang, Ilpo Viertola, Weiqiang Wang, Qiuhong Ke, Min Zhao, Jianfei Cai

英文摘要：Streaming video-to-audio (V2A) generation is essential for immersive interactive applications. However, real-time constraints leave existing streaming V2A methods far behind offline models: (1) visual frontends shrink their receptive fields to cut look-ahead latency, severely degrading audio-visual alignment; (2) causal autoregressive diffusion backends denoise each audio patch in isolation, impairing audio quality. To this end, we present LiveFoley, a streaming causal autoregressive diffusion architecture that addresses both bottlenecks with imperceptible latency. For the visual frontend, we introduce a Trailing Sliding Window that converts non-causal segment-level encoders into streaming modules without retraining. By simply realigning the window, it preserves precise audio-visual alignment, while a configurable hop size reduces the frontend look-ahead well below the human perceptual threshold. Building on this tunable hop size, we propose the Unified Varhop Framework to accommodate scenarios with diverse latency and computation budgets. For the audio backend, we explore widening the denoising window of the local diffusion transformer (LocDiT) head in two temporal directions. History LocDiT adds past context to improve quality at no latency cost and serves as our default. Alternatively, Future-Warmup LocDiT breaks the autoregressive deadlock with an approximate future condition, further improving generation quality at the cost of an acceptable backend look-ahead. Extensive experiments demonstrate that LiveFoley matches offline models in temporal alignment and leads streaming systems in generation quality. Project page: this https URL


2. 安全、隐私与深度伪造音频 | 1 篇

4. GraphSeal: Graph-Augmented Speech Deepfake Detection via Temporal-Spectral Artifact Propagation

GraphSeal：基于时谱伪影传播的图增强语音深度伪造检测

AI 总结：GraphSeal是一种图增强语音深度伪造检测方法，通过构建时谱图传播伪影线索，在多个公开数据集上取得了优于或匹配现有方法的检测性能。

链接：https://arxiv.org/abs/2611.06171

机构：Pusan National University（釜山国立大学）； Shanghai Jiao Tong University（上海交通大学）； Hanyang University（汉阳大学）

作者：Lee Seung-woo, Bowen Qi, Kim Min-jun, Jang Won-young

英文摘要：Current speech deepfake detectors treat each utterance as a flat sequence of frames, limiting their ability to capture long-range spectral inconsistencies that span non-adjacent regions of an audio signal. We propose \textbf{GraphSeal}, which constructs a temporal-spectral graph over short speech segments and propagates artifact cues through graph attention layers before a global readout produces the bonafide-vs-spoof decision. Each node in the graph corresponds to a fixed-length segment; edges encode temporal adjacency and spectral similarity, letting the network attend to distant frames that share suspicious spectral patterns. A multi-head graph attention mechanism weights neighboring nodes according to both learned features and edge attributes, and a hierarchical readout aggregates node representations into a single utterance-level embedding for binary classification. GraphSeal is front-end agnostic: it operates on top of any self-supervised or handcrafted feature extractor. On ASVspoof 2019~LA, ASVspoof 2021~LA, ASVspoof~5, and the In-the-Wild dataset, GraphSeal achieves equal error rates of 0.52\%, 2.41\%, 8.17\%, and 5.24\%, respectively, matching or outperforming prior methods that rely on fixed-topology graph networks or purely sequential architectures.


3. 其他/综合语音音频 | 2 篇

5. SonoDyno: A Sonic Dynamometer for Engine Speed and Load Estimation from Audio

SonoDyno：一种用于从音频估计发动机转速与负载的声学测功仪

AI 总结：提出SonoDyno仅通过麦克风信号估计发动机转速、气缸数与负载强度，在未知发动机的测试中，其转速估计准确率达93%-97%，优于通用音高跟踪器。

链接：https://arxiv.org/abs/2611.06262

机构：Audio Research & Development, Impulse Audio Lab GmbH（音频研发，Impulse Audio Lab有限公司）

作者：Robin Doerfler, Beat Rossmy

英文摘要：Controllable engine sound synthesis, Active Sound Design and engine order analysis all build on the relationship between the sound of an engine and its momentary operating state: its speed and its load. Recordings of engine sound are abundant in archives, sample packs and online video, yet they rarely carry time-aligned annotations of that state, which limits their applicability to controllable engine sound generation. We present SonoDyno, which estimates the engine speed, the cylinder count and an engine-normalized load intensity from a microphone signal alone, with no tachometer, no bus access and no prior knowledge of the engine configuration. A convolutional scorer of 124,302 parameters reads the harmonic order energy of each frame at three time-frequency resolutions to jointly score cylinder count and speed, and a decoder imposes the rotational dynamics of the engine on the speed trajectory. A second head estimates the load intensity, the position of the torque between the engine's drag floor at 0 and its full-load ceiling at 1, from the order levels at the decoded speed. On recordings of engines it has never heard, with microphone position, engine type and cylinder count unknown, SonoDyno places 93 to 97% of the frames within 2% of the true speed, identifies the cylinder count with 80% accuracy and confuses the octave on fewer than 2% of the frames. On an unseen engine of a known type, the load intensity has a mean absolute error of 0.12, and the sign of the torque, driving or braking, is right on 91% of the frames. General pitch trackers fall behind even when the harmonic order of each engine is supplied: on our validation recordings and at a tolerance of 5%, the best of them places 62% of the frames within the tolerance, against 96% for SonoDyno.


6. Drowning in Noise: On Signal Saliency in Self-Supervised Learning for Ocean Acoustics

淹没在噪声中：海洋声学自监督学习中的信号显著性研究

AI 总结：该研究针对海洋声学SSL中噪声占比过高导致模型性能下降的问题，以BEATs为测试平台，发现30小时人工筛选的显著性数据预训练效果远超1240小时未过滤数据，证明信息稀疏领域数据筛选比数据规模更重要。

链接：https://arxiv.org/abs/2611.06725

机构：Carnegie Mellon University（卡内基梅隆大学）； Stanford（斯坦福大学）

作者：Shikhar Bharadwaj, Samuele Cornell, Shinji Watanabe, Malcolm Slaney

英文摘要：Self-supervised learning (SSL) for audio typically benefits from larger pre-training corpora, but this relationship assumes that training data carries meaningful learning signal. In ocean acoustics, this assumption is often violated: passive recordings are dominated by waves, wind, and silence, with biological vocalizations occupying a small fraction of the total duration. We show that this overrepresentation of noise causes SSL models to allocate representational capacity to acoustically trivial structure, degrading downstream performance. Using BEATs as a testbed, we demonstrate that pre-training on 30 hours of salient recordings (segments containing animal vocalizations) outperforms 1,240 hours of unfiltered data on a 31-class marine mammal classification benchmark. Under linear probing, the manually curated salient set achieves markedly better accuracy than the full corpus, despite using 41 times less data. Automatic saliency-based filtering improves over unfiltered pre-training, but still falls well short of the manually curated salient set. Our findings suggest that, in domains where informative content is temporally sparse, data curation matters more than data scale for SSL pre-training.