今日论文合集：cs.SD语音7篇，eess.AS音频处理11篇。
cs.SD语音
【1】 Uncertainty Quantification of Bandgaps in Acoustic Metamaterials with  Stochastic Geometric Defects and Material Properties标题：具有随机几何缺陷和材料性质的声学超材料带隙的不确定性量化链接：https://arxiv.org/abs/2310.12869
作者：Han Zhang,Rayehe Karimi Mahabadi,Cynthia Rudin,Johann Guilleminot,L. Catherine Brinson机构：Catherine Brinsona, Duke University, Durham, NC, USA摘要：本文研究了不确定性量化技术的效用，即频谱投影和多项式混沌扩展，在减少随机材料特性和几何缺陷的声超材料色散带响应特性的采样需求。一种新的方法编码的几何缺陷在一个可解释的，分辨率无关的输入空间概率分布的形成展示。通过蒙特卡罗、求积法则和稀疏网格采样与代理模型拟合相结合，在保持精确输出空间概率分布的同时，分别在一维和七维输入空间中实现了$\sim10^0$和$\sim10^1$的数量级采样缩减.摘要：This paper studies the utility of techniques within uncertainty quantification, namely spectral projection and polynomial chaos expansion, in reducing sampling needs for characterizing acoustic metamaterial dispersion band responses given stochastic material properties and geometric defects. A novel method of encoding geometric defects in an interpretable, resolution independent is showcased in the formation of input space probability distributions. Orders of magnitude sampling reductions down to $\sim10^0$ and $\sim10^1$ are achieved in the 1D and 7D input space scenarios respectively while maintaining accurate output space probability distributions through combining Monte Carlo, quadrature rule, and sparse grid sampling with surrogate model fitting.
【2】 Audio Editing with Non-Rigid Text Prompts标题：使用非硬性文本提示进行音频编辑链接：https://arxiv.org/abs/2310.12858
作者：Francesco Paissan,Zhepei Wang,Mirco Ravanelli,Paris Smaragdis,Cem Subakan机构：FBK,Amazon AWS,Concordia University,Universit´e de Montr´eal,Mila-Qu´ebec AI Institute, University of Illinois Urbana Champaign,Universit´e Laval摘要：在本文中，我们探讨了非刚性文本编辑的音频编辑。我们表明，建议的编辑管道能够创建音频编辑，保持忠实于输入音频。我们将探索执行添加、样式转换和内部绘制的文本提示。我们定量和定性地表明，编辑能够获得优于Audio-LDM，最近发布的文本提示音频生成模型的结果。定性检查的结果指出，我们的方法给出的编辑保持更忠实于输入音频在保持原始的音频事件的起始和偏移。摘要：In this paper, we explore audio-editing with non-rigid text edits. We show that the proposed editing pipeline is able to create audio edits that remain faithful to the input audio. We explore text prompts that perform addition, style transfer, and in-painting. We quantitatively and qualitatively show that the edits are able to obtain results which outperform Audio-LDM, a recently released text-prompted audio generation model. Qualitative inspection of the results points out that the edits given by our approach remain more faithful to the input audio in terms of keeping the original onsets and offsets of the audio events.
【3】 EmoDiarize: Speaker Diarization and Emotion Identification from Speech  Signals using Convolutional Neural Networks标题：EmoDiarize：基于卷积神经网络的语音说话人二值化与情感识别链接：https://arxiv.org/abs/2310.12851
作者：Hanan Hamza,Fiza Gafoor,Fathima Sithara,Gayathri Anil,V. S. Anoop机构：School of Digital Sciences, Kerala University of Digital Sciences-, Innovation and Technology, Thiruvananthapuram, India, Applied NLP Research Lab摘要：在先进的人工智能和人机交互时代，识别口语中的情感至关重要。本研究探讨了深度学习技术在语音情感识别中的集成，为说话人日记化和情感识别相关的挑战提供了全面的解决方案。它引入了一个框架，该框架结合了预先存在的说话人日志化管道和基于卷积神经网络（CNN）构建的情感识别模型，以实现更高的精度。该模型在来自五个语音情感数据集的数据上进行了训练，即RAVDESS，CREMA-D，SAVEE，TESS和Movie Clips，其中后者是专门为本研究创建的语音情感数据集。从每个样本中提取的特征包括Mel频率倒谱系数（MFCC）、过零率（ZCR）、均方根（RMS）和各种数据增强算法，如音调、噪声、拉伸和移位。这种特征提取方法旨在提高预测精度，同时降低计算复杂度。该模型产生的未加权准确率为63%，在准确识别语音信号中的情感状态方面表现出显着的效率。摘要：In the era of advanced artificial intelligence and human-computer interaction, identifying emotions in spoken language is paramount. This research explores the integration of deep learning techniques in speech emotion recognition, offering a comprehensive solution to the challenges associated with speaker diarization and emotion identification. It introduces a framework that combines a pre-existing speaker diarization pipeline and an emotion identification model built on a Convolutional Neural Network (CNN) to achieve higher precision. The proposed model was trained on data from five speech emotion datasets, namely, RAVDESS, CREMA-D, SAVEE, TESS, and Movie Clips, out of which the latter is a speech emotion dataset created specifically for this research. The features extracted from each sample include Mel Frequency Cepstral Coefficients (MFCC), Zero Crossing Rate (ZCR), Root Mean Square (RMS), and various data augmentation algorithms like pitch, noise, stretch, and shift. This feature extraction approach aims to enhance prediction accuracy while reducing computational complexity. The proposed model yields an unweighted accuracy of 63%, demonstrating remarkable efficiency in accurately identifying emotional states within speech signals.
【4】 Energy-Based Models For Speech Synthesis标题：基于能量的语音合成模型链接：https://arxiv.org/abs/2310.12765
作者：Wanli Sun,Zehai Tu,Anton Ragni机构：Department of Computer Science, University of Sheffield, Sheffield, UK摘要：近年来，人们对用于语音合成的非自回归（non-AR）模型（例如FastSpeech 2和扩散模型）产生了很大的兴趣。与AR模型不同，这些模型在输出之间不具有自回归依赖性，这使得推理效率高。本文扩展了可用的非AR模型的范围，另一个成员称为基于能量的模型（EBM）。本文介绍了如何噪声对比估计，这依赖于正负样本之间的比较，可以用来训练EBM。它提出了一些生成有效阴性样本的策略，包括使用高性能AR模型。它还描述了如何使用Langevin马尔可夫链蒙特卡罗（MCMC）从EBM中进行采样。Langevin MCMC的使用使得能够绘制EBM和当前流行的扩散模型之间的连接。在LJSpeech数据集上的实验表明，该方法在Tacotron 2上有所改进。摘要：Recently there has been a lot of interest in non-autoregressive (non-AR) models for speech synthesis, such as FastSpeech 2 and diffusion models. Unlike AR models, these models do not have autoregressive dependencies among outputs which makes inference efficient. This paper expands the range of available non-AR models with another member called energy-based models (EBMs). The paper describes how noise contrastive estimation, which relies on the comparison between positive and negative samples, can be used to train EBMs. It proposes a number of strategies for generating effective negative samples, including using high-performing AR models. It also describes how sampling from EBMs can be performed using Langevin Markov Chain Monte-Carlo (MCMC). The use of Langevin MCMC enables to draw connections between EBMs and currently popular diffusion models. Experiments on LJSpeech dataset show that the proposed approach offers improvements over Tacotron 2.
【5】 Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative  Editing标题：Loop Copilot：为音乐生成和迭代编辑指挥AI合奏链接：https://arxiv.org/abs/2310.12404
作者：Yixiao Zhang,Akira Maezawa,Gus Xia,Kazuhiko Yamamoto,Simon Dixon机构：Can you give me a smooth rock music loop with, a guitar and snare drums?, I want to add a saxophone track to this music., Sure., Added a saxophone solo.备注：Source code and demo video are available at \url{this https URL}摘要：创作音乐是一个反复的过程，在每个阶段都需要不同的方法。然而，现有的人工智能音乐系统在协调多个子系统以满足不同需求方面存在不足。为了解决这一差距，我们引入了Loop Copilot，这是一个新颖的系统，使用户能够通过交互式的多轮对话界面生成和迭代优化音乐。该系统使用大型语言模型来解释用户意图，并选择适当的AI模型来执行任务。每个后端模型都专门用于特定的任务，它们的输出被聚合以满足用户的需求。为了确保音乐的连贯性，基本属性被保存在一个集中的表中。我们通过半结构化的访谈和问卷调查评估所提出的系统的有效性，突出其效用不仅在促进音乐创作，但也有其更广泛的应用潜力。摘要：Creating music is iterative, requiring varied methods at each stage. However, existing AI music systems fall short in orchestrating multiple subsystems for diverse needs. To address this gap, we introduce Loop Copilot, a novel system that enables users to generate and iteratively refine music through an interactive, multi-round dialogue interface. The system uses a large language model to interpret user intentions and select appropriate AI models for task execution. Each backend model is specialized for a specific task, and their outputs are aggregated to meet the user's requirements. To ensure musical coherence, essential attributes are maintained in a centralized table. We evaluate the effectiveness of the proposed system through semi-structured interviews and questionnaires, highlighting its utility not only in facilitating music creation but also its potential for broader applications.
【6】 The CHiME-7 Challenge: System Description and Performance of NeMo Team's  DASR System标题：CHINE-7挑战：NEMO Team的DASR系统的系统描述和性能链接：https://arxiv.org/abs/2310.12378
作者：Tae Jin Park,He Huang,Ante Jukic,Kunal Dhawan,Krishna C. Puvvada,Nithin Koluguri,Nikolay Karpov,Aleksandr Laptev,Jagadeesh Balam,Boris Ginsburg机构：NVIDIA, Santa Clara, USA备注：None摘要：NVIDIA NeMo团队为第七届CHiME挑战赛远程自动语音识别（DASR）任务展示了其多通道语音识别系统，该任务专注于开发多通道、多说话者语音识别系统，该系统专为转录来自分布式麦克风和麦克风阵列的语音而量身定制。该系统主要包括以下集成模块：扬声器日志模块、多声道音频前端处理模块和ASR模块。这些组件共同建立了一个级联系统，精心处理多声道和多扬声器音频输入。此外，本文强调了全面的优化过程，显着提高了我们的系统的性能。我们团队的提交主要基于NeMo工具包，并将公开提供。摘要：We present the NVIDIA NeMo team's multi-channel speech recognition system for the 7th CHiME Challenge Distant Automatic Speech Recognition (DASR) Task, focusing on the development of a multi-channel, multi-speaker speech recognition system tailored to transcribe speech from distributed microphones and microphone arrays. The system predominantly comprises of the following integral modules: the Speaker Diarization Module, Multi-channel Audio Front-End Processing Module, and the ASR Module. These components collectively establish a cascading system, meticulously processing multi-channel and multi-speaker audio input. Moreover, this paper highlights the comprehensive optimization process that significantly enhanced our system's performance. Our team's submission is largely based on NeMo toolkits and will be publicly available.
【7】 Property-Aware Multi-Speaker Data Simulation: A Probabilistic Modelling  Technique for Synthetic Data Generation标题：属性感知多说话人数据模拟：一种合成数据生成的概率建模技术链接：https://arxiv.org/abs/2310.12371
作者：Tae Jin Park,He Huang,Coleman Hooper,Nithin Koluguri,Kunal Dhawan,Ante Jukic,Jagadeesh Balam,Boris Ginsburg机构：NVIDIA, Santa Clara, USA备注：None摘要：我们介绍了一个复杂的多扬声器语音数据模拟器，专门设计用于生成多扬声器语音录音。这个模拟器的一个显着特点是它的能力，通过调整统计参数来调制的沉默和重叠的分布。这种能力提供了一个量身定制的训练环境，用于开发适合于说话者日记和语音活动检测的神经模型。获取大量的说话人日记数据集往往是一个重大的挑战，特别是在多说话人的情况下。此外，语音数据的精确时间戳注释是训练说话人日志和语音活动检测的关键因素。我们提出的多扬声器模拟器解决这些问题，通过生成大规模的音频混合，保持统计特性与输入参数紧密对齐。我们证明了所提出的多扬声器模拟器生成的音频混合与统计特性，密切配合来自现实世界的统计输入参数。此外，我们还介绍了说话人日记和语音活动检测模型的有效性，这些模型专门在生成的模拟数据集上进行了训练。摘要：We introduce a sophisticated multi-speaker speech data simulator, specifically engineered to generate multi-speaker speech recordings. A notable feature of this simulator is its capacity to modulate the distribution of silence and overlap via the adjustment of statistical parameters. This capability offers a tailored training environment for developing neural models suited for speaker diarization and voice activity detection. The acquisition of substantial datasets for speaker diarization often presents a significant challenge, particularly in multi-speaker scenarios. Furthermore, the precise time stamp annotation of speech data is a critical factor for training both speaker diarization and voice activity detection. Our proposed multi-speaker simulator tackles these problems by generating large-scale audio mixtures that maintain statistical properties closely aligned with the input parameters. We demonstrate that the proposed multi-speaker simulator generates audio mixtures with statistical properties that closely align with the input parameters derived from real-world statistics. Additionally, we present the effectiveness of speaker diarization and voice activity detection models, which have been trained exclusively on the generated simulated datasets.
eess.AS音频处理
【1】 Deep Beamforming for Speech Enhancement and Speaker Localization with an  Array Response-Aware Loss Function标题：基于阵列响应感知损失函数的深度波束形成语音增强和说话人定位链接：https://arxiv.org/abs/2310.12837
作者：Hsinyu Chang,Yicheng Hsu,Mingsian R. Bai机构：Hsinchu, Taiwan, (e-mail:, Engineering and Electrical Engineering, National Tsing Hua University, mitigate this problem, an All Deep Learning MVDR, (ADL-MVDR) network is proposed in [,], where the matrix, operations are replaced by two recurrent neural networks备注：6 pages摘要：基于深度神经网络（DNN）的波束形成器的最新研究进展显示出在不利声学条件下语音增强的巨大前景。不同的网络架构和输入功能已被探索在估计波束成形权重。在本文中，我们提出了一种基于高效卷积递归网络（CNN）的深度波束形成器，该网络使用新的ARray Response-aWare（ARROW）损失函数进行训练。ARROW损失通过使用地面真实相对传递函数（RTF）来利用目标和干扰器的阵列响应。基于DNN的波束形成系统，通过监督学习用ARROW损失训练，能够联合执行语音增强和说话人定位。实验结果表明，所提出的深度波束形成器，训练与线性加权尺度不变的源噪声比（SI-SNR）和ARROW损失函数，实现语音增强和说话人定位相比，两个基线的优越性能。摘要：Recent research advances in deep neural network (DNN)-based beamformers have shown great promise for speech enhancement under adverse acoustic conditions. Different network architectures and input features have been explored in estimating beamforming weights. In this paper, we propose a deep beamformer based on an efficient convolutional recurrent network (CNN) trained with a novel ARray RespOnse-aWare (ARROW) loss function. The ARROW loss exploits the array responses of the target and interferer by using the ground truth relative transfer functions (RTFs). The DNN-based beamforming system, trained with ARROW loss through supervised learning, is able to perform speech enhancement and speaker localization jointly. Experimental results have shown that the proposed deep beamformer, trained with the linearly weighted scale-invariant source-to-noise ratio (SI-SNR) and ARROW loss functions, achieves superior performance in speech enhancement and speaker localization compared to two baselines.
【2】 On Feature Importance and Interpretability of Speaker Representations标题：论说话人表征的特征重要性和可解释性链接：https://arxiv.org/abs/2310.12599
作者：Frederik Rautenberg,Michael Kuhlmann,Jana Wiechmann,Fritz Seebauer,Petra Wagner,Reinhold Haeb-Umbach机构：Reinhold Haeb-Umbach 1 1Department of Communications Engineering,  Paderborn University Email,  Bielefeld University Email备注：Presented at the ITG conference on Speech Communication 2023摘要：无监督语音去纠缠的目的是分离语音信号中的快变成分和慢变成分。在这篇文章中，我们仔细研究了代表缓慢变化的信号分量的嵌入向量，通常称为扬声器嵌入向量。我们问，一个扬声器的声音的属性被捕获，并调查在何种程度上做个别嵌入向量分量的符号负责他们，使用Shapley值的概念。我们的研究结果表明，某些特定于扬声器的声学语音特性可以相当好地预测从扬声器嵌入，而调查更抽象的语音质量特征不能。摘要：Unsupervised speech disentanglement aims at separating fast varying from slowly varying components of a speech signal. In this contribution, we take a closer look at the embedding vector representing the slowly varying signal components, commonly named the speaker embedding vector. We ask, which properties of a speaker's voice are captured and investigate to which extent do individual embedding vector components sign responsible for them, using the concept of Shapley values. Our findings show that certain speaker-specific acoustic-phonetic properties can be fairly well predicted from the speaker embedding, while the investigated more abstract voice quality features cannot.
【3】 An Exploration of In-Context Learning for Speech Language Model标题：语音语言模型的语境学习探索链接：https://arxiv.org/abs/2310.12477
作者：Ming-Hao Hsu,Kai-Wei Chang,Shang-Wen Li,Hung-yi Lee机构：National Taiwan University, Meta AI备注：The first two authors contributed equally摘要：自从GPT-3在自然语言处理（NLP）领域的发展以来，上下文学习（ICL）在利用大型语言模型（LLM）方面发挥了重要作用。通过在输入处呈现LM话语标签演示，LM可以实现Few-Shot学习，而不依赖于梯度下降或需要显式修改其参数。这使得LM能够以黑盒的方式学习和适应。尽管ICL在NLP中取得了成功，但很少有人探索ICL在语音处理中的可能性。本研究提出了第一次探索的ICL与语音LM没有文本监督。我们首先表明，目前的语音LM不具有ICL能力。通过所提出的热身训练，语音LM可以因此对看不见的任务执行ICL。在这项工作中，我们验证了ICL语音LM语音分类任务的可行性。摘要：Ever since the development of GPT-3 in the natural language processing (NLP) field, in-context learning (ICL) has played an important role in utilizing large language models (LLMs). By presenting the LM utterance-label demonstrations at the input, the LM can accomplish few-shot learning without relying on gradient descent or requiring explicit modification of its parameters. This enables the LM to learn and adapt in a black-box manner. Despite the success of ICL in NLP, little work is exploring the possibility of ICL in speech processing. This study proposes the first exploration of ICL with a speech LM without text supervision. We first show that the current speech LM does not have the ICL capability. With the proposed warmup training, the speech LM can, therefore, perform ICL on unseen tasks. In this work, we verify the feasibility of ICL for speech LM on speech classification tasks.
【4】 A New Time Series Similarity Measure and Its Smart Grid Applications标题：一种新的时间序列相似性度量及其在智能电网中的应用链接：https://arxiv.org/abs/2310.12399
作者：Rui Yuan,S. Ali Pourmousavi,Wen L. Soong,Andrew J. Black,Jon A. R. Liisberg,Julian Lemos-Vinasco机构：∗ School of Electrical and Mechanical Engineering, The University of Adelaide, Adelaide, Australia, † School of Computer and Mathematical Sciences, ‡ Research & Department of Data Science, Watts AS, Køge, Zealand, Denmark备注：7 pages, 6 figures conference摘要：许多智能电网应用涉及数据挖掘、聚类、分类、识别和异常检测等。这些应用主要依赖于相似性的度量，即不同时间序列或时间序列的连续性之间的距离。常用的时间序列距离测量，即欧几里得距离（ED）和动态时间弯曲（DTW），不量化用电数据的时间动态方面的灵活性。因此，需要一种新的距离测量，其可以量化用于智能电网应用的电力时间序列的幅度和时间变化，例如，需求响应和负载分析。本文介绍了一种新的距离测量比较用电模式。该方法包括两个阶段，量化重塑一个时间序列到另一个所需的努力，同时考虑幅度和时间的变化。所提出的方法进行评估，对ED和DTW使用真实世界的数据在三个智能电网应用。整体而言，建议的措施优于ED和DTW在准确地识别最佳的负荷调度策略，不规则的用电量的异常天，并确定电力用户的电表（BTM）设备。摘要：Many smart grid applications involve data mining, clustering, classification, identification, and anomaly detection, among others. These applications primarily depend on the measurement of similarity, which is the distance between different time series or subsequences of a time series. The commonly used time series distance measures, namely Euclidean Distance (ED) and Dynamic Time Warping (DTW), do not quantify the flexible nature of electricity usage data in terms of temporal dynamics. As a result, there is a need for a new distance measure that can quantify both the amplitude and temporal changes of electricity time series for smart grid applications, e.g., demand response and load profiling. This paper introduces a novel distance measure to compare electricity usage patterns. The method consists of two phases that quantify the effort required to reshape one time series into another, considering both amplitude and temporal changes. The proposed method is evaluated against ED and DTW using real-world data in three smart grid applications. Overall, the proposed measure outperforms ED and DTW in accurately identifying the best load scheduling strategy, anomalous days with irregular electricity usage, and determining electricity users' behind-the-meter (BTM) equipment.
【5】 The CHiME-7 Challenge: System Description and Performance of NeMo Team's  DASR System标题：CHINE-7挑战：NEMO Team的DASR系统的系统描述和性能链接：https://arxiv.org/abs/2310.12378
作者：Tae Jin Park,He Huang,Ante Jukic,Kunal Dhawan,Krishna C. Puvvada,Nithin Koluguri,Nikolay Karpov,Aleksandr Laptev,Jagadeesh Balam,Boris Ginsburg机构：NVIDIA, Santa Clara, USA备注：None摘要：NVIDIA NeMo团队为第七届CHiME挑战赛远程自动语音识别（DASR）任务展示了其多通道语音识别系统，该任务专注于开发多通道、多说话者语音识别系统，该系统专为转录来自分布式麦克风和麦克风阵列的语音而量身定制。该系统主要包括以下集成模块：扬声器日志模块、多声道音频前端处理模块和ASR模块。这些组件共同建立了一个级联系统，精心处理多声道和多扬声器音频输入。此外，本文强调了全面的优化过程，显着提高了我们的系统的性能。我们团队的提交主要基于NeMo工具包，并将公开提供。摘要：We present the NVIDIA NeMo team's multi-channel speech recognition system for the 7th CHiME Challenge Distant Automatic Speech Recognition (DASR) Task, focusing on the development of a multi-channel, multi-speaker speech recognition system tailored to transcribe speech from distributed microphones and microphone arrays. The system predominantly comprises of the following integral modules: the Speaker Diarization Module, Multi-channel Audio Front-End Processing Module, and the ASR Module. These components collectively establish a cascading system, meticulously processing multi-channel and multi-speaker audio input. Moreover, this paper highlights the comprehensive optimization process that significantly enhanced our system's performance. Our team's submission is largely based on NeMo toolkits and will be publicly available.
【6】 Property-Aware Multi-Speaker Data Simulation: A Probabilistic Modelling  Technique for Synthetic Data Generation标题：属性感知多说话人数据模拟：一种合成数据生成的概率建模技术链接：https://arxiv.org/abs/2310.12371
作者：Tae Jin Park,He Huang,Coleman Hooper,Nithin Koluguri,Kunal Dhawan,Ante Jukic,Jagadeesh Balam,Boris Ginsburg机构：NVIDIA, Santa Clara, USA备注：None摘要：我们介绍了一个复杂的多扬声器语音数据模拟器，专门设计用于生成多扬声器语音录音。这个模拟器的一个显着特点是它的能力，通过调整统计参数来调制的沉默和重叠的分布。这种能力提供了一个量身定制的训练环境，用于开发适合于说话者日记和语音活动检测的神经模型。获取大量的说话人日记数据集往往是一个重大的挑战，特别是在多说话人的情况下。此外，语音数据的精确时间戳注释是训练说话人日志和语音活动检测的关键因素。我们提出的多扬声器模拟器解决这些问题，通过生成大规模的音频混合，保持统计特性与输入参数紧密对齐。我们证明了所提出的多扬声器模拟器生成的音频混合与统计特性，密切配合来自现实世界的统计输入参数。此外，我们还介绍了说话人日记和语音活动检测模型的有效性，这些模型专门在生成的模拟数据集上进行了训练。摘要：We introduce a sophisticated multi-speaker speech data simulator, specifically engineered to generate multi-speaker speech recordings. A notable feature of this simulator is its capacity to modulate the distribution of silence and overlap via the adjustment of statistical parameters. This capability offers a tailored training environment for developing neural models suited for speaker diarization and voice activity detection. The acquisition of substantial datasets for speaker diarization often presents a significant challenge, particularly in multi-speaker scenarios. Furthermore, the precise time stamp annotation of speech data is a critical factor for training both speaker diarization and voice activity detection. Our proposed multi-speaker simulator tackles these problems by generating large-scale audio mixtures that maintain statistical properties closely aligned with the input parameters. We demonstrate that the proposed multi-speaker simulator generates audio mixtures with statistical properties that closely align with the input parameters derived from real-world statistics. Additionally, we present the effectiveness of speaker diarization and voice activity detection models, which have been trained exclusively on the generated simulated datasets.
【7】 Uncertainty Quantification of Bandgaps in Acoustic Metamaterials with  Stochastic Geometric Defects and Material Properties标题：具有随机几何缺陷和材料性质的声学超材料带隙的不确定性量化链接：https://arxiv.org/abs/2310.12869
作者：Han Zhang,Rayehe Karimi Mahabadi,Cynthia Rudin,Johann Guilleminot,L. Catherine Brinson机构：Catherine Brinsona, Duke University, Durham, NC, USA摘要：本文研究了不确定性量化技术的效用，即频谱投影和多项式混沌扩展，在减少随机材料特性和几何缺陷的声超材料色散带响应特性的采样需求。一种新的方法编码的几何缺陷在一个可解释的，分辨率无关的输入空间概率分布的形成展示。通过蒙特卡罗、求积法则和稀疏网格采样与代理模型拟合相结合，在保持精确输出空间概率分布的同时，分别在一维和七维输入空间中实现了$\sim10^0$和$\sim10^1$的数量级采样缩减.摘要：This paper studies the utility of techniques within uncertainty quantification, namely spectral projection and polynomial chaos expansion, in reducing sampling needs for characterizing acoustic metamaterial dispersion band responses given stochastic material properties and geometric defects. A novel method of encoding geometric defects in an interpretable, resolution independent is showcased in the formation of input space probability distributions. Orders of magnitude sampling reductions down to $\sim10^0$ and $\sim10^1$ are achieved in the 1D and 7D input space scenarios respectively while maintaining accurate output space probability distributions through combining Monte Carlo, quadrature rule, and sparse grid sampling with surrogate model fitting.
【8】 Audio Editing with Non-Rigid Text Prompts标题：使用非硬性文本提示进行音频编辑链接：https://arxiv.org/abs/2310.12858
作者：Francesco Paissan,Zhepei Wang,Mirco Ravanelli,Paris Smaragdis,Cem Subakan机构：FBK,Amazon AWS,Concordia University,Universit´e de Montr´eal,Mila-Qu´ebec AI Institute, University of Illinois Urbana Champaign,Universit´e Laval摘要：在本文中，我们探讨了非刚性文本编辑的音频编辑。我们表明，建议的编辑管道能够创建音频编辑，保持忠实于输入音频。我们将探索执行添加、样式转换和内部绘制的文本提示。我们定量和定性地表明，编辑能够获得优于Audio-LDM，最近发布的文本提示音频生成模型的结果。定性检查的结果指出，我们的方法给出的编辑保持更忠实于输入音频在保持原始的音频事件的起始和偏移。摘要：In this paper, we explore audio-editing with non-rigid text edits. We show that the proposed editing pipeline is able to create audio edits that remain faithful to the input audio. We explore text prompts that perform addition, style transfer, and in-painting. We quantitatively and qualitatively show that the edits are able to obtain results which outperform Audio-LDM, a recently released text-prompted audio generation model. Qualitative inspection of the results points out that the edits given by our approach remain more faithful to the input audio in terms of keeping the original onsets and offsets of the audio events.
【9】 EmoDiarize: Speaker Diarization and Emotion Identification from Speech  Signals using Convolutional Neural Networks标题：EmoDiarize：基于卷积神经网络的语音说话人二值化与情感识别链接：https://arxiv.org/abs/2310.12851
作者：Hanan Hamza,Fiza Gafoor,Fathima Sithara,Gayathri Anil,V. S. Anoop机构：School of Digital Sciences, Kerala University of Digital Sciences-, Innovation and Technology, Thiruvananthapuram, India, Applied NLP Research Lab摘要：在先进的人工智能和人机交互时代，识别口语中的情感至关重要。本研究探讨了深度学习技术在语音情感识别中的集成，为说话人日记化和情感识别相关的挑战提供了全面的解决方案。它引入了一个框架，该框架结合了预先存在的说话人日志化管道和基于卷积神经网络（CNN）构建的情感识别模型，以实现更高的精度。该模型在来自五个语音情感数据集的数据上进行了训练，即RAVDESS，CREMA-D，SAVEE，TESS和Movie Clips，其中后者是专门为本研究创建的语音情感数据集。从每个样本中提取的特征包括Mel频率倒谱系数（MFCC）、过零率（ZCR）、均方根（RMS）和各种数据增强算法，如音调、噪声、拉伸和移位。这种特征提取方法旨在提高预测精度，同时降低计算复杂度。该模型产生的未加权准确率为63%，在准确识别语音信号中的情感状态方面表现出显着的效率。摘要：In the era of advanced artificial intelligence and human-computer interaction, identifying emotions in spoken language is paramount. This research explores the integration of deep learning techniques in speech emotion recognition, offering a comprehensive solution to the challenges associated with speaker diarization and emotion identification. It introduces a framework that combines a pre-existing speaker diarization pipeline and an emotion identification model built on a Convolutional Neural Network (CNN) to achieve higher precision. The proposed model was trained on data from five speech emotion datasets, namely, RAVDESS, CREMA-D, SAVEE, TESS, and Movie Clips, out of which the latter is a speech emotion dataset created specifically for this research. The features extracted from each sample include Mel Frequency Cepstral Coefficients (MFCC), Zero Crossing Rate (ZCR), Root Mean Square (RMS), and various data augmentation algorithms like pitch, noise, stretch, and shift. This feature extraction approach aims to enhance prediction accuracy while reducing computational complexity. The proposed model yields an unweighted accuracy of 63%, demonstrating remarkable efficiency in accurately identifying emotional states within speech signals.
【10】 Energy-Based Models For Speech Synthesis标题：基于能量的语音合成模型链接：https://arxiv.org/abs/2310.12765
作者：Wanli Sun,Zehai Tu,Anton Ragni机构：Department of Computer Science, University of Sheffield, Sheffield, UK摘要：近年来，人们对用于语音合成的非自回归（non-AR）模型（例如FastSpeech 2和扩散模型）产生了很大的兴趣。与AR模型不同，这些模型在输出之间不具有自回归依赖性，这使得推理效率高。本文扩展了可用的非AR模型的范围，另一个成员称为基于能量的模型（EBM）。本文介绍了如何噪声对比估计，这依赖于正负样本之间的比较，可以用来训练EBM。它提出了一些生成有效阴性样本的策略，包括使用高性能AR模型。它还描述了如何使用Langevin马尔可夫链蒙特卡罗（MCMC）从EBM中进行采样。Langevin MCMC的使用使得能够绘制EBM和当前流行的扩散模型之间的连接。在LJSpeech数据集上的实验表明，该方法在Tacotron 2上有所改进。摘要：Recently there has been a lot of interest in non-autoregressive (non-AR) models for speech synthesis, such as FastSpeech 2 and diffusion models. Unlike AR models, these models do not have autoregressive dependencies among outputs which makes inference efficient. This paper expands the range of available non-AR models with another member called energy-based models (EBMs). The paper describes how noise contrastive estimation, which relies on the comparison between positive and negative samples, can be used to train EBMs. It proposes a number of strategies for generating effective negative samples, including using high-performing AR models. It also describes how sampling from EBMs can be performed using Langevin Markov Chain Monte-Carlo (MCMC). The use of Langevin MCMC enables to draw connections between EBMs and currently popular diffusion models. Experiments on LJSpeech dataset show that the proposed approach offers improvements over Tacotron 2.
【11】 Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative  Editing标题：Loop Copilot：为音乐生成和迭代编辑指挥AI合奏链接：https://arxiv.org/abs/2310.12404
作者：Yixiao Zhang,Akira Maezawa,Gus Xia,Kazuhiko Yamamoto,Simon Dixon机构：Can you give me a smooth rock music loop with, a guitar and snare drums?, I want to add a saxophone track to this music., Sure., Added a saxophone solo.备注：Source code and demo video are available at \url{this https URL}摘要：创作音乐是一个反复的过程，在每个阶段都需要不同的方法。然而，现有的人工智能音乐系统在协调多个子系统以满足不同需求方面存在不足。为了解决这一差距，我们引入了Loop Copilot，这是一个新颖的系统，使用户能够通过交互式的多轮对话界面生成和迭代优化音乐。该系统使用大型语言模型来解释用户意图，并选择适当的AI模型来执行任务。每个后端模型都专门用于特定的任务，它们的输出被聚合以满足用户的需求。为了确保音乐的连贯性，基本属性被保存在一个集中的表中。我们通过半结构化的访谈和问卷调查评估所提出的系统的有效性，突出其效用不仅在促进音乐创作，但也有其更广泛的应用潜力。摘要：Creating music is iterative, requiring varied methods at each stage. However, existing AI music systems fall short in orchestrating multiple subsystems for diverse needs. To address this gap, we introduce Loop Copilot, a novel system that enables users to generate and iteratively refine music through an interactive, multi-round dialogue interface. The system uses a large language model to interpret user intentions and select appropriate AI models for task execution. Each backend model is specialized for a specific task, and their outputs are aggregated to meet the user's requirements. To ensure musical coherence, essential attributes are maintained in a centralized table. We evaluate the effectiveness of the proposed system through semi-structured interviews and questionnaires, highlighting its utility not only in facilitating music creation but also its potential for broader applications.
机器翻译，仅供参考