【1】 ESSYS* Sharing #UC: An Emotion-driven Audiovisual Installation
标题：ESSYS*分享#UC：情感驱动的视听装置
链接：https://arxiv.org/abs/2209.03338
作者：Sérgio M. Rebelo,Mariana Seiça,Pedro Martins,João Bicker,Penousal Machado摘要：我们展示ESSYS* Sharing #UC，这是一件视听装置艺术品，它反映了与大学和科英布拉市相关的情感背景，基于Twitter上共享的数据。该装置于2021年夏季和秋季在科英布拉艺术中心的一个城市艺术画廊展出。在安装空间中，人们可以看到一组印刷海报，显示推文并收听不断变化的环境声音。本视听产品是通过自主计算创造性方法创建的，该方法采用神经分类器来识别推文的情感背景，并使用该结果数据作为视听产品生成的原料。该装置的空间设计旨在促进一种方法和融合之间的在线和物理感知的同一地点。通过多个实验对该方法的性能进行了评估。此外，我们还进行了基于访谈的评估会议，以了解人们对装置元素，特别是海报设计的多样性、表现力和在其他商业和社会场景中的可能就业的体验。摘要：We present ESSYS* Sharing #UC, an audiovisual installation artwork that reflects upon the emotional context related to the university and the city of Coimbra, based on the data shared about them on Twitter. The installation was presented in an urban art gallery of C\'irculo de Artes Pl\'asticas de Coimbra during the summer and autumn of 2021. In the installation space, one may see a collection of typographic posters displaying the tweets and listening to an ever-changing ambient sound. The present audiovisuals are created by an autonomous computational creative approach, which employs a neural classifier to recognize the emotional context of a tweet and uses this resulting data as feedstock for the audiovisual generation. The installation's space is designed to promote an approach and blend between the online and physical perceptions of the same location. We applied multiple experiments with the proposed approach to evaluate the capability and performance. Also, we conduct interview-based evaluation sessions to understand how the installation elements, especially poster designs, are experienced by people regarding diversity, expressiveness and possible employment in other commercial and social scenarios.
【2】 Multimodal Speech Enhancement Using Burst Propagation
标题：基于突发传播的多模式语音增强
链接：https://arxiv.org/abs/2209.03275
作者：Leandro A. Passos,Ahmed Khubaib,Mohsin Raza,Ahsan Adeel摘要：本文提出了一种新的多模态语音增强方法MBURST，该方法考虑了关于前额叶皮层和其他脑区锥体细胞的最新神经学发现。所谓的突发传播实现了几个标准，以便以生物学上更合理的方式来解决信用分配问题：通过反馈操纵可塑性的符号和大小，通过不同的权重连接跨层复用反馈和前馈信息，近似反馈和前馈连接，以及线性化反馈信号。MBURST受益于这种学习噪声信号和视觉刺激之间的相关性的能力，从而通过放大相关信息和抑制噪声来赋予语音意义。在Grid Corpus和基于CHiME3的数据集上进行的实验表明，MBURST可以重现与基于多模态反向传播的基线相似的掩码重建，同时展示了出色的能量效率管理，将神经元放电率降低到最多\textbf{$70\%$}以下的值。这样的特征意味着更可持续的实现，适合于助听器或任何其它类似的嵌入式系统并且是期望的。摘要：This paper proposes the MBURST, a novel multimodal solution for audio-visual speech enhancements that consider the most recent neurological discoveries regarding pyramidal cells of the prefrontal cortex and other brain regions. The so-called burst propagation implements several criteria to address the credit assignment problem in a more biologically plausible manner: steering the sign and magnitude of plasticity through feedback, multiplexing the feedback and feedforward information across layers through different weight connections, approximating feedback and feedforward connections, and linearizing the feedback signals. MBURST benefits from such capabilities to learn correlations between the noisy signal and the visual stimuli, thus attributing meaning to the speech by amplifying relevant information and suppressing noise. Experiments conducted over a Grid Corpus and CHiME3-based dataset show that MBURST can reproduce similar mask reconstructions to the multimodal backpropagation-based baseline while demonstrating outstanding energy efficiency management, reducing the neuron firing rates to values up to \textbf{$70\%$} lower. Such a feature implies more sustainable implementations, suitable and desirable for hearing aids or any other similar embedded systems.
【3】 AudioLM: a Language Modeling Approach to Audio Generation
标题：AudioLM：一种用于音频生成的语言建模方法
链接：https://arxiv.org/abs/2209.03143
作者：Zalán Borsos,Raphaël Marinier,Damien Vincent,Eugene Kharitonov,Olivier Pietquin,Matt Sharifi,Olivier Teboul,David Grangier,Marco Tagliasacchi,Neil Zeghidour摘要：我们介绍AudioLM，一个用于生成具有长期一致性的高质量音频的框架。AudioLM将输入音频映射到离散标记序列，并将音频生成作为该表示空间中的语言建模任务。我们展示了现有的音频标记化器如何在重构质量和长期结构之间提供不同的折衷，并且我们提出了一种混合标记化方案来实现这两个目标。也就是说，我们利用对音频预先训练的掩蔽语言模型的离散化激活来捕获长期结构，并利用由神经音频编解码器产生的离散代码来实现高质量合成。通过对大量原始音频波形进行训练，AudioLM学会了在给出简短提示的情况下生成自然和连贯的延续。当在语音上进行训练时，在没有任何文字记录或注释的情况下，AudioLM生成在语法和语义上似乎合理的语音延续，同时还为未看过的说话者保持说话者身份和韵律。此外，我们展示了我们的方法如何通过生成连贯的钢琴音乐延续而扩展到语音之外，尽管没有任何音乐的符号表示进行训练。摘要：We introduce AudioLM, a framework for high-quality audio generation with long-term consistency. AudioLM maps the input audio to a sequence of discrete tokens and casts audio generation as a language modeling task in this representation space. We show how existing audio tokenizers provide different trade-offs between reconstruction quality and long-term structure, and we propose a hybrid tokenization scheme to achieve both objectives. Namely, we leverage the discretized activations of a masked language model pre-trained on audio to capture long-term structure and the discrete codes produced by a neural audio codec to achieve high-quality synthesis. By training on large corpora of raw audio waveforms, AudioLM learns to generate natural and coherent continuations given short prompts. When trained on speech, and without any transcript or annotation, AudioLM generates syntactically and semantically plausible speech continuations while also maintaining speaker identity and prosody for unseen speakers. Furthermore, we demonstrate how our approach extends beyond speech by generating coherent piano music continuations, despite being trained without any symbolic representation of music.
【4】 Improving Choral Music Separation through Expressive Synthesized Data from Sampled Instruments
标题：利用采样乐器的表现力合成数据改进合唱音乐分离
链接：https://arxiv.org/abs/2209.02871
作者：Ke Chen,Hao-Wen Dong,Yi Luo,Julian McAuley,Taylor Berg-Kirkpatrick,Miller Puckette,Shlomo Dubnov摘要：合唱音乐分离指的是提取声部的音轨（例如，女高音、女低音、男高音和男低音）。由于版权问题和数据集收集困难，以前的工作只能在几分钟的合唱音乐数据上训练和评估模型，因此数据集的缺乏阻碍了对该主题的研究。在这篇论文中，我们研究了使用合成训练数据对真实合唱音乐进行源分离的任务。我们做出了三大贡献：首先，我们提供了一种用于在乐器表现力的可控选项内从采样的乐器插件合成合唱音乐数据的自动流水线。This produces an 8.2-hour-long choral music dataset from the JSB Chorales Dataset and one can easily synthesize additional data.第二，我们在先前工作的合唱音乐分离数据集上进行实验来评估多种分离模型。据我们所知，这是第一次全面评估合唱音乐分离的实验。第三，实验结果表明，合成的合唱音乐数据质量良好，可以提高模型在真实合唱音乐数据集上的性能。这为合唱音乐分离研究提供了额外的实验统计和数据支持。摘要：Choral music separation refers to the task of extracting tracks of voice parts (e.g., soprano, alto, tenor, and bass) from mixed audio. The lack of datasets has impeded research on this topic as previous work has only been able to train and evaluate models on a few minutes of choral music data due to copyright issues and dataset collection difficulties. In this paper, we investigate the use of synthesized training data for the source separation task on real choral music. We make three contributions: first, we provide an automated pipeline for synthesizing choral music data from sampled instrument plugins within controllable options for instrument expressiveness. This produces an 8.2-hour-long choral music dataset from the JSB Chorales Dataset and one can easily synthesize additional data. Second, we conduct an experiment to evaluate multiple separation models on available choral music separation datasets from previous work. To the best of our knowledge, this is the first experiment to comprehensively evaluate choral music separation. Third, experiments demonstrate that the synthesized choral data is of sufficient quality to improve the model's performance on real choral music datasets. This provides additional experimental statistics and data support for the choral music separation study.
【5】 The Role of Voice Persona in Expressive Communication:An Argument for Relevance in Speech Synthesis Design
标题：语音人物角色在表现力交流中的作用：语音合成设计中的关联性论证
链接：https://arxiv.org/abs/2209.02855
作者：Camille Noufi,Lloyd May,Jonathan Berger摘要：我们提出了一种在合成语音中注入表现力的方法，通过对10位声乐研究和表演专家的访谈进行主题分析，为实时、交互式声乐角色的设计框架提供信息，该角色将产生令人信服的和适当的上下文相关的表达。所得到的语音音调被定义为存在于连续的、上下文相关的概率空间内的点。在合成语音中包括语音角色在广泛的应用中是重要的。特别令人感兴趣的是在增强和辅助通信（AAC）社区的潜在影响。最后，我们将介绍我们正在进行的研究，调查声音人物角色的主题，以及他们如何可以继续告知所提出的表达性语音合成设计框架。摘要：We present an approach to imbuing expressivity in a synthesized voice by acquiring a thematic analysis of 10 interviews with vocal studies and performance experts to inform the design framework for a real-time, interactive vocal persona that would generate compelling and appropriate contextually-dependent expression. The resultant tone of voice is defined as a point existing within a continuous, contextually-dependent probability space. The inclusion of voice persona in synthesized voice can be significant in a broad range of applications. Of particular interest is the potential impact in augmentative and assistive communication (AAC) community. Finally, we conclude with an introduction to our ongoing research investigating the themes of vocal persona and how they may continue to inform proposed expressive speech synthesis design frameworks.
【6】 Read it to me: An emotionally aware Speech Narration Application
标题：朗读给我听：一种情感感知的演讲叙事应用
链接：https://arxiv.org/abs/2209.02785
摘要：在这部作品中，我们尝试对音频进行情感风格迁移。特别地，MelGAN-VC体系结构被探索用于各种情绪对传递。然后，使用用于音频的基于LSTM的情感分类器对所生成的音频进行分类。我们发现，与“快乐”或“愤怒”相比，“悲伤”音频生成得更好，因为人们具有相似的悲伤表情。摘要：In this work we try to perform emotional style transfer on audios. In particular, MelGAN-VC architecture is explored for various emotion-pair transfers. The generated audio is then classified using an LSTM-based emotion classifier for audio. We find that "sad" audio is generated well as compared to "happy" or "anger" as people have similar expressions of sadness.
【7】 Instrument Separation of Symbolic Music by Explicitly Guided Diffusion Model
标题：基于显式引导扩散模型的符号音乐乐器分离
链接：https://arxiv.org/abs/2209.02696
作者：Sangjun Han,Hyeongrae Ihm,DaeHan Ahn,Woohyung Lim备注：Submitted to NeurIPS 2022 Workshop on Machine Learning for Creativity and Design摘要：与计算机视觉中的彩色化类似，乐器分离是指分配乐器标签（例如钢琴、吉他...）涉及来自仅包含性能信息的未标记混合物的注释。为了解决这个问题，我们采用扩散模型，并明确地引导它们保持混合物和音乐之间的一致性。定量结果表明，该模型能够为具有创造性的多音轨符号音乐生成高保真度的样本。摘要：Similar to colorization in computer vision, instrument separation is to assign instrument labels (e.g. piano, guitar...) to notes from unlabeled mixtures which contain only performance information. To address the problem, we adopt diffusion models and explicitly guide them to preserve consistency between mixtures and music. The quantitative results show that our proposed model can generate high-fidelity samples for multitrack symbolic music with creativity.
【1】 Rapid dynamic speech imaging at 3 Tesla using combination of a custom vocal tract coil, variable density spirals and manifold regularization
标题：使用定制声道线圈、可变密度螺旋和流形正则化相结合的3特斯拉快速动态语音成像
链接：https://arxiv.org/abs/2209.02768
作者：Rushdi Zahid Rusho,Abdul Haseeb Ahmed,Stanley Kruger,Wahidul Alam,David Meyer,David Howard,Ingo Titze,Mathews Jacob,Sajan Goud Lingala摘要：目的：改善3特斯拉下的动态语音成像。 方法：提出一种基于16通道声道线圈、变密度螺旋线（VDS）和流形正则化的新方法。使用短的读出持续时间螺旋（1.3ms长）来最小化对非共振的灵敏度。流形模型利用了共享相似声道姿势的帧之间的相似性，而没有显式的运动面元划分。重建被提出为基于SENSE的非局部软加权时间正则化方案。利用VDS的自导航功能了解管汇的结构。我们的方法与低秩和有限差分重建约束进行了比较，两名志愿者执行重复和任意的说话任务。由语音研究领域的三位专家对混叠伪影、空间模糊和时间模糊类别进行盲法图像质量评估。 结果：我们实现了www.example.com的空间分辨率2.4mm2/pixel，单层成像的时间分辨率为17.4 ms/帧，并行3层成像的时间分辨率为52.2 ms/帧。对重复性和流利性口语任务的流形方案的隐式运动装仓进行了演示。与低秩和时域有限差分方法相比，流形方法在模拟发音运动方面提供了更高的保真度。这反映在空间和时间模糊类别的图像质量评分较高。我们的技术显示了微弱的假像伪影，但与假像伪影类别中的其他方法相比，提供了更小的四分位数范围评分。 结论：定制声道线圈、可变密度螺旋和流形正则化的协同组合能够在3特斯拉下实现稳健的动态语音成像。摘要：Purpose: To improve dynamic speech imaging at 3 Tesla. Methods: A novel scheme combining a 16-channel vocal tract coil, variable density spirals (VDS), and manifold regularization was developed. Short readout duration spirals (1.3 ms long) were used to minimize sensitivity to off-resonance. The manifold model leveraged similarities between frames sharing similar vocal tract postures without explicit motion binning. Reconstruction was posed as a SENSE-based non-local soft weighted temporal regularization scheme. The self-navigating capability of VDS was leveraged to learn the structure of the manifold. Our approach was compared against low-rank and finite difference reconstruction constraints on two volunteers performing repetitive and arbitrary speaking tasks. Blinded image quality evaluation in the categories of alias artifacts, spatial blurring, and temporal blurring were performed by three experts in voice research. Results: We achieved a spatial resolution of 2.4mm2/pixel and a temporal resolution of 17.4 ms/frame for single slice imaging, and 52.2 ms/frame for concurrent 3-slice imaging. Implicit motion binning of the manifold scheme for both repetitive and fluent speaking tasks was demonstrated. The manifold scheme provided superior fidelity in modeling articulatory motion compared to low rank and temporal finite difference schemes. This was reflected by higher image quality scores in spatial and temporal blurring categories. Our technique exhibited faint alias artifacts, but offered a reduced interquartile range of scores compared to other methods in alias artifact category. Conclusion: Synergistic combination of a custom vocal-tract coil, variable density spirals and manifold regularization enables robust dynamic speech imaging at 3 Tesla.
【2】 ESSYS* Sharing #UC: An Emotion-driven Audiovisual Installation
标题：ESSYS*分享#UC：情感驱动的视听装置
链接：https://arxiv.org/abs/2209.03338
* 与cs.SD语音【1】为同一篇
作者：Sérgio M. Rebelo,Mariana Seiça,Pedro Martins,João Bicker,Penousal Machado摘要：我们展示ESSYS* Sharing #UC，这是一件视听装置艺术品，它反映了与大学和科英布拉市相关的情感背景，基于Twitter上共享的数据。该装置于2021年夏季和秋季在科英布拉艺术中心的一个城市艺术画廊展出。在安装空间中，人们可以看到一组印刷海报，显示推文并收听不断变化的环境声音。本视听产品是通过自主计算创造性方法创建的，该方法采用神经分类器来识别推文的情感背景，并使用该结果数据作为视听产品生成的原料。该装置的空间设计旨在促进一种方法和融合之间的在线和物理感知的同一地点。通过多个实验对该方法的性能进行了评估。此外，我们还进行了基于访谈的评估会议，以了解人们对装置元素，特别是海报设计的多样性、表现力和在其他商业和社会场景中的可能就业的体验。摘要：We present ESSYS* Sharing #UC, an audiovisual installation artwork that reflects upon the emotional context related to the university and the city of Coimbra, based on the data shared about them on Twitter. The installation was presented in an urban art gallery of C\'irculo de Artes Pl\'asticas de Coimbra during the summer and autumn of 2021. In the installation space, one may see a collection of typographic posters displaying the tweets and listening to an ever-changing ambient sound. The present audiovisuals are created by an autonomous computational creative approach, which employs a neural classifier to recognize the emotional context of a tweet and uses this resulting data as feedstock for the audiovisual generation. The installation's space is designed to promote an approach and blend between the online and physical perceptions of the same location. We applied multiple experiments with the proposed approach to evaluate the capability and performance. Also, we conduct interview-based evaluation sessions to understand how the installation elements, especially poster designs, are experienced by people regarding diversity, expressiveness and possible employment in other commercial and social scenarios.
【3】 Multimodal Speech Enhancement Using Burst Propagation
标题：基于突发传播的多模式语音增强
链接：https://arxiv.org/abs/2209.03275
* 与cs.SD语音【2】为同一篇
作者：Leandro A. Passos,Ahmed Khubaib,Mohsin Raza,Ahsan Adeel摘要：本文提出了一种新的多模态语音增强方法MBURST，该方法考虑了关于前额叶皮层和其他脑区锥体细胞的最新神经学发现。所谓的突发传播实现了几个标准，以便以生物学上更合理的方式来解决信用分配问题：通过反馈操纵可塑性的符号和大小，通过不同的权重连接跨层复用反馈和前馈信息，近似反馈和前馈连接，以及线性化反馈信号。MBURST受益于这种学习噪声信号和视觉刺激之间的相关性的能力，从而通过放大相关信息和抑制噪声来赋予语音意义。在Grid Corpus和基于CHiME3的数据集上进行的实验表明，MBURST可以重现与基于多模态反向传播的基线相似的掩码重建，同时展示了出色的能量效率管理，将神经元放电率降低到最多\textbf{$70\%$}以下的值。这样的特征意味着更可持续的实现，适合于助听器或任何其它类似的嵌入式系统并且是期望的。摘要：This paper proposes the MBURST, a novel multimodal solution for audio-visual speech enhancements that consider the most recent neurological discoveries regarding pyramidal cells of the prefrontal cortex and other brain regions. The so-called burst propagation implements several criteria to address the credit assignment problem in a more biologically plausible manner: steering the sign and magnitude of plasticity through feedback, multiplexing the feedback and feedforward information across layers through different weight connections, approximating feedback and feedforward connections, and linearizing the feedback signals. MBURST benefits from such capabilities to learn correlations between the noisy signal and the visual stimuli, thus attributing meaning to the speech by amplifying relevant information and suppressing noise. Experiments conducted over a Grid Corpus and CHiME3-based dataset show that MBURST can reproduce similar mask reconstructions to the multimodal backpropagation-based baseline while demonstrating outstanding energy efficiency management, reducing the neuron firing rates to values up to \textbf{$70\%$} lower. Such a feature implies more sustainable implementations, suitable and desirable for hearing aids or any other similar embedded systems.
【4】 AudioLM: a Language Modeling Approach to Audio Generation
标题：AudioLM：一种用于音频生成的语言建模方法
链接：https://arxiv.org/abs/2209.03143
* 与cs.SD语音【3】为同一篇
作者：Zalán Borsos,Raphaël Marinier,Damien Vincent,Eugene Kharitonov,Olivier Pietquin,Matt Sharifi,Olivier Teboul,David Grangier,Marco Tagliasacchi,Neil Zeghidour摘要：我们介绍AudioLM，一个用于生成具有长期一致性的高质量音频的框架。AudioLM将输入音频映射到离散标记序列，并将音频生成作为该表示空间中的语言建模任务。我们展示了现有的音频标记化器如何在重构质量和长期结构之间提供不同的折衷，并且我们提出了一种混合标记化方案来实现这两个目标。也就是说，我们利用对音频预先训练的掩蔽语言模型的离散化激活来捕获长期结构，并利用由神经音频编解码器产生的离散代码来实现高质量合成。通过对大量原始音频波形进行训练，AudioLM学会了在给出简短提示的情况下生成自然和连贯的延续。当在语音上进行训练时，在没有任何文字记录或注释的情况下，AudioLM生成在语法和语义上似乎合理的语音延续，同时还为未看过的说话者保持说话者身份和韵律。此外，我们展示了我们的方法如何通过生成连贯的钢琴音乐延续而扩展到语音之外，尽管没有任何音乐的符号表示进行训练。摘要：We introduce AudioLM, a framework for high-quality audio generation with long-term consistency. AudioLM maps the input audio to a sequence of discrete tokens and casts audio generation as a language modeling task in this representation space. We show how existing audio tokenizers provide different trade-offs between reconstruction quality and long-term structure, and we propose a hybrid tokenization scheme to achieve both objectives. Namely, we leverage the discretized activations of a masked language model pre-trained on audio to capture long-term structure and the discrete codes produced by a neural audio codec to achieve high-quality synthesis. By training on large corpora of raw audio waveforms, AudioLM learns to generate natural and coherent continuations given short prompts. When trained on speech, and without any transcript or annotation, AudioLM generates syntactically and semantically plausible speech continuations while also maintaining speaker identity and prosody for unseen speakers. Furthermore, we demonstrate how our approach extends beyond speech by generating coherent piano music continuations, despite being trained without any symbolic representation of music.
【5】 Improving Choral Music Separation through Expressive Synthesized Data from Sampled Instruments
标题：利用采样乐器的表现力合成数据改进合唱音乐分离
链接：https://arxiv.org/abs/2209.02871
* 与cs.SD语音【4】为同一篇
作者：Ke Chen,Hao-Wen Dong,Yi Luo,Julian McAuley,Taylor Berg-Kirkpatrick,Miller Puckette,Shlomo Dubnov摘要：合唱音乐分离指的是提取声部的音轨（例如，女高音、女低音、男高音和男低音）。由于版权问题和数据集收集困难，以前的工作只能在几分钟的合唱音乐数据上训练和评估模型，因此数据集的缺乏阻碍了对该主题的研究。在这篇论文中，我们研究了使用合成训练数据对真实合唱音乐进行源分离的任务。我们做出了三大贡献：首先，我们提供了一种用于在乐器表现力的可控选项内从采样的乐器插件合成合唱音乐数据的自动流水线。This produces an 8.2-hour-long choral music dataset from the JSB Chorales Dataset and one can easily synthesize additional data.第二，我们在先前工作的合唱音乐分离数据集上进行实验来评估多种分离模型。据我们所知，这是第一次全面评估合唱音乐分离的实验。第三，实验结果表明，合成的合唱音乐数据质量良好，可以提高模型在真实合唱音乐数据集上的性能。这为合唱音乐分离研究提供了额外的实验统计和数据支持。摘要：Choral music separation refers to the task of extracting tracks of voice parts (e.g., soprano, alto, tenor, and bass) from mixed audio. The lack of datasets has impeded research on this topic as previous work has only been able to train and evaluate models on a few minutes of choral music data due to copyright issues and dataset collection difficulties. In this paper, we investigate the use of synthesized training data for the source separation task on real choral music. We make three contributions: first, we provide an automated pipeline for synthesizing choral music data from sampled instrument plugins within controllable options for instrument expressiveness. This produces an 8.2-hour-long choral music dataset from the JSB Chorales Dataset and one can easily synthesize additional data. Second, we conduct an experiment to evaluate multiple separation models on available choral music separation datasets from previous work. To the best of our knowledge, this is the first experiment to comprehensively evaluate choral music separation. Third, experiments demonstrate that the synthesized choral data is of sufficient quality to improve the model's performance on real choral music datasets. This provides additional experimental statistics and data support for the choral music separation study.
【6】 The Role of Voice Persona in Expressive Communication:An Argument for Relevance in Speech Synthesis Design
标题：语音人物角色在表现力交流中的作用：语音合成设计中的关联性论证
链接：https://arxiv.org/abs/2209.02855
* 与cs.SD语音【5】为同一篇
作者：Camille Noufi,Lloyd May,Jonathan Berger摘要：我们提出了一种在合成语音中注入表现力的方法，通过对10位声乐研究和表演专家的访谈进行主题分析，为实时、交互式声乐角色的设计框架提供信息，该角色将产生令人信服的和适当的上下文相关的表达。所得到的语音音调被定义为存在于连续的、上下文相关的概率空间内的点。在合成语音中包括语音角色在广泛的应用中是重要的。特别令人感兴趣的是在增强和辅助通信（AAC）社区的潜在影响。最后，我们将介绍我们正在进行的研究，调查声音人物角色的主题，以及他们如何可以继续告知所提出的表达性语音合成设计框架。摘要：We present an approach to imbuing expressivity in a synthesized voice by acquiring a thematic analysis of 10 interviews with vocal studies and performance experts to inform the design framework for a real-time, interactive vocal persona that would generate compelling and appropriate contextually-dependent expression. The resultant tone of voice is defined as a point existing within a continuous, contextually-dependent probability space. The inclusion of voice persona in synthesized voice can be significant in a broad range of applications. Of particular interest is the potential impact in augmentative and assistive communication (AAC) community. Finally, we conclude with an introduction to our ongoing research investigating the themes of vocal persona and how they may continue to inform proposed expressive speech synthesis design frameworks.
【7】 Read it to me: An emotionally aware Speech Narration Application
标题：朗读给我听：一种情感感知的演讲叙事应用
链接：https://arxiv.org/abs/2209.02785
* 与cs.SD语音【6】为同一篇
摘要：在这部作品中，我们尝试对音频进行情感风格迁移。特别地，MelGAN-VC体系结构被探索用于各种情绪对传递。然后，使用用于音频的基于LSTM的情感分类器对所生成的音频进行分类。我们发现，与“快乐”或“愤怒”相比，“悲伤”音频生成得更好，因为人们具有相似的悲伤表情。摘要：In this work we try to perform emotional style transfer on audios. In particular, MelGAN-VC architecture is explored for various emotion-pair transfers. The generated audio is then classified using an LSTM-based emotion classifier for audio. We find that "sad" audio is generated well as compared to "happy" or "anger" as people have similar expressions of sadness.
【8】 Instrument Separation of Symbolic Music by Explicitly Guided Diffusion Model
标题：基于显式引导扩散模型的符号音乐乐器分离
链接：https://arxiv.org/abs/2209.02696
* 与cs.SD语音【7】为同一篇
作者：Sangjun Han,Hyeongrae Ihm,DaeHan Ahn,Woohyung Lim备注：Submitted to NeurIPS 2022 Workshop on Machine Learning for Creativity and Design摘要：与计算机视觉中的彩色化类似，乐器分离是指分配乐器标签（例如钢琴、吉他...）涉及来自仅包含性能信息的未标记混合物的注释。为了解决这个问题，我们采用扩散模型，并明确地引导它们保持混合物和音乐之间的一致性。定量结果表明，该模型能够为具有创造性的多音轨符号音乐生成高保真度的样本。摘要：Similar to colorization in computer vision, instrument separation is to assign instrument labels (e.g. piano, guitar...) to notes from unlabeled mixtures which contain only performance information. To address the problem, we adopt diffusion models and explicitly guide them to preserve consistency between mixtures and music. The quantitative results show that our proposed model can generate high-fidelity samples for multitrack symbolic music with creativity.
机器翻译，仅供参考
每日语音速递