今日论文合集：cs.SD语音13篇，eess.AS音频处理11篇。【1】 Exploring the Viability of Synthetic Audio Data for Audio-Based Dialogue State Tracking标题：探索合成音频数据用于基于音频的对话状态跟踪的可行性链接：https://arxiv.org/abs/2312.01842
作者：Jihyun Lee,Yejin Jeon,Wonjun Lee,Yunsu Kim,Gary Geunbae Lee备注：Accepted in ASRU 2023摘要：在面向任务的对话系统中，对话状态跟踪在信息提取中起着至关重要的作用。然而，以前的研究仅限于文本模态，主要是由于缺乏真实的人类音频数据集。我们通过调查基于音频的DST的合成音频数据来解决这个问题。为此，我们开发了级联和端到端模型，用我们的合成音频数据集对它们进行训练，并在实际的人类语音数据上对其进行测试。为了便于评估量身定制的音频模态，我们引入了一种新的PhonemeF1来捕获发音相似性。实验结果表明，仅在合成数据集上训练的模型可以将其性能推广到人类语音数据。通过消除对人类语音数据收集的依赖，这些见解为基于音频的DST的重大实际进展铺平了道路。数据和代码可在https://github.com/JihyunLee1/E2E-DST上获得。摘要：Dialogue state tracking plays a crucial role in extracting information in task-oriented dialogue systems. However, preceding research are limited to textual modalities, primarily due to the shortage of authentic human audio datasets. We address this by investigating synthetic audio data for audio-based DST. To this end, we develop cascading and end-to-end models, train them with our synthetic audio dataset, and test them on actual human speech data. To facilitate evaluation tailored to audio modalities, we introduce a novel PhonemeF1 to capture pronunciation similarity. Experimental results showed that models trained solely on synthetic datasets can generalize their performance to human voice data. By eliminating the dependency on human speech data collection, these insights pave the way for significant practical advancements in audio-based DST. Data and code are available at https://github.com/JihyunLee1/E2E-DST.
【2】 A text-dependent speaker verification application framework based on Chinese numerical string corpus标题：基于中文数字串语料库的文本相关说话人确认应用框架链接：https://arxiv.org/abs/2312.01645作者：Litong Zheng,Feng Hong,Weijie Xu摘要：研究表明，文本相关说话人确认（TD-SV）往往优于文本无关的验证（TI-SV）在短语音场景。然而，收集大规模的固定文本语音数据是具有挑战性的，随着语音长度的增加，句子节奏和停顿等因素影响TDSV对文本序列的敏感性。基于这些因素，我们提出的假设，如更细粒度的池化方法的时间尺度和语音说话人嵌入和文本嵌入的解耦表示更适合TD-SV的策略。我们已经介绍了一个端到端的TD-SV系统的基础上的数据集，包括较长的中文数字字符串文本。它包含一个文本嵌入网络，一个扬声器嵌入网络和后端融合。首先，我们记录了一个由长中文数字文本组成的数据集，名为SHAL，该数据集在Open-SLR网站上公开。我们通过使用Tacotron 2和HiFi-GAN来增强数据集稀缺性来解决这个问题。接下来，我们介绍了一种具有文本嵌入和说话人嵌入的语音双重表示。在文本嵌入网络中，我们采用了增强的Transformer，并引入了三重损失，包括文本分类损失，CTC损失和解码器损失。对于说话人嵌入网络，我们改进了滑动窗口注意统计池（SWASP），并结合注意统计池（ASP）创建了一个多尺度池方法。最后，我们融合了文本嵌入和说话人嵌入。我们的池化方法在Hi-Mia和SHAL上分别实现了49.2%和75.0%的等错误率（EER）性能改进。摘要：Researches indicate that text-dependent speaker verification (TD-SV) often outperforms text-independent verification (TI-SV) in short speech scenarios. However, collecting large-scale fixed text speech data is challenging, and as speech length increases, factors like sentence rhythm and pauses affect TDSV's sensitivity to text sequence. Based on these factors, We propose the hypothesis that strategies such as more fine-grained pooling methods on time scales and decoupled representations of speech speaker embedding and text embedding are more suitable for TD-SV. We have introduced an end-to-end TD-SV system based on a dataset comprising longer Chinese numerical string texts. It contains a text embedding network, a speaker embedding network, and back-end fusion. First, we recorded a dataset consisting of long Chinese numerical text named SHAL, which is publicly available on the Open-SLR website. We addressed the issue of dataset scarcity by augmenting it using Tacotron2 and HiFi-GAN. Next, we introduced a dual representation of speech with text embedding and speaker embedding. In the text embedding network, we employed an enhanced Transformer and introduced a triple loss that includes text classification loss, CTC loss, and decoder loss. For the speaker embedding network, we enhanced a sliding window attentive statistics pooling (SWASP), combined with attentive statistics pooling (ASP) to create a multi-scale pooling method. Finally, we fused text embedding and speaker embedding. Our pooling methods achieved an equal error rate (EER) performance improvement of 49.2% on Hi-Mia and 75.0% on SHAL, respectively.【3】 Multimodal Speech Emotion Recognition Using Modality-specific Self-Supervised Frameworks标题：基于特定通道自监督框架的多通道语音情感识别链接：https://arxiv.org/abs/2312.01568作者：Rutherford Agbeshi Patamia,Paulo E. Santos,Kingsley Nketia Acheampong,Favour Ekong,Kwabena Sarpong,She Kun摘要：情感识别是辅助机器人领域的一个重要课题，因为需要让机器人具备理解人类行为的能力，从而促进它们在社会中的有效互动。因此，需要支持最佳人机通信的高效且可靠的情感识别系统。多模态（包括语音、音频、文本、图像和视频）通常用于情感识别任务。许多相关研究都是基于合并多种数据模式和利用低级数据表示来训练深度学习模型。然而，大多数现有的情感数据库都不够大（或复杂），不足以让机器学习方法学习详细的表示。本文探讨了特定于模态的预训练Transformer框架，用于语音和文本表示的自监督学习，以实现数据高效的情感识别，同时实现最先进的情感识别性能。该模型采用特征级融合，使用非言语线索数据点从运动捕捉，以提供多模态语音情感识别。该模型使用公开的IEMOCAP数据集进行训练，对四种情绪的总体准确率为77.58%，优于最先进的方法摘要：Emotion recognition is a topic of significant interest in assistive robotics due to the need to equip robots with the ability to comprehend human behavior, facilitating their effective interaction in our society. Consequently, efficient and dependable emotion recognition systems supporting optimal human-machine communication are required. Multi-modality (including speech, audio, text, images, and videos) is typically exploited in emotion recognition tasks. Much relevant research is based on merging multiple data modalities and training deep learning models utilizing low-level data representations. However, most existing emotion databases are not large (or complex) enough to allow machine learning approaches to learn detailed representations. This paper explores modalityspecific pre-trained transformer frameworks for self-supervised learning of speech and text representations for data-efficient emotion recognition while achieving state-of-the-art performance in recognizing emotions. This model applies feature-level fusion using nonverbal cue data points from motion capture to provide multimodal speech emotion recognition. The model was trained using the publicly available IEMOCAP dataset, achieving an overall accuracy of 77.58% for four emotions, outperforming state-of-the-art approaches【4】 Building Ears for Robots: Machine Hearing in the Age of Autonomy标题：为机器人制造耳朵：自主时代的机器听力链接：https://arxiv.org/abs/2312.01554作者：Xuan Zhong备注：11 pages, 6 figures The materials covered in this article were presented and discussed at the Hearing Seminar at Stanford University organized by Malcolm Slaney in October, 2023摘要：随着在不确定环境中工作的机器人数量的增加，机器人听觉系统成为一个重要的研究课题。本研究讨论了听觉系统对机器人意味着什么以及为什么它很重要。以机器人出租车为例，介绍了其硬件设计原理，并利用车外麦克风阵列检测警笛等异常声音事件。在此基础上，基于现代概率机器人的分类，初步开发了机器人听觉软件设计框架，作为决策过程的一部分。摘要：Robot hearing system is becoming an important topic due to the increasing number of field robots in uncertain environments. This study discusses what a hearing system means to a robot and why it is important. In particular, the hardware design principles are introduced with the example of robotaxi, on which exterior microphone arrays are used for detection of siren and other abnormal sound events. After that, a preliminary robot hearing software design framework is developed based on the taxonomy of modern probabilistic robotics as a part of decision processes.【5】 Bigger is not Always Better: The Effect of Context Size on Speech Pre-Training标题：越大不一定越好：语境大小对语音预训练的影响链接：https://arxiv.org/abs/2312.01515作者：Sean Robertson,Ewan Dunbar备注：Repository at this https URL This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible摘要：在自动语音识别（ASR）文献中，通常假设模型最好能够访问更宽的上下文窗口。然而，许多在监督环境中可能是正确的潜在原因并不一定转移到无监督学习的情况下。我们研究了使用自监督学习来实现高质量的预训练声学模型需要多少上下文。我们主要研究对比预测编码（CPC），我们调整它，以便能够在训练和推理过程中精确控制模型可见的上下文量。我们发现，在所得到的模型表示的电话可辨别性峰值在40毫秒左右的前一个上下文，并有太多的上下文（超过约320毫秒）大大降低了质量的表示。令人惊讶的是，我们发现，当预训练的表示用作冻结的输入特征时，这种模式也会转移到有监督的ASR。我们的研究结果指出，在当前的上游架构的设计，以更好地促进各种下游任务的潜在变化。摘要：It has been generally assumed in the automatic speech recognition (ASR) literature that it is better for models to have access to wider context windows. Yet, many of the potential reasons this might be true in the supervised setting do not necessarily transfer over to the case of unsupervised learning. We investigate how much context is necessary to achieve high-quality pre-trained acoustic models using self-supervised learning. We principally investigate contrastive predictive coding (CPC), which we adapt to be able to precisely control the amount of context visible to the model during training and inference. We find that phone discriminability in the resulting model representations peaks at around 40~ms of preceding context, and that having too much context (beyond around 320 ms) substantially degrades the quality of the representations. Surprisingly, we find that this pattern also transfers to supervised ASR when the pre-trained representations are used as frozen input features. Our results point to potential changes in the design of current upstream architectures to better facilitate a variety of downstream tasks.【6】 OpenVoice: Versatile Instant Voice Cloning标题：OpenVoice：多功能即时语音克隆链接：https://arxiv.org/abs/2312.01479作者：Zengyi Qin,Wenliang Zhao,Xumin Yu,Xin Sun备注：Technical Report摘要：我们介绍OpenVoice，这是一种多功能的语音克隆方法，只需要参考扬声器的一个简短的音频片段就可以复制他们的声音并生成多种语言的语音。OpenVoice在解决该领域的以下开放性挑战方面取得了重大进展：1）灵活的语音风格控制。OpenVoice支持对语音风格的精细控制，包括情感、口音、节奏、停顿和语调，以及复制参考说话者的音色。语音风格不直接从参考说话者的风格复制并受其约束。以前的方法缺乏在克隆后灵活操纵语音风格的能力。2)Zero-Shot跨语言语音克隆。OpenVoice实现了zero-shot跨语言语音克隆，适用于未包含在说话人-说话人训练集中的语言。与以前的方法不同，这些方法通常需要针对所有语言的大量非母语多语言（MSML）数据集，OpenVoice可以将语音克隆到一种新的语言中，而无需该语言的任何非母语训练数据。OpenVoice在计算上也很高效，成本比商业上可用的API低几十倍，后者的性能甚至更差。为了促进该领域的进一步研究，我们公开了源代码和训练模型。我们还在演示网站中提供定性结果。在公开发布之前，我们的OpenVoice内部版本在2023年5月至10月期间被全球用户使用了数千万次，作为MyShell.ai的后端。摘要：We introduce OpenVoice, a versatile voice cloning approach that requires only a short audio clip from the reference speaker to replicate their voice and generate speech in multiple languages. OpenVoice represents a significant advancement in addressing the following open challenges in the field: 1) Flexible Voice Style Control. OpenVoice enables granular control over voice styles, including emotion, accent, rhythm, pauses, and intonation, in addition to replicating the tone color of the reference speaker. The voice styles are not directly copied from and constrained by the style of the reference speaker. Previous approaches lacked the ability to flexibly manipulate voice styles after cloning. 2) Zero-Shot Cross-Lingual Voice Cloning. OpenVoice achieves zero-shot cross-lingual voice cloning for languages not included in the massive-speaker training set. Unlike previous approaches, which typically require extensive massive-speaker multi-lingual (MSML) dataset for all languages, OpenVoice can clone voices into a new language without any massive-speaker training data for that language. OpenVoice is also computationally efficient, costing tens of times less than commercially available APIs that offer even inferior performance. To foster further research in the field, we have made the source code and trained model publicly accessible. We also provide qualitative results in our demo website. Prior to its public release, our internal version of OpenVoice was used tens of millions of times by users worldwide between May and October 2023, serving as the backend of MyShell.ai.【7】 A Semi-Supervised Deep Learning Approach to Dataset Collection for Query-By-Humming Task标题：一种基于半监督深度学习的哼唱查询数据集收集方法链接：https://arxiv.org/abs/2312.01092作者：Amantur Amatov,Dmitry Lamanov,Maksim Titov,Ivan Vovk,Ilya Makarov,Mikhail Kudinov摘要：哼唱查询（Query-by-Humming，QbH）是一项基于哼唱或演唱片段找到最相关歌曲的任务。尽管最近有成功的商业解决方案，但由于缺乏用于训练机器学习模型的高质量数据集，实现QbH系统仍然具有挑战性。在本文中，我们提出了一种深度学习数据收集技术，并介绍了Covers and Hummings Aligned Dataset（CHAD），这是一种新的数据集，包含18个小时的短音乐片段，与时间对齐的哼唱版本配对。为了扩展我们的数据集，我们采用了半监督模型训练管道，该管道利用QbH任务作为翻唱歌曲识别（CSI）任务的特殊情况。从在初始数据集上训练的模型开始，我们迭代地收集同一首歌的翻唱版本的片段组，并在扩展数据上重新训练模型。使用这个管道，我们收集了超过308小时的额外音乐片段，并与时间对齐的封面版本配对。最终模型成功地应用于QbH任务，并在基准数据集上取得了有竞争力的结果。我们的研究表明，所提出的数据集和训练管道可以有效地促进QbH系统的实现。摘要：Query-by-Humming (QbH) is a task that involves finding the most relevant song based on a hummed or sung fragment. Despite recent successful commercial solutions, implementing QbH systems remains challenging due to the lack of high-quality datasets for training machine learning models. In this paper, we propose a deep learning data collection technique and introduce Covers and Hummings Aligned Dataset (CHAD), a novel dataset that contains 18 hours of short music fragments, paired with time-aligned hummed versions. To expand our dataset, we employ a semi-supervised model training pipeline that leverages the QbH task as a specialized case of cover song identification (CSI) task. Starting with a model trained on the initial dataset, we iteratively collect groups of fragments of cover versions of the same song and retrain the model on the extended data. Using this pipeline, we collect over 308 hours of additional music fragments, paired with time-aligned cover versions. The final model is successfully applied to the QbH task and achieves competitive results on benchmark datasets. Our study shows that the proposed dataset and training pipeline can effectively facilitate the implementation of QbH systems.【8】 Acoustic Signal Analysis with Deep Neural Network for Detecting Fault Diagnosis in Industrial Machines标题：声信号深度神经网络分析在工业机械故障诊断中的应用链接：https://arxiv.org/abs/2312.01062作者：Mustafa Yurdakul,Sakir Tasdemir摘要：在早期阶段检测机器故障对于减少工业环境中操作过程的中断至关重要。最近，深度学习方法开始成为机器故障检测的首选。由于自动特征提取，深度学习在故障检测过程中提供了有效的解决方案。在这项研究中，设计了一个基于深度学习的系统来分析工业机器产生的声音信号。声学声音信号被转换成梅尔频谱图。为了对频谱图图像进行分类，使用了DenseNet-169模型，这是一种在图像分类任务中有效的深度学习架构。该模型使用迁移学习方法在MIMII数据集上进行训练，包括来自四种工业机器的声音。实验结果表明，该方法在不同的声噪声率水平下，准确率在97.17%~ 99.87%之间。摘要：Detecting machine malfunctions at an early stage is crucial for reducing interruptions in operational processes within industrial settings. Recently, the deep learning approach has started to be preferred for the detection of failures in machines. Deep learning provides an effective solution in fault detection processes thanks to automatic feature extraction. In this study, a deep learning-based system was designed to analyze the sound signals produced by industrial machines. Acoustic sound signals were converted into Mel spectrograms. For the purpose of classifying spectrogram images, the DenseNet-169 model, a deep learning architecture recognized for its effectiveness in image classification tasks, was used. The model was trained using the transfer learning method on the MIMII dataset including sounds from four types of industrial machines. The results showed that the proposed method reached an accuracy rate varying between 97.17% and 99.87% at different Sound Noise Rate levels.【9】 End-to-End Speech-to-Text Translation: A Survey标题：端到端语音到文本翻译：综述链接：https://arxiv.org/abs/2312.01053作者：Nivedita Sethiya,Chandresh Kumar Maurya备注：45 pages摘要：语音文本翻译是指将一种语言的语音信号转换为另一种语言的文本。它在各种领域中找到其应用，例如免提通信、听写、视频讲座转录和翻译，仅举几例。自动语音识别（ASR）和机器翻译（MT）模型在传统的ST翻译中发挥着至关重要的作用，能够将原始形式的口语转换为书面文本，并促进无缝的跨语言交流。ASR识别口语，而MT将转录的文本翻译成目标语言。这种分解的模型遭受级联误差传播和高资源和培训成本。因此，研究人员一直在探索用于ST翻译的端到端（E2E）模型。然而，据我们所知，当局并没有全面检讨东二东街的现有工程，因此，今次的调查是就这方面的工作进行讨论。我们的尝试一直是提供一个全面的审查模型，指标和用于ST任务的数据集，提供新的见解的挑战和未来的研究方向。我们相信这篇综述将有助于研究ST模型的各种应用的研究人员。摘要：Speech-to-text translation pertains to the task of converting speech signals in a language to text in another language. It finds its application in various domains, such as hands-free communication, dictation, video lecture transcription, and translation, to name a few. Automatic Speech Recognition (ASR), as well as Machine Translation(MT) models, play crucial roles in traditional ST translation, enabling the conversion of spoken language in its original form to written text and facilitating seamless cross-lingual communication. ASR recognizes spoken words, while MT translates the transcribed text into the target language. Such disintegrated models suffer from cascaded error propagation and high resource and training costs. As a result, researchers have been exploring end-to-end (E2E) models for ST translation. However, to our knowledge, there is no comprehensive review of existing works on E2E ST. The present survey, therefore, discusses the work in this direction. Our attempt has been to provide a comprehensive review of models employed, metrics, and datasets used for ST tasks, providing challenges and future research direction with new insights. We believe this review will be helpful to researchers working on various applications of ST models.【10】 Unveiling the Power of Audio-Visual Early Fusion Transformers with Dense Interactions through Masked Modeling标题：通过掩蔽建模揭示密集交互的视听早期融合Transformer的力量链接：https://arxiv.org/abs/2312.01017作者：Shentong Mo,Pedro Morgado摘要：人类具有整合听觉和视觉信息的非凡能力，能够更深入地了解周围环境。通过认知心理学和神经科学研究证明，这种早期的听觉和视觉线索的融合为开发多模态感知模型提供了有希望的潜力。然而，训练早期融合架构带来了重大挑战，因为增加的模型表达能力需要强大的学习框架来利用其增强的功能。在本文中，我们解决这一挑战，利用掩蔽重建框架，以前成功的单峰设置，训练视听编码器与早期融合。此外，我们提出了一个基于注意力的融合模块，捕获本地音频和视觉表示之间的交互，增强了模型捕获细粒度交互的能力。虽然有效，但随着局部表示的数量增加，该过程在计算上可能变得难以处理。因此，为了解决计算的复杂性，我们提出了一种替代的过程，分解的本地表示视听交互之前。对各种数据集的广泛评估表明，我们的方法在音频事件分类，视觉声音定位，声音分离和视听分割的优越性。这些贡献使深度集成的视听模型的有效训练成为可能，并显着提高了早期融合架构的实用性。摘要：Humans possess a remarkable ability to integrate auditory and visual information, enabling a deeper understanding of the surrounding environment. This early fusion of audio and visual cues, demonstrated through cognitive psychology and neuroscience research, offers promising potential for developing multimodal perception models. However, training early fusion architectures poses significant challenges, as the increased model expressivity requires robust learning frameworks to harness their enhanced capabilities. In this paper, we address this challenge by leveraging the masked reconstruction framework, previously successful in unimodal settings, to train audio-visual encoders with early fusion. Additionally, we propose an attention-based fusion module that captures interactions between local audio and visual representations, enhancing the model's ability to capture fine-grained interactions. While effective, this procedure can become computationally intractable, as the number of local representations increases. Thus, to address the computational complexity, we propose an alternative procedure that factorizes the local representations before representing audio-visual interactions. Extensive evaluations on a variety of datasets demonstrate the superiority of our approach in audio-event classification, visual sound localization, sound separation, and audio-visual segmentation. These contributions enable the efficient training of deeply integrated audio-visual models and significantly advance the usefulness of early fusion architectures.【11】 AV-RIR: Audio-Visual Room Impulse Response Estimation标题：AV-RIR：视听房间脉冲响应估计链接：https://arxiv.org/abs/2312.00834作者：Anton Ratnarajah,Sreyan Ghosh,Sonal Kumar,Purva Chiniya,Dinesh Manocha摘要：准确估计房间脉冲响应（RIR），捕捉环境的声学特性，是语音处理和AR/VR应用的重要。我们提出了AV-RIR，一种新的多模态多任务学习方法，从给定的混响语音信号及其相应环境的视觉线索中准确地估计RIR。AV-RIR建立在一种新型的基于神经编解码器的架构之上，该架构可以有效地捕获环境几何形状和材料属性，并通过使用多任务学习来解决语音去混响作为辅助任务。我们还提出了Geo-Mat功能，将材料信息增强到视觉线索和CRIP中，通过图像到RIR检索将估计RIR中的后期混响分量提高了86%。实证结果表明，AV-RIR定量优于以前的音频和视觉的方法，实现36% - 63%的改善，在各种声学指标的RIR估计。此外，它还在人类评估中获得了更高的偏好分数。作为一个辅助优势，AV-RIR的去混响语音在各种口语处理任务中表现出与最先进技术相媲美的性能，并优于真实世界AVSpeech数据集中的混响时间误差分数。合成混响语音和增强语音的定性示例可以在https://www.youtube.com/watch?上找到v=tTsKhviukAE。摘要：Accurate estimation of Room Impulse Response (RIR), which captures an environment's acoustic properties, is important for speech processing and AR/VR applications. We propose AV-RIR, a novel multi-modal multi-task learning approach to accurately estimate the RIR from a given reverberant speech signal and the visual cues of its corresponding environment. AV-RIR builds on a novel neural codec-based architecture that effectively captures environment geometry and materials properties and solves speech dereverberation as an auxiliary task by using multi-task learning. We also propose Geo-Mat features that augment material information into visual cues and CRIP that improves late reverberation components in the estimated RIR via image-to-RIR retrieval by 86%. Empirical results show that AV-RIR quantitatively outperforms previous audio-only and visual-only approaches by achieving 36% - 63% improvement across various acoustic metrics in RIR estimation. Additionally, it also achieves higher preference scores in human evaluation. As an auxiliary benefit, dereverbed speech from AV-RIR shows competitive performance with the state-of-the-art in various spoken language processing tasks and outperforms reverberation time error score in the real-world AVSpeech dataset. Qualitative examples of both synthesized reverberant speech and enhanced speech can be found at https://www.youtube.com/watch?v=tTsKhviukAE.【12】 Talent-Interview: Web-Client Cheating Detection for Online Exams标题：才艺面试：在线考试的网络客户端作弊检测链接：https://arxiv.org/abs/2312.00795作者：Mert Ege,Mustafa Ceyhan备注：This paper explains the workflow of Talent Interview project that is web-client cheating detection application摘要：新冠肺炎疫情之后，在线考试更具吸引力。此外，在招聘过程中，还使用了在线考试。然而，在线考试有更多的作弊可能性。每次考试都要找一个监考人，这会增加成本。在这一点上，自动监考系统检测可能的作弊状态。本文提出了一个端到端的系统和子模块，以获得更好的效果，在线监考。在我们的系统中使用的对象检测，人脸识别，人类的语音检测和分割。此外，我们提出的模型适用于用户的PC，这意味着基于客户端的系统。因此，消除了服务器成本。据我们所知，这是第一次使用基于客户端的在线监考系统进行招聘。新冠肺炎疫情之后，在线考试更具吸引力。此外，在招聘过程中，还使用了在线考试。然而，在线考试有更多的作弊可能性。每次考试都要找一个监考人，这会增加成本。在这一点上，自动监考系统检测可能的作弊状态。本文提出了一个端到端的系统和子模块，以获得更好的效果，在线监考。在我们的系统中使用的对象检测，人脸识别，人类的语音检测和分割。此外，我们提出的模型适用于用户的PC，这意味着基于客户端的系统。因此，消除了服务器成本。据我们所知，这是第一次使用基于客户端的在线监考系统进行招聘。此外，这个作弊系统在https://www.talent-interview.com/tr/上工作。摘要：Online exams are more attractive after the Covid-19 pandemic. Furthermore, during recruitment, online exams are used. However, there are more cheating possibilities for online exams. Assigning a proctor for each exam increases cost. At this point, automatic proctor systems detect possible cheating status. This article proposes an end-to-end system and submodules to get better results for online proctoring. Object detection, face recognition, human voice detection, and segmentation are used in our system. Furthermore, our proposed model works on the PCs of users, meaning a client-based system. So, server cost is eliminated. As far as we know, it is the first time the client-based online proctoring system has been used for recruitment. Online exams are more attractive after the Covid-19 pandemic. Furthermore, during recruitment, online exams are used. However, there are more cheating possibilities for online exams. Assigning a proctor for each exam increases cost. At this point, automatic proctor systems detect possible cheating status. This article proposes an end-to-end system and submodules to get better results for online proctoring. Object detection, face recognition, human voice detection, and segmentation are used in our system. Furthermore, our proposed model works on the PCs of users, meaning a client-based system. So, server cost is eliminated. As far as we know, it is the first time the client-based online proctoring system has been used for recruitment. Furthermore, this cheating system works at https://www.talent-interview.com/tr/.【13】 Head Orientation Estimation with Distributed Microphones Using Speech Radiation Patterns标题：基于语音辐射模式的分布式麦克风头部方位估计链接：https://arxiv.org/abs/2312.01808作者：Kaspar Müller,Bilgesu Çakmak,Paul Didier,Simon Doclo,Jan Østergaard,Tobias Wolff备注：6 pages, submitted to 57th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA, 2023摘要：确定说话者的头部方向不仅有利于各种语音信号处理应用，例如源定位或语音增强，而且还有助于直观的语音控制以及与智能环境或现代汽车助手的交互。大多数头部方向估计方法都是基于视觉线索的。然而，这需要通常不可用的相机系统。我们提出了一种方法，它纯粹使用音频信号捕获只有几个分布式麦克风周围的谈话者。具体而言，我们提出了一种新的方法，直接采用测量或建模的语音辐射模式，以推断说话者的方向在积极的语音周期的基础上的余弦相似性度量。此外，自动增益调整技术提出了未校准的，不规则的麦克风设置，如ad-hoc传感器网络。在实验中记录在消声和混响环境中的信号，所提出的方法优于国家的最先进的方法，使用测量或建模的语音辐射模式。摘要：Determining the head orientation of a talker is not only beneficial for various speech signal processing applications, such as source localization or speech enhancement, but also facilitates intuitive voice control and interaction with smart environments or modern car assistants. Most approaches for head orientation estimation are based on visual cues. However, this requires camera systems which often are not available. We present an approach which purely uses audio signals captured with only a few distributed microphones around the talker. Specifically, we propose a novel method that directly incorporates measured or modeled speech radiation patterns to infer the talker's orientation during active speech periods based on a cosine similarity measure. Moreover, an automatic gain adjustment technique is proposed for uncalibrated, irregular microphone setups, such as ad-hoc sensor networks. In experiments with signals recorded in both anechoic and reverberant environments, the proposed method outperforms state-of-the-art approaches, using either measured or modeled speech radiation patterns.【1】 Head Orientation Estimation with Distributed Microphones Using Speech Radiation Patterns标题：基于语音辐射模式的分布式麦克风头部方位估计链接：https://arxiv.org/abs/2312.01808作者：Kaspar Müller,Bilgesu Çakmak,Paul Didier,Simon Doclo,Jan Østergaard,Tobias Wolff备注：6 pages, submitted to 57th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA, 2023摘要：确定说话者的头部方向不仅有利于各种语音信号处理应用，例如源定位或语音增强，而且还有助于直观的语音控制以及与智能环境或现代汽车助手的交互。大多数头部方向估计方法都是基于视觉线索的。然而，这需要通常不可用的相机系统。我们提出了一种方法，它纯粹使用音频信号捕获只有几个分布式麦克风周围的谈话者。具体而言，我们提出了一种新的方法，直接采用测量或建模的语音辐射模式，以推断说话者的方向在积极的语音周期的基础上的余弦相似性度量。此外，自动增益调整技术提出了未校准的，不规则的麦克风设置，如ad-hoc传感器网络。在实验中记录在消声和混响环境中的信号，所提出的方法优于国家的最先进的方法，使用测量或建模的语音辐射模式。摘要：Determining the head orientation of a talker is not only beneficial for various speech signal processing applications, such as source localization or speech enhancement, but also facilitates intuitive voice control and interaction with smart environments or modern car assistants. Most approaches for head orientation estimation are based on visual cues. However, this requires camera systems which often are not available. We present an approach which purely uses audio signals captured with only a few distributed microphones around the talker. Specifically, we propose a novel method that directly incorporates measured or modeled speech radiation patterns to infer the talker's orientation during active speech periods based on a cosine similarity measure. Moreover, an automatic gain adjustment technique is proposed for uncalibrated, irregular microphone setups, such as ad-hoc sensor networks. In experiments with signals recorded in both anechoic and reverberant environments, the proposed method outperforms state-of-the-art approaches, using either measured or modeled speech radiation patterns.【2】 SEFGAN: Harvesting the Power of Normalizing Flows and GANs for Efficient High-Quality Speech Enhancement标题：SEFGAN：获取流和GAN标准化的力量，以实现高效的高质量语音增强链接：https://arxiv.org/abs/2312.01744作者：Martin Strauss,Nicola Pia,Nagashree K. S. Rao,Bernd Edler备注：Preprint. Accepted to IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA) 2023摘要：本文提出了SEFGAN，一种结合最大似然训练和生成对抗网络（GANs）的深度神经网络（DNN），用于有效的语音增强（SE）。为此，DNN被训练为使用归一化流（NF）作为GAN框架中的生成器来合成以噪声语音为条件的增强语音。虽然似然模型和GANs的组合并不简单，但SEFGAN证明了混合对抗和最大似然训练方法使模型能够保持高质量的音频生成和对数似然估计。我们的实验表明，这种方法大大优于基于NF的基线模型，而不会引入额外的复杂性的增强网络。使用计算指标和听力实验的比较表明，SEFGAN与其他国家的最先进的模型是有竞争力的。摘要：This paper proposes SEFGAN, a Deep Neural Network (DNN) combining maximum likelihood training and Generative Adversarial Networks (GANs) for efficient speech enhancement (SE). For this, a DNN is trained to synthesize the enhanced speech conditioned on noisy speech using a Normalizing Flow (NF) as generator in a GAN framework. While the combination of likelihood models and GANs is not trivial, SEFGAN demonstrates that a hybrid adversarial and maximum likelihood training approach enables the model to maintain high quality audio generation and log-likelihood estimation. Our experiments indicate that this approach strongly outperforms the baseline NF-based model without introducing additional complexity to the enhancement network. A comparison using computational metrics and a listening experiment reveals that SEFGAN is competitive with other state-of-the-art models.【3】 Exploring the Viability of Synthetic Audio Data for Audio-Based Dialogue State Tracking标题：探索合成音频数据用于基于音频的对话状态跟踪的可行性链接：https://arxiv.org/abs/2312.01842作者：Jihyun Lee,Yejin Jeon,Wonjun Lee,Yunsu Kim,Gary Geunbae Lee备注：Accepted in ASRU 2023摘要：在面向任务的对话系统中，对话状态跟踪在信息提取中起着至关重要的作用。然而，以前的研究仅限于文本模态，主要是由于缺乏真实的人类音频数据集。我们通过调查基于音频的DST的合成音频数据来解决这个问题。为此，我们开发了级联和端到端模型，用我们的合成音频数据集对它们进行训练，并在实际的人类语音数据上对其进行测试。为了便于评估量身定制的音频模态，我们引入了一种新的PhonemeF1来捕获发音相似性。实验结果表明，仅在合成数据集上训练的模型可以将其性能推广到人类语音数据。通过消除对人类语音数据收集的依赖，这些见解为基于音频的DST的重大实际进展铺平了道路。数据和代码可在https://github.com/JihyunLee1/E2E-DST上获得。摘要：Dialogue state tracking plays a crucial role in extracting information in task-oriented dialogue systems. However, preceding research are limited to textual modalities, primarily due to the shortage of authentic human audio datasets. We address this by investigating synthetic audio data for audio-based DST. To this end, we develop cascading and end-to-end models, train them with our synthetic audio dataset, and test them on actual human speech data. To facilitate evaluation tailored to audio modalities, we introduce a novel PhonemeF1 to capture pronunciation similarity. Experimental results showed that models trained solely on synthetic datasets can generalize their performance to human voice data. By eliminating the dependency on human speech data collection, these insights pave the way for significant practical advancements in audio-based DST. Data and code are available at https://github.com/JihyunLee1/E2E-DST.【4】 A text-dependent speaker verification application framework based on Chinese numerical string corpus标题：基于中文数字串语料库的文本相关说话人确认应用框架链接：https://arxiv.org/abs/2312.01645作者：Litong Zheng,Feng Hong,Weijie Xu摘要：研究表明，文本相关说话人确认（TD-SV）往往优于文本无关的验证（TI-SV）在短语音场景。然而，收集大规模的固定文本语音数据是具有挑战性的，随着语音长度的增加，句子节奏和停顿等因素影响TDSV对文本序列的敏感性。基于这些因素，我们提出的假设，如更细粒度的池化方法的时间尺度和语音说话人嵌入和文本嵌入的解耦表示更适合TD-SV的策略。我们已经介绍了一个端到端的TD-SV系统的基础上的数据集，包括较长的中文数字字符串文本。它包含一个文本嵌入网络，一个扬声器嵌入网络和后端融合。首先，我们记录了一个由长中文数字文本组成的数据集，名为SHAL，该数据集在Open-SLR网站上公开。我们通过使用Tacotron 2和HiFi-GAN来增强数据集稀缺性来解决这个问题。接下来，我们介绍了一种具有文本嵌入和说话人嵌入的语音双重表示。在文本嵌入网络中，我们采用了增强的Transformer，并引入了三重损失，包括文本分类损失，CTC损失和解码器损失。对于说话人嵌入网络，我们改进了滑动窗口注意统计池（SWASP），并结合注意统计池（ASP）创建了一个多尺度池方法。最后，我们融合了文本嵌入和说话人嵌入。我们的池化方法在Hi-Mia和SHAL上分别实现了49.2%和75.0%的等错误率（EER）性能改进。摘要：Researches indicate that text-dependent speaker verification (TD-SV) often outperforms text-independent verification (TI-SV) in short speech scenarios. However, collecting large-scale fixed text speech data is challenging, and as speech length increases, factors like sentence rhythm and pauses affect TDSV's sensitivity to text sequence. Based on these factors, We propose the hypothesis that strategies such as more fine-grained pooling methods on time scales and decoupled representations of speech speaker embedding and text embedding are more suitable for TD-SV. We have introduced an end-to-end TD-SV system based on a dataset comprising longer Chinese numerical string texts. It contains a text embedding network, a speaker embedding network, and back-end fusion. First, we recorded a dataset consisting of long Chinese numerical text named SHAL, which is publicly available on the Open-SLR website. We addressed the issue of dataset scarcity by augmenting it using Tacotron2 and HiFi-GAN. Next, we introduced a dual representation of speech with text embedding and speaker embedding. In the text embedding network, we employed an enhanced Transformer and introduced a triple loss that includes text classification loss, CTC loss, and decoder loss. For the speaker embedding network, we enhanced a sliding window attentive statistics pooling (SWASP), combined with attentive statistics pooling (ASP) to create a multi-scale pooling method. Finally, we fused text embedding and speaker embedding. Our pooling methods achieved an equal error rate (EER) performance improvement of 49.2% on Hi-Mia and 75.0% on SHAL, respectively.【5】 Multimodal Speech Emotion Recognition Using Modality-specific Self-Supervised Frameworks标题：基于特定模态自监督框架的多模态语音情感识别链接：https://arxiv.org/abs/2312.01568作者：Rutherford Agbeshi Patamia,Paulo E. Santos,Kingsley Nketia Acheampong,Favour Ekong,Kwabena Sarpong,She Kun摘要：情感识别是辅助机器人领域的一个重要课题，因为需要让机器人具备理解人类行为的能力，从而促进它们在社会中的有效互动。因此，需要支持最佳人机通信的高效且可靠的情感识别系统。多模态（包括语音、音频、文本、图像和视频）通常用于情感识别任务。许多相关研究都是基于合并多种数据模式和利用低级数据表示来训练深度学习模型。然而，大多数现有的情感数据库都不够大（或复杂），不足以让机器学习方法学习详细的表示。本文探讨了特定于模态的预训练Transformer框架，用于语音和文本表示的自监督学习，以实现数据高效的情感识别，同时实现最先进的情感识别性能。该模型采用特征级融合，使用非言语线索数据点从运动捕捉，以提供多模态语音情感识别。该模型使用公开的IEMOCAP数据集进行训练，对四种情绪的总体准确率为77.58%，优于最先进的方法摘要：Emotion recognition is a topic of significant interest in assistive robotics due to the need to equip robots with the ability to comprehend human behavior, facilitating their effective interaction in our society. Consequently, efficient and dependable emotion recognition systems supporting optimal human-machine communication are required. Multi-modality (including speech, audio, text, images, and videos) is typically exploited in emotion recognition tasks. Much relevant research is based on merging multiple data modalities and training deep learning models utilizing low-level data representations. However, most existing emotion databases are not large (or complex) enough to allow machine learning approaches to learn detailed representations. This paper explores modalityspecific pre-trained transformer frameworks for self-supervised learning of speech and text representations for data-efficient emotion recognition while achieving state-of-the-art performance in recognizing emotions. This model applies feature-level fusion using nonverbal cue data points from motion capture to provide multimodal speech emotion recognition. The model was trained using the publicly available IEMOCAP dataset, achieving an overall accuracy of 77.58% for four emotions, outperforming state-of-the-art approaches【6】 Building Ears for Robots: Machine Hearing in the Age of Autonomy标题：为机器人制造耳朵：自主时代的机器听力链接：https://arxiv.org/abs/2312.01554作者：Xuan Zhong备注：11 pages, 6 figures The materials covered in this article were presented and discussed at the Hearing Seminar at Stanford University organized by Malcolm Slaney in October, 2023摘要：随着在不确定环境中工作的机器人数量的增加，机器人听觉系统成为一个重要的研究课题。本研究讨论了听觉系统对机器人意味着什么以及为什么它很重要。以机器人出租车为例，介绍了其硬件设计原理，并利用车外麦克风阵列检测警笛等异常声音事件。在此基础上，基于现代概率机器人的分类，初步开发了机器人听觉软件设计框架，作为决策过程的一部分。摘要：Robot hearing system is becoming an important topic due to the increasing number of field robots in uncertain environments. This study discusses what a hearing system means to a robot and why it is important. In particular, the hardware design principles are introduced with the example of robotaxi, on which exterior microphone arrays are used for detection of siren and other abnormal sound events. After that, a preliminary robot hearing software design framework is developed based on the taxonomy of modern probabilistic robotics as a part of decision processes.【7】 Bigger is not Always Better: The Effect of Context Size on Speech Pre-Training标题：越大不一定越好：语境大小对语音预训练的影响链接：https://arxiv.org/abs/2312.01515作者：Sean Robertson,Ewan Dunbar备注：Repository at this https URL This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible摘要：在自动语音识别（ASR）文献中，通常假设模型最好能够访问更宽的上下文窗口。然而，许多在监督环境中可能是正确的潜在原因并不一定转移到无监督学习的情况下。我们研究了使用自监督学习来实现高质量的预训练声学模型需要多少上下文。我们主要研究对比预测编码（CPC），我们调整它，以便能够在训练和推理过程中精确控制模型可见的上下文量。我们发现，在所得到的模型表示的电话可辨别性峰值在40毫秒左右的前一个上下文，并有太多的上下文（超过约320毫秒）大大降低了质量的表示。令人惊讶的是，我们发现，当预训练的表示用作冻结的输入特征时，这种模式也会转移到有监督的ASR。我们的研究结果指出，在当前的上游架构的设计，以更好地促进各种下游任务的潜在变化。摘要：It has been generally assumed in the automatic speech recognition (ASR) literature that it is better for models to have access to wider context windows. Yet, many of the potential reasons this might be true in the supervised setting do not necessarily transfer over to the case of unsupervised learning. We investigate how much context is necessary to achieve high-quality pre-trained acoustic models using self-supervised learning. We principally investigate contrastive predictive coding (CPC), which we adapt to be able to precisely control the amount of context visible to the model during training and inference. We find that phone discriminability in the resulting model representations peaks at around 40~ms of preceding context, and that having too much context (beyond around 320 ms) substantially degrades the quality of the representations. Surprisingly, we find that this pattern also transfers to supervised ASR when the pre-trained representations are used as frozen input features. Our results point to potential changes in the design of current upstream architectures to better facilitate a variety of downstream tasks.【8】 OpenVoice: Versatile Instant Voice Cloning标题：OpenVoice：多功能即时语音克隆链接：https://arxiv.org/abs/2312.01479作者：Zengyi Qin,Wenliang Zhao,Xumin Yu,Xin Sun备注：Technical Report摘要：我们介绍OpenVoice，这是一种多功能的语音克隆方法，只需要参考扬声器的一个简短的音频片段就可以复制他们的声音并生成多种语言的语音。OpenVoice在解决该领域的以下开放性挑战方面取得了重大进展：1）灵活的语音风格控制。OpenVoice支持对语音风格的精细控制，包括情感、口音、节奏、停顿和语调，以及复制参考说话者的音色。语音风格不直接从参考说话者的风格复制并受其约束。以前的方法缺乏在克隆后灵活操纵语音风格的能力。2)Zero-Shot跨语言语音克隆。OpenVoice实现了zero-shot跨语言语音克隆，适用于未包含在说话人-说话人训练集中的语言。与以前的方法不同，这些方法通常需要针对所有语言的大量非母语多语言（MSML）数据集，OpenVoice可以将语音克隆到一种新的语言中，而无需该语言的任何非母语训练数据。OpenVoice在计算上也很高效，成本比商业上可用的API低几十倍，后者的性能甚至更差。为了促进该领域的进一步研究，我们公开了源代码和训练模型。我们还在演示网站中提供定性结果。在公开发布之前，我们的OpenVoice内部版本在2023年5月至10月期间被全球用户使用了数千万次，作为MyShell.ai的后端。摘要：We introduce OpenVoice, a versatile voice cloning approach that requires only a short audio clip from the reference speaker to replicate their voice and generate speech in multiple languages. OpenVoice represents a significant advancement in addressing the following open challenges in the field: 1) Flexible Voice Style Control. OpenVoice enables granular control over voice styles, including emotion, accent, rhythm, pauses, and intonation, in addition to replicating the tone color of the reference speaker. The voice styles are not directly copied from and constrained by the style of the reference speaker. Previous approaches lacked the ability to flexibly manipulate voice styles after cloning. 2) Zero-Shot Cross-Lingual Voice Cloning. OpenVoice achieves zero-shot cross-lingual voice cloning for languages not included in the massive-speaker training set. Unlike previous approaches, which typically require extensive massive-speaker multi-lingual (MSML) dataset for all languages, OpenVoice can clone voices into a new language without any massive-speaker training data for that language. OpenVoice is also computationally efficient, costing tens of times less than commercially available APIs that offer even inferior performance. To foster further research in the field, we have made the source code and trained model publicly accessible. We also provide qualitative results in our demo website. Prior to its public release, our internal version of OpenVoice was used tens of millions of times by users worldwide between May and October 2023, serving as the backend of MyShell.ai.【9】 A Semi-Supervised Deep Learning Approach to Dataset Collection for Query-By-Humming Task标题：一种基于半监督深度学习的哼唱查询数据集收集方法链接：https://arxiv.org/abs/2312.01092作者：Amantur Amatov,Dmitry Lamanov,Maksim Titov,Ivan Vovk,Ilya Makarov,Mikhail Kudinov摘要：哼唱查询（Query-by-Humming，QbH）是一项基于哼唱或演唱片段找到最相关歌曲的任务。尽管最近有成功的商业解决方案，但由于缺乏用于训练机器学习模型的高质量数据集，实现QbH系统仍然具有挑战性。在本文中，我们提出了一种深度学习数据收集技术，并介绍了Covers and Hummings Aligned Dataset（CHAD），这是一种新的数据集，包含18个小时的短音乐片段，与时间对齐的哼唱版本配对。为了扩展我们的数据集，我们采用了半监督模型训练管道，该管道利用QbH任务作为翻唱歌曲识别（CSI）任务的特殊情况。从在初始数据集上训练的模型开始，我们迭代地收集同一首歌的翻唱版本的片段组，并在扩展数据上重新训练模型。使用这个管道，我们收集了超过308小时的额外音乐片段，并与时间对齐的封面版本配对。最终模型成功地应用于QbH任务，并在基准数据集上取得了有竞争力的结果。我们的研究表明，所提出的数据集和训练管道可以有效地促进QbH系统的实现。摘要：Query-by-Humming (QbH) is a task that involves finding the most relevant song based on a hummed or sung fragment. Despite recent successful commercial solutions, implementing QbH systems remains challenging due to the lack of high-quality datasets for training machine learning models. In this paper, we propose a deep learning data collection technique and introduce Covers and Hummings Aligned Dataset (CHAD), a novel dataset that contains 18 hours of short music fragments, paired with time-aligned hummed versions. To expand our dataset, we employ a semi-supervised model training pipeline that leverages the QbH task as a specialized case of cover song identification (CSI) task. Starting with a model trained on the initial dataset, we iteratively collect groups of fragments of cover versions of the same song and retrain the model on the extended data. Using this pipeline, we collect over 308 hours of additional music fragments, paired with time-aligned cover versions. The final model is successfully applied to the QbH task and achieves competitive results on benchmark datasets. Our study shows that the proposed dataset and training pipeline can effectively facilitate the implementation of QbH systems.【10】 End-to-End Speech-to-Text Translation: A Survey标题：端到端语音到文本翻译：综述链接：https://arxiv.org/abs/2312.01053作者：Nivedita Sethiya,Chandresh Kumar Maurya备注：45 pages摘要：语音文本翻译是指将一种语言的语音信号转换为另一种语言的文本。它在各种领域中找到其应用，例如免提通信、听写、视频讲座转录和翻译，仅举几例。自动语音识别（ASR）和机器翻译（MT）模型在传统的ST翻译中发挥着至关重要的作用，能够将原始形式的口语转换为书面文本，并促进无缝的跨语言交流。ASR识别口语，而MT将转录的文本翻译成目标语言。这种分解的模型遭受级联误差传播和高资源和培训成本。因此，研究人员一直在探索用于ST翻译的端到端（E2E）模型。然而，据我们所知，当局并没有全面检讨东二东街的现有工程，因此，今次的调查是就这方面的工作进行讨论。我们的尝试一直是提供一个全面的审查模型，指标和用于ST任务的数据集，提供新的见解的挑战和未来的研究方向。我们相信这篇综述将有助于研究ST模型的各种应用的研究人员。摘要：Speech-to-text translation pertains to the task of converting speech signals in a language to text in another language. It finds its application in various domains, such as hands-free communication, dictation, video lecture transcription, and translation, to name a few. Automatic Speech Recognition (ASR), as well as Machine Translation(MT) models, play crucial roles in traditional ST translation, enabling the conversion of spoken language in its original form to written text and facilitating seamless cross-lingual communication. ASR recognizes spoken words, while MT translates the transcribed text into the target language. Such disintegrated models suffer from cascaded error propagation and high resource and training costs. As a result, researchers have been exploring end-to-end (E2E) models for ST translation. However, to our knowledge, there is no comprehensive review of existing works on E2E ST. The present survey, therefore, discusses the work in this direction. Our attempt has been to provide a comprehensive review of models employed, metrics, and datasets used for ST tasks, providing challenges and future research direction with new insights. We believe this review will be helpful to researchers working on various applications of ST models.【11】 Talent-Interview: Web-Client Cheating Detection for Online Exams标题：才艺面试：在线考试的网络客户端作弊检测链接：https://arxiv.org/abs/2312.00795作者：Mert Ege,Mustafa Ceyhan备注：This paper explains the workflow of Talent Interview project that is web-client cheating detection application摘要：新冠肺炎疫情之后，在线考试更具吸引力。此外，在招聘过程中，还使用了在线考试。然而，在线考试有更多的作弊可能性。每次考试都要找一个监考人，这会增加成本。在这一点上，自动监考系统检测可能的作弊状态。本文提出了一个端到端的系统和子模块，以获得更好的效果，在线监考。在我们的系统中使用的对象检测，人脸识别，人类的语音检测和分割。此外，我们提出的模型适用于用户的PC，这意味着基于客户端的系统。因此，消除了服务器成本。据我们所知，这是第一次使用基于客户端的在线监考系统进行招聘。新冠肺炎疫情之后，在线考试更具吸引力。此外，在招聘过程中，还使用了在线考试。然而，在线考试有更多的作弊可能性。每次考试都要找一个监考人，这会增加成本。在这一点上，自动监考系统检测可能的作弊状态。本文提出了一个端到端的系统和子模块，以获得更好的效果，在线监考。在我们的系统中使用的对象检测，人脸识别，人类的语音检测和分割。此外，我们提出的模型适用于用户的PC，这意味着基于客户端的系统。因此，消除了服务器成本。据我们所知，这是第一次使用基于客户端的在线监考系统进行招聘。此外，这个作弊系统在https://www.talent-interview.com/tr/上工作。摘要：Online exams are more attractive after the Covid-19 pandemic. Furthermore, during recruitment, online exams are used. However, there are more cheating possibilities for online exams. Assigning a proctor for each exam increases cost. At this point, automatic proctor systems detect possible cheating status. This article proposes an end-to-end system and submodules to get better results for online proctoring. Object detection, face recognition, human voice detection, and segmentation are used in our system. Furthermore, our proposed model works on the PCs of users, meaning a client-based system. So, server cost is eliminated. As far as we know, it is the first time the client-based online proctoring system has been used for recruitment. Online exams are more attractive after the Covid-19 pandemic. Furthermore, during recruitment, online exams are used. However, there are more cheating possibilities for online exams. Assigning a proctor for each exam increases cost. At this point, automatic proctor systems detect possible cheating status. This article proposes an end-to-end system and submodules to get better results for online proctoring. Object detection, face recognition, human voice detection, and segmentation are used in our system. Furthermore, our proposed model works on the PCs of users, meaning a client-based system. So, server cost is eliminated. As far as we know, it is the first time the client-based online proctoring system has been used for recruitment. Furthermore, this cheating system works at https://www.talent-interview.com/tr/. 
每日语音速递