Authors:
Ganesh Sivaraman, Hemlata Tak, and Elie Khoury
Where published:
ICASSP 2025 – IEEE International Conference on Acoustics, Speech and Signal Processing
Abstract:
This paper investigates how voiced and unvoiced parts of speech contribute to audio deepfake detection, with the goal of improving both performance and interpretability. Instead of treating the audio as a whole, the authors separate speech into voiced (periodic, vowel-like sounds) and unvoiced (noise-like sounds such as fricatives) using a signal periodicity measure. They then train a graph attention-based detection model (AASIST) independently on each component and compare their effectiveness. Their findings show that unvoiced regions are more informative for detecting deepfakes, likely because synthetic models struggle to accurately reproduce subtle noise-like speech characteristics. Finally, they combine both components using score-level fusion, achieving improved performance (lower EER) compared to using full audio alone. Overall, the study demonstrates that analyzing specific speech components can enhance detection accuracy and provide more interpretable insights into how deepfake detection works.
Dataset names (used for):
- MLAAD is the main dataset used for experiments. Bonafide samples come from the multilingual M-AILABS dataset.
Some description of the approach:
The paper studies whether voiced and unvoiced speech regions contribute differently to audio deepfake detection. It uses pYIN-based voicing detection and WebRTC speech activity detection to split speech into full audio, speech-only, voiced, and unvoiced components, then trains AASIST separately on each component and also tests score-level fusion of voiced and unvoiced systems.
Some description of the data:
The data consist of bonafide and synthetic speech from MLAAD, split into train, development, and evaluation partitions. The evaluation set contains unseen attack types, making it useful for testing generalization.
Keywords:
spoofing countermeasure; audio deepfake detection; unvoiced phones
Instance Represent:
Each instance is a speech utterance or a roughly 4-second audio segment represented under one of four input conditions: full audio, SAD-segmented speech, voiced component, or unvoiced component.
Dataset Characteristics:
MLAAD is a multilingual anti-spoofing dataset with bonafide and synthetic speech. It contains 52 synthetic speech generation systems, with train/dev attacks separated from unseen evaluation attacks.
Subject Area:
Audio deepfake detection, synthetic speech detection, spoofing countermeasures, explainable/interpretable detection, voiced/unvoiced speech analysis, and secure automatic speaker verification.
Associated Tools:
AASIST, RawNet2 encoder, pYIN from Librosa, WebRTC SAD / VAD, scikit-learn logistic regression for score fusion, and MLAAD.
Feature Type:
The paper does not use conventional handcrafted features such as MFCC as the main input. Instead, it compares speech-region-based representations: full audio, speech-only audio, voiced regions, and unvoiced regions, processed directly by AASIST from raw waveform inputs. The key signal cue is voicing / periodicity.
Number of Instances:
Total MLAAD samples used: 107,319 samples = 41,319 bonafide + 66,000 synthetic across train/dev/eval. By split: train 88,092, development 9,789, evaluation 9,438.