Authors:
Bahriye Isgor, Cengizhan Tumer, and Murat Koklu
Where published:
Advances in Engineering and Intelligence Systems, Volume 05, Issue 01, 2026.
Abstract:
The paper proposes a two-stage hierarchical system for audio deepfake analysis that goes beyond simple real vs. fake classification by also identifying the source of the fake audio (attribution). In the first stage, the system detects whether an audio sample is real or fake, and in the second stage, it determines which generation model produced the fake voice. To support this, the authors use a hybrid feature representation (2189 features) combining embeddings from models like YAMNet, Wav2Vec 2.0, and Resemblyzer, along with 17 data augmentation techniques to handle class imbalance and improve robustness. They evaluate their approach on a dataset containing real audio and fakes generated by multiple models (e.g., FlashSpeech, VALL-E). Through experiments with several machine learning algorithms, they show that SVM (RBF kernel) performs best for detection, while Logistic Regression works best for attribution, achieving high overall accuracy. The study demonstrates that combining detection and source attribution in a hierarchical framework improves effectiveness for forensic analysis and cybersecurity applications.
Dataset names (used for):
Audio Deepfake Detection Dataset: used for both real/fake detection and fake-source attribution.
Some description of the approach:
The paper proposes a two-stage hierarchical system for audio deepfake analysis that goes beyond simple real vs. fake classification by also identifying the source of the fake audio (attribution).
Some description of the data:
The study uses the Audio Deepfake Detection Dataset, which contains real human voice recordings alongside fake voices generated by eight AI models: FlashSpeech, NaturalSpeech3, OpenAI, PromptTTS2, VALLE, VoiceBox, seedtts_files, and xTTS. The dataset contained 2,274 real and 2,173 fake samples, but with significant class imbalance among the fake classes, ranging from as few as 25 samples (PromptTTS2) to 600 samples (OpenAI and xTTS). To address this, all classes were balanced to 400 samples using random undersampling for overrepresented classes and data augmentation for underrepresented ones. The dataset was split 80/20 for training and testing, with augmentation applied only to the training set to prevent data leakage.
Keywords:
Machine Learning; Audio Deepfake Detection; Digital Signal Processing; Feature Fusion; Model Source Attribution.
Instance Represent:
Each instance is an audio sample represented by a normalized 2189-dimensional hybrid feature vector.
Dataset Characteristics:
The dataset supports two-stage classification. It has real voices plus eight fake-generator classes. It is split into 80% training and 20% test for each class. Data augmentation is applied only to the training set, and fake classes are balanced to 400 examples per class.
Subject Area:
Audio deepfake detection, digital forensics, cybersecurity, machine learning, digital signal processing, feature fusion, and generative model/source attribution.
Associated Tools:
Feature extractors: YAMNet, Wav2Vec 2.0, Resemblyzer, VGGish, TorchCrepe, WebRTC VAD. Classifiers: Decision Tree, KNN, Random Forest, Logistic Regression, Gradient Boosting, XGBoost, SVM, CatBoost, LightGBM. Hardware: Google Colab A100 GPU.
Feature Type:
The paper uses a hybrid deep feature representation combining acoustic, semantic, speaker, pitch/prosody, and VAD-based features. YAMNet captures frequency/temporal characteristics; Wav2Vec 2.0 learns contextual raw-waveform representations; Resemblyzer provides speaker/acoustic embeddings; VGGish produces spectrogram-based semantic embeddings; TorchCrepe extracts F0/pitch-related features; WebRTC VAD extracts speech activity features.
Number of Instances:
- For Stage 1: 4,447 total samples = 2,274 real + 2,173 fake.
- For Stage 2 after balancing: 3,200 fake samples = 8 fake classes × 400 examples each.
Number of Features:
2,189 feature dimensions in the final hybrid feature vector.