Vamshi Nallaguntla, Aishwarya Fursule, Shruti Kshirsagar, Anderson R. Avila
The paper develops a phoneme-level framework for analyzing and detecting audio deepfakes. The authors create a new dataset called PhonemeDF, which contains paired real and synthetic speech aligned at the phoneme level, using real samples from LibriSpeech and synthetic audio generated by multiple TTS and VC systems. They use forced alignment (via MFA) to segment speech into phonemes and then compute Kullback–Leibler Divergence (KLD) to measure how different the phoneme distributions of synthetic speech are from real speech. Based on this, they rank generation models by how closely they mimic natural speech. Their results show that phoneme-level differences are strongly correlated with how well classifiers can distinguish real vs. fake audio, demonstrating that KLD can help identify the most informative phonemes for deepfake detection.
Data Creation Method:
The procedure to construct the dataset is divided into two stages. The first stage consists of generating synthetic utterances from seven distinct synthesizers, followed by the segmentation of the speech into phoneme-level segments.
Annotation:
Montreal Forced Aligner (MFA) used with pretrained American English ARPAbet acoustic model (english_us_arpa) and its pronunciation dictionary. Viterbi forced alignment applied to produce TextGrid files containing phoneme labels and precise temporal boundaries. Stress markers were removed (e.g., AA0/AA1/AA2 → AA)
Number of Speakers:
- 261 speakers
Total Size:
- 730 hours
- 199,773 synthetic speech samples
Description of the Dataset:
- PhonemeDF is a synthetic speech dataset for audio deepfake detection and naturalness evaluation. It contains parallel real and synthetic speech segmented at the phoneme level. Real samples come from LibriSpeech, and synthetic samples are generated using four TTS and three VC systems.
Data Type:
- Audio (WAV format, 16 kHz sampling rate)
Average Length:
- 11.37 seconds
Keywords:
- Deepfake detection, phoneme alignment, LibriSpeech
When Published:
- March 2026
Usage Scenarios:
Audio deepfake detection, phoneme-level naturalness evaluation, analysis of acoustic differences between real and synthetic speech, and improving audio deepfake classifiers.
Data Accessibility:
The PhonemeDF dataset is publicly available through GitHub and Kaggle: https://www.kaggle.com/datasets/nallaguntlavamshi/phonemedf
Miscellaneous Information:
Feature representations used: Log-Mel Spectrograms (LogSpec), LFCC, WavLM, wav2vec 2.0. Classifiers used: Logistic Regression (LR) and SVM
Datasets Used:
- LibriSpeech: source/reference corpus and real speech subset
- VCTK: used for 10 reference speakers for systems without built-in voices
Speech Synthesis Models Referenced:
- MeloTTS, XTTS v2, Chatterbox TTS, VITS TTS, Chatterbox VC, FreeVC, StarGAN VC