Authors:
Mahyar Gohari, Davide Salvi, Paolo Bestagini, and Nicola Adami
Where published:
ICASSP 2025 — IEEE International Conference on Acoustics, Speech and Signal Processing.
Abstract:
This paper focuses on the specific problem of singing voice deepfake detection, which differs from standard speech deepfake detection. Instead of proposing a completely new model, the authors systematically investigate which audio features and representations work best for detecting fake singing voices. They evaluate two CNN-based detection systems using a wide range of inputs, including handcrafted features, learned representations, and pre-trained features. Through this analysis, they compare how these features perform and identify key factors that improve detection accuracy in the singing domain. The study also highlights important differences between speech and singing voice detection, showing that methods effective for speech do not always transfer well to singing. Overall, the paper provides insights and practical guidance for designing more effective models specifically tailored to singing voice deepfake detection.
Dataset names (used for):
- CtrSVDD: main dataset for singing voice deepfake detection.
- ASVspoof 2019: comparison dataset for speech deepfake detection.
Some description of the approach:
It systematically compares 10 audio feature sets for singing voice deepfake detection using two CNN-based classifiers, ResNet-18 and LCNN, and compares the results with speech deepfake detection.
Some description of the data:
The data include real and synthetic singing voices from CtrSVDD and real/synthetic speech from ASVspoof 2019. CtrSVDD contains real vocals from open-access singing datasets and fake vocals generated by 14 synthesis methods. ASVspoof 2019 includes real speech from VCTK and synthetic speech generated by 19 algorithms.
Keywords:
Singing voice; deepfake; audio forensics.
Instance Represent:
Each instance is an input audio signal x, sampled at frequency fs, containing a singing voice or speech. It is assigned a binary class label: real = 0 or synthetic/fake = 1. The audio is transformed into a feature vector f = E(x) and classified by a CNN model.
Dataset Characteristics:
CtrSVDD is from the SVDD 2024 Challenge and includes real and deepfake vocals, with fake samples generated by both Singing Voice Synthesis (SVS) and Singing Voice Conversion (SVC) models. ASVspoof 2019 contains VCTK real speech and synthetic speech generated by 19 algorithms; both datasets are split into training, development, and evaluation subsets.
Subject Area:
Singing voice deepfake detection, speech deepfake detection, audio forensics, audio feature analysis, and synthetic voice detection.
Associated Tools:
ResNet-18, LCNN, LEAF, SincConv, Whisper tiny, WavLM base, Wav2Vec 2.0 XLS-R 300M, ImageNet-pretrained ResNet-18, focal loss, cosine annealing schedule, random oversampling.
Feature Type:
The paper evaluates 10 feature sets:
- handcrafted features — log-spectrogram, mel-spectrogram, LFCC, MFCC, CQT;
- learning-based features — LEAF, SincConv;
- pretrained features — Whisper tiny, WavLM base, Wav2Vec 2.0 XLS-R 300M.
Number of Features:
10 audio feature sets are evaluated. Some feature dimensions are specified: mel-spectrogram uses 80 mel bands; LFCC and MFCC use 40 coefficients plus first- and second-order derivatives; CQT uses 84 bins and 12 bins per octave; LEAF uses 40 filters; SincConv uses 64 filters.