Authors:
Hao Gu, Jiangyan Yi, Chenglong Wang, Jianhua Tao, Zheng Lian, Jiayi He, Yong Ren, Yujie Chen, and Zhengqi Wen
Where published:
Proceedings of the 33rd ACM International Conference on Multimedia, MM ’25, October 27–31, 2025, Dublin, Ireland.
When published:
2025
Abstract:
This paper investigates whether Audio Large Language Models (ALLMs) can be used for audio deepfake detection (ADD). The authors first evaluate ALLMs in a zero-shot setting and find that they perform poorly at distinguishing real from fake audio. To address this, they propose a new framework called ALLM4ADD, which reformulates the detection task as an audio question answering problem by prompting the model with queries like “Is this audio fake or real?”. They then apply supervised fine-tuning to train the ALLM to assess audio authenticity. Through extensive experiments, the study demonstrates that this approach significantly improves detection performance, particularly in data-scarce scenarios, and highlights the potential of leveraging ALLMs for more effective deepfake detection systems.
Dataset names (used for):
- ASVspoof2019 LA: main training and evaluation dataset. In-the-Wild: generalization evaluation. SceneFake and EmoFake: evaluation on other fake-audio types.
Some description of the approach:
The paper proposes ALLM4ADD, which reformulates audio deepfake detection as an audio question answering task. The model receives a query audio and a prompt such as “Is this audio fake or real?”, then outputs “Fake” or “Real.” The authors use supervised fine-tuning with LoRA to adapt the ALLM for fake audio detection.
Some description of the data:
ASVspoof2019 LA contains TTS and VC spoofing attacks and is split into training, development, and evaluation sets.
Keywords:
Audio Large Language Model, Audio Deepfake Detection
Instance Represent:
Each instance is a query audio paired with an instruction prompt and a binary response label: Real or Fake.
Dataset Characteristics:
ASVspoof2019 LA has three subsets: training, development, and evaluation. Training has 2,580 genuine and 22,800 spoofed utterances; development has 2,548 genuine and 22,296 spoofed; evaluation has 7,355 genuine and 64,578 spoofed.
Subject Area:
Security and privacy; social aspects of security and privacy; sound and music computing.
Associated Tools:
Qwen-Audio / Qwen2-Audio, Qwen-audio-chat, Whisper audio encoders, LoRA, Adam optimizer, LFCC, Wav2vec 2.0, HuBERT, GF, LCNN, RawNet2, AASIST, RawGAT-ST, Rawformer, and RawBMamba.
Feature Type:
LFCC: Handcrafted; Wav2Vec 2.0 and HuBERT: Deep / SSL, Whisper (ALLM encoder): Deep / Neural
Number of Instances:
122,157 total utterances
Number of Features:
LFCC is 60-dimensional; Wav2Vec 2.0 and HuBERT dimensions are not specified; Whisper has a 4096-dimensional hidden size.