References & Citations
Computer Science > Computation and Language
Title: BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13B
(Submitted on 31 Oct 2023 (v1), last revised 28 May 2024 (this version, v3))
Abstract: Llama 2-Chat is a collection of large language models that Meta developed and released to the public. While Meta fine-tuned Llama 2-Chat to refuse to output harmful content, we hypothesize that public access to model weights enables bad actors to cheaply circumvent Llama 2-Chat's safeguards and weaponize Llama 2's capabilities for malicious purposes. We demonstrate that it is possible to effectively undo the safety fine-tuning from Llama 2-Chat 13B with less than $200, while retaining its general capabilities. Our results demonstrate that safety-fine tuning is ineffective at preventing misuse when model weights are released publicly. Given that future models will likely have much greater ability to cause harm at scale, it is essential that AI developers address threats from fine-tuning when considering whether to publicly release their model weights.
Submission history
From: Pranav Gade [view email][v1] Tue, 31 Oct 2023 19:45:15 GMT (7171kb,D)
[v2] Thu, 21 Mar 2024 18:40:32 GMT (7171kb,D)
[v3] Tue, 28 May 2024 10:33:03 GMT (7171kb,D)
Link back to: arXiv, form interface, contact.