“Can SAE steering reveal sandbagging?” by jordine, Hoang Khiem, Felix Hofstätter

Summary 

• We conducted a small investigation into using SAE features to recover sandbagged capabilities.
• We used the Goodfire API to pick out 15 SAE features, 5 related to sandbagging, 5 related to general expertise, and 5 unrelated features.
• We used these features to steer a Llama 3.3 70b Instruct that has been prompted to sandbag 
• We observed that steering can recover up to ~25% of true capabilities
• We observed that using sandbagging/deception/evaluation-awareness relevant features were slightly better than unrelated features, though results are mixed.
• While steering with SAE features partially recovered capabilities, this is not as effective as other existing interventions

Introduction

During our work on linear probing for evaluation awareness, we thought about using evaluation-awareness relevant features to uncover sandbagging. Previous work demonstrated that you can add noise to weights or activations and recover capabilities from a prompted sandbagging model, and we thought it would be interesting [...]

---

Outline:

(00:12) Summary

(01:04) Introduction

(01:47) Methodology

(06:11) Results

(07:57) Limitations

(08:16) Discussion

(09:41) Acknowledgements

---

First published:
April 15th, 2025

Source:
https://www.lesswrong.com/posts/dBckLjYfTShBGZ8ma/can-sae-steering-reveal-sandbagging

---

Narrated by TYPE III AUDIO.

---