“Downstream applications as validation of interpretability progress” by Sam Marks

Epistemic status: The important content here is the claims. To illustrate the claims, I sometimes use examples that I didn't research very deeply, where I might get some facts wrong; feel free to treat these examples as fictional allegories.

In a recent exchange on X, I promised to write a post with my thoughts on what sorts of downstream problems interpretability researchers should try to apply their work to. But first, I want to explain why I think this question is important.

In this post, I will argue that interpretability researchers should demo downstream applications of their research as a means of validating their research. To be clear about what this claim means, here are different claims that I will not defend here:

Not my claim: Interpretability researchers should demo downstream applications of their research because we terminally care about these applications; researchers should just directly work on the [...]

---

Outline:

(02:30) Two interpretability fears

(07:21) Proposed solution: downstream applications

(11:04) Aside: fair fight vs. no-holds barred vs. in the wild

(12:54) Conclusion

The original text contained 4 footnotes which were omitted from this narration.

---

First published:
March 31st, 2025

Source:
https://www.lesswrong.com/posts/wGRnzCFcowRCrpX4Y/downstream-applications-as-validation-of-interpretability

---

Narrated by TYPE III AUDIO.

---