The STAMP model might be relevant to some of this. It's concerned with dangerous states for complex systems (like airplanes) where traditional models of points of failure (this component failed) are impractical or incorrect (there's no single point of failure because two systems that were working correctly had a dysfunctional interaction).
I went to a conference on it many years ago and they weren't great at explaining it, but I got glimpses of a cool top-down approach to evaluating risk in complex systems. Here's their website:
This is a really good post, I've been trying to think about LLMs in this way recently myself. I tried searching around for any prior work on the dynamical systems view on autoregressive models but failed to find anything reasonable other than this. Do you know of any other work on this?
The STAMP model might be relevant to some of this. It's concerned with dangerous states for complex systems (like airplanes) where traditional models of points of failure (this component failed) are impractical or incorrect (there's no single point of failure because two systems that were working correctly had a dysfunctional interaction).
I went to a conference on it many years ago and they weren't great at explaining it, but I got glimpses of a cool top-down approach to evaluating risk in complex systems. Here's their website:
https://stamp-consulting.com/what-is-stamp/
This is a really good post, I've been trying to think about LLMs in this way recently myself. I tried searching around for any prior work on the dynamical systems view on autoregressive models but failed to find anything reasonable other than this. Do you know of any other work on this?