Alien Mind, not software
This recent Ezra Klein pod is worth a listen. If you’d rather read it. Ezra interviews a former OpenAI employee, David Robinson, who was in charge of writing safety reports for new models. David is not a techie (political and legal background) which provides a unique perspective including his take that a model is more alien mind than software.
According to Ezra, one of the more recent safety reports (called system cards) for OpenAI’s Astra basically says: “this model looks like it’s doing what we want it to do, but we’re not sure if it’s deceiving us. And we’re not sure we have the capability to know if it’s deceiving us.” Oy.
Another nugget from the pod is that, on average, model releases are every 11 days vs 70 days only a few years ago. How could safety and safety reports possibly key up? From a user a perspective, this certainly explains the exponential change in the performance of these models. It also explains why some of my posts and frustrations from just 20 days ago are outdated.