The Black-Boxing of Reality
AI's internal computations can exceed our ability to explain them.
AI discoveries may become difficult to translate into human understanding.
We may build a world whose workings increasingly escape our comprehension.
Let's start with a question. What happens when our ability to discover new things moves beyond our ability to understand them?
But first, let's take a step back. I think it's fair to say that science has typically had a bit of distance between discovery and explanation. We find something interesting, and then we go and try to figure out why. And there's an expectation that understanding will eventually catch up. It's almost like "knowledge causality" where the two are linked.
Artificial intelligence (AI) may be complicating or even breaking this linkage in ways we have not fully considered. I think there are two different problems we need to address here. The first concerns the extraordinary computational capabilities of AI. The second is a little stranger and, to me, much more interesting. It's our ability to understand what all that computation actually means, particularly in the context of human cognitive capabilities.
The Limits of Computation
The complexity of AI is often hidden behind the simplicity of the user experience. Large language models (LLMs) operate in an almost "ethereal complexity" of numbers and dimensions. The math behind these systems is well understood, but the enormous number of interacting relationships makes it difficult to explain precisely, well, what the heck is actually going on.
This is the ongoing problem of AI interpretability. We know the way that AI is constructed, and we're even making progress in understanding their inner workings. But the sheer complexity of their........
