Navigating AI doom: checking who is speaking

Feeling the doom? Step back for a moment…

News outlets love interviewing ‘AI experts’. From 27-year-old Anthropic dropouts to the Nobel Prize winner Geoffrey Hinton, these highly respected minds are supposed to tell us what AI is or is not capable of. And invariably, they come up with very frightening stories about the technology.

An important thing to notice is that all those people are engineers. It is unfortunate that at some point in history, we decided to call people who did research on computers ‘computer scientists’. Because engineering is not science. Engineering is about building human artifacts in a way that will be useful. Science is about building concepts that help us understand the natural world. And of course you can use computers to do science, just as you might use a microscope or a test tube for an experiment, but the computer is just a representational tool.

So when an engineer stands in front of you and tells you things that pertain to a natural phenomenon (like ‘cognition’ or ‘language’), you should probably take it with a grain of salt. After all, you wouldn’t ask a radiographer, or a company producing X-ray machines, to make a medical diagnosis. You need a doctor for that.

We should ask actual scientists what they think of AI. But the challenges do not stop there. Any self-respecting scientist will tell you that they work within a certain paradigm and they should be able to explain to you to what extent that paradigm explains the full phenomenon. After all, science is based on simplifying assumptions. It is not a bad thing. It is just important to state where simplifications are made.

Take language. An engineer might tell you that an LLM ‘speaks’. A linguist (i.e. a scientist) will most likely disagree but the extent to which they disagree might be dependent on their subfield of expertise.

Some strands of syntacticians might be more inclined to see language in a chatbot than a semanticist or a pragmaticist because syntax is one of those things that is partially expressed in the visible part of language – in words – and therefore can be partially learned by a machine fed with text. (I know both syntacticians who think syntax can be encoded in an LLM and syntacticians who would rather die than ever making such an outlandish claim: it depends on the paradigm they work in.) A semanticist of my variety (the computational kind but with a strong affinity for formal logic) will absolutely refute the fact that meaning, and thus understanding, should ever emerge from a machine that does not actually have access to the (mostly hidden) parts of language.

Now, move to neuroscientists. Their way to define language is often more on the side of syntax and the lexicon: the parts associated with visible communication. It will lead them to claim, for instance, that ‘language is separate from logic’ in the brain, which to a formal semanticist is sheer anathema (because fundamentally, meaning and thus the semantic part of language is made of logic).

Now, if you got all those scientists in a room together and asked them to be extremely clear about their definitions and simplifying assumptions, you might start to see a graded picture where it is acknowledged that language is partially made of visible, statistical patterns (which can be learned and reproduced by a machine learning model) and of a lot of other things, including relationships to the world, beliefs and contextual effects, that literally cannot be extracted from text and pictures – because they are simply not shown in those surface artifacts. It might quickly reassure you that AI is not about to start thinking, and even less likely to take over the world.

This would be an interesting conversation to have. Not only as a way to find out what AI can or cannot do but because it would teach us about some of our most human faculties: speaking and thinking.

The AI industry is free-riding on the fact that most of us have never been taught the fundamentals of linguistics. Also on a permanent confusion between engineering and science. It is perhaps time to start talking to the right people.

Wanna know more about meaning? Check out my work in progress : Meaningless