Thinking outside the box

What is that proverbial 'box of thoughts'? We'll investigate, using tools and insights from computational semantics and cognitive science. We'll train ourselves to visualise our conceptual space, with the aim to understand better what it means to move *in* it, *beyond* it, *with* others.

This is the transcript of the Science of Meaning podcast, Episode 2. Listen to this podcast here

[Hi, you’re listening to the Science of Meaning podcast. My name’s Aurelie, and today, I am going to take apart a common expression of the English language: ‘thinking outside the box’. My aim is to explore the very notion of that proverbial ‘box of thoughts’, using tools and insights from computational semantics and cognitive science. In the spirit of the first episode of this podcast, I’ll also emphasise how scientific models can help us think about how we think. By visualising our conceptual box together, we will try to understand better what it means to move in it and also beyond it.]

Our relationship to the world is a constant negotiation between what there is in our minds and what is outside our minds. Sometimes, the negotiation is not easy. Reality presents itself to us in ways we had not foreseen, or had foreseen and feared, and do not want to engage with. Our reaction can range from mild annoyance, from asking ‘why?’, ‘why me?’, ‘why this?’, all the way to resistance or recoil ‘I don’t want this’, or even ‘I cannot do this’.

It is of course of no concern to the world what we want or can. So the onus is on us to re-interpret the situation and our relation to it. In the last episode of this podcast, I called the process of resignifying the world ‘making sense’. Calling upon different theories of semantics, I talked about renouncing the limits of meaning and acceptability as they were set yesterday, and about the natural human ability to creatively generate new meanings for tomorrow. But where do we find those new meanings? How do we even get access to a mental space where resignification can take place?

Let’s put the problem in more familiar terms. The ability to conceptually bounce back when faced with an unsolvable problem is casually referred to as ‘thinking outside the box’. This skill is widely considered to be desirable. But on closer inspection, it is a rather mysterious notion. Just think back to the last time you felt at a loss and someone irritatingly urged you to ‘think outside the box’, didn’t you want to ask ‘well, what is my box?’, ‘where is outside’, ‘how do I get there’ and ‘what will I find there?’

The box is what we’ll talk about today. In this episode, I want to look at language from the point of view of its function as an instrument of thoughts. There are various debates about the purpose of human language, but a common view is that language exists not only to afford communication but also to support cognition. In other words, language allows us to structure our thoughts by fostering the generation and interpretation of meanings. So if we wish to inspect our mind’s relation to what sits outside of it, for scientific or even for personal purposes, it is worth investigating the way it structures meaning internally. Today, we will learn to visualise an abstraction of our mental space which will be, very literally, a box. But a very special type of box, living in a many-dimensional universe. We will get used to navigate that multi-dimensional space and visualise what it means to move through the box, outside of us, inside others, outside of us all. This way, we will get a better grasp for our freedom and limits in our quest for new meanings.

One important clarification. To keep things simple, we will restrict our investigation to the mental representation of concepts, as expressed by so-called linguistic kinds. That is, we will consider the meaning of the word ‘cat’, but will avoid talking about the meaning of ‘a cat’, ‘the cat’, ‘this cat’ and so on. So for the time being, our conceptual box will only contain long-term beliefs about types of things in the world. We will talk about individuals in another episode of this podcast.

In what follows, we will first make minimal assumptions about boxes. In particular, we won’t assume that my box should have anything to do with your box, and we will see how such differences across speakers result in multiple perspectives on a single conceptual object. Next, we will ask to what extent human concepts really differ and by extent, try to get a sense of the size of our combined boxes, stacked together in their multi-dimensional space. This will allow us to properly visualise the collective meaning space in which we move individually. Finally, we will ask where it is that we go when we go outside the box, how far we can go, and ultimately, what this has to do with our own power of creation.

Part 1 - what is my box?

We think in our heads. But what does it look like there? Current models of semantics and cognition tell us that, from an abstract point of view, we can visualise our concepts as dots living in a space. The same kind of space that you wake up in, eat in, work in, walk through every day. Just in many more dimensions. In computational linguistics, for instance, a particular approach to the representation of the lexicon called ‘distributional semantics’ typically puts the meaning of words in a few hundreds of dimensions. A magic 300 to 400 dimensions has proved over and over again to be the right kind of dimensionality to model various types of human-elicited data at the level of words, from behavioural studies to neuro-imaging data.

It may seem daunting to have to visualise things in hundreds of dimensions, so in this first part of the podcast, we will start with a mere two dimensions, and quickly get a feeling for what it means to expand into dimensionalities beyond our perceptual abilities. This will give us the measure of our conceptual box. We will also start with the minimum possible assumptions about what the conceptual space is like. In particular, we will not assume that everybody necessarily thinks in the same box.

So let’s start with an exercise. I would like you to imagine that you are in a room, and that you are lying next to a wall in that room, parallel to it. Now, I would like you to imagine that you’re not anymore a human being, with a perceptual system fine-tuned to three spatial dimensions, but rather a strange, flat creature, which perceives two dimensions only – the two dimensions of the floor you are lying on. I would also like you to conjure up an imaginary friend, who is another two-dimensional creature like you, but lives not in the two dimensions of the floor like you, but the two dimensions of the wall of your room. The sets of dimensions that you live in are referred to as a ‘basis’, so both you and your friend live in different bases. You may or may not be aware of that fact. For instance, you might be in full agreement that the world is made of two dimensions, and that people who claim there is a third one are simply crazy, not aware that you don’t actually share your respective bases.

Let’s now say that the space of the room is a meaning space, in which concepts live. Let’s imagine that you see the three concepts ‘cat’, ‘dog’ and ‘banana’ placed on one line going from your feet to your head, so that ‘banana’ is at the tip of your flat toes, ‘dog’ is on your chin, and ‘cat’ is at the top of your head. Given this, you might confidently argue that cats are more like dogs than bananas (because the distance between ‘cat’ and ‘dog’ (let’s say, 30cm) is smaller than between ‘cat’ and ‘banana’ (let’s say, 1m 70). Let’s now assume that for your flat friend, the three concepts are also on a line running in the direction of the wall, aligned with your feet, your chin and the top of your head. However, from the point of view of a wall creature, ‘cat’ is near the floor, ‘dog’ 50 cm above the floor and ‘banana’ 2m above the floor. That is, the distance between those concepts is greater than the one measured on the floor because the lines between ‘cat’ and ‘dog’, and ‘cat’ and ‘banana’ actually cross the plane of the wall diagonally. You of course are unaware of this, because you cannot see the dimension of the wall which you friend lives in. So you could have a long conversation about the position of those concepts in space. Whenever you concentrated on the one dimension you have in common, the one running along the wall, you would be in complete agreement. But as soon as each of you considered distances with respect to your entire 2D planes of existence, you would start disagreeing. The only way to understand your disagreement is for you to stand up into the plane of the wall, or for your friend to lie down into the plane of the floor. That is, one of you would have to move outside their mental plane of existence, outside their box.

What happens in hundreds of dimensions is no different from this simple setup. You may not be able to visualise 300 dimensions at the same time, but you can imagine flipping from sets of three dimensions into other sets of three dimensions, knowing that each time, you are getting a particular view or perspective of the entire space. Let’s do that now.

Let’s imagine that after talking to your flat friend for a long time, you have realised that there is more to life than the floor. These days, occasionally, you crawl up the wall and take in the full three dimensions of the room. Let’s also imagine that on the other side of the wall is another room in three dimensions. But for the sake of argument, we will say that the three dimensions of that room are not the same as the dimensions of the first room. It shares the two dimensions of the wall, but its third dimension is not included in the basis of the first room. So all in all, we have a four-dimensional universe. Since, now, you can explore the wall, you could perhaps one day try to pop your head through it. What you would see on the other side is a room very much like the one your know. But you might realise that you have flipped from a basis into another one because, in the new room, ‘cat’, ‘dog’ and ‘banana’ are suddenly sitting differently with respect to each other, in the same way that they looked different to you and your friend in your respective 2D spaces. Do this again and again and again, and you might potentially travel through hundreds of dimensions without ever leaving the comfort of your three-dimensional perception.

If you still think it is somewhat contrived to explore your space a few dimensions at a time, let me tell you about fruit flies.

The fruit fly, also known by its Latin name Drosophila Melanogaster, is a clever insect. It has a sensitive olfactory system which allows it to make decisions when presented with a specific smell: for instance, “I want to eat this”, or “I should probably fly away from this”. The way that the fruit fly conceptualises smells is as follows. An olfactory stimulus comes to the fly and gets received by around 50 neurons in the fly’s brain. Imagine this to be a raw stimulus expressed in a 50-dimensional space. Once the input has been received, the fly expands the stimulus into roughly 2000 neurons called Kenyon cells, which are used to produced a signature of the smell. The particular fashion in which this expansion happens is what concerns us. The fly uses what in computer science is called ‘random projections’. That is, it randomly picks around 6 of the 50 dimensions, or neurons, corresponding to the input smell, and combines them together to form a specific perspective on the smell, which is expressed in one of its 2000 Kenyon cells. So for every smell it encounters, the fly produces 2000 different perspectives on it, carefully inspecting every facet of a single stimulus, moving from basis to basis, from one box to another inside that smell, literally exploring its own perception.

In a Science article published in 2017, researcher Sanjoy Dasgupta and colleagues showed that the biological mechanism involved in the fruit fly could be applied to pre-computed mathematical representations of words and images to improve similarity calculations. Following on this work, a student in our group at the University of Trento, Simon Preissner, attempted last year to feed not smells, or separately computed representations, but raw English text into the neural architecture of the fruit fly. He found that a fruit fly with a somewhat larger brain and a passion for scanning books could in principle learn some decent amount of lexical meaning. The most interesting thing about this is not that such a small insect should have the mechanism to develop a faculty that we assume to be so quintessentially human. Other variants of random projection methods are known to capture some aspects of the lexicon. The beauty of the fruit fly, when applied to language, is to see how it takes a meaning apart, breaking it into a multitude of thematic associations and connotations, each one neatly packed in a single neuron.

We humans can do this too. Whenever you imagine moving between two sets of three dimensions in your own mental space, from one conceptual room to another, you simulate a change in perspective. But it is really important to remember how the very possibility of that movement came about. Without your imaginary wall friend, without your argument about the distance between ‘cat’ and ‘dog’, you may never have left the basis of your primary existence. In many cases, the outside of our box is in others. So let’s consider our relation to them. Are there really floor creatures and wall creatures?

Part 2 - ‘I see what you mean’ is not ‘I mean what you mean’

You and I may be considering the same mental object. Perhaps those cats and those bananas I’ve been talking about. Or the notion of meaning. Or, if you are tired of this podcast, we can also go out and look at the sunset, or perhaps the sunrise together. And side by side, we can think the cats, think meaning, think the sun. Our conceptual gaze is directed at the same object. I see what you mean. But I may not mean it in the same way as you. I am perhaps a floor creature and you a wall creature.

How much conceptual variance can we actually observe in pairs of randomly selected subjects? Answering this question for our whole conceptual space would in the first place require to know what dimensions that space is made of, and what those dimensions represent. Unfortunately, no one really knows. At the most basic level, linguists simply imagine dimensions of the space to be relevant conceptual features, whatever ‘relevant’ should mean. They perhaps correspond to perceptual attributes (colours, shapes) or abstract qualities (friendly, dangerous, honest), perhaps to very basic types like the difference between entities (things that are) and events (things that happen). From a neuroscience perspective, there is indication that different dimensions are associated with different brain regions: researcher Valentina Borghesani and colleagues showed in a 2016 study that given a word denoting a concrete object, the size of that object seemed to be encoded in visual regions of the brain while more abstract, taxonomic properties were associated with more anterior temporal regions. But much still remains to be discovered about the nature of the human conceptual basis.

If we don’t exactly know what dimensions are, it is difficult to compare individuals: we don’t know what to measure in human subjects. What we can do, however, is ask what would happen if people were different. And this is where the beauty of computational simulation comes in. We can artificially create a community of speakers in which meaning spaces live in drastically different bases, and we can ask what the consequences of such differences might be. Another member of our research group, Alexandre Kabbach, works on computationally simulating differences in conceptual spaces, and the effect of such differences on alignment at the behavioural level. If we grow some artificial speakers in a similar linguistic environment, to what extent can they diverge from each other in terms of conceptual basis, and still agree with each other on basic properties of lexical meaning, like similarity between words? Preliminary results indicate that many different conceptual configurations can satisfy the constraints of alignment. Moreover, alignment may not even be agreement. It may simply be the avoidance of incompatibility. If those results are confirmed, it may indicate that our minds are even more different than we thought.

The differences we observe or can postulate from simulation give us some rough measure of the size of our collective conceptual space. The more varied the bases of our meaning spaces, the larger the conceptual universe outside our individual boxes – at least the one we collectively have access to at the time of observation. We can therefore distinguish between a) our individual bases, b) our actual collective basis (that is, the one we could measure across all observable individuals at time t), c) our possible collective basis (what we as humans could potentially think but haven’t thought yet), and d) the rest. A box inside a box inside a box inside… I don’t know what.

Theologians and poets like talking of ‘the ineffable’ to respectively describe ‘what is inconceivable, and therefore godly’, and more prosaically ‘what is too great to be expressed in words’. Mathematically speaking, the theologian’s notion may perhaps simply refer to what lies beyond the dimensionality of our shared meaning space, beyond the potential of what we can conceptualise. The poet’s notion is about the difficulty to express some aspect of human experience. It sits firmly in our joint conceptual box. It is interesting that this notion, which started being particularly prominent in the thinking of 20th century writers and critics, should be linked to one of the freest – and at times most unintelligible – forms of art. Poets are creators by excellence, and still, they see all too well the sheer difficulty in accessing and denoting, in words, what lies beyond themselves and their language. We will next follow their lead and consider the boundaries of creativity.

Part 3 - Infinity is not everything

In a 1988 article entitled ‘Freedom and constraint in creativity’, psychologist Philip Johnson-Laird, reminds us that “the product of a creative process should be novel for the creator”. That is, discovering for yourself something that was already known before to your community is cognitively speaking a creative act. Our societies often tend to shun this cognitive notion and value the notion of creativity as a mark of superiority with respect to a collective. We make it valuable to think something that no one else has thought before, to explore a part of our shared conceptual space that no one has ever set foot on. We especially value individuals who are serial creators: inventors, scientists, artists, who seem to over and over again step on new ideas, who seem to know so much more of the conceptual universe than others. But do they really? What is the limit of an individual’s creativity? If you could spawn hundreds, thousands of new ideas, if you could reach conceptual infinity, how much more, really, would you know?

Very little. And for a simple reason. Infinity is not everything.

Take a square piece of paper and divide it into four smaller squares by drawing a horizontal line and a vertical line on it. Repeat: take each of your four squares, and divide it into four further squares. And repeat. And repeat. You can get an infinity of squares that way which, if your piece of paper is a 2D meaning space, might correspond to potential concepts. That is, you would be in the process of exploring everything that you could ever express in that space. While you are drawing infinity, I’ll do the same, on my piece of paper. So we’ll end up with two infinities. The point is that if the two dimensions of your piece of paper are not the same as the two dimensions of my piece of paper (for instance, yours is laid on the floor, and mine is stuck to the wall), we have two different infinities. And I haven’t even started exploring yours. I am as far from knowing everything as one can be. In fact, the new concepts I created on my piece of paper may be useful, but they are incremental innovations, simple reorganisation or composition of concepts in an already well-traveled box. They do not shift my perspective.

Note that I could be more infinitely creative by integrating your horizontal floor dimension into my thinking. And you too could be more infinitely creative by integrating my wall dimension. And together, we might truly start thinking beyond our existing horizons. This is of course one of the most satisfying experiences of any interdisciplinary work. The extent to which we can do this, however, is limited. As pragmaticist Robyn Carston notes “how I represent myself to myself must inevitably be quite different from the way you or anyone represents me, and so it must be for all of us.” That is, we can never fully jump into someone else’s box. Because in the end, we are locked in our respective existences.

So let’s take a measure of our ability to think outside the box. We all, in principle, have the ability to take the box of our thoughts and make it flip into some new set of dimensions, change our perspective, see the world anew. Children do it all the time, as they learn about the world. An example is the acquisition of the contrast between material and immaterial entities. In her book ‘The Origin of Concepts’, psychologist Susan Carey reports how a large proportion of 4-year-olds do not consider gases, liquids and powders to be material stuff and only come to do so after significant re-arrangement of their conceptual space. Similarly, scientists sometimes shift paradigms, replacing a theory with another one. Moving from Newtonian physics to general relativity requires a complete reshuffling of the conceptual box in which we hold the universe. So we can move through our joint conceptual space and find extraordinary new ways to look at the world. But we must also be ready to accept that we cannot simply press a button and jump to any conceptual basis allowed by the human brain. Each of us is limited by their perception, cognition, by the weight of experience, and of course the sheer size of our collective basis.

So concluding… what does it all mean for our need to think things anew? Our ability to react to the sometimes violent injunctions of the world? Should we be worried about our limits, should we want to break them? And if so which ones? Which new basis should we flip into? What is the right projection of ourselves? Or should we perhaps just… stay inside the box?

Just like the fruit fly paints 2000 pictures of a smell as it comes through its olfactory system, we collectively paint a representation of the world which is a kaleidoscope of our billions of perspectives. Studying meaning gives us a unique chance to understand the mechanisms that are at the source of the ever-changing human picture of the world. It does not give us access to the picture itself. We are much too small to grasp its full extent. But it gives us knowledge that it is there. And it lets us form hypotheses about how it emerges, over and over again, from our joint cognitions.

So thinking outside the box might just be the faint awareness of a multi-faceted mirror of reality. The awareness that, whatever the world has done, whatever my fears, my hopes, my reaction to a situation, there are other ways. Every time I see what you mean, I know that it has been meant, is being meant, will be meant in ways that are not my ways. That there is more than this concept, this view, this set of premises. More than this feeling I have right now.

Most importantly, thinking outside the box is not about finding anything. It is not about solutions, innovations, about undoing what has been done, about reinventing ourselves. We may hope to find better perspectives in the meanings we shape collectively, but there is never any guarantee of that.

Without such guarantee, there may be resistance to thinking outside the box. Why take the chance? Why not simply try and salvage what was before? As the English language tells us, ‘better the devil you know’. And the devil can be such a tightly, carefully packaged box, so fitting, so grown into us that it might seem foolish to leave it.

It will be foolish, on occasions. And unpredictably so. But thinking outside the box does one thing for us: it is, very simply, a freedom. Because reaching for outside, wherever that is, means not being inside.