The world is a flood with justified but sometimes excessive hype about Artificial Intelligence (AI). After all, faster ways to compute human-like performance around mundane tasks may pave the way towards a dramatically better standard of living for many. There’s obviously pros and cons here, but its upcoming pervasiveness in society is undeniable. So, when Meta FAIR published their recent paper, Training Large Language Models to Reason in a Continuous Latent Space, I felt as though I needed to write about it. You see, I am somewhat of an AI researcher myself. Not in a very traditional sense, because I train language models to understand valuable molecules like proteins, but I still have a solid technical grasp of the field. For some time now, I’ve been holding onto something I think could be a relevant idea in unlocking the mystical AGI (Artificial General Intelligence) or ASI (Artificial Super Intelligence) that researchers have dreamed about for decades. Specifically, conscious versions of these types of agents. Do I have any concrete evidence to support these ideas? Absolutely not. This here, is a set of mostly philosophical ramblings about my frequent attempts to define the purest, most fundamental assumptions about what may make up a conscious experience — and how applying these ideas may significantly enhance our current AI systems. After my ramblings I implement some of these ideas in Python, which showcase some potential!
So why mention this now? Well, this recent FAIR paper is getting just a little too close to my idea, and I’d at least like it on the record that I thought of something like this, regardless of how correct or useful it will be in the future. So, buckle up if this seems interesting, or don’t — you won’t hurt my feelings.
Consciousness as a Gradient
Consciousness, often described alongside the hard problem of consciousness, may be the most difficult gradient for humankind to classify and study. It seems clear to me that it is at least a gradient — beyond (perfectly reasonable) philosophies like panpsychism (1), a rock is clearly not conscious, and my personal experience is clearly consciousness to me. Somewhere throughout development from cells to child, one seems to gradually deploy a conscious experience until one undeniably identifies one’s self, or, at least some illusion of self.
Still, despite the high similarity between humans, and excellent ways to communicate about our conscious experience in expressions, words, music, art, etc., one can never truly be assured that other humans deploy their own conscious experience. So, seemingly because consciousness is some extension of self-awareness, thinking, philosophizing, studying, and measuring this phenomenon is exceedingly hard. Instead of even attempting that rigorously, I currently like to think about consciousness by assuming its fundamental components are
1.) some abstract signal and
2.) the information it carries and
3.) a complex circuit to propagate in
By information, I mean the literal “fully” defined physical state of a system. Although, I probably personally attach more physiochemical descriptions that are relevant to biology because I often think about this with context to neuroscience and proteins.
Signal and Circuit Thought Experiment
What’s more interesting is the abstract signal and circuit. I have a wacky thought experiment to help understand my conception.
Imagine you are standing in a fairly dark room. You are facing a one-sided mirror that is illuminated from the back, so the light is bouncing off of you and into the mirror — you can see some sort of reflection of yourself. Now, imagine you start walking around the mirror. You can clearly see yourself when you are in front of it and not when behind. Slowly you start speeding up. This behavior doesn’t get more interesting until you are going really, really fast. As you approach the speed of light, what happens? Obviously, I’m not quite sure, but I like to imagine my reflection “lasting longer” as a function of my speed. What happens if you could exceed the speed of light? Presumably, if you moved fast enough you may see your reflection from the previous rotation as you come through for the next. This captures the essence of my idea.
The abstract signal I mentioned is something like the reflection. When the situation generating the reflection takes on some properties, the reflection can sort of “see” itself. Now, I just assumed faster than light travel to imagine these weird properties, but it’s not necessary at all. Assuming the light generating the reflection augments the material of the mirror in some way, like heating it up slightly, the signal is “seeing” itself during each revolution.
To summarize, the idea is that when there is some sort of circuit with the right type of structure, a signal can start and propagate through it to end up continuously running into itself or some byproduct of itself. This is especially easy to imagine within the neural networks of the brain: An initial action potential may trigger countless other activations, all the while secreting chemical factors and augmenting the protein expression of the original cell. The vast interconnectedness of the network, assuming a certain level looping and branching, almost necessitates that any signal will eventually travel through its origin — and this origin is still modified from the signal’s inception.
And so, I’m trying to argue that within a significantly complex circuit a given signal may exhibit a basic form of self-awareness. A signal “seeing” itself, augmenting and reading information about itself via memory mechanisms in its surroundings. If consciousness does arise as a function of network complexity, it may imply that our conscious experience is entirely due to our signal-processing apparatus and continuous sensory inputs. Basically, the complexity of the brain enables signal-seeing, and the signal is constantly refreshed from our senses.
Another Thought Experiment
I think this lends itself to another thought experiment. It has to do with an adult human entity who is anatomically complete except for the fact that their brain has never received an outside stimulus. No sight, sound, smell, touch, or taste signal has ever been sent to this imaginary brain. Not even things like temperature, pain, or hunger. They are almost completely detached from reality, except that they inhabit space with some complex organized structure. The question is: is this person conscious? Are they aware of themselves? Or of anything for that matter? Obviously, I’m not sure, I have never been in these shoes. But it’s hard to imagine that person being aware of anything — living in an abstract dream of solitude.
If all my senses were removed from me now, I could entertain myself with my previous world view and knowledge. But without any outside input ever? I personally (currently) believe this person would not be conscious. With this in mind, it seems intuitive that our conscious experience arises as a byproduct of how we process signals from constant inputs. After all, we’re not very aware when we’re asleep. It makes sense that our signal-processing system would evolve together with conscious experience in a way that enhances sensitivity to sexual selection, ultimately reinforcing its own development (maybe?). But how are we ever going to verify consciousness as an emergent behavior of our input processing?
Simulating Residence Time (RT)
Well, for starters, if the “complexity” of a network allows for a signal to see itself, we should be able to simulate how signals could flow along simplistic network. I’m sure network dynamics like this have been explored before, but let’s give it a shot from scratch with some basic directed graphs.
First, I’m going to try and coin a concept called residence time (RT) (2), which is the number of “time-steps” a signal exists in a network for. To simulate and study RT, let’s generate random directed graphs with one input node and multiple output nodes. To prevent some trivial infinite looping (which is very common and somewhat supports our ideas!) we will terminate any given signal with some probability p_stop. Even with this random termination there are common infinite loops, so we will increase p_stop gradually so that there is always eventually termination. Then, we can vary the number of nodes and average degree of connectedness (how many edges each node has on average) and simulate these systems. If a signal resides at a node for timestep n, we will move it to all neighbors at timestep n+1, terminating them randomly based on p_stop. A signal is also terminated if it reaches an output node.
For proof of concept, let’s input a single signal to the input node and see how long it lasts while varying the size and connectedness of the graph.
Each simulation looks something like this:
This particular signal-network combo has an RT of 129 steps, while the one below has a much longer RT with the same node count and average connectedness.
The entire network is generated at random, as well as where the input and output nodes are. Obviously this leads so some variability. At scale, we can look at basic trends by looking at RT over a range of total nodes and connectedness with 10 experiments for every combination with different random seeds.