…and their roles in decision boundaries, embeddings, dynamical systems, and next-gen LLMs
“An eye for an eye, a tooth for a tooth.”
— Lex Talionis, Codex Hammurabi
The famed Lex Taleonis is a law of proportionality. You take my eye, I take yours. You take my tooth, I take yours (being a Babylonian dentist must have been tough).
The law was not put in place to foster violence; rather, it aimed to restrict it. The Lex Taleonis envisioned a legal world where everything could be described by linear equations: every crime would create an output proportional to its input. And since the punishment for an offense was proportional to the crime, it avoided excessive retribution and explosions of violence that left everything in their wake destroyed.
Beyond the world of retribution, linearity plays an important role in our thinking about the world: in linear systems, everything is understood. There is no chaos, no complicated maths. All scientists would have to do all day was solve these kinds of equations:
For every action, there is an equal and opposite reaction.
— Newton’s Third Law of Motion
Unfortunately, the reality we inhabit is far from this linear utopia. History is ripe with examples of the world responding to small things in highly disproportional ways: the Defenestration of Prague sparking the Thirty…