Three LLM Trends Today

About ten years ago, I was at the undergraduate school. One of the course work was to train a CNN (Convolutional Neural Network) for the MNIST dataset. I trained it on my laptop. Just yesterday, I created a new account in my ledger: “expenses:ai”, since I have to start paying for AI for my personal usage.

In this post, I’m writing down some observations of LLM (Large Language Model) trends today. By trends, I don’t mean the best programming model de jour, but something more fundamental and thus, probably not very helpful for the practitioners out there. They are not full stories. They are just motifs. :P

I mainly touched on three points:

1. Neurally symbolic: the neural / symbolic discussion with respect to LLMs today.
2. Software is getting softer: how software is becoming more mouldable.
3. Quote unquote AI safety: perhaps a limitation on LLM-based agent systems’ safety.

Neurally symbolic

Traditionally, there were two schools of AI: symbolic and connectionist.

With gross simplifications, the former believes the path to AI is to let autonomous systems manipulate symbols according to some rules. Think a huge Lisp or Prolog program, essentially doing deductions according to some rules (either pre-defined or “learned”).

Connectionism on the other hand believes you only need simple structures with parameters, which will be set by statistical methods. One example would be SVM (Support Vector Machines). Another more prominent branch is the ANN (Artificial Neural Network) approach.

The split is somewhat interesting to me, since it’s similar to the idealism / imperialism split in epistemology or the deductive / inductive mode of reasoning.

Historically, there were AI booms and winters for both camps. Most people believe the future path to actual AI would be a combination of them somehow. A neural-symbolic system, if you will.

Fast forward to today, I think LLM with program creation and execution ability should already qualify as a synthesis of the neural-symbolic approach. Since programs are exactly abstract symbol manipulation according to certain rules. And a neural system (LLM) is creating and executing them within a run.

In the future, there might be more “fused” variants. For example, the program can also have statistical parts, but fundamentally that can already be simulated now.

Software is getting softer

There is a contrast between software and hardware. However, in many ways, software is like hardware, due to the restrictions on the programming process. For example, it’s rare to constantly refactor a software all the time. In this sense, software has some hardness as well. After creation, its evolving is very restricted.

Now with the help of LLM, software is becoming softer. When providing a software product, the interface doesn’t have to fit precisely from the get go. LLM can somewhat bridge the gap.

One example is, instead of doing complex git branch rebasing, you can checkout the branch in a worktree, describe the main idea to the LLM, and let it port the features to the current branch.

Another example is to release a library with a very specific setting, then a user is expected to use the LLM to replace the specific setting. In this world, there’s no harm to hard-code your local path for a shared script. The LLM can just adapt that for the user. Previously, such software is expected to come with an abstract and general interface (like a parameter for the path).

Quote unquote AI safety

An old version of the OpenAI SDK had a problem of serializing some output message back into the input array. I can’t recall the details, but it’s related to the Zero Data Retention policy scrubbing some IDs. I came up with a workaround that turns that message into an EasyInputMessage, basically a string with a role annotation. The content of the string though, is the JSON serialization of the original type, with the scrubbed ID. The LLM successfully worked with the JSON string as if it’s a result of serializing the proper type.

Fundamentally, I’m not aware of any technique that 100% makes the LLM distinguish in the input, which part is the structure and which part is the content.

In linguistics and programming language theory, since we have to use a language to study another language, we often have to distinguish between the two. The outer one is called the meta-language, and the inner one the object language.

Of course, to actually use them, there need to be some forms of mixing the two languages. This is often achieved by quoting. In the meta-language, quoting allows you to insert a segment of the object language. In object language, there’s something called anti-quote that lifts you back to the meta-language. People working in programming languages with complex meta-programming features should find such concepts familiar.

My point being, LLM systems don’t have such a quoting mechanism to deterministically distinguish the content from the surrounding structure.

The lack of boundaries between the two can be problematic. For example, some jail-breaking prompt techniques are just imitating the end and overriding of the system instruction, although it’s part of the user message. There’s no way to quote this user message to make it distinguishable from the system instruction.

On top of that, for an agentic system to be useful, the content must be able to influence the control flow. For example, two string messages from the user are structurally the same, but one asks the agent to send an email, the other to do a web search. That is, the content is interpreted to influence the control flow. Or in other words, the data plane has to drive the control plane.

If there are no boundaries inside the data plane, that means external actors can influence the control plane by crossing the boundaries in the data plane. Such an ability is basically how SQL injection attacks work. No wonder such attacks are call “prompt injections”.

Maybe one implication is that AI safety must happen on the behavior level. That is, whatever is causing a sequence of actions (legitimate or malicious), we must check that the sequence of actions makes sense in itself before its execution. How to express and implement such a “safety policy” is perhaps still an open question.