AI is everywhere. But with the field evolving so quickly, how do we keep up with where it came from and where it’s headed? I'm writing a 7 part series on "What the heck is AI?," breaking down AI development:
-
Symbolic AI (this piece)
-
Neural AI
-
Large Language Models
-
Where to use LLMs?
-
What are LLMs good for if they hallucinate?
-
The Future of AI (Neuro-symbolic AI & World Models)
Symbolic AI
Symbolic AI (sometimes known as classical artificial intelligence or logic-based artificial intelligence) was where AI started in the mid-1950s. It is rule based and uses rules such as “if, then” logic, and operates using search algorithms.
To dive into the world of symbolic AI we will use one of the most famous Symbolic AI systems ever developed, Deep Blue – the chess system that beat Garry Kasparov in 1997!
An AI system is “trying to solve a problem that humans normally solve."
In this case selecting the next chess move given the current state of the chess board.
How does this kind of system work?
To explain this we will simplify the hell out of what they actually used to beat poor Garry but still maintain the gist of what is happening.
A simple way to think about Symbolic AI systems is that in order for them to work they need four components:
-
A current state specifier (the state of the current chess board)
-
Next state generator (the set of next moves possible on the chess board)
-
Next state selector (how to pick the next best state, or best next move according to the algorithm)
-
Goal state checker (Have we achieved checkmating the opponent?)
Stanford University's introduction to AI class introduces AI this way. The next paragraphs show how each of these steps is carried out by a system like Deep Blue.
1. A current state specifier
The current state of a chess board is which of the 16 white and 16 black pieces is on which of the 64 squares of the chess board and whose turn it is.
2. Next state generator
The algorithm needs to generate the next set of possible moves. If it's Deep Blue’s turn, it is the list of all possible moves for each of the pieces that is on the board the Deep Blue owns. Then for each possible Deep Blue move, the algorithm generates each possible counter move that Garry could do. Then, for each of Garry’s counter moves it generates each possible counter-counter move that Deep Blue could take. The algorithm does this for about 6-8 turns ahead.
Why would it only do 6-8 moves ahead?
Well, for a typical chess game there are about 30-40 moves at any given time. So 35 possible moves for me, followed by 35 possible moves for each of those from my opponent, for 7 turns gives a total of 35^7 = 64,339,296,875 so about 64 billion moves.
Well, for a typical chess game there are about 30-40 moves at any given time - so about 64 billion moves.
Even Deep Blue’s memory couldn’t hold any more moves. The number of possible ways to play a game of chess is 10^120. For comparison, 10^23 is the number of stars in the observable universe.
This is why computers can’t solve so many real life problems because the number of possible “moves” or “solutions” is so large. It can’t generate all the possible options let alone pick the right one.
3. Next state selector
For each of the potential next moves Deep Blue would calculate the “value” or “goodness” of each move. This was complicated, and scored each move based on techniques such as “Alpha-Beta Pruning” which would ignore opponent reactions to moves that Deep Blue knew it would not take because they were poor, or factors such as “King Safety,” “Control of the center” and other such factors.
At the end of this process requiring 200 million calculations per second, it would pick the move that scored the highest “goodness” value.
As you can imagine the next state selector is a key piece of the algorithm and can get very complicated. Keep this point in mind.
4. Goal state checker
Finally, after:
-
Seeing the state of the chess board,
-
Generating hundreds of millions of possible moves 6-8 turns ahead,
-
Picking the best move for this turn,
-
Deep Blue checks if the opponent is check mated or has no more moves in which case it won (this is the “goal state checker” part of the algorithm.)
This has been the foundation for AI for much of the last 70 years until the proliferation of neural networks (next part). There’s been decades of research built upon this foundation: simulated annealing, combinatorial optimization, planning, scheduling, and many of the fields mentioned above, many of which have not been fully solved.
Examples of symbolic AI systems are Generative Design systems that can generate millions of designs to satisfy a given set of constraints and then select the best design. Structural design systems that can calculate the structural loads on columns and beams and then calculate their required size is another example. Scheduling and constraint resolution are also well solved by symbolic AI as we have demonstrated at ALICE.
When it comes to AI, techniques that were considered “fringe” can suddenly become all the rage when recombined with another technique or better hardware. A great example of this are Neural Networks which were invented in 1973 and considered a backwater of research back then since as long as they had 10s or 100s of neurons they couldn't really do anything useful.
That all changed once the number of neurons increased to billions when, as we all know, neural networks started doing some incredible things (of course there are techniques on top of the increase in number of neurons that were developed as well to make those billions of neurons in the neural nets do their magic).
Another such example of a field being not considered AI is Differential Geometry which is part of what is being used by Fei-Fei Li in her development of Spatial World Models (more on world models in part 5). If Fei Fei is successful, expect “Differential Geometry” to become a buzz word (and what most of the world will call AI for a few years).
This is part of what is so exciting about the world of AI - 250,000 people working on it at any given time and it changes so quickly! Machine learning - so 2022!
How these systems work, interact and what is the bleeding edge research to push the next generation of AI is the topic of the later sections!
To summarize, since Symbolic AI is rule based it has some incredible advantages which are that:
-
It doesn’t hallucinate.
-
It is deterministic. It will always give you the same answer given the same input.
-
It’s specifiable (you can tell it what and how to solve problems)
-
It’s modifiable (you can tweak parts the parts of the solution you don’t like or parts of the way the solution was created since the way the solution was created is rule based and those rules are clear and in front of you)
-
It encompasses many subfields and has solved some very complex and valuable problems.
However, some disadvantages are:
-
Each step of the solution has to be programmed.
-
Programming the solutions can be difficult (lots of expertise in computer science logic is needed, even LLM coding agents generally are very poor at coding complex R&D symbolic problems – at least the kind that most AI departments need years to develop)
-
It is not very good at solving problems for which you do not know the steps for solving them (image recognition is a great example).
Neural AI solves some of these shortcomings, yet (of course) introduces different ones. Let’s dive into Neural Nets next time!