What is Reasoning?

(This is part 1 in a series on reasoning in artificial intelligence)

Starting in the 1960s, AI research has diverged into two camps — classical approaches like rule-based systems, and machine learning. The early advantage went to the classical techniques. But, especially, in the last couple of decades, the trend has seen a near complete reversal, with machine learning being the primary source of major advances in AI. ML has been so successful that it is in the mainstream with applications in vision, audio, and language. But, more complex tasks like Reasoning are still very much outside the comfort zone of what is possible today. Classical approaches, on the other hand, have focused nearly exclusively on higher-order problems like reasoning. I believe we are at an inflection point now — one where classical and machine learning approaches have developed enough that bringing them together can push AI to the next level.

Artificial Intelligence can do some amazing things right now. It can answer our questions, drive our cars, make world leaders sing, create movies from a single picture, and even create art. All of these abilities are examples of prediction — taking an image, audio, or text as input and returning a category, text, or control commands as output. These types of predictions are not trivial and can be the result of arbitrarily complicated pipelines and use combinations of methods from control systems, traditional machine learning, and deep learning. But in all cases, the steps in these pipelines are fixed (though they may be skipped) and the resulting model (or pipeline) is targeted at a very narrow problem (question answering, image classification, controlling a vehicle). You can’t ask a question answering model to control a car, or a game playing model to detect tumors in MRIs. Every AI today is purpose-built and heavily trained to achieve that one (narrow) function.

Now compare that to how the human brain works — we can drive cars, make coffee, answer questions, play video games, and most importantly, learn to do very different and new things. We can even learn to do old tasks differently. And we learn to do these tasks from very few examples while continuing to get better as we keep doing them.

What about AlphaGo (Alphastar)?

AlphaGo is a tremendous achievement in our understanding of how to construct machine learning models that can learn complex tasks. AlphaGo uses one of the most successful discoveries in the past two decades — deep neural networks — and it uses these networks effectively to learn how to solve Go. But, someone sat down and crafted the networks and reinforcement learning strategy that trained AlphaGo to learn how to play Go. And after it became the best at Go, AlphaGo had to be completely redone to create AlphaStar, the AI that plays Starcraft II. (Link to a simple explanation of how AlphaGo works)

So how is reasoning different?

At its simplest, reasoning can be thought of as at least one level above prediction in complexity — if prediction is a matter of training a model given the right architecture and pipeline, reasoning is about figuring out the architecture and pipeline itself. Put another way, if the solution to a problem is a sequence of prediction (and other engineering) steps, then reasoning is the ability to construct these sequences of steps. Reasoning can be applied to many different types of tasks. When you reason about how to do something, it’s called Problem Solving. When you reason about why something happened, it’s called Causal Reasoning.

Why is reasoning a more difficult problem than prediction?

Intuitively, it makes sense that constructing a sequence of steps on top of training this sequence is more difficult than just the training itself. But to understand how much more difficult reasoning is, we have to use an analogy… Consider the problem of making coffee. Given a coffee maker, milk, coffee beans, a grinder, a frother and a set of instructions on what to do, making a latte is just a question of finding the right amounts of each ingredient to use (the amount of coffee, the burr on the grinder, how much milk to use, how long to froth, the ratio of milk to froth and so on). This step requires some trial and error, but after a few tries you get it right. Prediction is similar. You are given a bunch of resources and instructions on how to use them and in what order, and prediction is the process of finding the right amounts of each ingredient to use.

Now imagine you are simply given the ingredients and the tools to make coffee. You don’t just pour water over beans. You heat the water, grind the beans, put the coffee in the filter placed over a mug, and then you pour the water over the beans. You don’t simply run the frother independently of the milk, you have to make sure the frother is in the milk to make the foam. This problem is definitely harder and is about the level at which autoML works.

But what if you are not even given the ingredients or the tools. Someone points you in the direction of a coffee shop (or a supermarket) and asks you to make coffee. You have to pick the right ingredients, the right tools, the right settings, and find the right amounts of each ingredient to make coffee from a cafe (or warehouse) full of things. That’s the complexity of reasoning.

So, what is reasoning?

I will talk about the technical aspects of reasoning including representation and problem spaces, policies, reward and loss functions, and,of course, AutoML. But for now, I want to keep it simple and provide four qualities that a technique/algorithm/model should satisfy to qualify as a reasoning system.

The four qualities of a reasoning system:

1. It is a self-contained model or system
2. that can solve novel problems
3. from a limited number of trials
4. while getting better through practice.

By self-contained, I simply mean that it is not modified externally (by a modeler/programmer/user/scientist) for different tasks. It could be a collection of models or algorithms or systems as long as it is untouched while learning new tasks.

It should be able to solve problems it has never seen before either through examples, or instruction, or even trial-and-error.

It should solve these new problems in a limited time-frame and from a small number of examples

And, finally, it should get better through practice. We haven’t talked about this quality earlier but it is necessary because qualities 1–3 can be achieved through trial-and-error (for large values of limited).

What’s the difference between Prediction, Inference, and Reasoning

Reasoning is targeted (or goal-directed) inference where inputs are carefully selected, organized, and the inference machinery is used creatively to generate the desired result.

Why focus on Reasoning?

Machine learning has gotten pretty good at prediction tasks especially when the problem is well-defined and there is a large amount of data available. But these systems are incredibly brittle, prone to adversarial attacks, and don’t degrade gracefully. One way to make them more resilient is to augment their capabilities with ways to identify and recover from failure, and that involves trying to solve novel problems as they arise without external agency.

What’s the deal with classical AI?

There is ongoing research in ML targeting every quality I’ve listed above — life-long learning to build a single model that learns multiple tasks, one-shot and zero-shot learning to solve new problems from limited data, various RL techniques that allow an agent to get better through repetition, autoML techniques that try to learn the most effective pipeline to solve the problem, and so on. But the fact remains that there is a large gap between what these systems are capable of currently and what is needed to build systems that can reason.

Classical AI has been working on this problem for decades and has numerous systems that can do reasoning in much larger domains than what is possible in ML. More importantly, classical AI and ML have complementary strengths. Classical AI is good at manipulating discrete units of information but has no way of connecting these units to the world which exists in pixels and audio. ML is excellent at converting pixels and audio streams and characters to other discrete units of information but not good at manipulating them. What’s missing is that these systems don’t talk very well to one another. The discrete units that come out of ML techniques like deep neural networks are not the kind of units that classical techniques use.

Given classical AI’s success with reasoning and ML’s success with perception and action, it seems a good time to revisit reasoning from a classical AI perspective to see what we can learn from our previous successes and failures.

What’s next in this series?

I know where I want to end this series — by covering some of the work currently being done to marry these two approaches but I don’t have a clear idea of the path I want to take between here and there. There are few topics I want to talk about in classical AI including representations, different kinds of reasoning, logic systems, physical symbol systems, and cognitive architectures. I hope to meander through these topics over the next few articles beginning with a brief history of classical AI.

Disclaimer: The views expressed in this article are my own and do not necessarily represent the views of my employer.