Chapter 28. Understanding the Solution

Key Topics

• Complexity of the Solution

• The Search Space

• Different Approaches to Finding a Solution

The importance of understanding the problem was discussed in Chapter 21, "Knowledge of the Problem." The lesson for Part IV provides insights into the solution itself. The role of the solution is to provide a direct mapping between the inputs and the outputs. The solution has many different aspects, so it's important to present the big picture before moving on. The solution consists of four different parts:

• Representation is the formalism used to store the solution, the structure of the expert knowledge about the problem.

• Training provides a way to find the right variation of the representation that manages to solve the problem.

• An instance is the result of the training, a particular set of internal values expressing the input/output mapping.

• Simulation is the algorithm that uses an instance to compute the correct response given a particular input pattern.

As always, it's easier to understand using an example or two. The concepts are depicted in Table 28.1. It's important to understand that AI techniques presented in this book provide design patterns for each of these four aspects of the solution. That said, we can generally consider the concepts separately during the design and development. (For instance, some training algorithms are interchangeable.)

Table 28.1. Different Aspects of Solutions to Problems

	Perceptron	Decision Tree
Representation	Connection weights between neurons	Decision nodes and branches
Training	Back propagation	Recursive partitioning
Instance	Values of weights	Decision criteria for each node
Simulation	Forward propagation of input pattern	Guided traversal of the tree

This chapter looks at the general concepts behind each of these four characteristics of solutions. This will help explain the general role of design when developing intelligent systems, and allow us to apply AI to novel problems. Sadly, it's not possible to discuss the process of simulation in general terms, because the algorithms depend on specific representations.

The remaining sections of this chapter cover the following topics:

• A description of AI solutions based on information theory. Specifically, we can express redundancy by compressing the mapping from inputs to outputs—exploiting patterns in the data. This is the idea behind the minimum length description principle, which helps determine the necessary complexity of the solution. This section also reveals the importance of representation of the solution.

• The representation in more detail, explaining search spaces as the number of possible instances of the solution. In a similar fashion to Chapter 21, the size and dimensionality of the search space is analyzed. A few examples reveal important properties of the search space (such as roughness and smoothness).

• An overview of the different design paradigms used to craft an AI system. These range from human solutions to expert tailored algorithms, including random and exhaustive search methods.

At the end of this chapter, we'll better understand the intricacies of AI solutions and have an enhanced comprehension of the approaches used.