Successful learning systems are based on strong understanding of both the problem and the solution. Chapter 21, "Knowledge of the Problem," and Chapter 28, "Understanding the Solution," cover this issue in depth. When it comes to adaptation, there are particular ways to deal with problems by modeling the system differently:
Fitness and Reward
When the learning is not supervised, fitness functions and reward signals define the problem—but can be a surprisingly large source of grief. Learning based on feedback is particularly difficult to achieve realistically and reliably. When done right, however, the design of such feedback mechanisms can assist the learning in many ways:
An AI architecture is capable of combining learning components—or fixed components—to provide the desired behaviors. For example, the learning component can be overridden using a subsumption architecture. The essential actions are identified and provided by fixed components while the rest is learned.
Another useful architectural pattern is to provide a fixed component to supervise the adaptation. This can be understood as a learning manager (sometimes referred to as a critic); it can adapt the learning rate, verify that key learning examples are not forgotten, or it can determine when the learning algorithm needs to process new patterns.