Naturally, each AI system has its own requirements, and thus each may benefit from a different implementation. However, common techniques tend to improve the development times and the quality of the final product.
The appropriate compromise between the various approaches will not sacrifice performance for development times, however.
All the components that are commonly used and have large slices of execution time should be implemented in native C++ code. The only way to identify these performance bottlenecks is through profiling. Standard tools may be used for this purpose, such as Intel's VTune or GNU's gprof, although it's extremely simple to include custom timing routines.
As Donald Knuth points out, "Premature optimization is the root of all evil." The optimizations of the modules should be done only after the system works fully and the bottlenecks have been identified. Ideally, any optimization procedure should not change the results, so ideally, robust unit tests should be available to identify adjustments that break the system.
Scripting languages, although they are arguably modern programming languages, have much to offer for the development of AI. With the support of either weak or dynamic typing, fewer lines need to be typed and the true purpose of the code shines through. AI algorithms are much faster to prototype and test.
Scripts are ideal for providing small parts of code that are not executed often, or that require strong customization (for instance, an AI dependent on the level of the game). Languages such as Lua or Python support such functionality, although their execution times are much slower than native code. This is the main reason for profiling the entire system and optimizing the crucial components.
Game engines in general—and AI systems in particular—are well suited to a data-driven approach. The entire system is driven by data files that are loaded during initialization, instead of integrating the information within the code. This separation is convenient for all the members of the development team (working independently), and generally leads to more robust systems (clean separation of the code and data).
In particular, the scripts themselves can be stored as files. The data for each of the AI modules is also easily stored on disk (data such as the rules of an expert system or the weights of a neural network, for instance).