How a Biochemical Metabolic Model can Contribute

to Intelligent Lifelike Behaviour

Eric Goodwin

CyberLife Technology Ltd

Quayside

Bridge Street

Cambridge

CB5 8AB

United Kingdom

eric.goodwin@cyberlife.co.uk

Abstract

Incorporating a model metabolism in an artificial organism gives it the potential to develop behaviour which is lifelike, interesting and entertaining to interact with. The physiological component of such a metabolism provides behavioural requirements and limitations that reflect those imposed on real-life organisms, and the psychological component provides motivation for apparently sensible actions. As an interface between an organism’s actions and its brain, a metabolism supports the learning processes of the organism, and as a reflection of the success of an organism’s behaviour patterns, the physiological metabolic state provides a good basis for the assessment of reproductive fitness.

Artificial Intelligences in Game Characters

CyberLife Technology Ltd believe that the future of computer entertainment is in the interactive entertainment product, without the rules imposed in more traditional computer games. These entertainment products will resemble places to play in, populated with things to play with. These playground worlds will be enriched with lifelike and intelligent organisms. Without attempting a definition of either of those characteristics, these organisms will have: unpredictable yet justifiable behaviour; freedom to do what they want; and ability to learn (to change their behaviour over time). We find the current AI philosophy based on pre-defined rule-sets too limiting. The autonomy that our organisms will exhibit is not possible with classical rule-based AI, so we are working on structures to replace or to be added to the current artificial intelligence architecture. Another problem with rule-sets is that as the environment of the organism gets more and more rich, the size of the rule-set required for the organism to act sensibly in it, grows impracticably large. We believe the answer lies in modelling biological systems such as neural nets and metabolisms (defined below). Our first entertainment product to exploit these concepts was Creatures (http://www.creatures.co.uk), conceived and written by Steve Grand (Grand, 1997). Encouragingly, this game has been a great success both commercially and within the artificial life community, and we have continued to develop the systems in further projects and products.

Any artificial intelligence in a virtual environment is usually associated with a representation of an individual character or body, whose actions are determined by the output of the ‘brain’. Because the inputs to the brain regulate its output, and hence the behaviour of the character, the nature of those inputs will determine the nature of the character’s behaviour. In general the input to the brain comes through various sensors located in the body. These can be analogues of the five major senses of humans, or more usually various levels of interpretation (or pre-processing) on top of these standard senses. (For instance, rather than presenting an array of colour values to a brain, this can be pre-interpreted as a set of known objects.) This level of input to the brain may be sufficient to give rise to the sort of behaviours expected from classic computer-game characters, but does not produce the behaviour exhibited by ‘Real-Life’ organisms.

Increasingly artificial and pre-interpreted senses can be introduced to better reflect the state of the character’s environment. However, until the character has an internal state which is made perceptible, it will appear unrealistically to think itself immortal, as if it had not evolved at all from its three-lived ancestors such as Pacman. It will not modify its actions as it takes damage in combat, or becomes weary through exertion. There is some advance made when a character (an adversary for instance) has a finite level of energy, and so can be outrun by sheer persistence. But this character can still not be called an organism, and more realism yet is possible. If the character is aware of internal physical states, such as its energy level and strength, and can take them into account, it may become more able to ensure its survival. If it could learn to walk more slowly, or lie in hiding when it was becoming weak, then it may remain an effective enemy for longer than if it fought until it dropped. If its level of damage was perceptible to the character then it could decide to retreat from a combat situation in order to heal itself rather than to carry on fighting. It would then be necessary to pursue retreating enemies in order to finish them off, or as enemies they would remain effective.

The examples given above can be modelled simplistically, with a set of ‘internal state variables’. In a game with a range of enemy types, each can be given a certain rate of energy production, so that it must rest for a certain while to overcome fatigue. Each can have a simple damage threshold at which point it will retreat from combat. Each can be given a certain rate of repair which will keep it out of action for some time before it is sufficiently healed to fight again. A character can have a defined strength making it capable of lifting or pushing objects of a certain weight. While simplistic in the extreme, such rules already represent a primitive metabolism.

Simple Artificial Metabolisms Restrict Behaviour

More realism can be achieved if these thresholds and rates are not hard-coded, but are variable as a function of a more involved metabolism. Eating well will give an organism greater endurance; if it eats while resting, it will replenish its energy levels faster. To heal properly requires the organism to avoid exertion and have a balanced dietary intake. Perhaps the foods that aid healing provide little energy, so are normally of little interest. An individual which is initially incapable of a certain task, due to its being too weak could have improved chances after exercise and a high-protein diet. These examples could all be effected via a rule set; however they are implemented, they represent a metabolism, and require a set of internal state values for the organism.

These physical state values can be altered according to the consequences of the organism’s actions. An organism’s metabolism is that set of algorithms which alters its internal state variables according to values of other variables, and in response to its actions and their consequences. The brain is then responsible for modifying its actions in response to the internal states, while the metabolism modulates factors of the organism’s capabilities such as strength and speed.

A metabolism can therefore contribute to varied and interesting behaviour, if the internal state of an organism is perceptible to its brain. In two instances of an identical environmental situation, the ‘best’ action determined by the brain may be different, as it will depend on the dynamic physical state of the organism. This can make a game character less predictable, a characteristic of intelligent lifelike behaviour. If an organism’s actions alter its internal states, metabolism is then a dynamic and realistic means to personalise individuals created equal, providing them with some identity.

If a metabolism design acknowledges the production of waste products, this can place an extra requirement on the behaviour of the organism. It may that if it does not periodically attend to ridding itself of waste products of its metabolism, it becomes victim of disease or discomfort, further limiting other behaviours. For this to be a reasonable behavioural requirement, the state of waste product build up would obviously have to be made perceptible to the organism’s brain.

Artificial Metabolisms Contribute to Learning

As a modulator of internal state values, providing the link between an organism’s actions or behaviour and its brain, metabolism supports its learning functions. To carry out actions suitable for its situation at the time, an organism must attend to both its internal and external environmental states. If a brain is to learn, and adapt its actions according to feedback received via the environment, it needs some means to differentiate between a good and a poor decision. What constitutes a good or poor decision will of course depend on the environment in which the creature is implemented.

A component of the design phase of a learning lifelike organism is to identify ‘risk and reward’ factors. These factors make up a primitive psychological internal state, as opposed to the physical states discussed so far. Examples of these factors drawn from the Creatures product are hunger, tiredness, boredom, temperature, and sex drive, inter alia. Each of these factors is represented as a variable whose value is perceptible to the brain, and any action the organism makes in the environment may affect the values of these variables. Actions which reduce a risk factor or increase a reward factor are encouraged, while actions that increase a risk factor or decrease a reward factor are discouraged. In this way the overall behaviour of the character will tend towards that which renders it most effective at its role in the environment, whether this is basic survival in an a-life simulation, tactical combat in a fighting simulation, or any other behaviour.

The two aspects of the metabolism which have been mentioned here (the physical state of the organism, and the psychological state) can be integrated for more interesting, less predictable behaviour. If the psychological state values are altered according to the physical state of the character, and the physical state is affected by the character’s actions in the environment, then this extra level of complexity should be reflected in the behaviour. This is represented in figure 1, where the environment affects the physical metabolism; the physical metabolism influences the psychological aspects of the metabolism; and all three are perceptible to the brain. An example of the nature of the interaction between the physical and psychological components of the metabolism may be the organism’s hunger being based on the changes in immediate energy availability, or the levels of stored food reserves

Based on the three types of input describing the current situation, the brain then determines what action is best carried out in the environment. This action is modulated by the physical metabolism, for instance to limit the strength of an action, or the speed of a movement.

Artificial Metabolisms Contribute to Evolution

Current genetic algorithms for the evolution of fit individuals rely on definite fitness functions imposed on the population by an extrinsic agent. The quality of each individual of a population of synthetic organisms is assessed using this fitness function as its metric. The individuals with the best results based on this test are then allowed to breed, and slowly the population improves in ‘quality’ as judged by the fitness function. As the population improves the same fitness function continues to apply. The ambition when designing a fitness function is that a simple function will support the development of a complex solution. The strength of the genetic algorithm system is its potential flexibility, allowing any behaviour to emerge that satisfies the fitness

function. However, the limitation is that a pre-defined fitness function may not support the ‘Red Queen’ scenario, where the evolutionary pressures on a population change constantly as it evolves. A well designed fitness function is highly capable of finding a single optimal solution to a well-defined problem, but does not always allow for ongoing, open-ended evolution. It is this continued co-evolution as an alternative to a genetic algorithm that produces not only solutions we would never have thought of, but solutions to problems we would never have thought of, and that we may even struggle to identify.

"… the process of designing an evaluation function for behaviour evolution …is delicate and laborious" (Mondada, and Floreano; 1996), especially if it is required to evolve a range of behaviours satisfying poorly defined requirements. Even the design phase of the fitness function itself may require many evolutionary generations before suitable candidates are found, which on the time scales involved is a large restriction. The knowledge gained during this design cycle is often never reported, and so their design has become a black art. Metrics supporting the development of multiple specific behaviours (e.g. go to green block, pick it up, place on blue block) are difficult to define, and often require incremental evolution. Metrics for measuring group behaviour (e.g. flocking, schooling, co-ordinated communal effort) are almost impossible to define.

The alternative to these overly complex fitness functions is to use a more de-centralised approach to genetic algorithms. Instead of a centralised global algorithm applied to the individuals ‘from above’ which measures the population and supervises the generation of offspring, each member is responsible for determining when it can reproduce. Several classic Alife works have already exploited this idea, with great success. Larry Yaeger’s Polyworld (Yaeger, 1994), and Tom Ray’s Tierra (Ray, 1991) are obvious examples. The factors determining whether an individual can reproduce are available in the integrated network of complex internal interactions that is the metabolism.

As a set of psychological and physiological internal state values, metabolism provides a lot of information about the success of an individual’s behaviour, which will be based on its intelligence. The state of a metabolism can be a good basis for the assessment of behavioural success, avoiding the need to measure the potential intelligence of a given brain architecture. If the architecture of the brain (or whatever structure supports the organism’s intelligence) is not hard-coded, but genetically specified and mutable, this allows for variation and evolution of learning capability. Basing reproductive fitness on a function of intelligence should lead to the evolution of brain architectures better able to exploit the current environment.

The breeding function will depend on the nature of the species being evolved. This paper has focussed on techniques for developing ‘generalist’ intelligent organisms, but similar concepts can be applied to the evolution of other ‘specialist’ entities. For instance, if one was evolving a pilot (control system) for a spaceship, the pilot and the ship could be treated and bred as a unit. The assessment of the pilot’s reproductive fitness would be based on the state of the ship, requiring an artificial metabolism to be developed for it. The pilot could have psychological reward/risk factors associated with transporting cargo over large distances and avoiding piracy, or with destroying large numbers of enemies while taking little damage. The physical state of the ship, such as the amount of fuel in the tank and the amount of damage to the hull, could also be perceptible to the pilot.

The metabolic state required for a more generalist organism to reproduce could have an anatomical basis: a non-damaged, functioning reproductive organ could be required. It could be based on a physiological requirement: an individual can only survive the demands of pregnancy if it has enough stored fat reserves for instance. It could also be dependent on the psychological aspects of the metabolism (the risk/reward factors): the individual may have to be happy and secure-feeling before it will engage in sexual behaviour.

Further advance can be made in the evolutionary realism when aspects of the metabolic state of other individuals are also perceptible. This can lead to mate selection, which will be an extra pressure speeding evolution.

However metabolism is modelled in an artificially intelligent organism, it will contribute to the interest and complexity of its behaviour. The physical aspects of metabolism act as a dynamic and realistic set of limitations or requirements on the capabilities of an organism, while the psychological aspects of metabolism provide vital support and input to the learning function of the brain. Both these aspects can contribute to the open evolution of intelligence, leading to more and more capable brain architectures. Clearly some acknowledgement of metabolism is of vital importance in the design of intelligent artificial organisms.

References

Grand, S.; Cliff, D.; Malhotra, A.; (1997); "Creatures: Artificial Life autonomous software agents for home entertainment." Proceedings of the First International Conference on Autonomous Agents.; W. Lewis Johnson (Ed) ; ACM Press.

Mondada, F., and Floreano, D; (1996); "Evolution of Neural Control Structures: Some Experiments on Mobile Robots"; Robotics and Autonomous Systems.

Ray, T. S.; (1991); "An approach to the synthesis of life."; Artificial Life II, Langton, C.; Taylor, C.; Farmer, J.D.; and Rasmussen, S. [eds]; Santa Fe Institute Studies in the Sciences of Complexity, vol. XI, 371-408. Redwood City, CA: Addison-Wesley.

Yaeger, L.; (1994); "Computational Genetics, Physiology, Metabolism, Neural Systems, Learning, Vision, and Behavior or Polyworld: Life in a New Context."; Artificial Life III, ed. Christopher G. Langton, SFI Studies in the Sciences of Complexity, Proc. Vol. XVII, Addison-Wesley.