Aesthetically Evolved Virtual Pets

Thomas S. Ray
Zoology, University of Oklahoma, Norman, Oklahoma, 73019
ATR Human Information Processing Research Laboratories
2-2-2 Hikaridai, Seika-cho Soraku-gun, Kyoto 619-0288 Japan,


Aesthetic, emotional, and empathetic selection are applied to a derivative of Karl Sims' Evolved Virtual Creatures. The resulting Creatures can be beautiful or strange, and provoke strong reactions in human observers. It may be possible to evolve virtual pets to which humans can form strong emotional bonds.


Evolution is the greatest creative process that we know. It created the deep beauty of living nature and the drama of the human mind. Even the arts are an epiphenomenon of evolution. While the theoretical science of evolution is less than two centuries old, the applied science of evolution dates back to the dawn of civilization. The domestication of plants and animals formed the agricultural basis for the origin of civilization. Thus through the breeding of plants and animals, we have been using evolution as a design tool for tens of thousands of years.

In the past decades, our experience in working with evolution has spread beyond the organic realm into the digital. Today we breed computer programs, digital organisms, and media art. The expansion of evolution into the digital medium has also expanded the nature of our relationship to the evolutionary process. The breeding of plants and animals has consisted of examing a collection of offspring in one generation, and selecting those that we will use as the parents of the next generation, "selective breeding". While selective breeding has been predominantly aimed at producing better quality food, it has also taken on an artistic quality in the breeding of ornamental plants, and animal pets.

Our relationship to digital evolution is much broader. It spans a wide spectrum of control over the process (Ray 1998, Ray In Press). At one extreme, we set up an artificial life system of self-replicating organisms which evolve by natural selection, and we keep our hands off, apart from observation of the process. At the other extreme, we control every aspect of reproduction and selection in the system. Even when we have total control over the system, the reproduction and selection can be fully automated, so that the process takes place with no human intervention, apart from starting the process and interpreting the results; the common practice in using genetic algorithms in engineering.

There also exist examples of digital evolution in which, like plant and animal breeding, the human examines a collection of offspring and selects those that will become the parents of the next generation. One of the earliest examples was Richard Dawkins' Blind Watchmaker program (Dawkins 1987). This approach was raised to an art form with Karl Sims' Genetic Images (Sims 1991) and Galapagos (Sims 1997). In Genetic Images and Galapagos, as in the breeding of ornamental plants, selection is based on human aesthetics.

The work presented here uses aesthetic selection, as in Sims' Genetic Images, but applies it to another system originally developed by Sims, known as Evolved Virtual Creatures (Sims 1994a, Sims 1994b, download 9Mb video). In Sims' Evolved Virtual Creatures, a genetic system defines a developmental process that generates a body built from blocks, articulating joints which connect the blocks, and a neural circuitry to control movements of the joints. The Evolved Virtual Creatures (here referred to as "Creatures") are embedded in a simulation of "real" physics, in which they experience the forces of inertia, friction, and optionally gravity or fluid drag. The Creatures can be evolved to swim, walk, jump, follow objects, compete for possession of objects, and potentially many other tasks.

Human observers have strong emotional, empathetic, or aesthetic reactions when observing Karl Sims' Creatures. Sims evolved the Creatures for the tasks listed above, with no human intervention in the evolution. The project presented here introduces the ability for humans to guide the course of evolution by selecting Creatures based on the human's emotional responses to the Creatures in a fashion analogous to the breeding of animal pets. It is believed that this will lead to the evolution of Creatures which evoke even stronger emotional, empathetic, and aesthetic responses. Humans may be able to evolve virtual pets to which they form strong emotional bonds.



Karl Sims has commented that the most difficult part of his Evolved Virtual Creatures project was the simulation of physics. It was the hardest part of the code to write, and the physics simulation consumed more than 95% of the CPU cycles in running the system. Karl recommended the use of a physics library, and MathEngine was chosen for this project.

Morphology Genome

Implementation of the Creatures began with the definition of the genetic structures that control the Creature morphology. The genome defines a directed graph of connected boxes. Each genome contains a randomly determined number of boxes. The dimensions and colors of the boxes are determined at random. Each box has a random number of connections to other boxes. Each connection is randomly assigned to one of the other boxes in the genome. For each connection, the locations of the contact points on each of the two connected boxes are determined at random, and the Euler coordinates of the resting joint angle of the connection between the boxes is determined at random. (Figure from Sims, K. 1994a).

The genome is then translated into a phenotype, by tracing through the randomly connected boxes. If a box connects to itself, the directed graph is directly recursive. Recursion can be indirect, involving more than one box, when a series of connections eventually lead back to a previous box. The genome includes a randomly determined limit on direct recursion. Where direct recursions occurs, a random number of gradients are applied over the recursive chain of boxes. There are about twenty characters that may be affected by gradients, such as color, box dimensions, and the locations, angles and stiffness of joints between boxes.

The algorithm that translates genotypes into phenotypes includes a limit on the total number of boxes, which effectively limits indirect recursion. Tracing of the genome connections begins with a randomly determined root box, which has a randomly determined number of connections to child boxes. The child boxes also contain a random number (possibly zero) of connections to child boxes. Thus each box (except the root) is connected to one "parent" box, and a random number of child boxes.

Actuator Joints

Each "child" box, has one joint to a parent box. The joint has genetically defined resting angles with respect to both the child and parent boxes, and the joint contains a spring that restores it to its original orientation when perturbed.

The child box includes an actuator that controls the Euler angles (yaw, pitch, roll) on the joint. There are effectively eight possible actuators, depending on which of the Euler angles are being manipulated: none; yaw; pitch; roll; yaw+pitch; yaw+roll; pitch+roll; yaw+pitch+roll. Which angles are actually being manipulated, depends on the outputs from the control circuit (see next section).

Control Circuit Genome

In addition to the genome which defines the morphology, there is also a genome which determines the structure of a control circuitry. The control circuitry includes sensors, processors, and effectors. The effectors control the three Euler angles, yaw, pitch, and roll, on the actuator joints between the boxes. There are also effectors which output color values. Currently implemented sensors sense the time, color, Euler angles of the actuator joints, position, velocity, angular momentum, and forces on individual boxes or the creature as a whole, and the relative position of an object. The processor elements include a large number of computational primitives such as add, multiply, greater than, log, sine, tangent, arccosine, exponent, square root, bitwise and, etc.

Like the morphological genome, the control genome is a randomly constructed directed graph. A complete control circuitry is randomly created for each box in the genome. In addition, one "global" control circuit is created that is not associated with any box. The control circuit directed graphs are nested within the directed graph of the morphology.

For each circuit, a random number of circuit elements (neurons) is determined. The type of each neuron is determined at random (e.g., color effector, velocity sensor, square root processor). The number of connections for each neuron depends on its type. The neurons are connected at random. The connections are assigned at random to one of four pools of neurons: in the global circuit, in the local circuit, in the parent box, in a daughter box. Connections to other neurons in the local circuit can be determined at the genotype level, but connections to neurons in circuits in other boxes, or in the global circuit must be determined at the time that the genotype is translated into the phenotype.

Collision Detection

In the absence of collision detection, it is common for the morphology to result in forms in which boxes inter-penetrate. To be able to detect and limit inter-penetration of boxes, two collision detection algorithms have been implemented. However, neither algorithm provides satisfactory results. The system is generally run with collision detection turned off.


In order to support swimming behavior, a simple fluid dynamic is implemented by adding a force normal to a surface, proportional to its area and velocity, thereby simulating fluid drag. This simulated fluid drag can be turned on or off. While the simulated fluid drag does permit natural looking swimming behavior, sometimes Creatures accumulate unnatural quantities of momentum, either linear momentum or angular momentum.

Also, because collision detection is generally not used, sometimes, several boxes occupy roughly the same space. Because we do not disregard hidden surfaces when calculating fluid drag, several boxes which occupy the same location can generate a very strong fluid drag force, causing these structures to attain unnaturally high linear or angular velocities.

Clearly the fluid dynamics algorithms are not completely accurate and need further development. However, the objective in this project is not to create an accurate simulation of "real" physics. The goal is to create Creatures that provoke a strong emotional, aesthetic, or empathetic response in humans. In spite of the flaws in the fluid dynamics simulation, the current system produces beautiful and interesting movements.

In order to support walking behavior, gravity, a ground plane, and frictional forces are provided. Gravity can be turned on or off and the ground plane can be permeable or impermeable.


In order to allow evolution, a rich set of genetic operators has been implemented, affecting all genetically determined characters of both the morphology and behavioral components of the genome. The user can select any creature, then produce genetic variants from it, and select among those variants for the parent of the next generation. In this way, the user can direct the evolution through aesthetic selection.

In addition, a genetic algorithm has been implemented to permit the automated evolution of certain behaviors. Karl Sims used a GA to automate evolution with selection for: swimming speed, crawling/walking speed, jumping height, ability to follow.

Sims also used GA selection for possession of a block to create a co-evolutionary scenario.

Rather than using GA selection for competition, I will evolve my creatures to love, by selecting for moving towards one another, then maximum surface contact, then continued relative movement while in contact.

At this time, only a single fitness function has been implemented in the GA, which selects for the ability to follow an object which periodically moves to random positions relative to the creature. This is the first of the three steps leading to the evolution of the appearance of affection.


The directed graph structure of the genome provides a rich set of morphology and behavior in the randomly generated Creatures. Many of these Creatures show aesthetically appealing shapes, colors, and movements. Many of them are capable of significant amounts of swimming and crawling, even without the influence of evolution to enhance their ability to move.

A Zoo of many hundreds interesting genomes has been accumulated. Initially, most of these were found by random search, without the aid of evolution. Yet, evolution has proven to be a powerful tool in enhancing the aesthetic qualities of the Creatures. Most of my favorite creatures in the Zoo have been enhanced by some amount of selective breeding. Let me cite a few examples of favorite creatures from the Zoo.

FlowerWave and descendants

I came across a rather beautiful form which I originally named CoilFlower, and I saved CoilFlower01 and CoilFlower02 in the Zoo. CoilFlower01 is like a flower blooming rhythmicly, and CoilFlower02 reminds me of a wave crashing upon itself. Some time later while browsing the Zoo, I developed a fondness for them, and decided to try to evolve them. I also decided that they deserved a better name, so I renamed the series FlowerWave.

I was particularly interested in trying to develop some nice neuron color effector activity, which I was able to do. However, in the process of selecting for nice color changes, I lost the beautiful shape. I decided to use a new tool I had developed to selectively focus mutations. I turned off mutations to the body, and allowed only mutations to the neural nets. In this way I evolved the behavior while keeping the morphology fixed. So much for the Coevolution of Brains and Bodies. The FowerWave series was evolved to be viewed with the Color Effectors - Neuron, and the Body Shape - Ellipsoid settings. It is important to use these settings when viewing them. The striking thing about their neuron color effectors is that colors flow out from the base of the flower, moving from petal to petal.

After a great deal of selection I came upon FlowerWave36. This combines very interesting neural color changes with great form and movement. Colors flow up from the base.

Taking that series a little further, I came upon FlowerWave73. Colors pulse in brightness, and change in hue.

At this point I thought it would be interesting to hold the neural net fixed and evolve the body. After some further selection I came upon FlowerWave45.

After evolving further in this way, I decided it was time to let both the brain and the body evolve together. Eventually this led me to FlowerWave56, which has an amazing sequence of color flows.

This formed the beginning of the color flow series, which is the lineage which has been subjected to the greatest amount of aesthetic selection, over hundreds of generations. Initially, the color flowed in discrete steps down the length of the body. That is, a certain color value jumped from box to box down the chain. I wanted to see if it would be possible for the colors to flow in a more continuous fashion, with color values changing gradually, while flowing down the chain. This proved to be possbile. Also, there evolved many touch sensitive creatures, in which a touch causes a pulse of color to flow down the chain. These creatures can be recognized in the Zoo by the names ColorFlowB, ColorFlow2Head, ColorBiFlow, ColorSense, CoilFlower. These are some of the most aesthetically spectacular creatures in the Zoo (they must be viewed with the "Color - Neuron" setting). "Help - The Zoo" contains a listing of these creatures, and their individual characteristics, such as sensitivity to touch or movement. The images below are (in order): ColorFlowB28, ColorFlowB53, ColorBiFlow14, ColorBiFlow14, ColorSense05, ColorFlow2Head03, CoilFlower04, CoilFlower05, CoilFlower06, CoilFlower06.


The genetic algorithm has been used to evolve creatures which follow an object which appears on the screen as a green ball. About a dozen of these are included in the zoo, and can be recognized in that their names begin with "Attraction", and take forms like "AttractionDd22894", where the number 22,894 means that it is the 22,894th genome evaluated by the genetic algorithm. The images below are (in order): AttractionAa14986, AttractionDc13096, AttractionDd22894, AttractionEb09635, AttractionFa17134, AttractionFb14291, AttractionFe10397, AttractionGd15080.


Ladder13 is the result of my first experience with evolution of the Creatures. While generating Creatures at random, I came across a striking form, consisting of six flat panels in a row, and twelve bars arranged with six on each of the two sides. While the form appealed to me, it was almost completely motionless. I decided to try to select for motion. This turned out to be feasible, and in the next few generations the motions of all parts increased gradually.

Then I chanced across an individual that exploded (with the MathEngine physics library, structures will explode if the forces are too great). Explosion is an unfortunate result as it means the disappearance of the creature. However, explosion is also a kind of motion, and in fact, the motion was much greater than any creature in the series up to this point. I decided to select the exploding creature, and attempt to reduce the explosive tendency through further breeding.

In successive generations, the time before explosion increased gradually, until the explosion was completely eliminated. However, the resulting creature, Ladder13, retains a fascinating residue of the explosive tendency. It is perfectly balanced on the edge between exploding and pulling back together. Soon after birth, the flat panels begin shaking, and then the chain of panels invert, by sliding through each other. The whole collection of parts scatter and begin spinning as if in a whirlwind. When the scattered parts have become completely disordered, the flat panels may begin shaking again, and the shaking somehow pulls the parts back into their original positions. The spinning may then pause momentarily, before the chain of panels inverts again, and the collection of parts again spins and scatters into disorder. It is interesting to view this creature with the Neuron Color Effectors setting.

Magnets02 and LittleArmShakey also show this fascinating balance between exploding and pulling back together. However, unlike Ladder13, these two will eventually explode. If they do explode quickly, just start them again, they usually last a long time before exploding.


A chain of fifty five slender panels in a spectrum of colors, SkyDiver04 exhibits perhaps five distinct patterns of form and behavior. It is born with its long body extended into a beautiful pattern of loops and curves (the first form/behavior), but it immediately contracts its body into a sort of triangular accordion (the second form/behavior). It remains in this form for several minutes, while gently breathing, and ever so slowly twisting its body more and more. Then suddenly, it bursts into motion, and reorganizes its body into a tapered coil, pentagonal at the small end and hexagonal at the large end (the third form/behavior).

If left undisturbed for a few minutes, it may begin a spring-like dance, while retaining generally the same tapered form (the forth form/behavior). The head and tail simultaneously begin a strong contraction, which cause waves to pass through the body, from the tail towards the head and from the head towards the tail, the two waves meeting in the thickest part of the body.

The third and fourth forms are morphologically the same, but behaviorally different (the third rests, while the fourth contracts). The Creature spontaneously alternates between the third and fourth forms, but evidently remains in those two forms indefinitely if left undisturbed. Here is a frequency distribution of the interval between contractions during the first hour. If left undisturbed for several hours, it seems that the Creature dissipates its energy and stops the contractions.

If the second form is disturbed, it will hasten its transformation into the third form. Disturbing the third/fourth forms will cause it to transform back into the second form. Sometimes these disturbed transformations result in hybrid forms.

Incidently, SkyDiver04 is born on the side of the y-z plane corresponding to the negative x-axis. If, at any stage of its life, the Creature is dragged across the y-z plane into the positive region of the x-axis (use the x key), it begins a graceful dance made of loops (the fifth form). It appears that it will remain in the fifth form as long as it is on the positive x side of the y-z plane. If it is dragged back to the negative x side of the y-z plane, it will transform back into forms two through four. If the body spans the y-z plane, it may produce mixed results.

The images below show the first form of SkyDiver04:

The images below show the second form of SkyDiver04:

The images below show the third form of SkyDiver04:

The images below show the fifth form of SkyDiver04:


Searching through random genomes, I came across a chain of five purple blocks, that quickly spun out of control. A little selection lead to a chain of seven red flat panels in a slightly twisted configuration, like a propeller. They also tend to begin spinning, as if they may spin out of control and explode. But as the spinning accelerates, the chain flexes subtly, bringing the spinning momentarily to a halt, and then the spin begins again in a new direction, always accelerating, flexing, slowing to a stop, and accelerating again.


An active tapered chain of fifteen flat square slabs in changing shades of purple, and a passive tail of about forty eight little bars arranged in a criss-cross pattern, some light blue, some changing shades of pink and yellow. The massive tapered body and the delicate tail join at a blue-grey block, around which they dance elegantly. The passive tail, follows the lead of the active body, as if in an alien ballet.


A tapered chain of about forty eight blocks, initially arranged in an inverted yellow-green coil, like a tornado. The coil quickly unwinds into a snake, shifting through every color in the spectrum. The snake writhes, twists, and swims for an uncertain amount of time, before suddenly coiling again into a writhing chaotic mass of blocks of quickly changing colors.

After some time, this highly disordered coil arranges into highly geometric form, each time different. Sometimes it may have the shape of a sky scraper building of perfectly square tapered tiers with golden flashing panels. Other times it may be a twisted tower of light blue tapered blocks. It settles into a completely motionless geometric form, from which it apparently will never move again, unless touched.

When touched by applying a force to some part of the body, it begins moving again. The degree of movement depends on the strength and duration of the touch, and chance. It may completely uncoil into a snake again, and it always eventually assumes another motionless geometric form, each time different.

The eleven images below are a time sequence of the typical behavior of Twister02:

The image below shows a rare open-coiled final resting form of Twister02:


A chain of over sixty blocks, of two kinds, found by random search, with no enhancement through breeding. This worm twists and writhes, exhibiting a most remarkable display of changing colors.


Also found by random search, five blue or purplish spastically moving blocks arranged roughly in a square connected by six brown flat panels, and attached to an arm of seven green or brown blocks which occasionally flex and then make a sudden jerk.


While the shapes and colors can be beautiful and interesting, it is the movement that makes these creatures compelling. In the natural world, many things move, but only living things move by their own power and volition. Thus when we see artifacts which move by their own power and volition, we may be prone to feel that they are alive. But this depends on many aspects of their appearance and movement. Automatic doors sense our approach and open of their own power and volition, yet we don't feel that they are alive. They are merely mechanical devices.

Some of the Creatures of this work have such simple appearance and movements, as to provoke a similar response, they seem mechanical. Yet other Creatures have more organic forms, and more subtle movements that cause them to appear very lifelike. The range of forms and behaviors that we observe among the Creatures shows some overlap with familiar organic life forms. For example, we see many Creatures with snake-like forms and movements. Yet there are also many forms that are completely unlike any known life form. Yet, in their own unique way, they seem alive, like alien life forms. But of course they are not alive, they are dynamical systems which play with our perceptions of what is alive.

The wide range of forms and behaviors among the Creatures provokes a wide range of responses from the human user. The Creatures may appear beautiful, elegant, sensuous, nervous, bizarre, strange. Selective breeding can enhance any of these qualities.

Because this work has not attempted an accurate simulation of "real" physics, the Creatures sometimes seem otherworldly. The body parts interpenetrate, or completely separate, or fly apart. Observing these strange Creatures takes us into another world. They may still seem alive, but completely on their own terms.


This work was supported by a research contract between the Nippon Telegraph and Telephone Corporation and the University of Oklahoma. I thank Tim Taylor for bringing me up to speed with MathEngine. I thank MathEngine for making their software available and providing extensive technical support.

Software Availability

This software is available for free download at:


Dawkins, Richard. 1987. The Blind Watchmaker. Norton.

Ray, T. S. 1998. La vita artificiale. In: Frontiere Della Vita, Estratto Dal Volume I. Gilbert, Walter, and Glauco Tocchini Valentini, [eds.], 109-125. Istituto della Enciclopedia Italiana, Fondata da Giovanni Treccani.

Ray, T. S. In press. Artificial Life. In: "From Atoms to Mind", Gilbert, Walter, and Glauco Tocchini Valentini, [eds.]. Istituto della Enciclopedia Italiana Treccani. Rome.

Sims, K. 1991. "Artificial Evolution for Computer Graphics," Computer Graphics (Siggraph '91 proceedings), Vol.25, No.4, July 1991, pp.319-328.

Sims, K. 1994a. "Evolving Virtual Creatures," Computer Graphics (Siggraph '94) Annual Conference Proceedings, July 1994, pp.15-22. New York: ACM Siggraph.

Sims, K. 1994b. "Evolving 3D Morphology and Behavior by Competition," Artificial Life IV Proceedings, R. Brooks and P.Maes [eds.], MIT Press, 1994, pp.28-39.

Sims, K. 1997. "Galapagos".