IJCEM International Journal of Computational Engineering & Management, Vol. 12, April 2011 ISSN (Online): 2230-7893 www.IJCEM.org
Neural Networks Touching New Horizons in Video Games
Neetu Nasheir Duhan Department of Computer Science and Application Hindu Girls College Sonipat -131001, Haryana, INDIA neetuduhan@rediffmail.com
Abstract Neural Networks are touching new horizons in video games. While a detailed description of neural systems seems currently unattainable, progress is made towards a better understanding of basic mechanisms. Neural networks offer some key advantages over more traditional AI techniques. First, using a neural network may allow game developers to simplify the coding of complex state machines or rules-based systems by relegating key decision-making processes to one or more trained neural networks. Second, neural networks offer the potential for the game's AI to adapt as the game is played. Modern video games are usually played in real time, allow very complex player interaction and provide a rich and dynamic virtual environment. Neural Net can use a technique for increasing the probability that a population will remember old skills as they learn new ones. This paper is presenting Learning, Augmented Topology and many facets of neural network with video games. Keywords: Neural Networks, Video Games, Learning, Augmented Topology, complex state machines 1. Introduction Global video games software market to reach $91.96 billion by 2015, according to new report by global industry analysts. To meet the competitions in video game industry neural network is a powerful real or virtual device, modeled after the human brain.
In Neural Network several interconnected elements process information simultaneously, adapting and learning from past patterns. Over the past 20 years, the neural network has been a vibrant area of AI research, leading to new algorithms that have been used in a variety of disciplines. Neural Networks applications are being implemented in Video games to produce the illusionary effect of intelligence augmentation in order to give the player a good game play experience. It is used to create exciting playing strategies which keep the players focused and interested in the game. Players are provided exciting opponents, more intelligent creatures that inhabit the world of their games, which exhibit interesting behavior. The main purpose is that boredom of repetition is avoided. In the last couple decade there is a great evolution in the computer game industry. When two dimensional games were saturating the market, the introduction of 3Dtechnology really made the concept of a game world entered to the mainstream. A games story line no longer consists of only one primary character. It must consist of many, all playing an important role in the conflict resolution of their virtual world. In most modern video games, character behavior is scripted; no matter how many times the player exploits a weakness, that weakness is never repaired. But with neural networks game characters could learn through interacting with the player, behavior could improve as the game is played, keeping it interesting. One of the most compelling yet least exploited technologies is machine learning. Thus, there is an unexplored opportunity to make video games more interesting and realistic, and to build entirely new genres. Neural Networks make good controllers for video game agents because they can compute arbitrarily complex functions, can both learn and perform in the presence of noisy inputs, and generalize their behavior to previously unseen inputs. Thus, Neural Network is a good match for video games. I am presenting various facets of Neural Network with video games. 2. Games using Neural Network IJCEM www.ijcem.org

Picture 1

Blackjack or twenty-one is a card game where the player attempts to beat the dealer, by obtaining a sum of card values that is equal to or less than 21 so that his total is higher than the dealer's. This game is using Learning strategies in neural network. The 20Q AI uses a true artificial neural network to pick the questions and to guess. After the player has answered the twenty questions posed (sometimes fewer), 20Q makes a guess. If it is incorrect, it asks more questions, then guesses again. It makes guesses based on what it has learned; it is not programmed with information or what the inventor thinks.
3. Learning Learning is process by which the free parameters of a neural network are adapted through a process of simulation by the environment in which the network is embedded. The type of learning is determined by the manner in which the parameter changes take place. I am starting with mathematical model of a neuron as in fig 1.
Figure 1 -- Mathematical model of a neuron put in the form of an equation, this reduces to:

Picture 2 Answers to any question are based on players interpretations of the questions asked. Pathfinding in realtime by giving awareness of the virtual world around through sensors. The machine learning technique is used in this type of games. Neural network is working as data base in video game Tic-Tac-Toe. Computer Go game is using a genetic algorithm known as Symbiotic Adaptive NeuroEvolution (SANE) to evolve an artificial neural network. In most modern video games, character behavior is scripted; no matter how many times the player exploits a weakness, that weakness is never repaired. Yet if game characters could learn through interacting with the player, behavior could improve during gameplay, keeping it interesting. NERO video game is using the real-time NeuroEvolution of Augmenting Topologies (rtNEAT) method for evolving increasingly complex artificial neural networks in real time, as a game is being played. The rtNEAT method allows agents to change and improve during the game. Colin McRae Rally 2 uses neural networks to train the non-player vehicles to drive realistically on the track, and Creatures uses neural networks along with evolutionary algorithms to dynamically evolve unique behaviours for game creatures. Other interesting recent examples of the use of neural networks within games include an approach for strategic decision making. A game named First Person Shooters using a method involving a multi-layer perceptron network. Equation 1
the induced local field of a neuron is the output of the summation unit, as indicated in the diagram. If just the induced local field was propagated to other neurons, then a neural network could perform only simple, linear calculations. To enable more complex computation, the idea of a decision function was introduced. McCulloch and Pitts introduced one of the simplest decision functions in 1943. The most widely used decision function is the sigmoid function given by:

Equation

the sigmoid function has important properties that make it well-suited for use as a decision function to train networks. Other decision functions like the hyperbolic tangent are sometimes used as well. A neural network consists of one or more neurons connected into one or more layers as in fig 2. The first layer nodes are called input nodes, the third-layer node is called an output node, and nodes in the layers in between the input and output layers are called hidden nodes.

IJCEM www.ijcem.org

Equation 5 Each of the terms in Equation 5 can be reduced so we get:

Figure

Three-Layer

Neural

Network
Notice the input labeled, x6, on the first node in the hidden layer. The fixed input (x6) is not driven by any other neurons, but is labeled as being a constant value of one. This is referred to as a bias and is used to adjust the firing characteristics of the neuron. The neural networks we'll be dealing with will be structurally similar to the one in Figure 2. The output of a single neuron (we've added a parameter for a particular set of data, k):

Where?() signifies differentiation with respect to the argument. Adjustments to the weights can be written using the delta rule:
Here, is a learning-rate parameter that varies from 0 to 1. It determines the rate at which weights are changed to move "up the gradient". If? is 0, no learning will take place. We can re-write Equation 7 to include what is known as the local gradient, (k):
Equation 3 Note here that x = y, the output from neuron i, if neuron j is not an input neuron. Also, w is the weight connecting output of neuron i as an input to neuron j. We want to determine how to change the values of the various weights, w (k), when the output, y (k), doesn't agree with the result we expect or require from a given set of inputs, x (k). Formally, let d (k) be the desired output for a given set of inputs, k. Then, we can look at the error function, e (k) =d (k)-y (k). We want to modify the weights to reduce the error (ideally to zero). We can look at the error energy as a function of the error:
Here, Equation 8 can be used directly to update the weights of a neuron in the output layer of a neural network. For neurons in hidden and inputs layers of a network, the calculations are slightly more complex. To calculate the weight changes for these neurons, we use what is known as the back-propagation formula. I won't go through the details of the derivation, but the formula for the local gradient reduces to:
Adjusting the weights now becomes a problem of minimizing? (k). I want to look at the gradient of the error
energy with respect to the various weights,. Combining Equation 3 and Equation 4 and using the chain rule (and recalling that y(k)=?(v(k)) and v(k)=?w(n)y(n) ), we can expand this derivative to something more manageable:
In this formula, w (k) represents the weights connecting the output of neuron, j, to an input of neuron n. Once we've calculated the local gradient, j, for this neuron, we can use
Equation 8 to calculate the weight changes. To compute the weight changes for all the neurons in a network, we start with the output layer. Using Equation 8 we first compute the weight changes for all the neurons in the output layer of the network. Then, using Equation 8 and Equation 9 we compute the weight changes for the hidden layer closest to the output layer. We use these equations again for each additional hidden layer working from outputs toward inputs and from right to left, until weight changes for all the neurons in the network have been calculated. Finally we apply the weight changes to the weights, at which point we can recomputed the network output to see if we've gotten closer to the desired result. Network training can occur in several different ways: The weight changes can be accumulated over several input patterns and then applied after all input patterns have been presented to the network. The weight changes can be applied to the network after each input pattern is presented.

candidate solutions during the evolution process. This addition addressed concern that unbounded automated growth would generate unnecessary structure. 5. Real Time Evolution of Augmented Topology
In 2003 Stanley devised an extension to NEAT that allows evolution to occur in real time rather than through an iteration of generations as used by most genetic algorithm. The basic idea is to put the population under constant evaluation with a "lifetime" timer on each individual in the population. When a network's timer expires its current fitness measure is examined to see whether it falls near the bottom of the population, and if so it is discarded and replaced by a new network bred from two high-fitness parents. A timer is set for the new network and it is placed in the population to participate in the ongoing evaluations. 6. Complex State Machines State Machine is the most basic building block of complex real-time systems Over the years, Real time systems have increased in complexity. As the systems get complex they get harder and harder to develop as it needs a completely different mindset and tools to develop complex systems. Current state of biological systems represent system architectures that have evolved over millions of years. Most biological systems are several order of magnitudes more complex than man made system. Nature has a few billion years of experience in developing complex Real time systems. In complex real-time video games all state machine objects in a sub-system share common characteristics. These characteristics can be abstracted in a common base state machine class. The individual state machines inherit from the base state machine class. The only way to develop complex software systems is to follow an evolutionary software release development schedule. The first release implements the basic cross-section of the system. Subsequent releases refine the software in small steps. This development lifecycle minimizes development risk by integrating the tricks of game at every stage of development. Sometimes sudden changes in game environment, technology or competitive landscape require sudden changes, similar to mutations. Good Neural Net in games is all about organizing what can become a very complex system into manageable chunks, and to make these modules as intuitive and reusable as possible. Micro threads allow us to choose the granularity of our modules ourselves, and choose the level that makes most sence conceptually. Neural net allow to work on different modules of behaviors in complex state machine of a video game flexibly and build up different entity brains from reusable behaviors. 7. Experience
Wrap up this section by noting that I've only discussed one type of learning process: back-propagation using errorcorrection learning. Other types of learning processes include memory-based learning, Hebbian learning and competitive learning. 4. Augmented Topologies Augmenting Topologies is a genetic algorithm for evolving neural networks. Developed by Ken Stanley at University of Texas at Austin and published under the GPL; it integrates with Guile, a GNU scheme interpreter. Ken Stanley's Neuro Evolution of Augmented Topology is considered the base reference for implementations of the NEAT algorithm. Conventional neural network topology is defined by the developer, and the genetic algorithm is used to modify weights in the network. The complexity of such a network stays constant through the evolution process, as the number of nodes and connections between nodes remains constant. The NEAT approach begins with perceptron like structure, with no hidden neurons. It is a simplistic feed-forward network of input neurons and output neurons, representing the input and output signals. As the evolution progresses, the topology of the network may be augmented by adding a neuron along an existing connection, or by adding a new connection between previously unconnected neurons. 4.1 Phased Pruning An extension of Ken Stanley's NEAT, developed by Colin Green, adds periodic pruning of the network topologies of

Neural nets are not programmed and they learn by experience. Learn from experience is possible by adjusting weighted connections in a network. A computer program for video game is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience. The core objective of a learner is to generalize from its experience. The training examples from its experience come from some generally unknown probability distribution and the learner has to extract from them something more general, something about that distribution that allows it to produce useful answers in new cases. 8. Applications If trained carefully, Neural Networks may exhibit some capability for generalization beyond the training data, that is, to produce approximately correct results for new cases that were not used for training. (LaMothe, 1999) gives a couple of examples of NN videogame applications:
of indirection in the control or behavior of an object. Basically, we might have a number of control variables, but we only have crisp responses for a number of certain combinations that we can teach the net with. However, using a neural net on the output, we can obtain other responses that are in the same ballpark as our well defined ones. 9. Conclusion
Neural Nets are tools that we can use in whatever way we like with video games. The key is to use them in cool ways that make our games simpler and more interesting. So that users respond more intelligently. Thus, experiment with neural networks and come up with novel ways in which they can add realism to upcoming game titles or enhance the productivity applications. 10. Acknowledgments
Environmental scanning and classification. A neural net can be feed with information that could be interpreted as vision or auditory information. This information can then be used to select an output response or teach the net. These responses can be learned in realtime and updated to optimize the response. Memory. A neural net can be used by game creatures as a form of memory. The neural net can learn through experience a set of responses. Then when a new experience occurs, the net can respond with something that is the best guess at what should be done. Behavioral control. The output of a neural net can be used to control the actions of a game creature. The inputs can be various variables in the game engine. The net can then control the behavior of the creature. Response mapping. Neural nets are really good at association, which is the mapping of one space to another. Association comes in two flavors: auto association, which is the mapping of an input with itself and heterassociation, which is the mapping of an input with something else. Response mapping uses a neural net at the back end or output to create another layer

My heartfelt gratitude goes to my father Mr. Raje Ram, Mother Mrs. Rattan Mala, Husband Mr. Krishan Duhan who gave me full support and a lot of time to work with my paper. My Special thanks due to my 8 year son, Bhumit, who is not getting my time during research work. Id like to thank Mr. Pawan Paruthi who guided me through my research work. 11. [1] References Kenneth O. Stanley and Risto Miikkulainen(2002). "Evolving Neural Networks Through Augmenting Topologies". Evolutionary Computation 10 (2): 99-127 Haykin, S., 1999. Neural Networks: A Comprehensive Foundation, 2nd Ed. New Jersey: Prentice Hall. Newell, Allen, and Herbert Simon, "The logic theory machine: a complex information processing system," IRE Transactions on Information Theory IT_2, 3: 61_79, 1956. David M. Bourg, Glenn Seemann, AI for Game Developers. A. Perez-Uribe and E. Sanchez, "Blackjack as a Test Bed for Learning Strategies in Neural Networks", Proceedings of the IEEE International Joint Conference on Neural Networks IJCNN'98 (to appear

[6] a b Announcement [7]

Goddard

Engineering

Colloquium

Official 20Q Q&A

[8] Grahm, Ross., Pathfinding in Computer Games, in proceedings of ITB Journal, 2004 [9] Lubberts, Alex. Co-Evolving a Go-Playing Neural Network.The University of Texas at Austin, 2001. [10] 2 [11] [12] Mark A. DeLoura, Games Programming Gems www.tropicalcoder.com /Neural Network.htm www.eventhelix.com