Creating Neural Networks for games

Artificial Intelligence is an important component for modern games. A particularly powerful concept in AI is machine learning, and Neural Networks are one method of many machine learning techniques.

The project I created in order to demonstrate a neural network within a game is L-Robot. You can see a video I created that demonstrates L-Robot below:

How does it work?

Neural networks are a technique generally used for pattern recognition. A large neural network can compute huge amounts of data and are proficient in finding solutions for problems such as character recognition, image recognition or even stock market predictions. But how can we use neural networks in games?

The neural network implemented into L-Robot is his brain, which controls how L-Robot moves. The network takes in two inputs and processes these inputs through a number of layers of neurons to produce an output.

The inputs for the network are:

  • The current fitness
  • The position of L-Robot relative to the closest platform

These numbers are processed to output these values:

  • A number to determine whether L-Robot should move left or right
  • A number to determine whether L-Robot should jump

The network is processed every frame and will run for 150 frames. If a solution is not found within this time, then a genetic algorithm is used to evolve the network.

Genetic Algorithm

The genetic algorithm within the project is used to evolve the weights of the neural network. A generation of thirty genes is created at the start of the program, each gene representing a different brain. After a generation has completed its run through the game, the genetic algorithm is executed.

The genetic algorithm uses roulette-wheel selection to create a new population of genes. Roulette-wheel selection consists of creating a mating pool of copies of each gene. The number of copies of each gene correlate directly to the fitness value of the gene – i.e. if a gene has 60% fitness, then 60 copies are pushed onto the mating pool – this promotes elitism, meaning the next population has a good chance of being more successful than the previous.

After a mating pool is created, random genes are selected to crossbreed a new child gene. These new children are then used to create a new population of genes to be given to the neural network. This is how the neural network is adapted and how L-Robot learns.

If you would like more information, leave a comment or email me at:

John_comrie@hotmail.co.uk

Multi-threading in C++

To demonstrate the use of multi-threading in programs, here is a problem:

Problem:

Customers enter a bank and need to be served by one of three bankers. Each banker has a queue of customers who they need to serve.

A program that does not make use of threads would not be able to make use of all the bankers at the same time. However, a program that makes use of threads would implement at least 1 thread per banker, and therefore allowing the bankers to operate simultaneously.

To solve the problem, I have created a program that starts by asking the user to insert the amount of customers to be generated. The customers could take a random time to generate, but for testing purposes the time taken was set to a static number of 800ms.

threads

When a customer is generated, it is added to the queue with the least
amount of customers. Each banker takes 200ms – 400ms to tend to each customer, and will tend to the next customer in its queue immediately after.

At the start of the program, the user can also enter the amount of threads to be used to generate the customers. The program itself creates one thread for each banker – meaning depending on the machine being used, increasing the amount of threads for the customer generation could have little to no impact. Using an 8 core machine, the increase in performance plateaued at around 5-6 threads as expected.

Data Collected:

statsgraph

 

Here is a gif of the program running using 40 customers and 6 threads:

ezgif-com-195ccf0f36

How it works:

To safely generate customers using multithreading, a channel class is used. The channel has a read and write function which both use a number of mutexes and a semaphore to operate safely.

The channel will ‘write’ a customer to a specific queue after waiting the static amount of time it takes to generate a customer.

The channel will ‘read’ the data when a customer is sent to the next free banker.

The semaphore inside the channel class signals when a customer has been ‘written’, and will ‘wait’ when the banker is free.

A mutex is also used for the print function to lock the cout function. This will avoid multiple threads trying to print at the same time.

The program decreases in performance when the number of threads is increased to a significant number such as 10,000 – creating a huge overhead of threads being created while the program is running.

Network programming in Java for Android

There are a number of ways to demonstrate networking features for mobile devices. Almost everyone nowadays has a smartphone that has access to the internet – allowing for developers to utilise these features to create interesting games and useful apps.

In my 3rd year of university I created an android game which makes use of several different networking features:

  • Local databases
  • Remote databases
  • Location services

Overview

The game itself is a simple clone of doodle jump. The game uses the accelerometer in the device to calculate the orientation of the device – this is how the player controls the game. Here is a short gif of the game being played:

ezgif-com-35b65d6b8b

Databases

When you play the game, your score is always automatically added to a database local to the device- This is the local highscores. From this point, you have a choice to add it to a remote database which is the global highscores.

To test the remote database, I used wampserver which hosts a server on the computer it is running on. From here, I connected my phone to the network and accessed the database. To communicate to the server, the app uses PHP files to retrieve and alter the database files.

Location services

To demonstrate the use of location services for a mobile device, the longitude and latitude co-ordinates of the current location is sent along with the players score to the database. Although this is not the ideal use for location services, it is used solely as a demonstration of the implementation.

After a location has been sent to the database, the player can access the database and click on any of the scores. When the player clicks on the score, the co-ordinates are sent to a maps activity, which shows the location on google maps.

DirectX11 Programming with Shaders

In order to demonstrate a number of different hlsl shaders in DirectX11, I created a 3D scene using a framework I was given by a lecturer in my 3rd year of university.

Aspects of the scene:

  • Basic vertex manipulation – Vertex shaders.
  • Shadows using shadow mapping – Depth shader, shadow shader.
  • Height map that makes use of tessellation – Hull shader, domain shader, pixel shader.
  • Gaussian blur post-processing – horizontal and vertical blur shaders.

Basic vertex manipulation shader:

To create a basic simulation of flowing water, a large plane can be passed through a vertex manipulation shader that makes use of a small formula that manipulates the Y co-ordinate of each vertex based on a cosine wave. The cosine formula is added to a time variable which gives movement to the wave. On top of this, a water texture is added to the plane. The final product is shown in the following gif:

ezgif-com-c9465e262a

 

Shadow Mapping:

To simulate shadows within the scene, a flat plane and a model are passed through a depth shader which calculates the depth of each pixel from the point of view of the light that casts the shadow.

The depth texture is given to the shadow shaders, along with the texture of each object and the position of the light. The resulting shadow map is then given to the pixel shader which determines that anything in the shadow map cannot be in shadow – thus the resulting pixels are lit – leaving the rest of the scene in shadow.

capture

 

Tessellated Height Map:

Tessellation is used in games to create very highly detailed textures. As an example, I passed a height map through a tessellator in order to show the difference that tessellation can make.

To begin with, the vertex shader receives and processes the control points, ready to send to the hull shader. The constant hull shader receives all of the control points and makes use of the tessellation factor that is passed in. It will then output the tessellation factor for each edge and interior of the patch. The control point hull shader sets up each control point for the tessellator. The tessellator now only has to subdivide each patch based on the tessellation factors that were calculated in the hull shader stage. The new vertices are then passed onto the domain shader stage.

The domain shader will act as a vertex shader and calculate the vertex manipulation based on a height map texture that is passed in. Using the tessellated vertices, the domain shader makes use of linear interpolation to calculate the position of the vertices, texture co-ordinates and normals of the plane. The texture co-ordinates are then used to increase the height of the Y vertices, based on the colour of the heightmap texture.

After the vertex manipulation, the pixel shader applies light calculations using a texture and two separate lights.

Single Tessellation:

capture2

8x Tessellation

Capture3.PNG

 

Post Processing – Gaussian Blur

Post-processing in general can be quite taxing on the performance of any program. To alleviate this slightly, the process of gaussian blur in this program down-samples the scene texture before blurring and up-samples the scene texture again after blurring.

The blurring process is also split into two separate shaders – horizontal blurring and vertical blurring. This significantly cuts down the amount of samples per pixel compared to if the process was done all in one.

The scene before and after the gaussian blur is applied:

Augmented Reality for the PlayStation Vita

For a module in my 4th year of university, one of my assignments involved creating an application that makes use of a technology and attempt to innovate.

The technology I chose to base my application around was augmented reality. Using the PlayStation Vita development kits that the university provides, I created a prototype for a multiplayer cat-and-mouse game.

The game makes use of the PS vita camera and the Sony marker tracking framework. The markers are used as reference points to simulate the 3D game in the real world.

Below is a short gif of the prototype:

giphy

The objective of the game is to either achieve 30 points by standing on your house, or to bait the cat into chasing and catching the other player.