Woot

Finished


- Posted from iPhone.

Debugging

Spent the day debugging my neural net code and fixed some critical bugs where the list of neurons in each layer were going out of the scope of the back propagation loop giving me bogus deltas. The bug had a knock on effect that would offset the result of the entire neural net. I plan to be reading training data from a text file by the end of a week so that I can actually use the training data I'm pulling from the quake game engine. Then it's just a matter of putting it in to the actual game engine itself and everything should be finished.

Events

Been watching tv for the last 3 hours while my project collects training data. I wonder how many events it has recorded.


- Posted from iPhone.

ioQuake3 compiles

I spend the guts of today trying to get ioquake3 to compile and I've finally managed to get it working. The Microsoft visual studio solution will compile now creating the different game engine components. I've had to write a batch file that will copy the dlls and the exe to the necessary folders so that it will run.

I also wrote my very first quake3 mod today. It's a simple modification that prints "Thinking Bitch" to the console every time a bot thinks.

The batch file is as follows:

@echo OFF

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\cgame_release\cgamex86.dll" "C:\Program Files (x86)\ioquake3\missionpack\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\q3_ui_release\uix86.dll" "C:\Program Files (x86)\ioquake3\missionpack\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\game_release\qagamex86.dll" "C:\Program Files (x86)\ioquake3\missionpack\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\quake3_release\ioquake3.exe" "C:\Program Files (x86)\ioquake3\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\cgame_release\cgamex86.dll" "C:\Users\Andrew\AppData\Roaming\Quake3\baseq3\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\q3_ui_release\uix86.dll" "C:\Users\Andrew\AppData\Roaming\Quake3\baseq3\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\game_release\qagamex86.dll" "C:\Users\Andrew\AppData\Roaming\Quake3\baseq3\"

"C:\Program Files (x86)\ioquake3\ioquake3.exe"

@echo ON

Wish I could use inheritance -_-

Working on my Bayesian graph and having problems with writing the structures.

Code is something like this:

typedef struct _b_Node

{

//Identifier, will be an action

int id;

//Linked list infrastructure

struct _b_Node *next;

b_NodeList *connections;

//Data

float *probs;

} b_Node;

typedef struct _b_NodeList

{

b_Node *head;

b_Node *tail;

int size;

} b_NodeList;

typedef struct _b_Graph

{

b_NodeList * nodes;

} b_Graph;

Problem is that when the preprocessor goes to compile the header file, the list of pointers to nodes hasn't been defined yet. Where this problem could be avoided completely in an OOP language with inheritance, there's no way to order these. A solution might be using a void pointer to a list instead. I'm not entirely how efficient casting pointers is though, or how much extra work I'm going to have to do to ensure it is robust at the memory level.

Heh, Ulster University

Ulster University are offering phds in AI where they are using reinforcement learning to make diverse adaptive characters in games. That is essintially sums up my final year project.

http://www.compeng.ulster.ac.uk/rgs/displayPhDProposal.php?id=264&ri=4

The closest thing to UML that I've got

After some careful considerations I have reviewed my high level design. I think I was looking at things the wrong way. Instead of the Bayesian network classifying actions to take given events, that will be the neural networks job. the Bayesian network's purpose is to recognize events.

Events will trigger queries to compute the probability of actions a bot might take, and they will be fed to the neural net, alongside botstate data from the bot using the neural net to make decisions, and each output neurons of the neural network should classify the best action to take.

This approach should make taking training data from the game engine a lot easier, however in order to get the neural net working the way it should I need to have a fully functional Bayesian Net.

The actions and variables I've decided to compute the probability for in the Bayesian net are as follows:

Probable Actions:

Seeking Health

Seeking Armor

Seeking Ammo

Seeking Enemy

Running Away

Variables

Health

Ammo

Armor

Extracting training data from quake

I'm writing a fiction that will log data about a bots state given an event that can be used as training data. Going to maybe run a couple of thousand games and log as much of it as possible so I have a large training set.

I'm working with the actual game engine now.

- Posted from iPhone.

Variables and states in a Bayesian net

I'm trying to decide on which states of player behavior I want to model the probabilities for in quake.

A few variables off the top of my head would be health level, armor level, ammo level, weapons used. Behaviors might be seek health, seek armor, seek ammo, seek ammo, seek weapon, run away, camp.

Each of the variables would be nodes on a graph, and the behaviors might be nodes on the graph also that are conditionally dependant on the variables.
- Posted from iPhone.

Neural Net Component almost finished, Emphasis on Bayesian Net.

Things I have left to do is finish up making the function calls to construct the net scalable, reading in properties from a config file. I also need to finish the function call that loads training patterns from another file. Last of all I need to debugging the neural net back propagation algorithm, and write a component that acts as a medium between the ANN and the Bayesian network.

In terms of implementing the Bayesian net, I'm currently working on the topology of the Bayesian net and constructing it. An idea would be to read this from the same config file the neural net reads from. Ideally all of the code should be re-usable so that Bayesian nets can be constructed use that is not constricted to the goal I have in mind for this project.

New Problem

Having a new problem with my training algorithm where the loop that's supposed to loop through all of the training patterns isn't doing so for some reason. It's a really strange one. I'm using the visual debugger in Visual Studio and for some reason the index of the for loop is always at the last index and I've verified this with printf statements inside the loop. In my test case the loop should execute the code inside 7 times and it's only being executed once.

I'm pretty sure its visual studio doing something at compile time, possibly some kind of auto optimization because of the number of loops in the function or something, but that's just a wild guess. Going to investigate further tomorrow. It's annoying as hell though.

Calculating target output / input delta

Seems I'm having a problem calculating the error amount which is a simple sum squared calculation, and it's because the outputs of the neural net are exceeding 1.0f which means there's something wrong with the flow of data in the net.

TODO: Weekend plan

Start writting up design chapter of thesis and write up on implementation so far.


- Posted from iPhone.

Training and Bayesian net topology

Got the first part of my training algorithm implemented today. Looking at reading information about the topology of the net from a config file. This might include variables such as the number of layers, number of neurons in each layer, paths to training data files and net storage XML files. I need to ensure that my current functions for constructing the net allow for scaling the number of layers which might mean a modification of the net structure.

I plan to finish the back propogation training algorithm this week and get stuck in to the Bayesian net code asap.

ideas

Today I had an interesting chat with someone who did a project which used ANNs who faced a similar problem as one of the problems I face, where there are many librarys and frameworks for using ANNs which use a different programming language to the one the project was using. In his case he solved the problem using an XML layer between the component feeding the ANN (C#) and the ANN itself (Java), where as the approach I've taken is programming the ANN from first principals myself. The argument in support of my approach is that I can optamise my implementation for use with quake reducing the bulkyness you might get with using a framework or api. The counter point I suppose is the time consumed implementing an ANN from scratch.

Net activations

Currently working on an algorithm for iterativly running through a layer of neurons calculating the output for each.

Stuff to fix

Currently to seed the neural network with data or weights I'm using the function to generate ID's for the layers, which due to the structure of how I initialise each layer means that layer id's vary by 11 units.

The XML file's generated can be parsed, however I need to correct some of the indentation, so it will be able to be read properly.

Currently the XML looks like this:

XML woot woot!

Today I've been working on an algorithm to generate xml files of a neural network for storing the data in the neural network to persist through game sessions.

It's almost finished. I just need to write the operations to read from the xml file to reconstruct the neural net.

Implementation Progress.

Figured I'd give a little update with regards to my progression in the implementation.

I'm currently working on the implementation of the neural net. The following is from the comments of the function which will initialize a layer of the net.

This function will initialise a layer and return a pointer to it. To initialise a layer, the following must be initialized.

Neural List structure variables:

-Number of Neurons

-Head Neuron of list

-Tail Neuron of list

Layer structure variables:

-ID for the layer, simple index

-List of Neurons in the layer(neural list as above)

-Reference to Input layer for the layer being initialized.

-Reference to Output layer for the layer being initialized.

Memory is assigned dynamically and the network is constructed. A layer seeding function will then be called to seed random values in to the weights of the net.

Update on Neural Net implimentation

I've been working on building a neural network with McCulloch and Pitts model neurons (MCP). Each neuron has a set of inputs, a set of weights associated with each input and a threshold. If the sum of the weight and the input when normalized is above the threshold of the neuron, that neuron is said to have fired.