Debugging

Spent the day debugging my neural net code and fixed some critical bugs where the list of neurons in each layer were going out of the scope of the back propagation loop giving me bogus deltas. The bug had a knock on effect that would offset the result of the entire neural net. I plan to be reading training data from a text file by the end of a week so that I can actually use the training data I'm pulling from the quake game engine. Then it's just a matter of putting it in to the actual game engine itself and everything should be finished.

Events

Been watching tv for the last 3 hours while my project collects training data. I wonder how many events it has recorded.


- Posted from iPhone.

ioQuake3 compiles

I spend the guts of today trying to get ioquake3 to compile and I've finally managed to get it working. The Microsoft visual studio solution will compile now creating the different game engine components. I've had to write a batch file that will copy the dlls and the exe to the necessary folders so that it will run.

I also wrote my very first quake3 mod today. It's a simple modification that prints "Thinking Bitch" to the console every time a bot thinks.

The batch file is as follows:

@echo OFF

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\cgame_release\cgamex86.dll" "C:\Program Files (x86)\ioquake3\missionpack\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\q3_ui_release\uix86.dll" "C:\Program Files (x86)\ioquake3\missionpack\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\game_release\qagamex86.dll" "C:\Program Files (x86)\ioquake3\missionpack\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\quake3_release\ioquake3.exe" "C:\Program Files (x86)\ioquake3\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\cgame_release\cgamex86.dll" "C:\Users\Andrew\AppData\Roaming\Quake3\baseq3\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\q3_ui_release\uix86.dll" "C:\Users\Andrew\AppData\Roaming\Quake3\baseq3\"

copy "C:\Users\Andrew\Documents\College\DT228 4\FYP\quake3\build\game_release\qagamex86.dll" "C:\Users\Andrew\AppData\Roaming\Quake3\baseq3\"

"C:\Program Files (x86)\ioquake3\ioquake3.exe"

@echo ON

Wish I could use inheritance -_-

Working on my Bayesian graph and having problems with writing the structures.

Code is something like this:

typedef struct _b_Node

{

//Identifier, will be an action

int id;

//Linked list infrastructure

struct _b_Node *next;

b_NodeList *connections;

//Data

float *probs;

} b_Node;

typedef struct _b_NodeList

{

b_Node *head;

b_Node *tail;

int size;

} b_NodeList;

typedef struct _b_Graph

{

b_NodeList * nodes;

} b_Graph;

Problem is that when the preprocessor goes to compile the header file, the list of pointers to nodes hasn't been defined yet. Where this problem could be avoided completely in an OOP language with inheritance, there's no way to order these. A solution might be using a void pointer to a list instead. I'm not entirely how efficient casting pointers is though, or how much extra work I'm going to have to do to ensure it is robust at the memory level.

Heh, Ulster University

Ulster University are offering phds in AI where they are using reinforcement learning to make diverse adaptive characters in games. That is essintially sums up my final year project.

http://www.compeng.ulster.ac.uk/rgs/displayPhDProposal.php?id=264&ri=4

The closest thing to UML that I've got

After some careful considerations I have reviewed my high level design. I think I was looking at things the wrong way. Instead of the Bayesian network classifying actions to take given events, that will be the neural networks job. the Bayesian network's purpose is to recognize events.

Events will trigger queries to compute the probability of actions a bot might take, and they will be fed to the neural net, alongside botstate data from the bot using the neural net to make decisions, and each output neurons of the neural network should classify the best action to take.

This approach should make taking training data from the game engine a lot easier, however in order to get the neural net working the way it should I need to have a fully functional Bayesian Net.

The actions and variables I've decided to compute the probability for in the Bayesian net are as follows:

Probable Actions:

Seeking Health

Seeking Armor

Seeking Ammo

Seeking Enemy

Running Away

Variables

Health

Ammo

Armor

Extracting training data from quake

I'm writing a fiction that will log data about a bots state given an event that can be used as training data. Going to maybe run a couple of thousand games and log as much of it as possible so I have a large training set.

I'm working with the actual game engine now.

- Posted from iPhone.

Variables and states in a Bayesian net

I'm trying to decide on which states of player behavior I want to model the probabilities for in quake.

A few variables off the top of my head would be health level, armor level, ammo level, weapons used. Behaviors might be seek health, seek armor, seek ammo, seek ammo, seek weapon, run away, camp.

Each of the variables would be nodes on a graph, and the behaviors might be nodes on the graph also that are conditionally dependant on the variables.
- Posted from iPhone.

Neural Net Component almost finished, Emphasis on Bayesian Net.

Things I have left to do is finish up making the function calls to construct the net scalable, reading in properties from a config file. I also need to finish the function call that loads training patterns from another file. Last of all I need to debugging the neural net back propagation algorithm, and write a component that acts as a medium between the ANN and the Bayesian network.

In terms of implementing the Bayesian net, I'm currently working on the topology of the Bayesian net and constructing it. An idea would be to read this from the same config file the neural net reads from. Ideally all of the code should be re-usable so that Bayesian nets can be constructed use that is not constricted to the goal I have in mind for this project.

New Problem

Having a new problem with my training algorithm where the loop that's supposed to loop through all of the training patterns isn't doing so for some reason. It's a really strange one. I'm using the visual debugger in Visual Studio and for some reason the index of the for loop is always at the last index and I've verified this with printf statements inside the loop. In my test case the loop should execute the code inside 7 times and it's only being executed once.

I'm pretty sure its visual studio doing something at compile time, possibly some kind of auto optimization because of the number of loops in the function or something, but that's just a wild guess. Going to investigate further tomorrow. It's annoying as hell though.

Calculating target output / input delta

Seems I'm having a problem calculating the error amount which is a simple sum squared calculation, and it's because the outputs of the neural net are exceeding 1.0f which means there's something wrong with the flow of data in the net.

TODO: Weekend plan

Start writting up design chapter of thesis and write up on implementation so far.


- Posted from iPhone.

Training and Bayesian net topology

Got the first part of my training algorithm implemented today. Looking at reading information about the topology of the net from a config file. This might include variables such as the number of layers, number of neurons in each layer, paths to training data files and net storage XML files. I need to ensure that my current functions for constructing the net allow for scaling the number of layers which might mean a modification of the net structure.

I plan to finish the back propogation training algorithm this week and get stuck in to the Bayesian net code asap.