Skip to main content

Competition Objective

Submitted by thavidu on

Okay so i'm a little confused as to the actual objectives of this competition.. from what i've gathered, we're making a game and also programming the AI controlled units in the game. Of which there are to be over 1000+ AI controlled units.
But how are we we being judged? Is the goal of the competition to make a game that is better than everyone elses with smarter AI, or the goal to make the smartest AI you can make?
If the latter it doesn't really make sense since we're all making different games anyway.

Basically i'm just a little confused as to how we are going to be compared to everyone else in order to judge the winner? What does our game have to achieve?

Submitted by Helo on Mon, 06/10/08 - 3:31 PM Permalink

Taken straight from the challenge brief:

The Judges will be reviewing your work with the following criteria:

- Quality of code framework
- Code & data efficiency
- Levels of player interaction
- Unique & clever uses of AI

You will not be judged on the graphics quality of the work submitted.

Submitted by thavidu on Mon, 06/10/08 - 4:30 PM Permalink

- Quality of code framework
- Code & data efficiency

but how could you compare those two when the 'game's we make will be vastly different?

Submitted by Helo on Mon, 06/10/08 - 4:56 PM Permalink

Doesn't matter what you make, if the code is poor its poor.
Maybe you create a game that is the most fun to play but in the end if it doesn't score well in all the judging criteria then it won't win.

Submitted by thavidu on Mon, 06/10/08 - 5:29 PM Permalink

Ok first of all, i'm assuming people doing this competition are competent enough programmers to be able to code cleanly, so your statement about the code being poor just seems irrelevant to me.
The first two clauses of the judging criteria just don't seem to be things you can compare in this competition because they way you code it is unique to the product/game you're making. We haven't been told that this game is to support extensions or comply to standard X or standard Y or anything of the sort; thus the first criteria is the most generic criteria possible.
The second criteria i'm going to assume is related to resource usage, ie. CPU and memory. And also slightly more generally to say you won't need the 100% resource usage of a top-end computer just to run Snake.

The second two criteria seem to be related to gameplay, but it is not clear how this is going to be judged? Thats really what i'm trying to find out in this thread. Is player interaction measured by how fun it is to play? the UI? What players have the ability to do? Its just that the more I look at it its sounding like a Game design competition not an AI programming challenge.
And as for unique & clever uses of AI, what does this mean? Is it a measure of how intelligent the AI are in the game? Are we programming bots that play like humans or is AI here defined as how intelligent the units in an RTS game are at carrying out their tasks? its so broad! an extra-intelligent AI could have more knowledge about its environment than the human player and also be able to cheat...

The competition brief is quite vague isn't it? Making it more specific at this stage wouldn't be possible but i'd like to have some sort of a better idea as to what they want and how they will be judging it.
For example you could create a chess game with a chess player AI (which could be coded cleanly, efficiently, have a good UI and the AI opponent could be pretty smart), but you could also create a whacky game of your own design where you're shooting things in the most complex world ever, where the physics of every object accurately reflects science on earth and the AI is so smart it recursively knows the next 5 branched levels of possible game state and acts accordingly. But how would you compare the two?

If this is an AI programming challenge, how do you compare two AIs when the game they are being played on is not common?
From what i'm guessing, it seems like a Game Design competition with a specified focus in its AI components where we have to make a Fun (? does it have to be) game (how complex?) where there are 1000+ (quoted) AI controlled objects/units/whatever that doesn't take up excessive ammounts of resources (but what is considered excessive?)

Actually, regarding the 1000+ AI unit requirement, what does that entail? do a thousand units that move forward or left depending on the colour of the square they're standing on count as 1000+ AI units?

Sorry its so long, and I would really like to enter the comp but what are we measuring :(

Submitted by pmayes on Mon, 06/10/08 - 9:52 PM Permalink

I'm not going to comment on the design stuff, because I think one of the organisers could answer it better, but in regards to the quality of code it is quite easy to measure (in my opinion).

In my job I frequently look at code written by other programmers who are all 'competent enough' but their code is not always easy to follow, easy to debug, well commented or particularly well written! The code still works, obviously, but it can take a little while to figure out what it is doing (or trying to do!).

With that in mind, I think it makes sense to add it as a criteria to be judged against. If the reviewer can't look at your code and understand what you are doing then you aren't coding cleanly :)

Hope that helps.

Submitted by StephenWade on Tue, 07/10/08 - 1:35 AM Permalink

I know I will come across as a grumpy old man, but let's face it. I am one

Brief answer to a few of your inane quandaries

1. Does it have to be a game?

In the competition specification it says "player interaction". Player implies the verb play, and play means game. At least, this is my take on it.

2. Player interaction ?

Fun may not be the top priority. My initial understanding is that it would be likely that this is more about the AI having not only a level of reaction to player input, but some 'perceivable consequence' - probably making it both a design and programming consideration.

3. Unique/clever use of AI ?

Two things I can think of that might come into the judging process. First is that AI already has many established methods, although it might seem like a 'design challenge' it's also true that to apply these solutions to new problems requires thorough understanding of the behaviour of the method. Judging whether the AI 'appears' smart is how I am going to look at it.

I think, regarding say your chess example that I would think that an AI being 'successful' is not equivalent to an AI being 'smart'. Knowing more branches of the game than a human can see is a dull, boring, obvious method for AI.

4. How complex a game ?

I think this is only implicitly part of the judging criteria, I'm not sure. It's likely (although not necessarily true) that a simpler game will have simpler AI requirements, and hence may effect what you can actually achieve AI wise. Hence, affect your result.

5. What is considered excessive ?

My logic :

We are given that we will be able to use DX9. Most systems that support the features of DX9 are probably 2-3 years old at least. If it doesn't run in real time on that kind of system, it's probably excessive. I am fairly sure it'd be a bad idea to write for a target platform of a mobile phone.

Okay - I will now spend less time responding, and more time coding

I suggest we all follow suit :)

Submitted by WilliamDocherty on Tue, 14/10/08 - 6:04 PM Permalink

HI

As someone was already kind enough to point out, the submissions will be reviewed using these criteria:

- Quality of code framework
- Code & data efficiency
- Levels of player interaction
- Unique & clever uses of AI

Some additional notes:

Quality of code framework
- Has nothing to do with coding standards or coding style
- Has everything to do with the architecture & quality of implementation
- This criteria will reward solutions that are architecturally sound

Code & data efficiency
- Solving unique code challenges within reasonable CPU performance bounds & memory restrictions is an every day task when developing games It is very easy to solve many problems by ignoring real-time aspects & reasonable memory use.
- This judging criteria will reward solutions that recognise the importance of code & data efficiency

levels of player interaction
- In game development it's all about player interaction for the end user
- This criteria will reward solutions that show good use of player interaction

Unique & clever uses of AI
- In game development a unique/clever feature may become a USP for your title. Pushing the boundaries of what's being achieved before or implementing an old system well using a new technique.
- This criteria rewards solutions that offer something new

Regarding some of the other questions:
A solution that moves 1000 units forward and left based on the colour of the square they are standing on, will not fulfill the criteria outlined above, for example, being unique or clever.

Proving a solution that plays chess, would be clever and may score well, however it wouldn't be unique or demonstrate 1000 units.

The specification for the task is deliberately open to interpretation, but is closed enough to limit the solutions that can be presented.

Cheers
William Docherty
Tehcnical Director
THQ Studio Oz

Submitted by souri on Wed, 12/11/08 - 2:29 PM Permalink

I may have to extend the deadline to match the modeller challenge's finishing date. I'm going overseas and won't be back in the country until the 13th or 14th of December. So yeh, you may get an extra few days :o

Posted by thavidu on

Okay so i'm a little confused as to the actual objectives of this competition.. from what i've gathered, we're making a game and also programming the AI controlled units in the game. Of which there are to be over 1000+ AI controlled units.
But how are we we being judged? Is the goal of the competition to make a game that is better than everyone elses with smarter AI, or the goal to make the smartest AI you can make?
If the latter it doesn't really make sense since we're all making different games anyway.

Basically i'm just a little confused as to how we are going to be compared to everyone else in order to judge the winner? What does our game have to achieve?


Submitted by Helo on Mon, 06/10/08 - 3:31 PM Permalink

Taken straight from the challenge brief:

The Judges will be reviewing your work with the following criteria:

- Quality of code framework
- Code & data efficiency
- Levels of player interaction
- Unique & clever uses of AI

You will not be judged on the graphics quality of the work submitted.

Submitted by thavidu on Mon, 06/10/08 - 4:30 PM Permalink

- Quality of code framework
- Code & data efficiency

but how could you compare those two when the 'game's we make will be vastly different?

Submitted by Helo on Mon, 06/10/08 - 4:56 PM Permalink

Doesn't matter what you make, if the code is poor its poor.
Maybe you create a game that is the most fun to play but in the end if it doesn't score well in all the judging criteria then it won't win.

Submitted by thavidu on Mon, 06/10/08 - 5:29 PM Permalink

Ok first of all, i'm assuming people doing this competition are competent enough programmers to be able to code cleanly, so your statement about the code being poor just seems irrelevant to me.
The first two clauses of the judging criteria just don't seem to be things you can compare in this competition because they way you code it is unique to the product/game you're making. We haven't been told that this game is to support extensions or comply to standard X or standard Y or anything of the sort; thus the first criteria is the most generic criteria possible.
The second criteria i'm going to assume is related to resource usage, ie. CPU and memory. And also slightly more generally to say you won't need the 100% resource usage of a top-end computer just to run Snake.

The second two criteria seem to be related to gameplay, but it is not clear how this is going to be judged? Thats really what i'm trying to find out in this thread. Is player interaction measured by how fun it is to play? the UI? What players have the ability to do? Its just that the more I look at it its sounding like a Game design competition not an AI programming challenge.
And as for unique & clever uses of AI, what does this mean? Is it a measure of how intelligent the AI are in the game? Are we programming bots that play like humans or is AI here defined as how intelligent the units in an RTS game are at carrying out their tasks? its so broad! an extra-intelligent AI could have more knowledge about its environment than the human player and also be able to cheat...

The competition brief is quite vague isn't it? Making it more specific at this stage wouldn't be possible but i'd like to have some sort of a better idea as to what they want and how they will be judging it.
For example you could create a chess game with a chess player AI (which could be coded cleanly, efficiently, have a good UI and the AI opponent could be pretty smart), but you could also create a whacky game of your own design where you're shooting things in the most complex world ever, where the physics of every object accurately reflects science on earth and the AI is so smart it recursively knows the next 5 branched levels of possible game state and acts accordingly. But how would you compare the two?

If this is an AI programming challenge, how do you compare two AIs when the game they are being played on is not common?
From what i'm guessing, it seems like a Game Design competition with a specified focus in its AI components where we have to make a Fun (? does it have to be) game (how complex?) where there are 1000+ (quoted) AI controlled objects/units/whatever that doesn't take up excessive ammounts of resources (but what is considered excessive?)

Actually, regarding the 1000+ AI unit requirement, what does that entail? do a thousand units that move forward or left depending on the colour of the square they're standing on count as 1000+ AI units?

Sorry its so long, and I would really like to enter the comp but what are we measuring :(

Submitted by pmayes on Mon, 06/10/08 - 9:52 PM Permalink

I'm not going to comment on the design stuff, because I think one of the organisers could answer it better, but in regards to the quality of code it is quite easy to measure (in my opinion).

In my job I frequently look at code written by other programmers who are all 'competent enough' but their code is not always easy to follow, easy to debug, well commented or particularly well written! The code still works, obviously, but it can take a little while to figure out what it is doing (or trying to do!).

With that in mind, I think it makes sense to add it as a criteria to be judged against. If the reviewer can't look at your code and understand what you are doing then you aren't coding cleanly :)

Hope that helps.

Submitted by StephenWade on Tue, 07/10/08 - 1:35 AM Permalink

I know I will come across as a grumpy old man, but let's face it. I am one

Brief answer to a few of your inane quandaries

1. Does it have to be a game?

In the competition specification it says "player interaction". Player implies the verb play, and play means game. At least, this is my take on it.

2. Player interaction ?

Fun may not be the top priority. My initial understanding is that it would be likely that this is more about the AI having not only a level of reaction to player input, but some 'perceivable consequence' - probably making it both a design and programming consideration.

3. Unique/clever use of AI ?

Two things I can think of that might come into the judging process. First is that AI already has many established methods, although it might seem like a 'design challenge' it's also true that to apply these solutions to new problems requires thorough understanding of the behaviour of the method. Judging whether the AI 'appears' smart is how I am going to look at it.

I think, regarding say your chess example that I would think that an AI being 'successful' is not equivalent to an AI being 'smart'. Knowing more branches of the game than a human can see is a dull, boring, obvious method for AI.

4. How complex a game ?

I think this is only implicitly part of the judging criteria, I'm not sure. It's likely (although not necessarily true) that a simpler game will have simpler AI requirements, and hence may effect what you can actually achieve AI wise. Hence, affect your result.

5. What is considered excessive ?

My logic :

We are given that we will be able to use DX9. Most systems that support the features of DX9 are probably 2-3 years old at least. If it doesn't run in real time on that kind of system, it's probably excessive. I am fairly sure it'd be a bad idea to write for a target platform of a mobile phone.

Okay - I will now spend less time responding, and more time coding

I suggest we all follow suit :)

Submitted by WilliamDocherty on Tue, 14/10/08 - 6:04 PM Permalink

HI

As someone was already kind enough to point out, the submissions will be reviewed using these criteria:

- Quality of code framework
- Code & data efficiency
- Levels of player interaction
- Unique & clever uses of AI

Some additional notes:

Quality of code framework
- Has nothing to do with coding standards or coding style
- Has everything to do with the architecture & quality of implementation
- This criteria will reward solutions that are architecturally sound

Code & data efficiency
- Solving unique code challenges within reasonable CPU performance bounds & memory restrictions is an every day task when developing games It is very easy to solve many problems by ignoring real-time aspects & reasonable memory use.
- This judging criteria will reward solutions that recognise the importance of code & data efficiency

levels of player interaction
- In game development it's all about player interaction for the end user
- This criteria will reward solutions that show good use of player interaction

Unique & clever uses of AI
- In game development a unique/clever feature may become a USP for your title. Pushing the boundaries of what's being achieved before or implementing an old system well using a new technique.
- This criteria rewards solutions that offer something new

Regarding some of the other questions:
A solution that moves 1000 units forward and left based on the colour of the square they are standing on, will not fulfill the criteria outlined above, for example, being unique or clever.

Proving a solution that plays chess, would be clever and may score well, however it wouldn't be unique or demonstrate 1000 units.

The specification for the task is deliberately open to interpretation, but is closed enough to limit the solutions that can be presented.

Cheers
William Docherty
Tehcnical Director
THQ Studio Oz

Submitted by souri on Wed, 12/11/08 - 2:29 PM Permalink

I may have to extend the deadline to match the modeller challenge's finishing date. I'm going overseas and won't be back in the country until the 13th or 14th of December. So yeh, you may get an extra few days :o