Automatic test run for bots

AI discussion, ideas, and SDK help.
Post Reply
inonzuk
Luxer
Posts: 20
Joined: Thu Apr 19, 2007 7:36 am

Automatic test run for bots

Post by inonzuk » Mon Jun 11, 2007 2:45 am

Hello there,

I wrote a new bot and I want to test it in a large test run (something like 1000 trails against other bots).

How can I do it automaticaly?

Thanks again,
Inon

User avatar
rip
Luxer
Posts: 352
Joined: Wed Mar 31, 2004 10:22 am
Location: Vienna Austria
Contact:

Trialing

Post by rip » Mon Jun 11, 2007 3:21 am

It would help if we knew under what OS and IDE you did your development.

For MacOS X.3+, using XCode,

Create a new custom executable (in the project pane, ctrl+click on "Executables" and select "New Custom Executable" from the context Menu.

Change the name to something like "Superfast"
For the path, navigate to your Lux executable. Select it. Change the path from .../Lux to .../Lux.app/Contents/MacOS/LuxDelux and "Finish"

In the Info window, Arguments, +, and type "-superfast". Ensure that it is checked in the Arguments window.

In the Info window, Debugging, turn off "Break on Debugger() and DbgString()".

To play the same map over and over and over, open the Finder and navigate to ~/Library/Application Support/Lux/Boards/Saved. Create a sub directory ("bottest" for example). Copy the map(s) you want to play into that directory.

That's that. Now set the active executable to Superfast, and execute it.
When Lux pops up, change all players to bots, answer "No" to the pop up this causes. In the preferences, set "autostart next game". Select any map from the map dropdown that is in bottest (they are clearly shown).

Play. Go outside and smell the roses. How many games your bots will play per hour is indeterminate (for example, if I play Bug with a Reaper and a BotOfDoom, four games out of seven I end up with a static map and 1x10^6 armies per the three remaining players, and only 150 get played in 8 hours. If I play Bug against five "hard" bots, I will get through about 1000 games in that time).

Some of the above is generic (bottest directory, Lux preferences, etc), but starting Lux with -superfast will depend on your set up.

HTH,
rip
Like I said, we need more info to get you

inonzuk
Luxer
Posts: 20
Joined: Thu Apr 19, 2007 7:36 am

Post by inonzuk » Mon Jun 11, 2007 3:40 am

Hi rip,

I'm working with Windows, and Eclipse as my IDE.

Inon

User avatar
dustin
Lux Creator
Lux Creator
Posts: 10498
Joined: Thu May 15, 2003 2:01 am
Location: Cascadia
Contact:

Post by dustin » Mon Jun 11, 2007 4:17 am

The simplest way:

Open the Lux preferences. Set the bot speed to the fastest. Go to the Game Options tab and select "Auto-start next game". Save the prefs and close. (You probably also want to set Explosions to None, turn off Flash bonus takeovers, and turn off sounds.)

Then set all the players to be bots and start a game. It will play game-after-game. Stop it whenever you want. Use the local Record Book (under the File menu) to check which bots won how many games.

inonzuk
Luxer
Posts: 20
Joined: Thu Apr 19, 2007 7:36 am

Post by inonzuk » Mon Jun 11, 2007 4:20 am

Thanks Dustin and rip,

I'll give it a try.

User avatar
dustin
Lux Creator
Lux Creator
Posts: 10498
Joined: Thu May 15, 2003 2:01 am
Location: Cascadia
Contact:

Post by dustin » Mon Jun 11, 2007 4:37 am

Another way (that will go lots faster) is to start Lux from the command line and give it the -superfast option. The command on Windows would be like:

Code: Select all

java -jar LuxCore.jar -superfast
This will turn off many of the graphics, so you can run a ton of games fast.

User avatar
Bertrand
Reaper Creator
Posts: 568
Joined: Mon Nov 28, 2005 4:35 pm
Location: Montreal

Re: Trialing

Post by Bertrand » Mon Jun 11, 2007 5:55 am

rip wrote:for example, if I play Bug with a Reaper and a BotOfDoom, four games out of seven I end up with a static map and 1x10^6 armies per the three remaining players, and only 150 get played in 8 hours.
What settings are you using? I test Reaper this way with botOfDoom and deadlocked games never happen.
inonzuk wrote:I wrote a new bot and I want to test it in a large test run (something like 1000 trails against other bots).
On more trick, if you use Dustin's method without the -superfast option: you can start multiple games with ctrl-N. I often run 4 at the same time.

inonzuk
Luxer
Posts: 20
Joined: Thu Apr 19, 2007 7:36 am

Post by inonzuk » Mon Jun 11, 2007 6:00 am

I'm running it on an quite old computer (1.3 Gh), and my agent is working recursivly, so it takes him a long time to make a turn.

User avatar
rip
Luxer
Posts: 352
Joined: Wed Mar 31, 2004 10:22 am
Location: Vienna Austria
Contact:

Re: Trialing

Post by rip » Tue Jun 12, 2007 11:44 am

Bertrand wrote:
rip wrote:for example, if I play Bug with a Reaper and a BotOfDoom, four games out of seven I end up with a static map and 1x10^6 armies per the three remaining players, and only 150 get played in 8 hours.
What settings are you using? I test Reaper this way with botOfDoom and deadlocked games never happen.
If you want to know what settings Reaper and BoD are working under, that would be "Classic Risk, 456..., static bonuses" and "bog standard, as delivered". As for Bug...That would be telling. And what I /failed/ to mention is that when Bug maneuvers Reaper and BoD into that deadlock, Bug wins.

I will say that Bug has /no/ code which is enemy agent specific, ie she does not look for "Reaper", "BotOfDoom", or "Human". She just play the game. In games with Bug, Reaper and four others, or Bug, BotOfDoom and four others, Bug wins about two thirds of the games that Reaper/BotOfDoom don't take. So she isn't yet into that league.

But I'm working on it.

As for four games at a time, my kit is rather dated. Once I'm the next
Internet Millionaire, I'll have enough money to buy something modern...

rip

User avatar
Bertrand
Reaper Creator
Posts: 568
Joined: Mon Nov 28, 2005 4:35 pm
Location: Montreal

Re: Trialing

Post by Bertrand » Tue Jun 12, 2007 7:41 pm

rip wrote:"Classic Risk, 456..., static bonuses"
OK now I see.... Low income games often end up deadlocked.

On the classic map I mostly use 4-6-8-10-15 10% to test Reaper. Those are, I think, the most popular settings. On large maps I use 4-6-8 5%.

Post Reply