Friday, January 25, 2008

Engine Chess (Book Hand Tuning I)



http://www.frayerchess.com/



Ok, you have a tournament book that is focused and plays a narrow variety of opening moves to a good depth. Now you need to hand tune it and keep it current. But what exactly are you tuning it to? The answer is; primarily your entire playing system. Which includes your hardware and its capabilities, your engine and its configuration settings, as well as the ever changing lines being played by your opponents.

There is no such thing as a perfect book that has all the answers to all the lines. A good opening book is a thing that is in constant flux. While in the human chess world an opening repertoire may serve a player for many years with only minor changes, it is not so in engine chess. Computers have a way of finding their way around seemingly impassable winning lines with frightening elegances and speed.

To get started; go on line and play 50-60 games at your desired time control. A hand tuned book is most certainly sensitive to time control changes. (by that I mean use one book for 3-0 play one for 16-0 and another for 60-0 + times) You may use the same book but with different tuning. Do not fiddle around with your engine configuration settings to much. (If you are using Rybka 2.3.2a the default settings are the strongest)

Now take the games that you have just played and go off line and into the engine GUI. Open up the My Internet Machine Games database file and do a bit of weeding. At first just go ahead and delete all the draws and games that go fewer than 35-40 moves. Now physically look at the games that you won. Games that you won from opponents that had inferior hardware to yours can be deleted. This should leave you with the games that you lost and a few that you won.

Take these games and get to work, the idea is to alter the opening book manually not only to avoid the loses but to find a winning line form before the point your evaluation number went negative. Make sure that you have the opening book that you wish to alter loaded and go to the book display. Right click the mouse on the book display window and activate allow move adding. Now any moves that you make manually while the engine is in infinite analysis mode will be added to the book. (as green play in tour moves)

If the book line went wrong before the engine engaged it may be necessary to mark some book moves red and try a deferent way. (book moves are turn red by right clicking on the moves and choosing the do not play in tournament option)

When the engine analysis mode is used (in most engines) it reverts to a brute force search. Where as in normal play all kinds of pruning tricks are used to reduce the number of moves the engine must look at. My point is that when going over played games in analyses mode you can often find strong moves that Rybka discarded in early pruning as being unproductive. It take some time and patients to get the search depth to an effective level. (time dependent on your hardware of course) When you are analyzing a game that you have lost, that had an average search depth of 16 ply you will probably need to go 19-20 ply in a brute force search starting several moves before the detectable swing in positional value to find a good refuting line.(more incomprehensible explanations on the dark art of book tuning early in the next week)

I noticed this week that some Mephisto computer ads appeared on this blog. I had one of these in the early 80s and was very found of it. It was not the model that moved the peaces on its own but was made of solid wood and was quite elegant, as well as a strong practice partner. Many of the fine products linked with this site may be worth checking out…

Wednesday, January 23, 2008

Engine Chess (Hand Tuned Book)







Book settings:

(Frayer Tour Book)
Use Book: on
Tournament Book: on
Varity of Play: max - (all the way to left)
Influence of learn Value: max + (all the way to right)
Learning Strength: max - (all the way to left)
Minimum games: 0
Up to move: 100

These are the settings for a hand tuned book. I recommend that you take the time to learn how to fix the losing lines (as they are reputed in online play) and reinforce the winning ones.
You can of course turn on the book learn function, but this only stops the book from playing a line that it has consistently lost in, it does not find a solution to the problem. In general it is my opinion that a good book with the learn function turned on will only become more drawlish.

With out a doubt the most useful aspect of book making, to the average engine player is hand tinning. Most of the commercial opening books (the books that come with engines and GUIs) can be tweaked into competitive shape by this technique.

What exactly is meant by the term hand tuning. Well to me its more than just turning some book moves red and others green. It is coming to the point in the book line where things started going wrong and doing deep analysis in infinite time mode to find a better line. Sometime one must go several moves back from the point where the positional value changed towards the negative to get back on track.

I will talk more about book development in next few weeks. Please feel free to share my Tour Book 14.0v (which by the way is made for 3-0 time controls) and send me an email if you have specific questions or seek calefaction of my thinking. kevin@frayerchess.com

Also, while you are here at my Blog check out the wonderful new advertisers that have taken an interest in supporting my writing. (I receive revenue every time you click on an advertisers site) Perhaps they can keep me in the volcanically grown Panamanian coffee that is required to work on this Blog.

Monday, January 21, 2008

Engine Chess (The Three Components)

As we separate the game of chess down into three parts (The opening, the middle game, and the end game) so too do I divide Engine Chess into three components. My thinking is, that to have a strong playing system, that is capable of sustaining a uniformly high Elo rating, detailed attention should be given to each of these aspects of engine chess.

1. Hardware: CPUs, RAM, Hard disk drives, Motherboards.

2. Software: UCI Engines, GUIs, Databases.

3. Books and EGTBs: Opening books, End game table bases.

The relative importance of these aspects seems to be in constant flux. (so I have not listed them here in order of weight) It also seems to be a highly debated point as engine players approach the game from different points of view. I am certain that players come from several different areas of expertise. Most notably Computers, Programming, and Chess. Some players enjoy seeing how there powerful computer hardware fairs in competition with other machines. Some like to use many different programs, tweaking them and in some cases fiddling with the code. Other come from the world of chess and are enticed by the strangely beautiful games produced by engine play. This I believe is the fulcrum that the fledgling sport can utilize to grow, its attraction to several different groups of enthusiasts.

Hardware

As they once put on old maps beyond the explored areas “past this point there be monsters” Have no illusions that your old desktop PC with a Pentium in it will be competitive in online engine play. (that’s not to say you can’t still have fun) However there are some real monsters out there. We are now in the era of 64bit multiple core CPUs. The current median standard at this time is Intel’s Quad core 64bit chips. Search depths with these processors even at blitz time control often exceed 21 half moves. Over clocking is common and is a dark art in its self. Some 16-32 core machines lurk in the shadows ready to eat your lunch or Elo as the case my be.

Engine hash size is no longer as dependent upon RAM as it once was. The modern chess engines make use of CPUs L2 catch. (which is much larger in the new CPUs) O no, maybe I should not have said that. (this seems to be one of Rybka little secrets) The L2 catch has traditionally been used for video processing the reason for this is its much faster for the program to save temporary data and retrieve it. The L2 runs at the full speed of the CPU and the RAM modules on the motherboard only a fraction of that speed.

In fact it may be that the latest release of Rybka 2.3.2a has its non-configurable L2 hash size set at 128Mb. How this size was arrived at and whether or not it is the optimum setting seems to be preparatory knowledge. (If this is just idle speculation on my part perhaps Vas will add a comment and clarify the issue)

As for Hard Disk Drives; What I do is keep one just for engine play online. The only things I put on it are a stripped down windows operating system, (I like XPpro 64) The GUI (I like Fritz 9 with latest update) a few UCI engines and all the EGTBs that I will be using in play. (I can just get it all on a 150Gb 10,000rpm Raptor) A fast HD does seem to aide EGTB accesses speed.

Software

The chess engine has reached a state of development as to be unfathomable to us ordinary mortals. Rybka 2.3.2a is by far the strongest commercially available engine at this time. There are some contenders especially in long time controls. Zappa Chess engine by Anthony Cozzie recently beat Rybka in an exhibition match in Mexico It is believed that the Zappa Mexico program used was better at its usage of more than 4 core CPUs. (This is alluding to the perceived problem that Rybka has in correct scaling above 4 cores)

Most of the GUIs are vary good; Fritz 9-10-11 are all compatible with UCI engine as are Shredder and Hiarcs.

ChessBase 9.0 is the premier database program available. Although some what expensive it is worth its price to the serious game collector. Its ability to manipulate game bases is awesome.
I feel that it is important to mention here that one should always buy these programs from the copyright holder. Not only does profiting form ones intellectual property act as an incentive for further improvement in the case of the engines you will want the authors to send you periodic updates.

Books and EGTBs

For me the opening book is the heart and soul of this kind of chess. I will be writing more in the weeks to come on my techniques and suggestions on how to make and develop books for chess engines. For right now let me just say that every one should try to make their own books. It gives your engine games a distinct caricature and unique stile that is a reflection of you own opening theories. There is much satisfaction to be had when your lines work out and much work to be done when they do not.

All the commercially available chess engines come with fairly comprehensive opening books. Although in most cases they are to broad and to shallow in scope. (The books that come with Rybka and Fritz seem to me to be intended to play against human opponents) It is relatively easy to tweak these already wide-ranging books into a more focused repertoire. This probably is where most players should start in the quest to take control of the stile that their engines will play. (Much more about opening books in later entries)

EGTBs are simply the game of chess worked out to its conclusion when only a few peaces are left on the board. All 3-4-5 and most 6 man bases are available some where on the net. However you must be aware that having all 3-4-5-6 man egtbs will require you to have at least 1.4Tb of storage space. No mater how fast your Hard Drives are this will prove to be to taxing on your system in fast time controls.

I recommend getting ChessBase Endgame Turbo 3 Nalimov Tablebases it comes on 9 DVDs and has all 3-4-5 man and a few 6 man. Once loaded onto your HD it will be about 42Gb in size. If configured properly in your GUI it will give you about +30 Elo in engine play. A list of the most commonly accruing 6 man end games can be found on the internet. If you feel brave and have the time and space you can use this list to download a further 100Gb of egtbs from the net. (This will take some time) 140-150Gb of the right bases can get you a +70 Elo boost in fast time controls.

Sunday, January 20, 2008

Computers in Chess (Engine Chess)

Just before the turn of the new century I began to concern myself exclusively with Engine Chess. By the term, (engine chess) I mean one chess program playing another chess program. The winboard GUI had facilitated this but it was not until the Arena GUI was made available to the public in 2001 that we had a graphic interface specifically designed to play one chess engine against another. Arena not only was compatible with the wiboard protocols but also the newer UCI engines. Everything was configurable; the time controls, the number of games in a match, the engine parameters and of course the opening books along with their weighted learning capabilities.

One could now take their computer hardware, chess engines and opining books onto the internet and play against other computers and their human handlers. Commercial chess programs, particularly Chessbase’s Fritz 7-8 came along with strong client based engines and very pleasing graphics. In September of 2001 Chessbase’s player site, Playchess.com opened a room specifically for engine play and competitive engine chess came into existences. An Elo rating system based on individual computers, programs and the way their users configured them now was possible rather than be based on how they did against human Masters, This made judgments of relative engine strength more viable and contributed to programming development.

Now I know what some are thinking that there had been WCCC tournaments held every few years since 1974. But in fact these events were and still are primarily for showcasing commercial programs and cutting edge computer hardware. In my world view, these elite events held in exotic locations do not constitute Engine Chess being played in public venues by ordinary individuals that we enjoy today.

The commercial programs quickly proved themselves to be as strong their advertising departments had clamed. Fritz, Shredder, Junior, Hiarcs and many others all proved that they could play hauntingly beautiful chess. In early 2006 Vasik Rajlich’s UCI engine Rybka (little fish) rose to the top and is currently the dominant chess program in the world.

One of the things I realized early about engine chess was that all things be equal, (both players having similar hardware and using the strongest program available) it is the opening book that made the difference. Not only does the opening book set the tone of the game but its manipulation is best way for the human operator to communicate with his program. Even if an engine has detailed configuration setting, (material value of the pieces, king safety, aggressive vs. defensive stile) these are esoteric and communicate with the program in a none human way. Only the engine book guided by a human intelligence can channel a chess engines positional decision making power to its fullest potential.

It is possible to move a huge mound of dirt by hand. Using shovels, wheelbarrows and human muscle. History is replete with such accomplishments. Today however we attack the mound with bulldozers and trucks making short work of it. It is no disrespect to the laborers that would have done this feat in the past. As we move forward in 21st century applying these computer programs, theses engines of the mind to our beloved game of chess we mean not to diminish the accomplishments of the great human players but only to advance the game , the art that is chess.