How I Built a Working Poker Bot
Introduction
Several years ago, a client asked me to come up with a prototype for a real-money online poker bot. That's right: a piece of software you park on your computer while it goes out to a site like PokerStars or Full Tilt and plays no-limit Holdem for you, at 4 or 14 different tables, for real-money stakes.
If you're a poker player, and particularly if you're an online poker player, you've probably heard rumors about the rise of the poker bots. Unfortunately there's very little hard information out there (for obvious reasons) about how to build one of these bots. In fact, many so-called authorities still dismiss poker bots as a relic of the overactive poker player's imagination.
Well, I'm here to tell you that online poker bots are 100% real, and I know this because I've built one. And if I can build one, well. Anybody can build one. What's more, over the course of this multi-part article, I'll show you how. But first, a teaser (click on the image for a larger version):
That, ladies and gents, is a picture of a full-featured poker bot managing three play-money tables (note: this same bot also handles real-money tables) at an honest-to-goodness, real-money online poker site. Of course, it could be any site. The bot implementation I'm going to reveal will work at all major online poker sites, including Poker Stars, Full Tilt, Party Poker, Ultimate Bet, and most other major venues.
Why are you giving this information out?
I debated for a long time whether or not to make this information public, as I'm a poker player myself and have no desire to see the game ruined by an avalanche of poker bots. It's not that building a poker bot is some sort of black magic, known only to the privileged few. Any competent programmer can build one. But this information hasn't, so far as I know, been collected and presented in one place, certainly not as a "How To" complete with sample code. So the question I struggled with was this: is it irresponsible to publicize this information, such that every Internet script kiddie out there now has the ammunition he needs to actually build a bot?
After thinking about it, I've decided that keeping the technology of poker bot building secret is like declaring that only criminals can carry handguns. The fact is, there are people in the world right now who are doing this:
Poker bots, underground online poker boiler rooms, and collusion are a reality. That doesn't mean online poker's not worth playing, just that it pays to be educated about what's possible. Furthermore, there should be public discussion regarding what to do about it because one thing's certain: computers and programming languages aren't exactly going to be getting less powerful. The rise of the poker bots is a virtual certainty. I'd like to see the major online poker venues open up their famously vague "bot detection" and "anti-collusion" strategies to public scrutiny, as cryptography and security providers learned to do years ago. The best security algorithms and techniques all have the weight of public review behind them and I don't see how online poker's any different.
But even assuming all that weren't the case:
- Poker bots already exist on the open market. Do a little creative Internet searching.
- The poker community suffers from an irrational fear of bots. I'd gladly risk my money against most homegrown bots and trust me: you would too.
- I believe that bots are actually good for the game of poker. Mike Caro, "the Mad Genius of Poker," expressed a similar idea years ago.
- Any programmer worth his salt can build a bot with or without this document. They already have.
If you're visiting this page from 2 + 2 or another poker community, and you want to stay on top of this article (which will be in several parts), you can subscribe to the Coding the Wheel RSS feed or get it in your email inbox as I don't participate in these communities often. For easy digestibility, I'll be organizing these posts using a question and answer format, as there's a lot of highly technical material to cover.
Now, without further ado, let's talk about the basics. If you're not a programmer, fair warning: highly technical, possibly excruciatingly boring material ahead.
Basic poker bot responsibilities
At a very high level, the poker bot is best analyzed according to the classic model of information handling: Input, Processing, Output.
You'll find that your programming tasks decompose rather nicely into these three basic stages.
Input. The input to the system is the poker client software itself, including all its windows, log files, and hand histories, as well as internal (often private) state maintained by the running executable. The goal of the input stage is to interrogate the poker client and produce an accurate model of the table state - your hole cards, names and stack sizes of your opponents, current bets, and so forth.
Processing. The processing stage runs independently of the other two stages. It's job is to take the table model assembled during the Input phase, and figure out whether to fold, check, bet, raise, or call. That's it. The code that performs this analysis should (ideally) know nothing about screen scraping or interrogating other applications. All it knows is how to take an abstract model of a poker table (probably expressed as some sort of PokerTable class) and determine which betting action to make.
Output. Once the processing stage has made a decision, the Output stage takes over. It's tasked with clicking the correct buttons on the screen, or simulating whatever user input is necessary in order to actually make the action occur on a given poker site/client.
How does the bot figure out what its hole cards (and the board cards) are?
This is a broad question which it's better to break down into particulars. First of all, there's a very easy way to detect hole cards via a screen-scraping or "poor-man's OCR" approach. You don't have to be an image-recognition expert. All you have to know is how to get the color of a handful of different pixels on the screen. Or to put it another way, for any given card in the deck, there are a handful of pixels you can test which will uniquely identify that card.
That's fairly easy to implement, and requires zero knowledge of OCR, image recognition, graphics processing, etc. But depending on the specific poker site, pulling card rank and suit information might be even easier. On some sites, the hole cards will be emitted into the real-time game summary info:
Occasionally you'll find that hole cards are emitted into the log file. Poker Stars, for example, conveniently emits this information into its log file, and it does so in real time (meaning you can snoop on it in real time, and in the next installment, I'll show you how):
MSG_TABLE_SUBSCR_ACTION
MSG_TABLE_SUBSCR_DEALPLAYERCARDS
sit1
nCards=2
sit3
nCards=2
sit5
nCards=2
sit6
nCards=2
sit7
nCards=2
dealerPos=3
TableAnimation::dealPlayerCards
MSG_TABLE_PLAYERCARDS 00260C82
::: 8s <-- Hole Card 1, Cool!
::: 13c <-- Hole Card 2, Cool!
Last but not least, hole cards are always included in the hand history for a given game:
*** HOLE CARDS ***
Dealt to CodingTheWheel [Qs 9h]
MargeLeb: calls 10
ke4njd: calls 10
diamondlover2nite: calls 10
franklg454: folds
WhoAmINot: calls 5
CodingTheWheel: checks
*** FLOP *** [4h 7c Qd]
WhoAmINot: checks
CodingTheWheel: bets 10
The only problem is that, in many cases, the hand history file isn't emitted until the end of the hand.
How should the poker bot be structured, as a single EXE, a bunch of DLLs, what?
You will need:
- An executable file (.EXE) to display the bot's UI, and to contain the processing logic (the stuff that knows how to play poker).
- A dynamic link library (.DLL) to handle the Input (screen scraping) and Output (clicking buttons) processing. You'll inject this DLL into the poker client's process so that your code is effectively running as part of PokerStars, or FullTilt, or whatever site you're using. This will make your life a lot easier both when it comes to collecting data as well as doing things like simulating genuine user input.
Those two pieces are essential. Other than that, you're free to structure things however you want. I'll have more to say on this as we get into the nitty-gritty details of the implementation.
How do I inject my code into the poker client process?
There are a number of well-documented techniques for injecting your code - for example, a DLL you've written - into another application's address space. The method I used, and the method I'm going to recommend you use, is by installing what's known as a Windows Hook and specifically a CBT Hook. The relevant Windows API is SetWindowsHookEx, and here's the actual source code. If you're familiar with C++ and the Windows API, it should be straightforward:
// This is the CBT hook procedure. The HCBT_CREATEWND notification generally
// doesn't give us any useful information about the window because WM_CREATE
// hasn't been called for the window yet. So instead we don't consider the
// window as created until it's gotten it's first WM_ACTIVATE (note: this is
// how it works on Poker Stars, the behavior may need to be changed for other sites)
///////////////////////////////////////////////////////////////////////////////
LRESULT CALLBACK PokerBotCBTProc(int nCode, WPARAM wParam, LPARAM lParam)
**
if (nCode < 0)
**
return CallNextHookEx(g_hHook, nCode, wParam, lParam);
**
else if (theInjector.getVenue() != Venue_Unknown) // ignore this bit of code for now..
**
// Since we can't use DllMain, perform initialization the first time the hook is called.
if (g_bFirstTime)
**
theInjector.inject();
bFirstTime = false;
**
// These are the only notifications we're interested in passing on.
if (nCode == HCBT_ACTIVATE)
return (LRESULT) theInjector.HandleIt(Hook_Activate, (HWND)wParam);
else if (nCode == HCBT_CREATEWND)
return (LRESULT) theInjector.HandleIt(Hook_Create, (HWND)wParam);
else if (nCode == HCBT_DESTROYWND)
return theInjector.HandleIt(Hook_Destroy, (HWND)wParam);
**
// Return 0 to allow window creation/destruction/activation to proceed as normal.
return 0;
**
///////////////////////////////////////////////////////////////////////////////
// Publically exported hook installation function. The bot will call this on
// startup in order to inject this DLL (the DLL that contains this function)
// into the address space of every process, including every poker client process,
// on the machine.
///////////////////////////////////////////////////////////////////////////////
bool OPCHOOK_API InstallHooks()
**
// Actually install the hook...
g_hHook = SetWindowsHookEx(WH_CBT, (HOOKPROC) AutoCBTProc, hInstance, 0);
return g_hHook != NULL;
**
The above source code would live in a DLL: the DLL you plan on injecting into the poker client's address space. This DLL will also contain whatever code you write to handle the "input" (screen-scraping) and "output" (button-clicking) stages of the bot.
An even better way is to use a two-stage injection process. The problem with global CBT hooks (or any other kind of global hook for that matter) is that they cause the hook DLL to be loaded into the address space of every process on the machine. If your hook DLL is very fat (for example, if it contains a bunch of code to do screen scraping and so forth) this can impact system performance as your DLL will get mapped into processes you care nothing about, like Notepad.exe.
So to get around that, make the hook DLL - the DLL that gets loaded into every process - as lightweight as possible. Then, whenever the hook DLL detects that it's been loaded by a poker client process, such as POKERSTARS.EXE, have it explicitly load another DLL (again, written by you) containing the bulk of the bot I/O processing code. This in fact is the purpose of the theInjector object in the above code sample - it figures out if the DLL is being mapped into a poker client process and, if so, uses LoadLibrary to load the actual DLL that knows how to do things like screen-scraping and so forth.
A picture might help to clarify:
Here, INJECT.DLL would be a lightweight DLL, written by you, which contains the CBT Hook procedure and installation code I showed you above. POKERBOT.DLL is the fat, messy DLL, also written by you, that contains the poker bot's screen-scraping logic.
Do things this way, and the drag on the system shouldn't even be noticeable, other than a brief load period when you first install the hook. This method is a lot easier than most other methods of DLL injection, and more importantly, it's supported across all the major Windows operating systems.
How do I retrieve the game summary text from the poker table window?
Almost every online poker client displays a small window in which game summary text is displayed:
Now, depending on which poker client you're using, this window may or may not be a standard Windows edit box or rich text control.
If it is a standard Windows control, you can get the handle (HWND) to the window, and then get its text via the GetWindowText API. Furthermore, you can do this even if the window you're interrogating is owned by another process.
But what you'll find is that many poker clients don't use "normal" Windows controls. They may write their own custom display controls, or they may subclass a standard Windows control and cause WM_GETTEXT to return an empty string.
In that case you have at least three options, none of them trivial:
- You can investigate the control at the binary level. No matter how customized the control, somewhere in memory it's maintaining a string or a list of string which contains the game summary text. Since your code will be running inside the poker client's process, you're free to do whatever you want to do - investigate different areas of memory, subclass the control, etc.
- You can use API hooking to hook the core Windows APIs that every control uses to display text: DrawText, ExtTextOut, etc.
- You can use full-fledged OCR to analyze the text window and return the text. But this is probably overkill, especially since text tends to run through the summary window rapidly.
When building the bot, I went with the second approach: API Hooking. Once you know how to hook a particular Windows API, so that whenever POKERSTARS.EXE thinks it's calling DrawText, it's actually calling your custom version of DrawText, which snoops on the text before passing the call on to the original DrawText, it's a simple matter to examine the output coordinates to determine, aha: this text is being written to the summary window; this text is being written to the title bar; etc.
This is a deep enough topic that I'll roll it into its own installment along with specific code examples.
Hook a Windows API? Instrumentation? What does that mean?
Every Windows application in the world has to call into the Windows API to get things done: open files, create windows, display text, etc. Even language-specific libraries such as the C run time library or the C++ standard library internally will use the OS-provided facilities to work with things like files, memory, and so forth.
API hooking or "instrumentation" is the process of intercepting the function calls that an application (any application) makes, and redirecting them to a custom function defined by you. Specifically, you're going to intercept some of the calls that the poker application makes to the Windows API...
- DrawText
- ExtTextOut
- WriteFile
- etc
...and redirect these calls to a custom "interceptor" function, written by you. Your code thus gets a chance to examine the parameters of the call (which could be a string containing the player's name, for example) and do any other work you desire. When your "preprocessing" is done, you'll pass control back to the original API the poker application thought it was calling in the first place so that everything works transparently.
This technique can be used to extract all manner of useful internal information from any application, not just online poker clients. The best part is: you no longer have to write custom assembler code to achieve this. Instead you'll use a third-party library, and one of the best is a little-known Microsoft Research Project, Detours. Download it. Learn it. Love it. Learning how to accomplish API instrumentation means that ultimately, there's nothing the poker client can really hide from you - but that doesn't mean you'll be able to snoop around and figure out your opponent's hole cards. Don't even try. Unless the implementors have been sloppy, that information won't exist anywhere on your machine until your opponents have actually flipped their hole cards over.
And yes, if there's sufficient interest I'll put together a dedicated post with sample code showing exactly how to instrument a poker application.
Generally speaking, how do I go about harvesting data from an online poker game?
We've covered a few of the specifics so far, and we'll cover many, many more in future installments. But specific techniques aside, getting information from the poker client is an exercise in detective work. First of all, be aware of just how much information is available:
- Visual Table State. Everything a human player sees when playing online poker: his hole cards; the names, stack sizes, and betting actions of all players at the table; the position of the button; and so forth..
- Summary Text. Each table usually displays a text summary area which captures various betting actions, the beginning and end of new hands, etc.
- Action Buttons. These are the buttons the user clicks in order to Fold, Raise, Call, etc. Note that we can use the presence or absence of various action buttons to infer table state.
- Log File. Many poker clients output a log file which may contain helpful information.
- Hand Histories. Most poker clients output a formal "hand history file" which contains a complete description of a single hand of poker.
- Internal Stuff. Internally, most poker client makes standard calls to the C run-time library, the C++ standard library, and the Windows API. You can eavesdrop on these calls through a process known as API "hooking" or instrumentation. Additionally, poker clients, like all software, use memory to store things. Things like player names, cards, and betting actions.
- And more. Stuff I may not know about, or may have chosen not to mention.
The bot's job is simply to eavesdrop on that information, and analyze it to produce an accurate model of the table's state at any given point in time. So any technique now or in the future which allows you to do that is potentially a technique you'll want to leverage in the bot. Later, I'll suggest an architecture whereby a number of different interrogators can be used to query poker tables in a simple, extensible, and above all tweakable way.
How do I create a bot that's intelligent enough to play winning poker?
That's the million-dollar question. I could tell you that the folks over at the University of Alberta's Computer Poker Research Group have made impressive headway towards producing a winning poker bot. I could tell you that extensible rules-based systems like the one I implemented for my bot...
...are a lot more powerful than you think. I could tell you a lot of things, and we'll investigate how to leverage some of the poker bot frameworks that are already out there, and how to combine those with your own custom rulesets. But understand one thing: you don't have to create a winning poker bot in order to make money with a poker bot. All you have to do is create a bot that's capable of breaking even.
If you can create a bot that breaks even - neither wins nor loses money - rakeback deals and specific programs like the Poker Stars Supernove Elite will ensure that you get a fairly hefty payday per bot account. I mean many tens of thousands of dollars per year, per account. And nothing except the logistical nightmare of it all restricts you to a single bot account; why not have ten; or a hundred?
And indeed, unbeknownst to the rest of the world, somewhere, someone probably does. But not me, and not you; and that's a disparity in basic firepower I'd like to see remedied, since none of the online poker sites have really stepped up to the plate and either a) made bots legal (similar to the way that they're legal on Internet Chess Club) or b) put effective prevention measures in place.
What skills will I need to write a bot?
Well, you'll want to be well-versed in the nuances of C++ and the Windows API, at a minimum.
In addition to that, you'll either need to be familiar with, or get familiar with, an assortment of Windows development topics that reads like a chapter out of Richter (whose books I highly recommend purchasing and studying
- Windowing & GDI
- Windows Hooks
- Kernel objects
- DLL Injection (in general: the injecting of code into other processes)
- API Instrumentation (via Detours or similar libraries)
- Inter-process Communication (IPC)
- Multithreading & synchronization
- Simulating user input
- Regular expressions (probably through Boost)
- Spy++
While it would probably be possible to build a bot using C#, VB.NET, or any other language, you'll find that some of the powers you'll need are only available through specific Windows APIs, and getting access to these APIs from a managed language is a little clunky. Another reason you'll want to use native C++ is that you'll need to sneek some of your code into the client poker process, and it's a lot cleaner to inject a small DLL than it is to inject all the machinery necessary to get managed code to run inside another (native) process.
Conclusion
This post has only scratched the surface of building a full-fledged poker bot. Hopefully it's given you food for thought and possibly whetted your appetite for the mountain of details left to be discussed. Assuming there's sufficient interest, I'll be posting a series of installments, each describing at a detailed level how to accomplish one specific poker bot task. If you found this document through a link on 2 + 2 or one of the other poker forums, you can subscribe to this site in a reader or get it in your email inbox, as I rarely post (or reply) on the poker forums these days.
Reply With Quote
. Greg Hoglund and Gary McGraw. If you actually plan on building an online poker bot, this is the book for you. Discusses many of the same techniques discussed in this series, but with a focus on World of Warcraft and other MMORPGs. All of the techniques discussed apply to poker, however.
. Greg Hoglund and Gary McGraw. Companion volume to the above, with an emphasis on local applications (e.g., the poker client application running on your machine).
. David Sklansky. Quite possibly the most influential book in poker, written by the father of modern poker theory, David Sklansky. The value of this book is that it explains poker theory in a way that's universally applicable to all poker games. Required reading for all poker players and poker bot programmers.
. Bill Chen and Jerrod Ankenman. Difficult but extremely rewarding exposition of poker mathematics. In order to program a fluent poker AI you'll want to understand a lot, though not necessarily all, of this material.
. Matt Flynn, Sunny Mehta, and Ed Miller. In my opinion this is the best book on no-limit Hold'em ever written, and the notion of planning hands around commitment and SPR squares very well with the poker bot AI we'll get into later.
. David Sklansky and Ed Miller. Another excellent no-limit Hold'em book.
. Mason Malmuth. A somewhat dry, rarely-read masterpiece, with some non-trivial mathematics. Not everything here is directly applicable to playing, or building software for, online poker, but it's all useful.
. Ed Miller. Introductory text. There are many good introductions to poker and to Texas Hold'em; this is one of the better ones, and introduces a short-stacked strategy that will be useful in deriving a competitive bot AI for real-money games, down the road.
Originally Posted by MrX
!