Making a bot

Well. This has got out of hand. I’m having…fun? …maybe?

Playwright

I did a bunch of investigation into ways to control your browser using commands. Playwright and Chromedp came out on top for what I was looking to do. I have a little bit of familiarity now in Go, so I picked this over using Python which seems very common to do this sort of thing.

Playwright acts as the ‘hands’ of the bot, it allows me to run a separate Chrome browser and click, move and set certain parts of the page.

GoCV

This is the ’eyes’ of the bot.

Screentop is built using React, PixiJS and Typescript and Apollo. It’s just a sandbox and isn’t really a ’true’ game engine. The elements are drawn onto a canvas making them hard to select. I haven’t been able to extract any form of Gamestate API from the site as it’s merely drawing a representation of the game, rather than storing the game at a particularly useful level for playtesting.

GenAI

I initially wanted to see how it would be to hook this up to the requests I’d send it to see if it could begin to formulate actual strategies to play against. It definitely has been able to spit out convincing high level strategies, but the more I’d prompt it, the more I realised it did not understand the details. Of which there are a lot to keep track of.

Chapter 1: Just some simple automation

I set off with a simple goal: get a bot to log into a game room. This was the easy part. Using the playwright-go library, the script could:

Launch a Chrome browser.
Navigate to the game URL.
Fill in the bot’s name.
Click the “Join” button.

After resolving a few simple “strict mode violation” errors by making the button selectors more specific, I had a bot that could successfully sit at the table. This seemed pretty simple, maybe it was going to be easy..It was far from it.

Chapter 2: The wall, the canvas

The first major roadblock appeared immediately. The entire game board, from the hex grid to the resource counters, was rendered inside a single HTML

This led to the first major strategic pivot: if we couldn’t interact with elements directly, we would have to interact with the canvas at specific X/Y coordinates, just like a human player. But to do that, the bot needed to know what it was looking at. It needed “eyes.”

Chapter 3: The hunt for game state

Before resorting to computer vision, I embarked on a long and frustrating hunt for a “game state object.” My thinking was that surely the game’s React front-end had a master JSON object in memory that held the location of every piece on the board. This led us deep into the browser’s developer tools, learning a bunch of debugging tools I hadn’t touched before.

Chased some red herrings for a while, including:

A getState() function in a React component, but it returned the application’s rendering state, not the game state, and was filled with circular references that crashed JSON.stringify.
Getting stuck in the entire React component tree, only to discover that Screentop.gg uses a custom react-reconciler for canvas. The game pieces were not DOM elements and their state was not in the React tree I could see.

The conclusion was there was no single, readable game state object I could access. The “state” was locked away inside a custom rendering engine.

Chapter 4: C++ dependency hell

With no state to read, I pivoted to a hybrid approach using local Computer Vision with GoCV (OpenCV). This is where the journey descended into the nine circles of C++ dependency hell.

The problem is simple but was painful to solve: GoCv is a Go wrapper around the OpenCV library, which is written in C++. To make them work together on Windows, you must have a perfectly configured C++ build environment. Every step of this process failed, teaching me some critical lessons:

Missing Engines: The first attempt failed with leptonica/allheaders.h: No such file or directory. Lesson: The Go packages are just translators; you must install the actual C++ engines (OpenCV and Tesseract) on your system.
The Wrong Compiler: After installing the official OpenCV binaries, there were undefined reference errors. Lesson: The official OpenCV build uses the MSVC compiler, but Go’s C++ bridge (cgo) on Windows requires the MinGW compiler. The two are incompatible.
Long Compile: The solution was to compile OpenCV from scratch using the correct MinGW toolchain. This involved installing the correct MinGW build from WinLibs, installing CMake, and running the win_build_opencv.cmd script—a process that took nearly two hours of 100% CPU usage.
The Linker’s: Even after a successful compile, the program failed with a massive wall of undefined reference errors. Lesson: The win_build_opencv.cmd script creates dozens of individual library files (.lib), and you must provide the linker with the complete, explicit list of every single one it needs. A single missing library causes the entire build to fail.

This phase was a brutal, multi-day battle against error messages and the unforgiving C++ toolchains. But once it was set up and sorted, I haven’t had to touch it since.

Chapter 5: Memory corruption

Even with a compiled program, a mysterious bug remained: exit status 0xc0000374 (Heap Corruption). The program would crash the moment it tried to use Playwright and GoCV in the same script.

The two C++ packages - the Chromium browser engine and the OpenCV library - created an overload ’taint’ in the program’s memory (particularly for my old laptop). They cannot coexist in the same process space, even one after the other.

This led to a rearrange of how my program/bot was laid out:

taker.go: A tiny Go program whose only job is to import Playwright, take a screenshot, and immediately exit, freeing all browser-related memory.
analyser.go: A separate Go program whose only job is to import GoCV and analyze the screenshot file.
A master run_bot.bat script that runs these two programs in sequence, ensuring memory isolation and the right build environments.

Chapter 6: Board calibration

With the architecture mostly solved, the next challenge was to reliably map the board. The initial attempts were using computer vision and failed spectacularly, with template matching finding thousands of false positives and shape detection picking up UI elements.

The solution was a mathematical calibrator. This tool uses no computer vision itself. Instead, it asks the user to provide the pixel coordinates of three “anchor” hexes (A1, L1, and A17). Using these three points, it applies hexagonal vector mathematics to calculate the precise, pixel-perfect center of all 102 hexes on the board, saving the result to a permanent calibration.json file.

Lessons Learned

This journey from a simple idea to a working bot was a deep dive into debugging and software engineering. Some of the key takeaways so far were:

Trust the Error: Most error messages, from strict mode violation to undefined reference to 0xc0000374, were telling the absolute truth. The solution was always to understand the error, not to work around it. Working in Go is particularly handy for this, as the error messages are quite verbose. Certainly running into far less ‘uncaught reference errors’ from some Javascript projects I’ve been attempting.
Isolate the Problems: The stable solution only emerged after we completely isolated the conflicting libraries into separate programs.
Use the Right Tool for the Job: Ultimately settled on a hybrid approach: Playwright for browser control, Go for orchestration, and a future AI model for the one thing it’s best at (complex OCR), all guided by a mathematically perfect calibration map.
Patience

In the end, we didn’t just build a bot, I have conquered a notoriously difficult C++ build environment, designed a robust, multi-process architecture to handle deep-seated memory conflicts. And that is a victory in itself.