aipat.ch· patching the gaps in AI
← Back to all projects
Harness VNC · 06

VNCPool

vncPool is a pool of virtual Linux desktops that bots on my server can check out, run a browser or a desktop app inside, and actually test their own work. With OCR and a zoom tool on top so Claude can see where things are on the screen, not just blur. One of the early tools and one of the most useful.

Harness In progress go · x11 · docker

About this project

One of the early tools I built, and one of the most useful. I noticed pretty quickly that Claude didn't really check what it was making, so it needed somewhere to run the things it wrote. Selenium and the other usual suspects were either headless or not quite what I wanted. vncPool is a pool of virtual Linux desktops: a bot checks one out, runs its Chrome or its compiled app inside, exercises the features, and releases it when done. Real rendering, real window behaviour, not headless approximations.

Early on I noticed that Claude doesn't see very well. The screenshots it gets back are low resolution, which makes clicking accurately tricky. So I added an OCR tool for exact text coordinates, and a zoom tool for non-text regions. After that the tests stopped flaking and started actually testing things.

The surprise use case is pair programming. When I'm debugging with a Claude, I can ask it to spin up a session in vncPool and watch it work through noVNC in my browser, live. Seeing what the bot sees turns out to be genuinely effective. It's Linux-only today. A Windows sibling is in the pipeline, but Windows multi-session has its own headaches and isn't there yet.

Screenshots

VNCPool browser session rack
Web · desktop VNCPool browser session rack

Details

RefVNC · 06
ZoneHarness
StatusIn progress
Stackgo · x11 · docker
Source Not published