A cinematic devlog  ·  ESP32 × YOLOv8 × PID  ·  Night #1

PENCOGNITO

A WiFi rover. A fried board. A savior from a side pocket.
The story of a robot that learned to hunt — and strike.

Scroll to descend
"The vision was simple.
The night had other plans."
1
Fried Pi Board
2
L298N Motor Drivers
6
Bugs Survived
1
Element of Surprise
Prologue
The Idea

A Little Chaos
Machine

The original concept was almost embarrassingly simple: a Raspberry Pi, a pi-cam, two motor drivers, and a mandate to cause mischief. Pens were to be its primary target. A little desk gremlin, sneaking up on stationery. Nothing fancy.

We didn't know that the night had already written a different script. One with a much more dramatic second act.

Chapter I
23:14 — Night One

The Board
Got Fried.

It happened without ceremony. One moment the Raspberry Pi was the star of the show. The next — silence. A very specific kind of silence that every hardware hacker knows. The silence of a board that will never boot again.

"All hopes were lost.
Really? Twists again."
Fried Raspberry Pi board
SYSTEM FAILURE
// RPi POST: no signal — 23:14:07 — retrying... timeout
ESP32-CAM module
THE CAMEO
// ESP32-CAM OV2640 — side pocket deployment — status: ONLINE
Chapter II
23:31 — The Pivot

From Side
Pocket,
A Savior.

It was sitting in a side pocket the whole time. The ESP32-CAM module — a fingerprint-sized board with a camera, WiFi, and just enough computational grit to replace an entire Raspberry Pi setup. We just had to rebuild the whole system architecture around it. In one night.

Two web servers. One board. Port 80 for HTTP motor control. Port 81 for the MJPEG video stream. Splitting them kept everything responsive — the ESP32's HTTP library gets cranky trying to handle both simultaneously.

// System Architecture v2.0 (Post-Fried-Pi)
laptop_brain.py
Python / YOLOv8
⟶
WiFi (LAN)
HTTP Requests
⟶
ESP32-CAM
Port :80 Control
OpenCV Window
MJPEG Decode
⟵
WiFi Stream
MJPEG Frames
⟵
ESP32-CAM
Port :81 Stream
// The two-server split that saved the project
server.on("/forward", []() {
  setMotors(HIGH, LOW, HIGH, LOW);
  server.send(200, "text/plain", "OK");
});

// Port 81 — stream server (never mixes with :80)
streamServer.on("/stream", []() {
  streamMJPEG(); // continuous pump
});
Chapter III
The Bug Reel

Every Bug
That Broke
Our Spirit.

Not a changelog. A crime scene. Each bug, documented with the unflinching clarity of someone who had to live through it.

BUG-001 · ~00:10
The Ghost Driver
Hold W to go forward... let go... bot keeps going. OpenCV key events only fire on key-down. No "key released" signal. The bot was, effectively, haunted.
FIX: Ditched OpenCV input. Used GetAsyncKeyState() — live, physically-held keyboard state. Ghost exorcised.
BUG-002 · ~00:28
The Rogue Bot
GetAsyncKeyState() is system-wide. Doesn't care what window is focused. Click away to Google something — bot randomly starts driving into a wall.
FIX: Focus check bolted on. Bot only acts while its own window is in the foreground. Click away → dead stop.
LIMITATION-003 · ~01:00
No Speed Knob. Anywhere.
ESP32 firmware: no PWM speed API. Full-send or nothing. Driving at full send was basically uncontrollable — ricocheting off table legs.
FIX: Software duty cycle. Laptop pulses commands on/off rapidly. Tune the ratio → effective speed. Hardware is dumb. Software compensates.
BUG-004 · ~02:15
The Mirror Bug (Hall of Shame)
Hold red pen to the right → bot spins left. Every. Single. Time. Manual mode worked perfectly — camera feed was mirrored relative to chassis direction.
FIX: INVERT_STEERING = True. One flag. One line. Forty-five minutes of debugging. Classic.
BUG-005 · ~03:40
WiFi Temporal Paradox
Making decisions on stale frames. Commands take time to reach the bot. Bot takes time to physically stop. Loop kept reading where the target was, not where it is.
FIX: Two-phase timing loop. Send pulse → wait → explicit stop → settle → THEN read next frame. Give physics and network time to catch up.
BUG-006 · ~04:55 (Final Boss)
Wrong Port. Hours Lost.
Everything times out after reassembly. Laptop script pointed at port 80 for the video stream. Stream server lives on port 81. A 404 after that. Board answered — without the handler registered.
FIX: Fresh reflash. Serial Monitor on boot. Confirmed camera init. Confirmed handlers. It worked. We were too tired to celebrate.
Chapter IV
The Intelligence

Teaching It
To Hunt.

Plain color detection. HSV color space. Hunt for red. Press P and the bot switches into autonomous predator mode.

The naive version was "blob left → spin left, blob right → spin right." It worked. It looked terrible. All stop-turn-stop-go, overreacting to every wobble. So we built it properly.

P
Proportional
How far off target right now? React accordingly.
I
Integral
Persistent bias? Unused. This bot doesn't drift.
D
Derivative
How fast is the error changing? Dampen the overshoot.

Way off? Rotate in place. Moderately off? Arc drive — correct aim while still closing the distance. As the target fills the frame, forward pulses ease off. It doesn't come in hot.

PID tracking visualization
// target_locked: True — error_x: +0.03 — P: 0.8 · D: 0.2 — mode: ARC_DRIVE
Chapter V
The Payoff

Strike.

Once the bot closes the distance — once that bounding box fills enough of the frame to count as "arrived" — it stops. Then it fires the slash motor. One burst. The arm swings.

A second L298N. A separate motor mounted on top, hidden like a folded wing. Nobody knew it was there. The audience watched it crawl. They held their breath. Then they understood what the arm was for.

"We were able to create
that element of surprise."
The slash motor strikes a red pen
SLASH — MOTOR FIRE
// target_filled: 0.82 — status: ARRIVED — firing slash_motor...
×1
Chapter VI
The Upgrade

The Control
Panel
Grew Up.

Somewhere along the way the web page stopped being a random set of directional buttons. It became an actual control panel — live camera feed, manual drive controls, slasher arm on demand, and a target-picker right in the feed itself. Point at what you want hunted. Flip to autonomous. Watch it stalk.

Same brain. Same PID. Same arc-drive-vs-rotate logic. Now it aims at whatever you pointed it at instead of whatever happened to be red.

🎯
AUTO HUNT
Scans for red. Stalks. Strikes. No input needed.
📌
PICK TARGET
Click anything in the feed. It becomes the prey.
🕹️
MANUAL HUD
Drive it yourself. Fire the arm on demand.
Web Control HUD
CONTROL HUD v2.0
// browser_panel: ACTIVE — target_selected: True — slash_arm: ARMED
Confession
Why. Just... Why.

Stupidly
Over-Engineered
For a Non-Problem.

The entire job of this robot is "go near red thing, hit it." That's a photoresistor and a servo's worth of problem. Here's the tally of what it got instead:

01
A full PID control loop — separate P and D terms, tuned against a normalized centroid offset — deciding how to steer at a pen. Output further split into two behaviors (rotate-in-place vs. gentle arc-drive) depending on misalignment severity.
02
A hand-rolled software PWM. ESP32 firmware only knows full-on or full-off per motor. So the laptop fakes a speed knob by rapidly duty-cycling drive commands on and off. An entire fake throttle invented in software because the hardware refused to have one.
03
A full YOLOv8 object detection pipeline running live on the laptop, labeling everything in frame in real time, that the bot never once acts on. Pure decorative compute. Eye candy layered on top of a chase-and-swat robot.
04
A two-phase timing loop built just to survive WiFi lag. Pulse, wait for it to land, send explicit stop, hold a settle period, only then look at the next frame. A physics-grade accommodation for a device whose terminal action is "hit a pen."
05
A dedicated INVERT_STEERING boolean — earning its entire existence because a $3 camera module and a chassis disagreed about which direction counts as left.
06
A second independently wired motor for the slash — complete with its own re-arm / cooldown bookkeeping so it doesn't machine-gun a pen that's already been dealt with, plus a manual test-fire key just for bench-testing that one motor in isolation.
None of this needed to exist.
All of it exists anyway.
The Stack
System Integration

Everything Had
To Hold Hands.

A Raspberry Pi died mid-build. Two concurrent web servers on one microcontroller. WiFi carrying both video and control. OS-level keyboard hooks. Two independent vision pipelines. A tuned PID loop. A browser front end. All of it had to talk to each other — so a little rover could sneak up on a highlighter and knock it off the table.

Hardware
ESP32-CAM + OV2640 camera · Two L298N driver boards (locomotion + slash arm) · Chassis rebuilt after Pi death
Firmware
Two concurrent web servers on one MCU · Port :80 HTTP control routes · Port :81 MJPEG stream — split because the library couldn't handle both at once
Networking
Everything over WiFi — video and motor commands alike · Static IP so it doesn't wander on reboot · Purpose-built latency compensation loop
Vision
OpenCV HSV red-blob detection driving autonomous behavior · Separate YOLOv8 pipeline for on-screen labels · Two parallel stacks, two unrelated jobs, same feed
Control SW
PID controller turning vision output into steering · Manual override via raw Windows GetAsyncKeyState() · Foreground-window focus check so it doesn't self-drive on alt-tab
Interface
Browser-based control panel · Live feed + manual drive + target picker + slasher arm — one page anyone can walk up to and use. Live backend on laptop, frontend in browser, bot moving in the real world.
The Final Word
What It Actually Is

A Desktop
Chaos Mate.

A desktop-roaming, WiFi-controlled chaos gremlin built on an ESP32-CAM. By default it hunts down and ambushes anything red on your table. Hand it a specific target through the camera feed and it goes sneaky about that instead. Full browser control panel for driving it and firing the slasher arm yourself — whenever you just want to mess with things.

The Autonomous Predator
Scans around its deployed field. Found anything red — sneaks behind it, then slash. Sends it flying.
The Henchman
Assign it the target. May it be something on your friend's desktop — or your friend itself. It'll do the dirty mischievous job. Don't get caught.
The Personal Gremlin
Web-based control HUD with live camera feed. Drive it around and enjoy. Why do the dirty job yourself when you have a robotic pal ready for it?
Why would you build or gift something like this?
Only if you like a little mischievousness and thrill on your desktop.
The Reaction
The Payoff

Like Watching
Football.

They watched it crawl in. Slow. Deliberate. Tracking the pen without missing a beat. The room held its breath. Nobody knew what the arm was for — until it fired.

Audience watching with intensity
THEY DIDN'T SEE IT COMING
Raw Footage
Uncut

See It
Hunt.

The real footage. The actual bot. Tracking, closing in, and delivering the strike — as it happened on the night.