Leaf's Grand Blue Autofish

1. The start

   A macro that worked, and not much else

   I play Grand Blue, and its fishing is a long grind. I found 1vtt's open-source fishing macro, and it worked, but it was bare bones: a console window, a JSON file to edit by hand, and a calibration that was hard to get right. No quality-of-life features at all. My first thought was that I could do it better myself.

2. The goal

   Better for a player who never opened a terminal

   I wanted to put my UI and UX work and my computer vision experience into it. A real interface instead of a console, a calibration that walks you through every step, live numbers instead of scrolling text, detection that doesn't lose a bite to one bad frame, and a ready-made exe, so nobody has to install Python. 1vtt agreed to me building on their macro, and it grew into its own app.

3. The adventure

   Looking at the screen like a player

   The app only looks at the screen and only uses the mouse and keyboard, like a very attentive player. It never reads or changes the game's files or memory. It casts, clicks the SHAKE button until a fish bites, reads the rarity badge, keeps the reeling bar on the fish, then reads the catch message to log the fish, its mutations and what it's worth.

   YouTubeLeaf's Grand Blue Autofish in 22 secondsLoads the YouTube player

   Around the engine I designed the interface, all of it in HTML and CSS inside pywebview: a dashboard with an ocean scene, an Inventory of past sessions with charts and CSV export, a guided calibration with a picture for every step, and settings for themes, accent colours and a dyslexia-friendly font.

   Every part of it reacts when you use it, and it has its own sound effects, which I made myself: hovering, clicking, switching pages and toggles, starting and stopping, and a different catch sound for each band of rarity. Sound is a big part of how an interface feels, and I wanted this one to be cozy and a little cool. You can hear them in the demo above.

4. The difficulties

   Things the background can fake

   Grand Blue draws its interface over water, sand and night scenes, and most of it is see-through. The SHAKE button is a translucent disk, so its colour is really the scene's colour. The rarity badge floats above your character and moves with the camera. While reeling, a fish that ran off the edge of the box looked like a finished catch. And the catch message animates in and out, next to other text. Each of these needed a detector built on something the background can't fake, measured on recorded sessions instead of tuned by feel. The Technical tab goes through them one by one.

5. The end

   Free, public and still growing

   Version 0.1.0 went public on GitHub on 2 October 2026, with a Discord server for help, and 0.1.2 followed three days later: text read on the graphics card, faster screen capture and recovery when fishing gets stuck. It's free to download, and its source isn't published.

   The next version adds auto skills: the app spends your skill points in the order you pick and rebirths by itself. For now only supporters have it. With a test build of it I reached fifth place on Grand Blue's rebirth leaderboard in a few days.

   

What's next

• Bait and food

  Read the bait and food you hold from the game's screens, then use them by themselves following a priority list you set.

• Fish index

  Every species with its rarity and value, plus your own price for each, for when you trade it at a different value.

• Notifications

  A Windows notification or a Discord message for rare catches, the end of a session, errors and pauses.

• Safety checks

  Notice the AFK prompt, a server restart or being sent back to the menu, then stop and tell you.

• Fishes for you

  Casts, waits for a bite, clicks the SHAKE button, plays the reeling minigame and casts again, until you stop it.

• Reads every catch

  The fish's name, its mutations, its rarity and what it's worth, from the message the game shows on screen.

• Keeps your sessions

  A live dashboard while you fish, and an Inventory of past sessions with charts, a searchable log and CSV export.

• Lets some fish go

  Pick the rarities you don't want and the app lets them off the hook instead of reeling them in.

• Auto skills

  Spends skill points in the order you set and can rebirth by itself at the Fishing level you choose.

• Calibrates with you

  A guided walk-through teaches it where things are on your screen, with a picture for every step. Up to five profiles.

• Stays out of the way

  Pauses by itself when Roblox isn't the window in front, and the main actions have hotkeys that work in the game.

• Looks the way you like

  Dark, light or the Windows setting, an accent colour, a dyslexia-friendly font, and sound effects I made for every part of the interface.

The dashboard

The Fishing page shows the session as it happens: time, casts, catches, casts with no bite and the gold earned, then every catch with its rarity, how long the fight took, how close the bar stayed to the fish and its value.

Inventory

Every session is saved. The Inventory shows the gold earned and the gold per hour, the catch of the day, the mix of rarities and the full catch log, which you can search, filter, copy as a summary or export as CSV.

Auto skills

Coming in the next version, and for now only for supporters. You pick the order of the skills on a map of the whole skill tree, and when a level brings enough points, the app opens the tree between two casts, unlocks what's next, closes the menu and fishes on. It can also rebirth by itself at the Fishing level you choose, then start the order again from the top. Up to three orders can be saved and shared as a code.

Calibration and settings

The calibration is a step-by-step guide: you point things out on a still of your game, the app checks each one, and nothing changes until every step passes. Quick fixes redo a single piece from inside the game. Settings cover hotkeys, the rarities to skip, auto skills, timings, the detection engine, the look and the sound.

The fishing loop

The engine is a state machine that runs on screen captures alone. Every decision comes from pixels: there is no API and no memory access into the Roblox client.

1. Cast

   Moves to the calibrated spot of water, holds the mouse for the cast and releases.

2. Wait and SHAKE

   Watches the minigame area, finds and clicks each SHAKE button as it pops up, and dismisses stray popups. Recasts on a timeout.

3. Hook

   Reads the rarity badge above the character. A rarity you chose to skip is let go by putting the rod away and taking it out again.

4. Reel

   Tracks the bar and the fish every frame and decides when to hold and release with a predictive controller that aims a little ahead of the fish.

5. Read

   Reads the catch message with OCR: species, mutations and length, then works out the value. Mutations multiply and stack.

Architecture

• Engine

  OpenCV, NumPy

  The state machine and its detectors: HSV and shape analysis of small screen regions, masked template matching and connected components. No neural networks except the OCR.

• Session

  Python

  Everything around the engine: settings, calibration profiles, hotkeys, start, stop and pause, and saving each session as it goes.

• Interface

  pywebview

  Every window is HTML, CSS and plain JavaScript in Edge WebView2, fully offline, on one design system with dark and light themes.

• Capture and input

  mss, DirectX, SendInput

  Screen capture of the game window, and mouse and key input that is only sent while Roblox is the window in front.

• Text reading

  RapidOCR, ONNX Runtime

  Its models ship inside the package, so reading text never goes online. It runs on the graphics card when it can.

• Release

  Nuitka

  Compiled into a Windows exe with everything it needs, so players don't install Python.

Measured, not tuned by feel

When a fight or a SHAKE click goes wrong, the app saves what it saw: the last 90 frames of the reeling strip with what each frame was read as, or a screenshot of the missed SHAKE button with its scores. Every fix below started from those recordings, and every detector has a benchmark built from real screenshots of the game.

Two long runs as proof

Two sessions I left running on their own, read from the session files the app saves. On 2 October, a stress test with maximum luck and the best conditions I could set up. On 12 October, a test of auto skills: the app levelled up, spent its points and rebirthed six times without me. A catch count above the cast count comes from bonus fish, like Double Catch, which bring more than one fish per cast.

Challenges

1. Oct 5, 2026

   0.1.2: faster and more reliable

   Text is read on the graphics card and the screen is captured faster, with a new Settings > Engine tab and an Instant SHAKE speed. A calibration step for where fish names are read. When nothing is caught for several casts, the app puts the rod away and takes it out again by itself. Fewer false catches in dark scenes, very fast catches counted, and bonus fish like Second Splash logged as their own catch. The activity log is saved as it goes.

2. Oct 2, 2026

   0.1.1: opens after a browser download

   Fixed the app not opening on some PCs after a browser download, because Windows marks downloaded files. Error messages now point to the Discord server first.

3. Oct 2, 2026

   0.1.0: first public release

   Fishes by itself, reads each catch, keeps a live dashboard and an Inventory of past sessions, skips the rarities you choose, and calibrates with a picture for every step, with up to five profiles. Light and dark themes, an accent colour, a dyslexia-friendly font and sound effects.