Selected engineering work / Case study

Wildlife observation platform · Edge systems

MockingBird

A local-first wildlife observation platform: cameras and footage in, a searchable record of a place out.

MockingBird application showing a Visit page: a zoomable evidence frame of a nestling in green foliage, a banner noting three other subjects detected in the same encounter, and a suggestion panel beside the sidebar navigation for Today, Cameras, Review Queue, Bird Log, Place and Settings.
The running application. A Visit page on a demo instance: the best evidence frame, subjects tracked from one shared encounter, and a labelled identification suggestion. The instance was fed public-domain clips through the real detector and species model.
Role
Creator & Developer
Status
Pre-release · v0.1.0
Runs
On your own computer · local sign-in, no cloud account or service
Stack
React · FastAPI · SQLite · ONNX Runtime
Edge
Raspberry Pi 4 Camera Node
What it is
A wildlife observation platform for your own cameras. It connects live and recorded sources, detects and tracks animals, and organizes the result into Visits, a Bird Log, a daily Today view, insights and durable local history, with optional visual and acoustic identification.
What I built
The whole system as creator and developer: the React interface, FastAPI backend, computer-vision runtime, SQLite persistence and the Raspberry Pi Camera Node, plus the architecture, milestones and acceptance decision for each step.
Key engineering
One observation pipeline shared by live cameras, imported clips and a replay harness; a replaceable edge node; forward-only schema migrations with verified backups; and a hard line between model evidence and the reviewed record.
Evidence here
Screenshots of the real application, an architecture diagram, real feeder captures from a Camera Node, test and validation evidence from a Pi 4 and a GPU desktop, and a scope section that states what is validated, limited and planned.
01 · The product Point it at a place, and it keeps a record of the wildlife there.

The problem

Backyard and trail cameras produce thousands of mostly empty frames. Security-camera tools treat an animal as a notification, so nothing accumulates into an account of what actually visited, where and when.

The approach

Treat every encounter as structured data: a Visit tied to a camera, a Zone and a time, with its best evidence frames, tracked subjects and any identification. It runs as one local application with local sign-in and no cloud account or service.

02 · What it doesSix jobs, one application.

From the edge device to a daily view of the place, every capability shares the same pipeline and database.

  1. i.

    Connect

    A Raspberry Pi Camera Node, RTSP/ONVIF and USB cameras, still-image URLs, and imported video, photos or trail-cam folders. Draw Zones to focus each view.

  2. ii.

    Detect and track

    MegaDetector v5a on ONNX Runtime and a tracker assemble frames into Visits with their best evidence, from live cameras or recorded footage.

  3. iii.

    Identify

    Optional BioCLIP 2 visual suggestions and BirdNET acoustic context, stored as labelled evidence that can abstain.

  4. iv.

    Observe the place

    Today, live camera views, Visits, an Activity Library with search, filters, tags and collections, the Bird Log, Place Story and Insights by Zone and hour.

  5. v.

    Operate reliably

    Supervised capture, camera health checks, in-app alerts, storage retention, and verified backup, restore and upgrade paths.

  6. vi.

    Own the data

    History in SQLite on disk you control, local sign-in, no cloud account or service, no telemetry, reviewed-only exports, and a Local Data console.

03 · The applicationThe real product, screen by screen.

Screenshots of the running application on a demo instance: public-domain sample clips run through the real detector and species model, with nothing pre-confirmed. Tap any screenshot to open it full size.

MockingBird Visit page with a zoomable evidence frame of a nestling, a banner listing three other subjects in the same encounter, and a suggestion panel with Confirm species, Change and Not sure.
01 · Visits

Evidence, tracking and identification in one place.

Each Visit holds its best frames in a zoomable viewer, the other subjects detected in the same encounter, and an identification suggestion with alternatives. Splitting, merging and re-zoning a Visit are structural edits kept separate from species labels.

MockingBird Today screen titled Today in Your Yard, with a Running on this device chip, camera actions, a Today's activity panel and a Biodiversity here summary.
Today. Demo capture; recorded clips carry no capture time, so activity totals stay empty by design.
02 · Today

The daily view, and where the data comes from.

Today summarizes what visited, how often and where, with best moments, recent activity, camera health and storage status. Cameras and Zones are set up in the same interface: choose a source type, validate it, and draw the part of the view that matters.

03 · Bird Log and Local Data

A durable record, kept on your machine.

The Bird Log lists confirmed species and holds provisional suggestions apart. Local Data covers device-only access, notification categories, storage and retention, backup and restore, and a storage integrity check.

MockingBird Bird Log on a demo instance: a provisional Likely Northern Mockingbird suggestion kept apart from a person-confirmed Northern Mockingbird entry with one reviewed observation.
The Bird Log. One confirmed species, kept apart from a provisional suggestion.
MockingBird Local Data screen showing this-device-only access, sign-in always required, and notification categories.
Local Data. Demo capture: access, notifications, storage and recovery.

04 · The engineeringOne host process, replaceable cameras.

Frontend, backend, vision runtime, persistence and edge node, integrated into a single local product.

Architecture diagram. Sources (Camera Node, RTSP/ONVIF/USB cameras, imported media, optional microphone) feed an Observation Engine on the host. It produces Evidence; an Identification step suggests species; Human review turns evidence into Reviewed truth; only reviewed truth feeds Today, the Bird Log, Place Story, Insights and exports.

A simplified diagram of the system, drawn from the project’s README and architecture notes.

  • Frontend

    A React and TypeScript single-page app of 35 screens (Today, Cameras and Streams, Visits, Activity Library, Bird Log, Place Story, Insights, Local Data, Doctor and more) over a typed API client, with a Playwright browser suite of 53 specs.

  • Backend

    FastAPI served from one process alongside the UI: covering authentication, camera and media providers, a durable monitoring supervisor, background workers, analytics and exports.

  • Computer-vision runtime

    MegaDetector v5a runs as ONNX on an NVIDIA GPU, feeding a tracker and Zone-aware Visit admission. BioCLIP 2 ranks a closed species list on the CPU. Live and recorded video share the same detect, track, assemble and admit path.

  • Persistence

    SQLite on disk you control, with forward-only migrations (schema 62), a verified pre-upgrade backup and rollback-safe restore. Saved evidence is managed media with retention, orphan checks and an integrity scan.

  • Camera Node

    The reference edge device is a Raspberry Pi 4 running a small capture-and-transport service under systemd, with pairing, an installer, upgrade and recovery paths and host-clock calibration. It is never the system of record: a replacement node reattaches to the existing logical camera.

  • Acoustic context and test tooling

    An optional microphone provider lets BirdNET add sound-based evidence under its own consent, with non-commercial model weights never bundled. A replay harness runs recorded footage through the real admission stack in a disposable database, so a fix can be tested without waiting for another real bird.

05 · ScopeCurrent capabilities and validation.

Taken from the project’s own support matrix and roadmap, not from aspiration.

Built and validated

  • Windows 10 host with an NVIDIA RTX 4060 Ti, packaged as a ZIP.
  • Pi 4 Camera Node: install, reboot, upgrade, replacement and recovery.
  • RTSP and ONVIF cameras, image and video import, Zones, Today, Visits, Bird Log, Insights, Local Data, backup and restore.

Built, with limits

  • Live detection needs an NVIDIA GPU; without one, import and review still work.
  • Species suggestion is bird-only and uses a closed candidate list.
  • Legacy local cameras run two armed streams at once; Camera Node streams are exempt.
  • Linux runs from source only. USB cameras have had limited real-hardware acceptance.

Roadmap: not yet built

  • Home Assistant, Frigate, Blue Iris and cloud-camera integrations.
  • A packaged Linux or macOS host and a code-signed installer.
  • The first public 1.0 release.
  • Plants, habitat and environmental measurements as full observation domains.
A small brown finch clinging to the base of a tube feeder with blue ports, a log and grass behind.
The reviewed example. A finch at the feeder, 8 Aug 2026. This is the one capture here whose Visit carries a human review label (House Finch), which is why it can be named.

06 · A design decisionEvidence becomes the record in one way.

One engineering choice shapes the data model. Everything a model produces, whether detections, proposals, suggestions or imported labels, is stored as evidence. A single rule in the code decides what counts as reviewed truth, and it only accepts an explicit human decision. The Bird Log, insights and exports are built on that rule, so a model’s guess can’t slip into them unnoticed.

  1. Capture. The Camera Node sends frames and the Observation Engine admits a candidate Visit with its best evidence.
  2. Suggest. A species model ranks a closed list of candidates, and abstains when they’re too close to call.
  3. Review. A person confirms, corrects, or answers “unknown.” Fast Review makes this quick.
  4. Record. Only that decision feeds the Bird Log. Corrections stay put when later model runs suggest something new.

07 · Field capturesFrom the feeder.

Frames from a Camera Node watching a backyard feeder in summer 2026. They’re tight crops of ordinary motion-triggered captures, small and a little soft, so I show them small.

A small bird with a red head and breast perched on a clear tube feeder with blue perches, grass and a paved road behind.
Feeder, early evening. 2 Aug 2026.
Two small brown and red-headed birds at a tube feeder with a pole to one side, standing on a sloping log.
Two visitors, one Visit. Several animals can share an encounter, and MockingBird has to decide how to split them.
Several small birds at the perches of a tube feeder, cropped tightly into a circle.
A busy minute.
A small bird resting on a log next to a feeder, in a soft close crop.
On the log.
A small reddish bird perched at a feeder, seen close and slightly blurred.
Perch, mid-afternoon.

These are crops of motion-triggered frames, re-encoded with location data removed, and wider frames that show neighbouring property are left out. Apart from the reviewed finch above, the birds are unreviewed and no species is confirmed.

08 · Privacy & challengesYour yard stays your data.

Video and data live on disk you control. Nothing uploads automatically, sign-in is local, and there is no cloud account, cloud service or telemetry.

Secrets stay out

Secrets are redacted from status, logs and the support bundle, and a Local Only posture documents exactly which optional features may touch the network.

Zones, described plainly

Privacy and Ignore Zones are trigger-level exclusions, and the documentation says so. They stop a region from creating Visits; they aren’t pixel-level redaction.

Models earn their place

A candidate detector or classifier has to beat the incumbent on identical data before it’s promoted. The benchmark record shows several candidates evaluated and none promoted yet.

Challenges along the way

  • Fragmented Visits.One animal can become several Visits, and several can share one. Fixes for continuity, boundaries and multiple subjects have helped, though some over-splitting remains.
  • Honest time.Recorded video has no trustworthy capture time, so MockingBird leaves those Visits out of time-based analytics instead of inventing one.
  • A closed species list.It keeps suggestions explainable, but a bird outside the list gets a wrong or uncertain answer. Review is the safety net, not a bigger model.
  • Hardware reality.Live detection needs an NVIDIA GPU. Real-hardware validation so far covers one Pi 4 Camera Node and one Windows 10 desktop with an RTX 4060 Ti.

09 · How it was builtSpecified, constrained, accepted.

I created and developed MockingBird end to end: the product principles, the architecture, the milestone plan, and the acceptance decision for every step. Development used AI coding assistants working from written task packets, inside guardrails I wrote myself.

Real-hardware validation ran on my own equipment: a Raspberry Pi 4 Camera Node and a Windows 10 desktop with an NVIDIA RTX 4060 Ti.

Product & architecture

The product principles, the evidence-versus-record model, the roadmap, and the scope of each milestone.

Engineering process

A task packet before any edit, a router to the relevant design documents, tiered validation, and an acceptance record per milestone.

Validation & acceptance

Test tiers and hardware sessions run under those records. I decide whether a result is accepted, and which candidate models are not promoted.

Backend suite
≈6,100test functions across 355 files, run in tiers. The last full run had two failures: one pre-existing, one sensitive to machine load.
Browser suite
53Playwright specs, passing at the latest accepted milestone
Schema
62forward-only migrations with verified pre-upgrade backups

Test results come from the project’s accepted release records and weren’t re-run for this page. MockingBird is pre-release (v0.1.0) and the first public release isn’t out yet. The demo’s three clips illustrate the workflow and say nothing about how accurately it identifies birds.

Next: Apex, a quantitative research platform built on real market data.

Read the Apex case study Back to selected engineering work