This is a submission for the Hacktoberfest Open-Source AI Challenge Week 1: Touch Grass

What I Built

WildLens is a small field notebook that lives in your phone's browser. Point the camera at a plant, bird, animal or insect and it tells you what it thinks you're looking at. The identification runs entirely on your device, so your photos are never uploaded, and once the model has downloaded the app is built to keep working with no signal.

I'm a developer, so I'm well aware of the irony of building an app to get people off screens. That's why I wanted the screen to be the shortest part of the experience: you spot something on a walk, take a photo, get a name in a few seconds, and put the phone back in your pocket. Every result also comes with a short "Look closer" prompt, a small observation task that turns the photo into a reason to keep looking at the real thing.

I didn't want it to bluff, either. When WildLens isn't sure, it says so. It shows its top three guesses instead of one confident-sounding answer, and if it can't find a living subject in the photo, it tells you that. Anything hazardous gets a safety note, and every plant carries one fixed rule: never eat anything based on a photo.

What you find can be saved to a local field journal with your own notes and, if you choose, an approximate location. It stays on your device, and you can export or clear it whenever you like.

I gave it a quiet look on purpose: a parchment background, muted greens, a serif for scientific names, no gradients. It should feel like a notebook, not another app begging for attention.

It's for anyone who has stood on a trail or in a garden and wondered "what is that?"

Demo

Live app: https://wildlens-57g9.onrender.com/

It works best on a phone. Open it, allow camera access, and on first launch let it download the model (about 330 MB) over Wi-Fi. That's a one-time download, and the model is kept in your browser's cache afterwards.

How it went outside:
I tried to identify the plants in my garden and a sweet little cat that always comes to my garden. Then, when I went outside, I tried to identify the flowers that I saw there and kept them in my journal, it worked quite nicely and was amazing.

Code

How I Built It

WildLens is a React, TypeScript, Vite and Tailwind app, installable as a PWA, with the journal stored in IndexedDB.

The AI at its core is BioCLIP, an open-weight vision model from the Imageomics Institute (MIT licensed) that was trained on biodiversity images across the tree of life. I exported its image encoder to ONNX once, in Google Colab, and the app runs it in the browser with ONNX Runtime Web (WebAssembly). There's no server and no API key.

Identification works like this. The photo is center-cropped and normalised, the encoder turns it into a vector, and that vector is compared with precomputed vectors for each species. BioCLIP responds best to full taxonomic names, so a small build script looks up every species on GBIF, and I embed "a photo of" plus the taxonomic string. Those text embeddings ship as a small file, so the phone only ever runs the image half of the model. A softmax turns the similarities into probabilities, and a set of "not nature" prompts catches photos that shouldn't become a species (a living room should never be reported as a gecko). When no single species wins clearly, the scoring adds up probabilities by genus, family, order and class, so it can lean on "probably this family" instead of a wrong species.

The model file is far too big for GitHub, so it lives in a Hugging Face model repo. The app downloads it once and keeps it in the browser cache. The app itself is a static site on Render.

The first version didn't work well. I started with OpenAI's CLIP (ViT-B/32, 8-bit) through Transformers.js, and it was wrong almost every time. I grew the label list, rewrote the prompts and tried prompt ensembling to separate a cat from a cheetah. Each tweak helped a little, but none fixed the real problem: CLIP is a general-purpose model, and BioCLIP actually knows biology.

What saved me was keeping the whole UI behind one small NatureIdentifier interface. Switching models meant adding a new identifier class and updating the scoring, with no changes to any screen. After the swap, the answers finally started to make sense.

It's honest about its limits. It knows a few hundred species, not every organism on earth. Look-alikes (small brown birds, frogs, grasses) are hard for any model. And it's a notebook companion, not a safety tool.

Credits: BioCLIP (Imageomics Institute), OpenCLIP, ONNX Runtime Web, GBIF for taxonomy, and Hugging Face for hosting the model files.

Why Does Open Innovation Matter?

I started with the trail. If you're hiking with no signal, a cloud vision API is just a spinner. Because the model weights are open, I could put the model on the phone itself: no connection needed, no per-photo cost, no API key to leak and no rate limit to hit.

Privacy mattered just as much. A wildlife photo carries a time and often a place, which is exactly the kind of data I didn't want sitting on a server I don't control. With a local model, the photo never leaves the device.

Open also meant I wasn't locked in when my first choice turned out to be wrong. With a closed API, "this model isn't good at biology" is a support ticket. With open weights it was a change I could make myself. Species are just text too, so adding a bird means adding a line, not retraining anything.

WildLens is mostly other people's open work, glued together carefully: BioCLIP, ONNX Runtime, GBIF's taxonomy, Hugging Face's hosting. A closed API would have given me a black box that needed internet. Open innovation gave me a notebook that works at the edge of the trail.

My Agent Session

I used an AI coding assistant throughout, for scaffolding, for debugging the in-browser WebAssembly setup, and for the one-time model export. The decisions about what to build and what to keep were mine, and I tested the results myself.

Prize Categories

Best Use of Render: WildLens is deployed on Render as a static site, which serves the PWA front end. The model itself runs on the user's device.

Thanks for reading. If you try it, I'd love to hear what you found out there. Now go outside.