Introducing Nanobe
Today
I was working on filesystem design and wanted to search a folder the way I actually think about it, where the answer to one question might be a voice memo, pdf doc, or an image. Nothing I could run on a macbook did that well, so the exploring turned into post-training my own model.
Today I'm releasing Nanobe — an open weight multi-modal embedding model that runs locally on your machine. It's a fraction of the size of the models it's measured against, and it beats all of them.
Nanobe is a local lightweight model that can search across text, images, audio, and video. The model was built to run on consumer hardware — it runs on M1 chips and searches over files in a single index.
In practice, you can build tools that:
- Group visuals by style or concept
- Find a video from a sound in its soundtrack
- Search private files without sending them to a server
The model shows clear strength when working with mixed modalities, conceptual understanding, and style while remaining competitive in tasks from conventional benchmarks.
How the model works
Nanobe is built on Jina's omni embedding model. On top of that core it adds learned retrieval components, Meta Perception Encoder components for acoustic information, and a bunch of post-training that resulted in the quality you see reflected in the scores.
For audio, it understands both speech meaning and acoustic character — you can retrieve what someone said and what something sounds like. For video, it understands the soundtrack too.
The representations of audio and video are learned directly from the media, without generating transcripts or captions during indexing; and every page and every window of a recording gets its own vector, so a page or moment can pull the file up in the results.
One of the harder problems we encountered is competition between filetypes. Nanobe searches them together in one index, so an article about flamenco shouldn't outrank an actual flamenco recording for a music query. Pictures answer image queries, but they don't displace documents when the query asks for a document.
I think about it as a difference between being talented and being smart. Talent allows you to complete the task competently, while being smart allows you to deliver on the implied intent. Nanobe is designed to be a smart model.
Nanobe Bench
Nanobe Bench is something I feel had to be built because it allowed me to ask the questions beyond how well the model retrieves something. This benchmark is built to show how well the model works with mixed modalities, concepts and style understanding.
There are plenty of benchmarks that test modalities separately. Captions matched to images, queries matched to text. Searching a mixed folder is a different problem: several file types can answer the same query, related files compete, and the relevant content may sit deep inside a long recording or a long document rather than describe the file as a whole.
It also allowed us to catch regressions we wouldn't have noticed otherwise. In one of the experiments we kept shrinking the eval window size, and scores kept improving. Which is when I started to get doubtful about whether it's a real improvement — I had a hypothesis that the benchmark was biased towards content that described the file's general theme or appeared in the beginning. So we added questions about material deep inside files and exposed a lost capability — that capability was recovered with training and now we're sure Nanobe understands the whole file, even in detail. The interesting content is usually not in the beginning anyway.
The benchmark consists of real files across +120 themes, with related distractors and queries with multiple valid answers. The queries themselves cover concepts, style, purpose, retrieval, speech, acoustic descriptions, soundtrack-dependent video and more. We kept the benchmark out of training data for Nanobe.
During training I had a hypothesis that the model has learned more useful capabilities than its current retrieval rule can reliably use. I suspect the model is better in some ways we don't have a way to test yet.
What it takes to run locally
The weights occupy approximately 2 GB. Model was optimized for Apple Silicon, but also runs on other hardware. Quantization shrinks the package, and the packaged model was re-evaluated through the actual local runtime with no regressions.
Files are indexed ahead of search, so each query only touches stored representations. On a MacBook Pro (M4 Max, 36 GB), indexing a 1.1 GB folder of 125 mixed files takes 16 minutes, and a text query over 1,247 files returns in ~700ms.
We also include a reference implementation together with the weights, so you know how to run it to get similar performance to our benchmark scores.
How Nanobe was built
I'm not an AI researcher, but I knew there was a lot of space for improvement, so I orchestrated an autoresearch loop. For a month that I was working on this project, Fable 5 and 5.1 iterated over 150+ experiments and resulted in around 50 kept improvements. The research throughput and engineering was done almost entirely autonomously, I only steered the agents in the right direction.
I also tried GPT-5.6 Sol and GPT-6 Astra in the same autoresearch flow. To my surprise, neither produced meaningful score gains. Most of the time Sol and Astra spent either debugging their experiments or converging on a narrow set of strategies. I have a suspicion that Astra is not a model meant to be used by humans — maybe it's a model meant to be orchestrated in a swarm, so giving Astra to Fable as a subagent could produce more interesting results. Still, Astra was instrumental in finding and fixing bugs Fable had introduced.
The agents used their harnesses — Codex and Claude Code respectively. I only added my own autoresearch flow + SF Compute Autoresearch CLI to let agents buy and manage compute for training.
What's in the box
Model weights, a Python reference implementation with examples for text queries and reference-file queries, and support for common file types.
License
Nanobe is distributed under the MIT License. I believe reusable computational capabilities should be available without requiring each downstream user to negotiate permission to build with them. The Hugging Face model card explains the licensing position and Jina lineage.
Post scriptum
There are many problems worth working on today, but no matter what you're working on — it is by far the best time to build with care. If you’re a carpenter making a beautiful chest of drawers, you’re not going to put a piece of plywood on the back, even though nobody will ever see it. You’ll know it’s there, so you’re going to use a beautiful piece of wood on the back. For you to sleep well at night, the care has to be carried all the way through. It is the quality that I try to carry in all my work, and especially this release. I hope you will like to use and build with Nanobe just as much as I liked building it.
With hopes, questions, complaints, and confessions — please reach out to ag@nanobe.sh