From numeric-ID lookup to real search: BLIP + Django tags

Earlier this year at Kreative I spent some time rebuilding our internal stock-image gallery. The TLDR is in the case study; this post is the slightly longer technical version — what shape the problem took, why I chose BLIP, and the handful of design decisions that mattered more than the model choice itself.

The actual problem

The gallery already existed. It had search. The search worked. The problem was what it searched by: a numeric image ID.

That's fine for the one scenario where someone already knows the ID of the image they want. For every other scenario — "do we have any beach photos," "find that one with the orange sunset," "something cheerful for the newsletter header" — the search box was useless, and the fallback was scrolling. Scrolling 38,504 images across 54 folders. For a tool people reach for every day, that gap between what the UI implies and what it actually does gets expensive fast.

So the goal wasn't "add fancy AI to the gallery." The goal was "make the search box do what a search box is supposed to do." The AI part was just the cheapest way to get there without asking the team to hand-caption tens of thousands of images.

Why BLIP instead of a cloud vision API

The obvious alternative was OpenAI's vision model, Google Cloud Vision, or AWS Rekognition. All three would have worked. BLIP won on three fronts:

• The stock library doesn't leave our infrastructure. This matters more than people usually realise with a managed service; even when a provider swears they don't train on your data, "we sent our whole stock library to a third party" is a conversation nobody wants to have with legal twice.
• Cost is a one-time compute cost, not per-image forever. BLIP runs on a GPU we already had. Tagging the full library was essentially free past the first evening of compute.
• Reproducibility. The model doesn't change out from under me. If we want to re-tag the library a year from now with the same model and the same prompts, we can.

Downsides were real too: self-hosting means I own the inference pipeline, batching, and failure modes, which took longer than wiring up a cloud API would have. For a smaller library or a team without a GPU lying around, I'd happily recommend a managed service instead.

The pipeline

The tagging script is boring in a good way. Roughly:

1. Walk the Image table and feed BLIP-large in batches of 16, with 12 parallel downloads keeping the GPU fed.
2. For each image, run inference in fp16 and ask the model for a short descriptive caption.
3. Normalise the caption into a set of tags — lowercase, split on obvious boundaries, drop stopwords, collapse plurals, keep anything the model emitted with enough confidence.
4. Upsert tags into a Tag model and link them via a through table.
5. Skip anything already tagged on re-runs, and checkpoint between batches so a crash doesn't restart from zero.

Nothing exciting — the restraint is the point. Letting the model produce a caption and deriving tags from it, instead of asking the model for tags directly, meant the output was more consistent and easier to inspect while I was tuning things.

On a single RTX 3060 Ti this shape of pipeline sustains ~30–60 images/sec, which puts a full cold run of all 38,504 images at roughly 10–20 minutes. On the shipped run every image came out with both tags and a short description — 100% coverage, 7.9 tags per image on average (range 3–13), zero recorded fetch failures. Incremental re-runs are much faster because the skip-already-tagged check means only genuinely new images pay BLIP time.

The Django side

The schema change was two models:

class Tag(models.Model):
    name = models.CharField(max_length=80, unique=True)
    # ...aliases, curator flags, created_at, etc.

class ImageTag(models.Model):
    image = models.ForeignKey(Image, on_delete=models.CASCADE)
    tag = models.ForeignKey(Tag, on_delete=models.CASCADE)
    source = models.CharField(max_length=20)  # "blip", "manual", etc.
    confidence = models.FloatField(null=True)

    class Meta:
        unique_together = ("image", "tag")

The old search view was a single .filter(id=...). The new one takes a query string, splits it into terms, looks up tags for each term, and ranks results by how many of the query's tags each image matches — with ordering tiebreakers for recency and (where present) confidence. The whole thing is a couple of ORM calls backed by an index on tag.name, so it's quick even as the library grows.

A fallback branch handles the "no tags matched" case: rather than show nothing, I fuzzy-match the query against tag names, so a typo for "sunset" still returns sunsets.

What I'd do differently

Two things, in rough order of importance:

• Store the raw caption, not just the derived tags. I ended up adding that in a follow-up because it was useful for debugging (and later for building a "similar images" feature). It's the kind of thing you want from day one.
• Don't be clever about tag normalisation up front. My first pass tried to be too aggressive about collapsing variants ("beach," "beaches," "beachside"). The second pass was simpler and relied on tag aliases instead — easier to reason about, easier to curate.

The takeaway

The model wasn't the hard part. The hard part was deciding where in the stack the AI output should live — captions on the image row vs. tags in a separate table vs. a search index of some kind — and then committing to a schema that keeps that output useful as other features show up later.

If you're working on something similar and want to compare notes, say hi.