A 330 MB embedding model, a consumer GPU and an archive of screenshots: that is the combination that allowed one user to test semantic text search inside images without relying on cloud services. The model is EmbeddingGemma2, run through llama.cpp on an RX 7900GRE via Vulkan; the Q8_0 quantized size is around 330 MB and the output vector has 768 dimensions. On a set of 3,003 screenshots from the visual novel Hanasaki Workspring, indexing took 510 seconds, roughly 5.9 images per second with batch size 8 and resizing to 1,280 pixels.

The experiment, conducted on a few thousand images and explicitly non-scientific, still reveals useful behavior for anyone evaluating self-hosted retrieval. A query with the Korean sentence '์‚ผ๊ฐ๊ฑด์ด ๋จธ๋ฆฌ์— ์–น์–ด์กŒ๋‹ค' returned the correct candidate with cosine similarity 0.71156 and a margin of 0.00810 over the next result; the same request in English, 'triangular cloth hat was place on top of head', stopped at 0.62414, a sign that the model captures meaning across languages but with lower similarity.

The critical issue is not speed but sensitivity to wording. The author notes differences between 'blond' and 'blonde', a detail that may seem marginal but changes ranking in a homogeneous archive. In the screenshots used, 90% of the images contain a blonde girl: a generic visual attribute does not help discriminate and, in some cases, adding it to the query worsens similarity. In Korean, the presence of a rare word dominates the whole sentence: '์‚ผ๊ฐ๊ฑด' appears in only 2 images out of 3,003 and produces a clear jump in margin over subsequent candidates. That shifts attention from the model to the corpus and query strategy: indexing is not enough; teams need to understand which terms carry discriminatory power in their own dataset.

Structurally, the test signals that on-prem multimodal embedding has entered a practical phase. A 330 MB model with 768 dimensions and a 4,096-token context runs on a single consumer GPU and can index thousands of screenshots in minutes. No external APIs or uploads of potentially sensitive images are required. For business archives, technical documentation or mixed-language datasets, this lowers the barrier to self-hosting and keeps data under control. But it introduces a different operational cost: retrieval quality depends on how well the content is known and how queries are written. Teams adopting this approach must plan for iteration over descriptions, internal vocabularies and tests on their own corpus, not just a model choice.

In the short term, what loses ground is the assumption that a small model is interchangeable with a cloud multimodal embedding service. The difference is not in architecture but in predictability: the local model works, but its limits (textual dominance, weak visual traits, sensitivity to spelling variants) emerge precisely when the dataset is visually homogeneous. That is not a defect to fix in the abstract; it is a constraint to map onto the use case. For anyone evaluating on-prem deployment, this kind of test on one's own corpus is more informative than any generic benchmark. AI-RADAR explores these trade-offs in its /llm-onpremise framework.