Receipt scanning, semantic image search, and LLM inference are moving to phones, Macs, iPads, consoles, and consumer GPUs. The trend demonstrates local processing as a practical option for private documents and self-hosted workloads.
A jailbroken PS5 performs inference with Qwen 27B at around 20 tokens per second on its GPU. The Reddit report shows how consumer hardware with unified memory can support self-hosted inference for a 27B-parameter model, lowering the barrier for on-pr...
A 0.8B model reading receipts on the phone shows on-device inference as a concrete option for personal documents. The project combines fine-tuning on donated data, JSON output and a self-hosted release valve via an OpenAI-compatible server. For those...
A developer built a free Android app that uses a 0.8-billion-parameter vision model to read receipt photos and extract store, date, total, and line items into JSON. The model, fine-tuned on receipts, runs on the phone without sending data to third pa...
A community ranking evaluates LLMs that can run on consumer GPUs with 10-16GB of VRAM and 26-32GB of RAM. The criteria include minimum 10 tokens/s, narrative coherence, handling of typos and nuance, and frequency of slop patterns. The results highlig...
DigUp is a free, open-source Mac app that runs EmbeddingGemma 2 locally to search files by content. The 865 MB Q8_0 GGUF model runs on Apple Silicon through llama.cpp and Metal; search loads only the 250 MB text encoder and responds in about a tenth ...
LemonSeed Studio runs local Qwen3.8-27B inference on iPad Pro with an external AMD GPU in a Thunderbolt enclosure. The LSE engine, identical across iPadOS, macOS, and Linux, hits 158.5 tokens/s on R9700 with DFlash2 speculative decoding and 64.4 on S...
Four open decision models tested on a rented RTX 4090: Laya, d1, Clef-Flash and Lev read 9,534 words and flagged centipede names. Laya is fastest at 3.9 ms per word on llama.cpp; Lev is most accurate on the target but about 13x slower. The test shows...
EmbeddingGemma 2 maps text and images into a 768-dimensional space. Ported to ruNNtime, a TypeScript WebGPU inference library, it enables photo search by description with inference entirely on the browser GPU. No upload, data stays on device: a concr...
