LensVLM: Compressing Long Context as Images and Expanding Only Relevant Pages

Apple has posted a model called LensVLM-9B on Hugging Face. The title on the model page states the idea compactly: long context is compressed as images, and only the relevant pages are expanded. The submission drew 76 points and 7 comments on Hacker News, a small thread for a design question that deserves more attention.

Two costs of long context

Feeding a model a long document costs twice. Token count drives attention compute and memory, and the bill grows faster than linearly as context lengthens. The second cost is quieter: many documents are laid out, not written. PDFs, scans, slide decks and tables carry meaning in their structure — heading levels, column alignment, figure placement. Converting them to plain text throws that structure away before the model sees the content, and the conversion itself can introduce errors that are hard to trace later.

Vision offers a middle path. Render a page as an image, run it through a vision encoder, and the page becomes a bounded set of visual tokens, usually far fewer than the same page would take as text. What is lost is fine detail. Small type, formulas and dense tables degrade first, and those are often exactly the parts people need.

Keep the document compressed, develop only a few pages

The name carries the idea: a lens that compresses context into images, and an expansion step applied only to relevant pages. The document sits in a cheap representation, and the system decides which pages deserve richer treatment for the question at hand.

That folds retrieval and reading into a single pass. Scan globally with an inexpensive representation, decide which pages justify compute, then pay the high-resolution or high-token cost for those pages alone. Against the common two-stage pipeline — chunk the document, embed the chunks, retrieve a few passages — this keeps layout cues intact. Tables stay aligned. Headings stay above the text they govern. Those cues are frequently severed by text chunking, and they are often what makes a document legible in the first place.

What the public material does not answer

The verifiable detail is thin. The publisher is Apple, the model is LensVLM-9B, and the “9B” conventionally points to parameter count. Beyond that, the title and link do not say how a page is judged relevant, whether an expanded page is re-encoded as visual tokens or passed through OCR into text, or which long-document benchmarks the model was evaluated on.

Those details decide whether the approach holds up. A page-level relevance call is a commitment: get it wrong and the error flows straight into the answer, and it is harder to catch than a bad chunk match, because a compressed image is an opaque representation. Looking at it, you cannot easily tell what the model actually read. There is also a rendering tradeoff. Raise the resolution and the visual token count rises with it, eating into the savings the compression was meant to produce.

What form should context take?

The thread on Hacker News is small, but the question is not. Before a long document reaches a model, what should it look like? Plain text, page images, or some mixture of both depends on how sensitive the task is to detail.

Holding an entire document as images and developing only the pages that matter is one engineering answer to that question. It saves on pages nobody needs to read closely, and it bets that the model can make good relevance decisions from a low-resolution view. Whether that bet pays off will show up in evaluations, ablations, and how the thing behaves on real PDFs and scans.