Furst/furst-read
nak0x 3f71d324c9 Add search, feeds and listing views to the reader server
Extraction looks for prose, so a front page, a comment thread or a search
result page correctly yields almost nothing. These render the structure
instead, and every link they emit routes back through /read.

  /search?q=   results from a configurable HTML endpoint
  /feed?u=     RSS and Atom
  /read?u=     now picks a site view, falling back to a link index when a
               page has too little text to be an article

Feeds are scanned rather than parsed. A feed needs five fields per entry
and an HTML parser mangles XML, so this walks the tags directly: CDATA,
named and numeric entities, and Atom's preference for rel=alternate over
rel=self. No XML dependency.

Hacker News gets a real adapter: stories with score, author and a link
into the discussion, and comment threads rendered with their indent
preserved. Comment bodies go through the article renderer so links inside
them behave like every other link.

Search is deliberately engine-agnostic. Known result shapes are tried
first, then heading links, then any link, because every free HTML endpoint
eventually rate-limits a repeat visitor. When one answers with a challenge
page rather than results the reader says so and points at the config,
instead of showing an empty page; detection reads the body, since these
arrive as 200 or 202 rather than an error status. A blocked search is
never cached.

Pages that turn out to be feeds redirect to the feed view, and a page that
declares its own feed offers it.
2026-09-06 20:07:49 +02:00
..
src Add search, feeds and listing views to the reader server 2026-09-06 20:07:49 +02:00
Cargo.toml Add furst-read, an article extractor and minimal renderer 2026-09-06 19:35:30 +02:00
README.md Add furst-read, an article extractor and minimal renderer 2026-09-06 19:35:30 +02:00

furst-read

Fetch a page, throw away everything that is not the article, and render what is left as a small self-contained HTML document — no scripts, no stylesheets, no webfonts, no third-party requests.

This is the part of the toolset that does the real work. Choosing a lighter browser engine buys 23×. Not loading 3MB of JavaScript buys considerably more.

Use

furst-read <url>              # render and open in a browser (default)
furst-read --html <url>       # minimal HTML to stdout
furst-read --text <url>       # plain text to stdout
furst-read --out page.html <url>
furst-read --no-images <url>
curl -s <url> | furst-read --stdin --text <url>

The default writes to $XDG_CACHE_HOME/furst-read/<hash>.html and opens it in $FURST_BROWSER, $BROWSER, or the first light browser on $PATH. Because the file is named by a hash of the URL, revisiting a page reuses it instead of littering.

Slots straight into furst:

[[rule]]
name = "reader"
hosts = ["wikipedia.org", "news.ycombinator.com", "*.substack.com"]
run = ["furst-read", "{url}"]

How it finds the article

Readability's approach, reimplemented:

  1. Score p, pre, td and blockquote by how much prose they hold — character count and comma count, which boilerplate has neither of.
  2. Push each score up to the ancestors: parent takes it whole, grandparent half, and it thins out from there.
  3. Weight by class and id. post-content gains, sidebar loses.
  4. Multiply by 1 linkDensity, which is what separates a nav block from a paragraph.
  5. Take the winner, then pull in sibling nodes that also read like body copy, because articles are routinely split across several divs.

Then serialise against a tag whitelist. Wrapper elements contribute their children and no tag of their own, which is how the div soup disappears.

Details that matter in practice

  • Redirect stubs. A <meta http-equiv="refresh"> is followed, including when it is tucked inside <noscript>, which is how most static site generators write them.
  • Lazy images. A src holding a placeholder falls back to data-src and friends, so the images are the real ones.
  • srcset takes the first candidate, not the last — on old hardware the smallest variant is the one you want.
  • Character sets. Content-Type first, then <meta charset>, then UTF-8. Latin-1 pages are still common enough to matter.
  • Links are made absolute against the post-redirect URL, so the output works from a file:// path.
  • Scripts and styles are never counted as text, so a page cannot inflate its own score with an inline bundle.

Limits

  • Listing pages and forums have no single article to find. The extractor will return whatever scored highest, which may be very little.
  • Anything that renders client-side arrives empty; there is no JavaScript here and that is the point.
  • Paywalls and interstitials are not defeated, only rendered plainly.