nak0x
76d0906564
Following a link inside an extracted article used to drop the reader back onto the live site. The server rewrites in-page links to /read?u=... so browsing stays in reader mode, and caches rendered pages on disk, which takes a revisit from ~350ms to under a millisecond. The HTTP layer is blocking and hand-rolled: GET only, one response per connection, a fixed pool of four worker threads so memory stays predictable, and nothing but the loopback interface is ever bound. Routes so far: / home, from ~/.config/furst/home.toml /read?u= the article, with links routed back through the reader /go?u= hand the original URL to the heavy browser Pages that are not documents redirect to the original rather than being run through an article extractor, and pages with too little text to be an article say so and offer the escape hatches. Redirect stubs are followed before that judgement is made. furst-serve --open ensures a server is running, starting a detached one if needed, then execs a browser at the reader URL. Cache entries carry a schema number so a renderer change drops them rather than serving stale markup. |
||
|---|---|---|
| .. | ||
| src | ||
| Cargo.toml | ||
| README.md | ||
furst-read
Fetch a page, throw away everything that is not the article, and render what is left as a small self-contained HTML document — no scripts, no stylesheets, no webfonts, no third-party requests.
This is the part of the toolset that does the real work. Choosing a lighter browser engine buys 2–3×. Not loading 3MB of JavaScript buys considerably more.
Use
furst-read <url> # render and open in a browser (default)
furst-read --html <url> # minimal HTML to stdout
furst-read --text <url> # plain text to stdout
furst-read --out page.html <url>
furst-read --no-images <url>
curl -s <url> | furst-read --stdin --text <url>
The default writes to $XDG_CACHE_HOME/furst-read/<hash>.html and opens it in
$FURST_BROWSER, $BROWSER, or the first light browser on $PATH. Because the
file is named by a hash of the URL, revisiting a page reuses it instead of
littering.
Slots straight into furst:
[[rule]]
name = "reader"
hosts = ["wikipedia.org", "news.ycombinator.com", "*.substack.com"]
run = ["furst-read", "{url}"]
How it finds the article
Readability's approach, reimplemented:
- Score
p,pre,tdandblockquoteby how much prose they hold — character count and comma count, which boilerplate has neither of. - Push each score up to the ancestors: parent takes it whole, grandparent half, and it thins out from there.
- Weight by
classandid.post-contentgains,sidebarloses. - Multiply by
1 − linkDensity, which is what separates a nav block from a paragraph. - Take the winner, then pull in sibling nodes that also read like body copy, because articles are routinely split across several divs.
Then serialise against a tag whitelist. Wrapper elements contribute their children and no tag of their own, which is how the div soup disappears.
Details that matter in practice
- Redirect stubs. A
<meta http-equiv="refresh">is followed, including when it is tucked inside<noscript>, which is how most static site generators write them. - Lazy images. A
srcholding a placeholder falls back todata-srcand friends, so the images are the real ones. srcsettakes the first candidate, not the last — on old hardware the smallest variant is the one you want.- Character sets. Content-Type first, then
<meta charset>, then UTF-8. Latin-1 pages are still common enough to matter. - Links are made absolute against the post-redirect URL, so the output
works from a
file://path. - Scripts and styles are never counted as text, so a page cannot inflate its own score with an inline bundle.
Limits
- Listing pages and forums have no single article to find. The extractor will return whatever scored highest, which may be very little.
- Anything that renders client-side arrives empty; there is no JavaScript here and that is the point.
- Paywalls and interstitials are not defeated, only rendered plainly.