nak0x
af9723714d
furst-serve needs the fetch, extract and render stages, so they move behind
a lib target with the CLI as a thin consumer.
Adds Options::link_prefix. When set, http and https links are rewritten to
{prefix}{percent-encoded url} so that following one stays inside the reader
rather than dropping the viewer back onto the live site; other schemes are
left alone. render::body exposes the article markup without the standalone
document wrapper, for callers supplying their own page chrome, and CSS
becomes public so they can reuse it.
Percent encode and decode live in urljoin, next to the other URL handling.
|
||
|---|---|---|
| .. | ||
| src | ||
| Cargo.toml | ||
| README.md | ||
furst-read
Fetch a page, throw away everything that is not the article, and render what is left as a small self-contained HTML document — no scripts, no stylesheets, no webfonts, no third-party requests.
This is the part of the toolset that does the real work. Choosing a lighter browser engine buys 2–3×. Not loading 3MB of JavaScript buys considerably more.
Use
furst-read <url> # render and open in a browser (default)
furst-read --html <url> # minimal HTML to stdout
furst-read --text <url> # plain text to stdout
furst-read --out page.html <url>
furst-read --no-images <url>
curl -s <url> | furst-read --stdin --text <url>
The default writes to $XDG_CACHE_HOME/furst-read/<hash>.html and opens it in
$FURST_BROWSER, $BROWSER, or the first light browser on $PATH. Because the
file is named by a hash of the URL, revisiting a page reuses it instead of
littering.
Slots straight into furst:
[[rule]]
name = "reader"
hosts = ["wikipedia.org", "news.ycombinator.com", "*.substack.com"]
run = ["furst-read", "{url}"]
How it finds the article
Readability's approach, reimplemented:
- Score
p,pre,tdandblockquoteby how much prose they hold — character count and comma count, which boilerplate has neither of. - Push each score up to the ancestors: parent takes it whole, grandparent half, and it thins out from there.
- Weight by
classandid.post-contentgains,sidebarloses. - Multiply by
1 − linkDensity, which is what separates a nav block from a paragraph. - Take the winner, then pull in sibling nodes that also read like body copy, because articles are routinely split across several divs.
Then serialise against a tag whitelist. Wrapper elements contribute their children and no tag of their own, which is how the div soup disappears.
Details that matter in practice
- Redirect stubs. A
<meta http-equiv="refresh">is followed, including when it is tucked inside<noscript>, which is how most static site generators write them. - Lazy images. A
srcholding a placeholder falls back todata-srcand friends, so the images are the real ones. srcsettakes the first candidate, not the last — on old hardware the smallest variant is the one you want.- Character sets. Content-Type first, then
<meta charset>, then UTF-8. Latin-1 pages are still common enough to matter. - Links are made absolute against the post-redirect URL, so the output
works from a
file://path. - Scripts and styles are never counted as text, so a page cannot inflate its own score with an inline bundle.
Limits
- Listing pages and forums have no single article to find. The extractor will return whatever scored highest, which may be very little.
- Anything that renders client-side arrives empty; there is no JavaScript here and that is the point.
- Paywalls and interstitials are not defeated, only rendered plainly.