Some sites, like Slate and the BBC, have <div> elements enclosing single
<p> nodes. This weakens the paragraph's score from propagating higher in
the DOM during the scoring step. In the pre-process step, make this
replacement.
TODO: We may also want to consider doing the same thing for divs with no
block-level elements. I didn't implement it here because the function
_hasChildBlockElement() in Readability.js checks for non-block elements,
so the function in confusing and may be buggy? Not sure.
It is common for extra data to be in the title extracted title. It's
usually metadata after colons, separators (e.g '|', '-', '»', etc). Try
and strip those out, be be mindful of the resulting word length.
After the new body node is chosen as the readable root, iterate over
its sibling nodes, and append "related" siblings. This could be article
preambles, content split out by ads that were removed during the
preprocess step, etc.
We are maintaining a very minimal fork of kuchiki-rs at
https://github.com/keur/kuchiki/tree/speedreader. We expose two new
fields to API users: `score` and `is_candidate`. This removes the need
for both the nodes binary tree and the candidates binary tree. Kuchiki
has nice helper methods for accessing parents, descendants, and
ancestors, further removing the need for the Paths. The trees indexed by
`Path` structs were mostly hacking around the lack of API extensibility.
This is another step towards algorithmic parity with Readability.js.
In the current implementation, we root the new DOM at top scoring
candidate. With this patch, we keep 1 top candidate and 4 alternative
candidates. If 3/4 of those candidates have a score near the top
candidate, we start iterating backwards starting from the top candidate.
If there is a shared parent with any 3 of the alternative candidates, we
make that the new root.
This is important, because some articles on Vanity Fair score very high
with a deeply nested top candidate, and real content is deleted from the
new DOM.
This breaks the DOM structure and even gets rid of important content
spaces. We have a test case to cover a being deleted. Found on
rpp.pe
Refactor the test helper functions out of the main module.
The RcDom is deprecated and not recommended for production use.
Servo provides no guarantees about the stabality and security of the
Rcdom https://github.com/servo/html5ever/issues/385
Some sites, such as Medium and BBC, put <img> tags as text inside of a
<noscript> tag. They use this to achieve a blurred image effect, where
the original blurred image is replaced with the higher resolution image
after some Javascript on the page runs.
In reader mode we aren't interested in that functionality. For all
<noscript> tags, we parse the text and check if there is a single image
element embedded inside of it. If there is, we pull that into our DOM
during the preprocess step and replace the <noscript> with the <img>. If
the <noscript> tag had an <img> element as the preceding sibling we
remove that image to avoid rendering both the blurred and final image.
Chromium change:
https://source.chromium.org/chromium/chromium/src/+/989852ed58f849453d062bcd3f3eecb5ccc14df0
commit 989852ed58f849453d062bcd3f3eecb5ccc14df0
Author: Greg Thompson <grt@chromium.org>
Date: Mon Nov 23 22:32:27 2020 +0000
[FilePathWatcher] Use an enum rather than a bool for the type of watch.
No functional changes. This is in preparation for adding a new type for
macOS. An overload of the Watch method that takes a bool is left behind
for compatibility reasons. As a consequence, necessary casting is
introduced to pick the correct overload when using base::Bind*.
BUG=1147071
Kevin Kuehler