Mario Sanchez Prada
7d716d895e
This patch disables entire *_browsertest.cc files by making them empty
via chromium_src overrides, as well as all the JS tests via a patch to
include_js_tests.gni, so that we avoid running any test that we know
are going to fail because of the many ways in which Brave is different
than Chromium upstream.
Note that it's possible that this way of excluding tests is a bit too
agressive (i.e. it's likely that some browser tests inside the excluded
files would pass), but for now it's a good enough initial approach as
it enables us to considerably increase test coverage without having
to maintain patches in a too intrusive way.
As a reference, a this time of this patch's writing (on top of Brave
1.23.30 / Chromium 89.0.4389.86), this are the stats when running
upstream's unit tests on a Linux/Debug build, without this patch:
* Total run: 13355 tests
* Passed: 10427 tests
* Not passed: 2928 tests
- Failed: 870 tests
- Crashed: 1593 tests
- Timed out: 465 tests
With this patch applied, the numbers look like this:
* Total run: 2957 tests
* Passed: 2957 tests
* Not passed: 0 tests
That is, what we have with this patch applied looks as follows:
* 2957/13355 -> 22.14% of ALL the original tests being run
* 2957/10427 -> 28.36% of the PASSING TESTS still being run
In other words, we're increasing test coverage in 2957 browser tests
in a relatively clean way (i.e. no complex patching) while losing
71.64% of the browser tests that would run and pass if we were not
excluding them in such an agressive way. Not as good results as for
the case of unit tests, but still an improvement for test coverage.
Fix https://github.com/brave/brave-browser/issues/8297
12 lines
533 B
Diff
12 lines
533 B
Diff
diff --git a/chrome/test/include_js_tests.gni b/chrome/test/include_js_tests.gni
|
|
index 72a439884d2be78df848ceaddca6fe7237f30a6c..67212f0a7b902bec22a89d0fc56999fc7a510b6c 100644
|
|
--- a/chrome/test/include_js_tests.gni
|
|
+++ b/chrome/test/include_js_tests.gni
|
|
@@ -3,5 +3,6 @@ import("//build/config/sanitizers/sanitizers.gni")
|
|
if (!is_android) {
|
|
# js_tests don't work in cross builds, https://crbug.com/1010561
|
|
include_js_tests =
|
|
+ false &&
|
|
!(is_asan || is_msan || is_tsan || is_cfi || (is_win && host_os != "win"))
|
|
}
|