Bug 276428
| Summary: | Tools/Scripts/run-webkit-tests does not recognize WPT crash tests in a crashtest directory | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Frédéric Wang (:fredw) <fred.wang> |
| Component: | Tools / Tests | Assignee: | Nobody <webkit-unassigned> |
| Status: | RESOLVED FIXED | ||
| Severity: | Normal | CC: | ahmad.saleem792, gsnedders, harpsichen, ntim, webkit-bug-importer |
| Priority: | P2 | Keywords: | InRadar |
| Version: | WebKit Nightly Build | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Bug Depends on: | |||
| Bug Blocks: | 276414 | ||
Frédéric Wang (:fredw)
I understand we somehow support WPT crashtests now:
https://web-platform-tests.org/writing-tests/crashtest.html
https://searchfox.org/wubkat/search?q=crashtests&path=Tools%2FScripts&case=false®exp=false
However, when running
$ Tools/Scripts/run-webkit-tests imported/w3c/web-platform-tests/mathml/crashtests
I get "All tests skipped."
If instead I do
$ Tools/Scripts/run-webkit-tests imported/w3c/web-platform-tests/css/css-tables/crashtests
I get "All 7 tests ran as expected."
So only 7 of the 24 crash tests are considered, apparently only those with an explicit *-crash* suffix.
$ ls LayoutTests/imported/w3c/web-platform-tests/css/css-tables/crashtests/ | wc --lines
24
$ ls LayoutTests/imported/w3c/web-platform-tests/css/css-tables/crashtests/*-crash* | wc --lines
7
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Radar WebKit Bug Importer
<rdar://problem/131926145>
Ahmad Saleem
Something related but might be different - it also case with SVG crash tests and editing related test case. It can import but it does not generate 'expected test files'.
Sam Sneddon [:gsnedders]
https://github.com/WebKit/WebKit/blob/6eef85d192a8f02fa74fc620854c83dd0fff6fed/Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder.py#L612-L620 should be handling this, but clearly isn't, unless we're somehow also ignoring them.
Harry Chen
It seems that the tests aren't running since they're listed in `resource-files.json` wrongfully -- this commit [1] was the one that added them. I believe that the importer script used was on this revision [2], since the time lines up and it would also explain why `-crash.` tests aren't in `resource-files.json` but `crashtests` tests are (since the commit only checks for `-crash.` tests).
Since it looks like the logic was fixed as part of the commit in [3], I think just removing the tests from `resource-files.json` should be good enough.
[1]: https://github.com/WebKit/WebKit/commit/a31f8e4d6253f04c8d955e42985e6e018d3a1a86
[2]: https://github.com/WebKit/WebKit/commit/d8b1589e4915042608bb1a1a55bd291bcec76d51#diff-955991178e0e7971ad17ceface65532b503e98975aec31d0fc7093e0d80c3dc5R167-R169
[3]: https://github.com/WebKit/WebKit/commit/b70287103be9d7a826cb2697e7a951536dc2a04b
Harry Chen
Pull request: https://github.com/WebKit/WebKit/pull/34374
EWS
Committed 284409@main (5a279668079a): <https://commits.webkit.org/284409@main>
Reviewed commits have been landed. Closing PR #34374 and removing active labels.