Bug 258399 - TestExpectations should allow for glob-like matching of paths
Summary: TestExpectations should allow for glob-like matching of paths
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-06-22 03:55 PDT by Sam Sneddon [:gsnedders]
Modified: 2023-06-22 09:04 PDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sam Sneddon [:gsnedders] 2023-06-22 03:55:56 PDT
Currently we have a whole bunch of lines like:

webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-intersection-observer.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-invalidation.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-001.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-002.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-003.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-004.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-005.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-006.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-007.html [ Skip ]

And more below:

imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-010.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-011.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-mul-column-border-box.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-mul-column-content-box.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-svg.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-visual-box-and-value-with-border-radius.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-visual-box-and-value.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-visual-box.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-y-visible-x-svg.html [ Skip ]

Most of these lines were added in a commit containing: "Skip some tests, for example some rely on unimplemented overflow: clip property."

If we supported glob/fnmatch-like matching of paths we could instead just have a singular:

webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-* [ Skip ]

(Of course, this _also_ shows one of the big problems of how we skip things we don't implement—we _have_ implemented overflow: clip for two years now, but all these tests are still skipped. Auditing expectations where we have a bug linked that's now resolved would've flagged up some but not all of these.)
Comment 1 Radar WebKit Bug Importer 2023-06-22 03:56:08 PDT
<rdar://problem/111150877>
Comment 2 Alexey Proskuryakov 2023-06-22 09:04:27 PDT
The downside of this proposal is that grepping TestExpectations for "overflow-clip-margin-intersection-observer.html" won't work to find that we already have an expectation.

Of course, it's already not 100% (because the whole directory can have an expectation), but (1) it's still a quick sanity check that myself and others use all the time, and (2) when suspecting foul play, one can grep for parent directory names.

If we add pattern matching, then we also need better tools to tell one if a test is subject to expectations.