Bug 258399

Summary: TestExpectations should allow for glob-like matching of paths
Product: WebKit Reporter: Sam Sneddon [:gsnedders] <gsnedders>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Normal CC: ap, jbedard, jenner, Morningstar, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   

Description Sam Sneddon [:gsnedders] 2023-06-22 03:55:56 PDT 
Currently we have a whole bunch of lines like:

webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-intersection-observer.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-invalidation.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-001.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-002.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-003.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-004.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-005.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-006.html [ Skip ]
webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-007.html [ Skip ]

And more below:

imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-010.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-011.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-mul-column-border-box.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-mul-column-content-box.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-svg.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-visual-box-and-value-with-border-radius.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-visual-box-and-value.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-margin-visual-box.html [ Skip ]
imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-y-visible-x-svg.html [ Skip ]

Most of these lines were added in a commit containing: "Skip some tests, for example some rely on unimplemented overflow: clip property."

If we supported glob/fnmatch-like matching of paths we could instead just have a singular:

webkit.org/b/198230 imported/w3c/web-platform-tests/css/css-overflow/overflow-clip-* [ Skip ]

(Of course, this _also_ shows one of the big problems of how we skip things we don't implement—we _have_ implemented overflow: clip for two years now, but all these tests are still skipped. Auditing expectations where we have a bug linked that's now resolved would've flagged up some but not all of these.)
Comment 1 Radar WebKit Bug Importer 2023-06-22 03:56:08 PDT 
<rdar://problem/111150877>
Comment 2 Alexey Proskuryakov 2023-06-22 09:04:27 PDT 
The downside of this proposal is that grepping TestExpectations for "overflow-clip-margin-intersection-observer.html" won't work to find that we already have an expectation.

Of course, it's already not 100% (because the whole directory can have an expectation), but (1) it's still a quick sanity check that myself and others use all the time, and (2) when suspecting foul play, one can grep for parent directory names.

If we add pattern matching, then we also need better tools to tell one if a test is subject to expectations.