Bug 254403

Summary: Simplify baseline search paths (wk1, wk2, ios-simulator)
Product: WebKit Reporter: Sam Sneddon [:gsnedders] <gsnedders>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Normal CC: ap, jbedard, jenner, Morningstar, rackler, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on: 262074    
Bug Blocks:    
Attachments:
Description Flags
Current search paths none

Description Sam Sneddon [:gsnedders] 2023-03-24 04:26:01 PDT
Currently we have a multitude of pretty similar baseline search paths, and it's often unclear where expectations should go.

As an attempt to simplify this, I suggest:

* The default baseline always corresponds to wk2 (this avoids a necessity to have, e.g., "mac", "mac-wk1", "and "mac-wk2": it becomes clear the wk2 expectation belongs in "mac"). This also implies getting rid of "wk2" as a place with expectations, and potentially adding a "wk1" (though do any ports aside from Apple still maintain WK1 and thus test expectations?).

* We move to only having "ios" (and not also "ios-simulator", "ios-device"), as this split is meaningless in WebKit where nobody outside of Apple can run tests on non-Simulator iOS and hence the expectations need to be maintained by Apple anyway. (And, e.g., if someone puts a failing expectation in ios-simulator in WebKit then we can quite easily end up with duplicate expectations between ios-simulator and ios-device, which helps nobody.) Note jbedard did related work in 187099@main.
Comment 1 Radar WebKit Bug Importer 2023-03-24 04:26:11 PDT
<rdar://problem/107182061>
Comment 2 Sam Sneddon [:gsnedders] 2023-06-02 16:16:53 PDT
Created attachment 466573 [details]
Current search paths

I think this graph accurately shows the status quo?