Bug 262680

Summary: Tests with captionDisplayMode option are flaky on ios-wk2
Product: WebKit Reporter: Kueno <kohei.ueno119>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   

Description Kueno 2023-10-04 23:24:47 PDT
The `captionDisplayMode` option was introduced in https://bugs.webkit.org/show_bug.cgi?id=261460 (PR: https://github.com/WebKit/WebKit/pull/17689). However when I attempted to import WPT for webvtt/parsing with `captionDisplayMode` setting enabled, I found that the tests were failing intermittently on ios-wk2 (example: https://github.com/WebKit/WebKit/pull/17852#issuecomment-1723066317).

The flakiness of these tests appears to be due to the fact that the `CaptionUserPreferencesTestingModeToken` instance may not always be initialized before the caption mode is set (https://github.com/WebKit/WebKit/blob/659f85098cf4322282a775cbd51eb58a4ed34022/Tools/WebKitTestRunner/InjectedBundle/InjectedBundle.cpp#L583). This instance is usually initialized on the `Internals` class's constructor during the WebKitTestRunner setup (https://github.com/WebKit/WebKit/blob/659f85098cf4322282a775cbd51eb58a4ed34022/Tools/WebKitTestRunner/InjectedBundle/InjectedBundlePage.cpp#L888). In most cases, the `CaptionUserPreferencesTestingModeToken` instance is created in the `Internals` class constructore before `WKBundlePageSetCaptionDisplayMode` is called, but there are instances where it is not, particularly when tests are run concurrently. This leads to the tests failing.

To address this issue, the WebKitTestRunner needs to ensure that the `CaptionUserPreferencesTestingModeToken` instance is always initialized before `WKBundlePageSetCaptionDisplayMopde` is called, then eliminating the flakiness of the tests.
Comment 1 Kueno 2023-10-04 23:27:03 PDT
Pull request: https://github.com/WebKit/WebKit/pull/18675
Comment 2 Radar WebKit Bug Importer 2023-10-11 23:25:14 PDT
<rdar://problem/116848999>
Comment 3 EWS 2023-10-18 01:16:52 PDT
Committed 269455@main (9948a5b7f373): <https://commits.webkit.org/269455@main>

Reviewed commits have been landed. Closing PR #18675 and removing active labels.