Bug 261840

Summary:	REGRESSION(268166@main): [ macOS ] 4 API tests under TestWebKitAPI.WKInspectorExtensionHost: 2 constantly failing, 2 constantly timing out
Product:	WebKit	Reporter:	Ben Schwartz <ben_schwartz>
Component:	Web Inspector	Assignee:	Matt Woodrow <mattwoodrow>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	ap, inspector-bugzilla-changes, mattwoodrow, mishal.rehman1787, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified
See Also:	https://bugs.webkit.org/show_bug.cgi?id=261717

Description Ben Schwartz 2023-09-20 14:06:24 PDT

The following API tests are constantly failing on macOS:

TestWebKitAPI.WKInspectorExtensionHost.RegisterExtension
TestWebKitAPI.WKInspectorExtensionHost.UnegisterExtension

The following API tests are constantly timing out on macOS:

TestWebKitAPI.WKInspectorExtension.EvaluateScriptOnPage
TestWebKitAPI.WKInspectorExtension.ExtensionTabIsPersistent


ERROR LOG (for failures):

    TestWebKitAPI.WKInspectorExtensionHost.RegisterExtension
        
        /Volumes/Data/worker/Apple-Monterey-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKInspectorExtensionHost.mm:83
        Value of: error
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/Apple-Monterey-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKInspectorExtensionHost.mm:84
        Value of: !(extension)
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/Apple-Monterey-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKInspectorExtensionHost.mm:85
        Value of: [error.localizedFailureReason containsString:@"RegistrationFailed"]
          Actual: false
        Expected: true
        
    TestWebKitAPI.WKInspectorExtensionHost.UnregisterExtension
        
        /Volumes/Data/worker/Apple-Monterey-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKInspectorExtensionHost.mm:155
        Value of: error
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/Apple-Monterey-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKInspectorExtensionHost.mm:156
        Value of: [error.localizedFailureReason containsString:@"InvalidRequest"]
          Actual: false
        Expected: true

STDIO URL:

https://build.webkit.org/api/v2/logs/24835290/raw

REPRODUCIBILITY:

I was able to reproduce this issue on macOS Ventura (13.5, 22G74/arm64) ToT using the following command:

run-api-tests --verbose TestWebKitAPI.WKInspectorExtensionHost.RegisterExtension TestWebKitAPI.WKInspectorExtensionHost.UnregisterExtension TestWebKitAPI.WKInspectorExtension.EvaluateScriptOnPage TestWebKitAPI.WKInspectorExtension.ExtensionTabIsPersistent

REGRESSION:

The flakiness dashboard, bisection, and contextual information all indicate a likely regression point at 268166@main.

Comment 1 Radar WebKit Bug Importer 2023-09-20 14:07:10 PDT

<rdar://problem/115803479>

Comment 2 EWS 2023-09-20 16:25:16 PDT

Committed 268221@main (1a475262cc73): <https://commits.webkit.org/268221@main>

Reviewed commits have been landed. Closing PR #17994 and removing active labels.

Comment 3 Matt Woodrow 2023-09-20 16:44:12 PDT

Re-opening for pull request https://github.com/WebKit/WebKit/pull/17996

Comment 4 Alexey Proskuryakov 2023-09-20 17:10:47 PDT

This PR is in Closed state. What is the next step for this?

Comment 5 Matt Woodrow 2023-09-20 17:14:50 PDT

The change has been reverted, comment 2.