Bug 261840 - REGRESSION(268166@main): [ macOS ] 4 API tests under TestWebKitAPI.WKInspectorExtensionHost: 2 constantly failing, 2 constantly timing out
Summary: REGRESSION(268166@main): [ macOS ] 4 API tests under TestWebKitAPI.WKInspecto...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Inspector (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Matt Woodrow
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-09-20 14:06 PDT by Ben Schwartz
Modified: 2024-04-04 19:47 PDT (History)
6 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Ben Schwartz 2023-09-20 14:06:24 PDT 
The following API tests are constantly failing on macOS:

TestWebKitAPI.WKInspectorExtensionHost.RegisterExtension
TestWebKitAPI.WKInspectorExtensionHost.UnegisterExtension

The following API tests are constantly timing out on macOS:

TestWebKitAPI.WKInspectorExtension.EvaluateScriptOnPage
TestWebKitAPI.WKInspectorExtension.ExtensionTabIsPersistent


ERROR LOG (for failures):

    TestWebKitAPI.WKInspectorExtensionHost.RegisterExtension
        
        /Volumes/Data/worker/Apple-Monterey-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKInspectorExtensionHost.mm:83
        Value of: error
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/Apple-Monterey-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKInspectorExtensionHost.mm:84
        Value of: !(extension)
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/Apple-Monterey-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKInspectorExtensionHost.mm:85
        Value of: [error.localizedFailureReason containsString:@"RegistrationFailed"]
          Actual: false
        Expected: true
        
    TestWebKitAPI.WKInspectorExtensionHost.UnregisterExtension
        
        /Volumes/Data/worker/Apple-Monterey-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKInspectorExtensionHost.mm:155
        Value of: error
          Actual: false
        Expected: true
        
        
        /Volumes/Data/worker/Apple-Monterey-Release-Build/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKInspectorExtensionHost.mm:156
        Value of: [error.localizedFailureReason containsString:@"InvalidRequest"]
          Actual: false
        Expected: true

STDIO URL:

https://build.webkit.org/api/v2/logs/24835290/raw

REPRODUCIBILITY:

I was able to reproduce this issue on macOS Ventura (13.5, 22G74/arm64) ToT using the following command:

run-api-tests --verbose TestWebKitAPI.WKInspectorExtensionHost.RegisterExtension TestWebKitAPI.WKInspectorExtensionHost.UnregisterExtension TestWebKitAPI.WKInspectorExtension.EvaluateScriptOnPage TestWebKitAPI.WKInspectorExtension.ExtensionTabIsPersistent

REGRESSION:

The flakiness dashboard, bisection, and contextual information all indicate a likely regression point at 268166@main.
Comment 1 Radar WebKit Bug Importer 2023-09-20 14:07:10 PDT 
<rdar://problem/115803479>
Comment 2 EWS 2023-09-20 16:25:16 PDT 
Committed 268221@main (1a475262cc73): <https://commits.webkit.org/268221@main>

Reviewed commits have been landed. Closing PR #17994 and removing active labels.
Comment 3 Matt Woodrow 2023-09-20 16:44:12 PDT 
Re-opening for pull request https://github.com/WebKit/WebKit/pull/17996
Comment 4 Alexey Proskuryakov 2023-09-20 17:10:47 PDT 
This PR is in Closed state. What is the next step for this?
Comment 5 Matt Woodrow 2023-09-20 17:14:50 PDT 
The change has been reverted, comment 2.