Bug 257235 - [ iPhone XS+ ] 3X TestWebKitAPI.WKWebView (API-Tests) are constant failures
Summary: [ iPhone XS+ ] 3X TestWebKitAPI.WKWebView (API-Tests) are constant failures
Status: RESOLVED DUPLICATE of bug 257246
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKit API (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
: 257228 257239 (view as bug list)
Depends on:
Blocks:
 
Reported: 2023-05-23 16:02 PDT by Robert Jenner
Modified: 2023-05-24 17:12 PDT (History)
5 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Jenner 2023-05-23 16:02:06 PDT 
TestWebKitAPI.WKWebView.SnapshotAfterScreenUpdates
TestWebKitAPI.WKWebView.SnapshotImageBaseCase
TestWebKitAPI.WKWebView.SnapshotWithoutAfterScreenUpdates

are constant failures on iOS 16 Debug when run on iPhone XS simulator and higher. 

HISTORY:
https://results.webkit.org/?suite=api-tests&suite=api-tests&suite=api-tests&test=TestWebKitAPI.WKWebView.SnapshotAfterScreenUpdates&test=TestWebKitAPI.WKWebView.SnapshotImageBaseCase&test=TestWebKitAPI.WKWebView.SnapshotWithoutAfterScreenUpdates

According to the history, these are all passing. However, the simulator we are currently using is iPhone SE (3rd gen). These tests fail on iPhone XS and higher, and we want to change our current simulator for API tests from iPhone SE to iPhone 12. So these tests may need to be adjusted. 

STDIO FAILURE TEXT:
    TestWebKitAPI.WKWebView.SnapshotAfterScreenUpdates
        LEAK: 1 WebProcessPool

        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:507
        Expected equality of these values:
          0
          rgba[pixelIndex]
            Which is: '\xFF' (255)


        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:508
        Expected equality of these values:
          0
          rgba[pixelIndex + 1]
            Which is: '\xFF' (255)


    TestWebKitAPI.WKWebView.SnapshotImageBaseCase
        LEAK: 1 WebProcessPool

        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:292
        Expected equality of these values:
          0
          rgba[pixelIndex]
            Which is: '\xFF' (255)


        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:294
        Expected equality of these values:
          255
          rgba[pixelIndex + 2]
            Which is: '\0'


    TestWebKitAPI.WKWebView.SnapshotWithoutAfterScreenUpdates
        LEAK: 1 WebProcessPool

        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:570
        Expected equality of these values:
          0
          rgba[pixelIndex]
            Which is: '\xFF' (255)


        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:571
        Expected equality of these values:
          0
          rgba[pixelIndex + 1]
            Which is: '\xFF' (255)
Comment 1 Radar WebKit Bug Importer 2023-05-23 16:03:22 PDT 
<rdar://problem/109743641>
Comment 2 Robert Jenner 2023-05-23 16:05:12 PDT 
These are very easy to reproduce at iOS 16 Debug ToT running the tests as follows:


run-api-tests --iOS-simulator <test_goes_here>

You will specifically need to boot an iPhone XS or higher simulator to reproduce the failure. After you have chosen your booted simulator it will choose to run on that simulated device, and not the one that we have set for our testing.
Comment 3 Robert Jenner 2023-05-24 17:10:03 PDT 
*** Bug 257228 has been marked as a duplicate of this bug. ***
Comment 4 Robert Jenner 2023-05-24 17:10:28 PDT 
*** Bug 257239 has been marked as a duplicate of this bug. ***
Comment 5 Robert Jenner 2023-05-24 17:12:00 PDT 

*** This bug has been marked as a duplicate of bug 257246 ***