Bug 257235

Summary:	[ iPhone XS+ ] 3X TestWebKitAPI.WKWebView (API-Tests) are constant failures
Product:	WebKit	Reporter:	Robert Jenner <jenner>
Component:	WebKit API	Assignee:	Nobody <webkit-unassigned>
Status:	RESOLVED DUPLICATE
Severity:	Normal	CC:	bdakin, thorton, webkit-bot-watchers-bugzilla, webkit-bug-importer, wenson_hsieh
Priority:	P2	Keywords:	InRadar
Version:	WebKit Nightly Build
Hardware:	Unspecified
OS:	Unspecified

Description Robert Jenner 2023-05-23 16:02:06 PDT

TestWebKitAPI.WKWebView.SnapshotAfterScreenUpdates
TestWebKitAPI.WKWebView.SnapshotImageBaseCase
TestWebKitAPI.WKWebView.SnapshotWithoutAfterScreenUpdates

are constant failures on iOS 16 Debug when run on iPhone XS simulator and higher. 

HISTORY:
https://results.webkit.org/?suite=api-tests&suite=api-tests&suite=api-tests&test=TestWebKitAPI.WKWebView.SnapshotAfterScreenUpdates&test=TestWebKitAPI.WKWebView.SnapshotImageBaseCase&test=TestWebKitAPI.WKWebView.SnapshotWithoutAfterScreenUpdates

According to the history, these are all passing. However, the simulator we are currently using is iPhone SE (3rd gen). These tests fail on iPhone XS and higher, and we want to change our current simulator for API tests from iPhone SE to iPhone 12. So these tests may need to be adjusted. 

STDIO FAILURE TEXT:
    TestWebKitAPI.WKWebView.SnapshotAfterScreenUpdates
        LEAK: 1 WebProcessPool

        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:507
        Expected equality of these values:
          0
          rgba[pixelIndex]
            Which is: '\xFF' (255)


        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:508
        Expected equality of these values:
          0
          rgba[pixelIndex + 1]
            Which is: '\xFF' (255)


    TestWebKitAPI.WKWebView.SnapshotImageBaseCase
        LEAK: 1 WebProcessPool

        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:292
        Expected equality of these values:
          0
          rgba[pixelIndex]
            Which is: '\xFF' (255)


        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:294
        Expected equality of these values:
          255
          rgba[pixelIndex + 2]
            Which is: '\0'


    TestWebKitAPI.WKWebView.SnapshotWithoutAfterScreenUpdates
        LEAK: 1 WebProcessPool

        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:570
        Expected equality of these values:
          0
          rgba[pixelIndex]
            Which is: '\xFF' (255)


        /Volumes/Data/worker/trunk-sydneye-ios-simulator-debug-archive/build/OpenSource/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebViewSnapshot.mm:571
        Expected equality of these values:
          0
          rgba[pixelIndex + 1]
            Which is: '\xFF' (255)

Comment 1 Radar WebKit Bug Importer 2023-05-23 16:03:22 PDT

<rdar://problem/109743641>

Comment 2 Robert Jenner 2023-05-23 16:05:12 PDT

These are very easy to reproduce at iOS 16 Debug ToT running the tests as follows:


run-api-tests --iOS-simulator <test_goes_here>

You will specifically need to boot an iPhone XS or higher simulator to reproduce the failure. After you have chosen your booted simulator it will choose to run on that simulated device, and not the one that we have set for our testing.

Comment 3 Robert Jenner 2023-05-24 17:10:03 PDT

*** Bug 257228 has been marked as a duplicate of this bug. ***

Comment 4 Robert Jenner 2023-05-24 17:10:28 PDT

*** Bug 257239 has been marked as a duplicate of this bug. ***

Comment 5 Robert Jenner 2023-05-24 17:12:00 PDT


*** This bug has been marked as a duplicate of bug 257246 ***