Bug 254289

Summary: REGRESSION(259811@main-259818@main): [ iOS Debug ] 4X TestWebKitAPI.AppPrivacyReport (API-Tests) are failing
Product: WebKit Reporter: Robert Jenner <jenner>
Component: Tools / TestsAssignee: Ryan Haddad <ryanhaddad>
Status: REOPENED ---    
Severity: Normal CC: ap, bfulgham, webkit-bot-watchers-bugzilla, webkit-bug-importer, wenson_hsieh, wilander
Priority: P2 Keywords: InRadar
Version: WebKit Local Build   
Hardware: Unspecified   
OS: Unspecified   

Description Robert Jenner 2023-03-22 14:46:28 PDT 
The following 4X TestWebKitAPI.AppPrivacyReport (API-Tests) are constant timeouts on iOS Debug:

TestWebKitAPI.AppPrivacyReport.RestoreFromInteractionStateIsAppInitiated
TestWebKitAPI.AppPrivacyReport.RestoreFromInteractionStateIsNonAppInitiated
TestWebKitAPI.AppPrivacyReport.RestoreFromSessionStateIsAppInitiated
TestWebKitAPI.AppPrivacyReport.RestoreFromSessionStateIsNonAppInitiated


HISTORY:
https://results.webkit.org/?suite=api-tests&suite=api-tests&suite=api-tests&suite=api-tests&test=TestWebKitAPI.AppPrivacyReport.RestoreFromInteractionStateIsAppInitiated&test=TestWebKitAPI.AppPrivacyReport.RestoreFromInteractionStateIsNonAppInitiated&test=TestWebKitAPI.AppPrivacyReport.RestoreFromSessionStateIsAppInitiated&test=TestWebKitAPI.AppPrivacyReport.RestoreFromSessionStateIsNonAppInitiated

These tests appeared to be flaky timeouts, but switched to constant timeouts for debug iOS only between 259811@main and 259818@main
Comment 1 Radar WebKit Bug Importer 2023-03-22 14:46:46 PDT 
<rdar://problem/107077600>
Comment 2 Robert Jenner 2023-03-22 14:49:27 PDT 
The other interesting thing about these tests is that they do appear to be consistently timing out on EWS observed here:

https://ews-build.webkit.org/#/builders/13


However, they do not appear to be timing out on release in OpenSource/Post-commit testing.
Comment 3 Robert Jenner 2023-03-22 18:26:01 PDT 
I was able to reproduce the timeouts at iOS 16 Debug ToT running the tests as follows:


run-api-tests --ios-simulator <test_goes_here> --debug


I will now attempt to bisect and uncover the regression point.
Comment 4 Alexey Proskuryakov 2023-03-25 19:29:29 PDT 
This is marked RESOLVED/FIXED. but I see a false positive on EWS from these tests: https://ews-build.webkit.org/#/builders/13/builds/467
Comment 5 Ryan Haddad 2023-03-27 08:19:48 PDT 
Reopening.
Comment 6 Ryan Haddad 2023-03-27 08:24:56 PDT 
rdar://107268120
Comment 7 Ryan Haddad 2023-03-27 08:49:14 PDT 
Pull request: https://github.com/WebKit/WebKit/pull/12007
Comment 8 EWS 2023-03-27 08:53:43 PDT 
Test gardening commit 262152@main (1fa0bf2759a8): <https://commits.webkit.org/262152@main>

Reviewed commits have been landed. Closing PR #12007 and removing active labels.
Comment 9 Alexey Proskuryakov 2023-03-27 09:34:26 PDT 
Renaming to not say "constant timeouts", as the tests were flaky on EWS, causing false positives. But they definitely were 100% timing out in post commit.
Comment 10 EWS 2023-03-29 11:24:05 PDT 
Test gardening commit 259548.509@safari-7615-branch (96be2e666d51): <https://commits.webkit.org/259548.509@safari-7615-branch>

Reviewed commits have been landed. Closing PR #504 and removing active labels.
Comment 11 Ryan Haddad 2023-05-08 14:44:11 PDT 
Pull request: https://github.com/WebKit/WebKit/pull/13600
Comment 12 Ryan Haddad 2023-05-09 09:11:40 PDT 
(In reply to Ryan Haddad from comment #11)
> Pull request: https://github.com/WebKit/WebKit/pull/13600

^ this was to unskip the tests to see how they fared, they are still timing out.