Bug 254289 - REGRESSION(259811@main-259818@main): [ iOS Debug ] 4X TestWebKitAPI.AppPrivacyReport (API-Tests) are failing
Summary: REGRESSION(259811@main-259818@main): [ iOS Debug ] 4X TestWebKitAPI.AppPrivac...
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Local Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Ryan Haddad
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2023-03-22 14:46 PDT by Robert Jenner
Modified: 2023-05-09 09:11 PDT (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert Jenner 2023-03-22 14:46:28 PDT
The following 4X TestWebKitAPI.AppPrivacyReport (API-Tests) are constant timeouts on iOS Debug:

TestWebKitAPI.AppPrivacyReport.RestoreFromInteractionStateIsAppInitiated
TestWebKitAPI.AppPrivacyReport.RestoreFromInteractionStateIsNonAppInitiated
TestWebKitAPI.AppPrivacyReport.RestoreFromSessionStateIsAppInitiated
TestWebKitAPI.AppPrivacyReport.RestoreFromSessionStateIsNonAppInitiated


HISTORY:
https://results.webkit.org/?suite=api-tests&suite=api-tests&suite=api-tests&suite=api-tests&test=TestWebKitAPI.AppPrivacyReport.RestoreFromInteractionStateIsAppInitiated&test=TestWebKitAPI.AppPrivacyReport.RestoreFromInteractionStateIsNonAppInitiated&test=TestWebKitAPI.AppPrivacyReport.RestoreFromSessionStateIsAppInitiated&test=TestWebKitAPI.AppPrivacyReport.RestoreFromSessionStateIsNonAppInitiated

These tests appeared to be flaky timeouts, but switched to constant timeouts for debug iOS only between 259811@main and 259818@main
Comment 1 Radar WebKit Bug Importer 2023-03-22 14:46:46 PDT
<rdar://problem/107077600>
Comment 2 Robert Jenner 2023-03-22 14:49:27 PDT
The other interesting thing about these tests is that they do appear to be consistently timing out on EWS observed here:

https://ews-build.webkit.org/#/builders/13


However, they do not appear to be timing out on release in OpenSource/Post-commit testing.
Comment 3 Robert Jenner 2023-03-22 18:26:01 PDT
I was able to reproduce the timeouts at iOS 16 Debug ToT running the tests as follows:


run-api-tests --ios-simulator <test_goes_here> --debug


I will now attempt to bisect and uncover the regression point.
Comment 4 Alexey Proskuryakov 2023-03-25 19:29:29 PDT
This is marked RESOLVED/FIXED. but I see a false positive on EWS from these tests: https://ews-build.webkit.org/#/builders/13/builds/467
Comment 5 Ryan Haddad 2023-03-27 08:19:48 PDT
Reopening.
Comment 6 Ryan Haddad 2023-03-27 08:24:56 PDT
rdar://107268120
Comment 7 Ryan Haddad 2023-03-27 08:49:14 PDT
Pull request: https://github.com/WebKit/WebKit/pull/12007
Comment 8 EWS 2023-03-27 08:53:43 PDT
Test gardening commit 262152@main (1fa0bf2759a8): <https://commits.webkit.org/262152@main>

Reviewed commits have been landed. Closing PR #12007 and removing active labels.
Comment 9 Alexey Proskuryakov 2023-03-27 09:34:26 PDT
Renaming to not say "constant timeouts", as the tests were flaky on EWS, causing false positives. But they definitely were 100% timing out in post commit.
Comment 10 EWS 2023-03-29 11:24:05 PDT
Test gardening commit 259548.509@safari-7615-branch (96be2e666d51): <https://commits.webkit.org/259548.509@safari-7615-branch>

Reviewed commits have been landed. Closing PR #504 and removing active labels.
Comment 11 Ryan Haddad 2023-05-08 14:44:11 PDT
Pull request: https://github.com/WebKit/WebKit/pull/13600
Comment 12 Ryan Haddad 2023-05-09 09:11:40 PDT
(In reply to Ryan Haddad from comment #11)
> Pull request: https://github.com/WebKit/WebKit/pull/13600

^ this was to unskip the tests to see how they fared, they are still timing out.