Bug 263879

Summary: REGRESSION(269235@main?):[ Monterey+ ] http/tests/security/xss-DENIED-script-inject-into-inactive-window2-pson.html is flaky recently
Product: WebKit Reporter: Fujii Hironori <Hironori.Fujii>
Component: WebKit Misc.Assignee: sideshowbarker <mike>
Status: RESOLVED FIXED    
Severity: Normal CC: cdumez, ddkilzer, jenner, mike, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=261889
https://bugs.webkit.org/show_bug.cgi?id=265101

Description Fujii Hironori 2023-10-30 04:40:14 PDT
http/tests/security/xss-DENIED-script-inject-into-inactive-window2-pson.html is flaky recently

Buildbot: builder Apple-Sonoma-Debug-AppleSilicon-WK2-Tests build 176 (269273@main)
https://build.webkit.org/#/builders/934/builds/176

https://build.webkit.org/results/Apple-Sonoma-Debug-AppleSilicon-WK2-Tests/269273@main%20(176)/http/tests/security/xss-DENIED-script-inject-into-inactive-window2-pson-diff.txt

--- /Volumes/Data/worker/Apple-Sonoma-Debug-AppleSilicon-WK2-Tests/build/layout-test-results/http/tests/security/xss-DENIED-script-inject-into-inactive-window2-pson-expected.txt
+++ /Volumes/Data/worker/Apple-Sonoma-Debug-AppleSilicon-WK2-Tests/build/layout-test-results/http/tests/security/xss-DENIED-script-inject-into-inactive-window2-pson-actual.txt
@@ -1 +1 @@
-This page doesn't do anything special.
+XSS


Buildbot: builder Apple-Monterey-Release-AppleSilicon-WK1-Tests build 12901 (269333@main)
https://build.webkit.org/#/builders/377/builds/12901

Buildbot: builder Apple-Monterey-Debug-AppleSilicon-WK2-Tests build 7027 (269333@main)
https://build.webkit.org/#/builders/376/builds/7027
Comment 1 Fujii Hironori 2023-10-30 04:43:39 PDT
Pull request: https://github.com/WebKit/WebKit/pull/19712
Comment 2 EWS 2023-10-30 04:47:13 PDT
Test gardening commit 269933@main (c4f3c627500f): <https://commits.webkit.org/269933@main>

Reviewed commits have been landed. Closing PR #19712 and removing active labels.
Comment 3 Robert Jenner 2023-11-01 11:36:27 PDT
I was able to semi-reliably reproduce this failure running the test as follows on Sonoma Release ToT:

run-webkit-tests http/tests/security/xss-DENIED-script-inject-into-inactive-window2-pson.html --force --iterations 100
Comment 4 Robert Jenner 2023-11-01 11:44:49 PDT
This test was last modified at: https://commits.webkit.org/269235@main, which was on October 11th. The first recent flake of this test occurred shortly after on October 12th. It had not failed before that test run. So, I would say that our regression point appears to be 269235@main.
Comment 5 Robert Jenner 2023-11-01 11:46:13 PDT
Assigning to Michael Smith.
Comment 6 Radar WebKit Bug Importer 2023-11-06 03:41:15 PST
<rdar://problem/117992521>
Comment 7 sideshowbarker 2023-11-06 23:39:31 PST
(In reply to Robert Jenner from comment #3)
> I was able to semi-reliably reproduce this failure running the test as
> follows on Sonoma Release ToT:
> 
> run-webkit-tests
> http/tests/security/xss-DENIED-script-inject-into-inactive-window2-pson.html
> --force --iterations 100

In my environment, when I run that, it consistently fails the first time, and then passes the remaining 99 times. And if I give it --iterations 1000, it again consistently fails the first time and then passes the other 999 times.

If I run it without --iterations value, it consistently fails.

I don’t yet understand what makes it fail the first/only time, and I don’t yet understand what makes it instead pass any subsequent times. But I’ll keep trying to figure out what it ought to be doing.

It’s using both setTimeout and setInterval — but those don’t seem to cause the fails-the-first-time-but-passes-any-subsequent-times problem I’m seeing.

And to work as expected, it seems to need to be able to detect that that the innocent-victim.html content has been loaded. Or maybe it instead just needs to know that the innocent-victim.html content has been navigated to — because the source of that doesn’t contain script, and nothing is done with the HTML it does contain, so it seems like it doesn’t really matter whether it’s been loaded. And if so, then it seems like it would be sufficient just to check that the owner document has gotten navigated to the http://localhost:8000/security/resources/innocent-victim.html URL.

But when I change it to check for that instead, the test always times out.

I’ll keep trying to see if I can find something that actually works.
Comment 8 sideshowbarker 2023-11-07 00:56:12 PST
Pull request: https://github.com/WebKit/WebKit/pull/20092
Comment 9 EWS 2023-11-14 17:21:33 PST
Committed 270737@main (c586c6d289d7): <https://commits.webkit.org/270737@main>

Reviewed commits have been landed. Closing PR #20092 and removing active labels.