Bug 251347
| Summary: | [SOUP] Several WPT websockets tests are failing | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Diego Pino <dpino> |
| Component: | WebKitGTK | Assignee: | Nobody <webkit-unassigned> |
| Status: | RESOLVED CONFIGURATION CHANGED | ||
| Severity: | Normal | CC: | annevk, bugs-noreply, cgarcia, mcatanzaro, webkit-bug-importer, youennf |
| Priority: | P2 | ||
| Version: | WebKit Nightly Build | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
Diego Pino
The following WPT websockets tests are failing for WebKitGTK:
imported/w3c/web-platform-tests/websockets/constructor/010.html
imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017.html
imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018.html
imported/w3c/web-platform-tests/websockets/opening-handshake/001.html
imported/w3c/web-platform-tests/websockets/opening-handshake/003.html
imported/w3c/web-platform-tests/websockets/security/001.html
Tests results:
https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/constructor/010-diff.txt
--- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/constructor/010-expected.txt
+++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/constructor/010-actual.txt
@@ -1,4 +1,5 @@
-CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/handshake_protocol' failed: The server did not accept the WebSocket handshake.
-PASS WebSockets: protocol in response but no requested protocol
+Harness Error (TIMEOUT), message = null
+TIMEOUT WebSockets: protocol in response but no requested protocol Test timed out
+
https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017-diff.txt
--- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017-expected.txt
+++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017-actual.txt
@@ -1,4 +1,5 @@
-CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/echo_raw' failed: Received invalid WebSocket response from the server
-PASS WebSockets: this, e.target, e.currentTarget, e.eventPhase
+Harness Error (TIMEOUT), message = null
+TIMEOUT WebSockets: this, e.target, e.currentTarget, e.eventPhase Test timed out
+
https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018-diff.txt
--- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018-expected.txt
+++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018-actual.txt
@@ -1,4 +1,5 @@
-CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/echo_raw' failed: Received invalid WebSocket response from the server
-PASS open, message, error and close events
+Harness Error (TIMEOUT), message = null
+TIMEOUT open, message, error and close events Test timed out
+
https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/opening-handshake/001-diff.txt
--- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/001-expected.txt
+++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/001-actual.txt
@@ -1,4 +1,5 @@
-CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/invalid' failed: The server did not accept the WebSocket handshake.
-PASS WebSockets: invalid handshake
+Harness Error (TIMEOUT), message = null
+TIMEOUT WebSockets: invalid handshake Test timed out
+
https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/opening-handshake/003-diff.txt
--- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/003-expected.txt
+++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/003-actual.txt
@@ -1,3 +1,5 @@
-PASS WebSockets: origin
+Harness Error (TIMEOUT), message = null
+TIMEOUT WebSockets: origin Test timed out
+
https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/security/001-diff.txt
--- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/security/001-expected.txt
+++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/security/001-actual.txt
@@ -1,4 +1,5 @@
-CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/wrong_accept_key' failed: Server returned incorrect “Sec-WebSocket-Accept” key
-PASS WebSockets: wrong accept key
+Harness Error (TIMEOUT), message = null
+TIMEOUT WebSockets: wrong accept key Test timed out
+
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Carlos Garcia Campos
I can't reproduce these. I get failures because of the console message showing the error, but those could be just rebaselined.
Anne van Kesteren
I looked into this a bit and I cannot figure out why macOS/iOS don't include console messages and GTK/WPE do. Note that it impacts both the API as well as the channel code so it's not really about macOS/iOS moving to a newer backend as far as I can tell. (Also discussed in #gtk on Slack.)
If someone knows why that is the case it seems like this would be a relatively straightforward fix.
Anne van Kesteren
Additional information from Youenn:
> The console messages are put in stderr for some WPT folders
> since they were causing too much havoc. The code is
> happening in shouldDumpJSConsoleLogInStdErr in
> Tools/TestRunnerShared/TestFeatures.cpp
> I am not sure why GTK/WPE bots are not activating this option.
Seems like this is very much a GTK/WPE issue.
Diego Pino
The current tests filed under this bug are:
imported/w3c/web-platform-tests/websockets/Create-invalid-urls.any.html
imported/w3c/web-platform-tests/websockets/Create-invalid-urls.any.worker.html
imported/w3c/web-platform-tests/websockets/opening-handshake/003.html?default
imported/w3c/web-platform-tests/websockets/opening-handshake/003.html?wss
imported/w3c/web-platform-tests/websockets/keeping-connection-open/001.html?wss
imported/w3c/web-platform-tests/websockets/constructor/011.html?wss
imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/close/close-basic.html?wss
--
WebKitGTK and WPEWebKit get better actual results than the expected baseline in these tests: (a new baseline should be emitted)
imported/w3c/web-platform-tests/websockets/Create-invalid-urls.any.html
imported/w3c/web-platform-tests/websockets/Create-invalid-urls.any.worker.html
imported/w3c/web-platform-tests/websockets/opening-handshake/003.html?default
imported/w3c/web-platform-tests/websockets/opening-handshake/003.html?wss
This test should be marked as Slow since it expects a timeout of 20s (default timeout is 15s):
imported/w3c/web-platform-tests/websockets/keeping-connection-open/001.html?wss
This other test is managed in bug https://bugs.webkit.org/show_bug.cgi?id=206652:
imported/w3c/web-platform-tests/websockets/constructor/011.html?wss
This test is already passing:
imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/close/close-basic.html?wss
https://results.webkit.org/?platform=GTK&platform=WPE&&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebsockets%2Finterfaces%2FWebSocket%2Fclose%2Fclose-basic.html%3Fwss
Diego Pino
Updated test expectations in 299003@main. Currently there are not more tests filed under this bug. Closing bug.