RESOLVED CONFIGURATION CHANGED251347
[SOUP] Several WPT websockets tests are failing 
https://bugs.webkit.org/show_bug.cgi?id=251347
Summary [SOUP] Several WPT websockets tests are failing
Diego Pino
Reported 2023-01-29 23:28:44 PST
The following WPT websockets tests are failing for WebKitGTK: imported/w3c/web-platform-tests/websockets/constructor/010.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018.html imported/w3c/web-platform-tests/websockets/opening-handshake/001.html imported/w3c/web-platform-tests/websockets/opening-handshake/003.html imported/w3c/web-platform-tests/websockets/security/001.html Tests results: https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/constructor/010-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/constructor/010-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/constructor/010-actual.txt @@ -1,4 +1,5 @@ -CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/handshake_protocol' failed: The server did not accept the WebSocket handshake. -PASS WebSockets: protocol in response but no requested protocol +Harness Error (TIMEOUT), message = null +TIMEOUT WebSockets: protocol in response but no requested protocol Test timed out + https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017-actual.txt @@ -1,4 +1,5 @@ -CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/echo_raw' failed: Received invalid WebSocket response from the server -PASS WebSockets: this, e.target, e.currentTarget, e.eventPhase +Harness Error (TIMEOUT), message = null +TIMEOUT WebSockets: this, e.target, e.currentTarget, e.eventPhase Test timed out + https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018-actual.txt @@ -1,4 +1,5 @@ -CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/echo_raw' failed: Received invalid WebSocket response from the server -PASS open, message, error and close events +Harness Error (TIMEOUT), message = null +TIMEOUT open, message, error and close events Test timed out + https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/opening-handshake/001-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/001-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/001-actual.txt @@ -1,4 +1,5 @@ -CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/invalid' failed: The server did not accept the WebSocket handshake. -PASS WebSockets: invalid handshake +Harness Error (TIMEOUT), message = null +TIMEOUT WebSockets: invalid handshake Test timed out + https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/opening-handshake/003-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/003-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/003-actual.txt @@ -1,3 +1,5 @@ -PASS WebSockets: origin +Harness Error (TIMEOUT), message = null +TIMEOUT WebSockets: origin Test timed out + https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/security/001-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/security/001-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/security/001-actual.txt @@ -1,4 +1,5 @@ -CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/wrong_accept_key' failed: Server returned incorrect “Sec-WebSocket-Accept” key -PASS WebSockets: wrong accept key +Harness Error (TIMEOUT), message = null +TIMEOUT WebSockets: wrong accept key Test timed out +
Attachments
Add attachment proposed patch, testcase, etc.
Carlos Garcia Campos
Comment 1 2023-02-13 01:51:40 PST
I can't reproduce these. I get failures because of the console message showing the error, but those could be just rebaselined.
Anne van Kesteren
Comment 2 2023-05-10 03:03:07 PDT
I looked into this a bit and I cannot figure out why macOS/iOS don't include console messages and GTK/WPE do. Note that it impacts both the API as well as the channel code so it's not really about macOS/iOS moving to a newer backend as far as I can tell. (Also discussed in #gtk on Slack.) If someone knows why that is the case it seems like this would be a relatively straightforward fix.
Anne van Kesteren
Comment 3 2023-05-10 10:27:37 PDT
Additional information from Youenn: > The console messages are put in stderr for some WPT folders > since they were causing too much havoc. The code is > happening in shouldDumpJSConsoleLogInStdErr in > Tools/TestRunnerShared/TestFeatures.cpp > I am not sure why GTK/WPE bots are not activating this option. Seems like this is very much a GTK/WPE issue.
Diego Pino
Comment 4 2025-08-20 23:53:44 PDT
The current tests filed under this bug are: imported/w3c/web-platform-tests/websockets/Create-invalid-urls.any.html imported/w3c/web-platform-tests/websockets/Create-invalid-urls.any.worker.html imported/w3c/web-platform-tests/websockets/opening-handshake/003.html?default imported/w3c/web-platform-tests/websockets/opening-handshake/003.html?wss imported/w3c/web-platform-tests/websockets/keeping-connection-open/001.html?wss imported/w3c/web-platform-tests/websockets/constructor/011.html?wss imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/close/close-basic.html?wss -- WebKitGTK and WPEWebKit get better actual results than the expected baseline in these tests: (a new baseline should be emitted) imported/w3c/web-platform-tests/websockets/Create-invalid-urls.any.html imported/w3c/web-platform-tests/websockets/Create-invalid-urls.any.worker.html imported/w3c/web-platform-tests/websockets/opening-handshake/003.html?default imported/w3c/web-platform-tests/websockets/opening-handshake/003.html?wss This test should be marked as Slow since it expects a timeout of 20s (default timeout is 15s): imported/w3c/web-platform-tests/websockets/keeping-connection-open/001.html?wss This other test is managed in bug https://bugs.webkit.org/show_bug.cgi?id=206652: imported/w3c/web-platform-tests/websockets/constructor/011.html?wss This test is already passing: imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/close/close-basic.html?wss https://results.webkit.org/?platform=GTK&platform=WPE&&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fwebsockets%2Finterfaces%2FWebSocket%2Fclose%2Fclose-basic.html%3Fwss
Diego Pino
Comment 5 2025-08-21 03:39:06 PDT
Updated test expectations in 299003@main. Currently there are not more tests filed under this bug. Closing bug.
Note You need to log in before you can comment on or make changes to this bug.