The following WPT websockets tests are failing for WebKitGTK: imported/w3c/web-platform-tests/websockets/constructor/010.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017.html imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018.html imported/w3c/web-platform-tests/websockets/opening-handshake/001.html imported/w3c/web-platform-tests/websockets/opening-handshake/003.html imported/w3c/web-platform-tests/websockets/security/001.html Tests results: https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/constructor/010-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/constructor/010-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/constructor/010-actual.txt @@ -1,4 +1,5 @@ -CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/handshake_protocol' failed: The server did not accept the WebSocket handshake. -PASS WebSockets: protocol in response but no requested protocol +Harness Error (TIMEOUT), message = null +TIMEOUT WebSockets: protocol in response but no requested protocol Test timed out + https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/017-actual.txt @@ -1,4 +1,5 @@ -CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/echo_raw' failed: Received invalid WebSocket response from the server -PASS WebSockets: this, e.target, e.currentTarget, e.eventPhase +Harness Error (TIMEOUT), message = null +TIMEOUT WebSockets: this, e.target, e.currentTarget, e.eventPhase Test timed out + https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/interfaces/WebSocket/events/018-actual.txt @@ -1,4 +1,5 @@ -CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/echo_raw' failed: Received invalid WebSocket response from the server -PASS open, message, error and close events +Harness Error (TIMEOUT), message = null +TIMEOUT open, message, error and close events Test timed out + https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/opening-handshake/001-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/001-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/001-actual.txt @@ -1,4 +1,5 @@ -CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/invalid' failed: The server did not accept the WebSocket handshake. -PASS WebSockets: invalid handshake +Harness Error (TIMEOUT), message = null +TIMEOUT WebSockets: invalid handshake Test timed out + https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/opening-handshake/003-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/003-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/opening-handshake/003-actual.txt @@ -1,3 +1,5 @@ -PASS WebSockets: origin +Harness Error (TIMEOUT), message = null +TIMEOUT WebSockets: origin Test timed out + https://build.webkit.org/results/GTK-Linux-64-bit-Release-Tests/259541%40main%20%2810392%29/imported/w3c/web-platform-tests/websockets/security/001-diff.txt --- /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/security/001-expected.txt +++ /home/buildbot/worker/GTK-Linux-64-bit-Release-Tests/build/layout-test-results/imported/w3c/web-platform-tests/websockets/security/001-actual.txt @@ -1,4 +1,5 @@ -CONSOLE MESSAGE: WebSocket connection to 'ws://web-platform.test:49001/wrong_accept_key' failed: Server returned incorrect “Sec-WebSocket-Accept” key -PASS WebSockets: wrong accept key +Harness Error (TIMEOUT), message = null +TIMEOUT WebSockets: wrong accept key Test timed out +
I can't reproduce these. I get failures because of the console message showing the error, but those could be just rebaselined.
I looked into this a bit and I cannot figure out why macOS/iOS don't include console messages and GTK/WPE do. Note that it impacts both the API as well as the channel code so it's not really about macOS/iOS moving to a newer backend as far as I can tell. (Also discussed in #gtk on Slack.) If someone knows why that is the case it seems like this would be a relatively straightforward fix.
Additional information from Youenn: > The console messages are put in stderr for some WPT folders > since they were causing too much havoc. The code is > happening in shouldDumpJSConsoleLogInStdErr in > Tools/TestRunnerShared/TestFeatures.cpp > I am not sure why GTK/WPE bots are not activating this option. Seems like this is very much a GTK/WPE issue.