Bug 260020

Summary:	[ews] mac-wk2 build marked as green even though it found failure in stress mode
Product:	WebKit	Reporter:	Aakash Jain <aakash_jain>
Component:	Tools / Tests	Assignee:	Aakash Jain <aakash_jain>
Status:	RESOLVED FIXED
Severity:	Normal	CC:	aakash_jain, ap, jbedard, ryanhaddad, webkit-bug-importer
Priority:	P2	Keywords:	InRadar
Version:	Other
Hardware:	Unspecified
OS:	Unspecified

Description Aakash Jain 2023-08-10 03:36:07 PDT

[ews] mac-wk2 build marked as green even though it found stress test failure.

In mac-wk2 (and similar queues) we run both: 1) test-being modified in stress mode, 2) complete layout-test suites.

Ideally if anyone detects a failure, ews bubble should be marked as red.

However, it seems to be marked as green when complete layout-test run was successful (failures were flaky).

e.g.: 
https://ews-build.webkit.org/#/builders/36/builds/13373
https://ews-build.webkit.org/#/builders/10/builds/13207
for PR https://github.com/WebKit/WebKit/pull/16505 (hash e5920c3)

Note that we also have a separate stress queue which had red bubble because of this failure (https://ews-build.webkit.org/#/builders/8/builds/13063). So, the issue wasn't completely hidden.

Comment 1 Aakash Jain 2023-08-11 10:54:12 PDT

Pull request: https://github.com/WebKit/WebKit/pull/16611

Comment 2 EWS 2023-08-11 11:54:44 PDT

Committed 266824@main (ad0590629fe0): <https://commits.webkit.org/266824@main>

Reviewed commits have been landed. Closing PR #16611 and removing active labels.

Comment 3 Radar WebKit Bug Importer 2023-08-11 11:55:14 PDT

<rdar://problem/113763261>

Comment 4 Aakash Jain 2023-08-16 08:32:30 PDT

This change is working fine. 
e.g.: https://ews-build.webkit.org/#/builders/10/builds/13514 is marked as red now (with GitHub status-bubble also red in https://github.com/WebKit/WebKit/pull/16658#issuecomment-1676773405).