On the EWS layout test bots we are enabling core-dumps so we can get backtraces of the tests that crash. And we have configured a cronjob to automatically clean cores older than 1 hour. The issue is that sometimes when a patch tested causes too much crashes, due to the work the EWS for WPE and GTK work that repeats each failure 10 times then a lot of coredumps can be generated in a sort space of time. This may end causing the bot to run out of space. This just happened here: https://ews-build.webkit.org/#/builders/86/builds/182 The server started with 41GB of free disk space (use 40%). See step configuration Then the step 'layout-test' caused 97 crashes Then the step 'layout-tests-repeat-failures' tried to repeat each of those 97 crashes 11 times. So the Step would end generating 1067 coredumps :\ The server ran out of disk space and the run aborted due to this. The next run on this server failed due to git issues (no disk space). See: - https://ews-build.webkit.org/#/builders/86/builds/188 - https://ews-build.webkit.org/#/builders/86/builds/190 - https://ews-build.webkit.org/#/builders/86/builds/191 - https://ews-build.webkit.org/#/builders/86/builds/192 .. etc Until an hour later the cronjob was fired and cleaned those coredumps the server was unusable. So I think we need a way to configure the bots to just delete the cores after processing it and generating the backtrace.
Pull request: https://github.com/WebKit/WebKit/pull/10531
Committed 260694@main (59ff7ba2dbdb): <https://commits.webkit.org/260694@main> Reviewed commits have been landed. Closing PR #10531 and removing active labels.