Bug 35358

Summary:

new-run-webkit-tests: tests are randomly timing out when run

Product:

WebKit

Reporter:

Eric Seidel (no email) <eric>

Component:

Tools / Tests

Assignee:

Dirk Pranke <dpranke>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

dglazkov, dpranke, mihaip, ojan

Priority:

Version:

528+ (Nightly build)

Hardware:

OS:

OS X 10.5

Bug Depends on:

Bug Blocks:

34984

Attachments:

Description	Flags
results.json	none
expectations.json (also needed by the layout test dashboard)	none

Eric Seidel (no email)

Reported 2010-02-24 13:40:19 PST

Created attachment 49430 [details] results.json tests are randomly timing out when run with run-chromium-webkit-tests I ran: while true; do run-chromium-webkit-tests --platform=mac-leopard --time-out-ms=15000; done In a loop overnight. The results that I got back strongly indicate that there is a problem with the harness, as every run or so at least one test will randomly time out. See the attached results.json file. (I can be loaded in layout test dashboard by dropping it in the right place ones tree (which Ojan could explain).

Attachments
results.json (157.85 KB, text/plain) 2010-02-24 13:40 PST, Eric Seidel (no email)	no flags	Details
expectations.json (also needed by the layout test dashboard) (6.08 KB, text/plain) 2010-02-24 13:40 PST, Eric Seidel (no email)	no flags	Details
View All Add attachment proposed patch, testcase, etc.

Eric Seidel (no email)

Comment 1 2010-02-24 13:40:44 PST

Created attachment 49431 [details] expectations.json (also needed by the layout test dashboard)

Dirk Pranke

Comment 2 2010-02-24 14:36:20 PST

define "problem with the harness". What makes you think it's the harness's fault, as opposed to DRT doing unpredictable things under timing pressure and concurrency (or apache doing weird things, for that matter)? Or does "problem with the harness" include such issues?

Eric Seidel (no email)

Comment 3 2010-02-24 14:38:56 PST

Problem with the hardness would include all that, yes. :)

Dirk Pranke

Comment 4 2010-02-24 14:49:57 PST

Ah, see, I don't consider it to be a problem with the harness if DRT is being flaky under load (since there's nothing the harness can do to fix that except reduce the load). I consider that to be a problem w/ DRT. On the other hand, if it's apache, that's a problem w/ the harness.

Eric Seidel (no email)

Comment 5 2010-02-24 14:51:17 PST

At this point, we don't know what the problem is. Why a test occasionally seems to lock up.

Eric Seidel (no email)

Comment 6 2010-02-24 15:08:56 PST

I appear to get random timeouts even with --num-test-shells=1. Which points more fingers at the actual python scripts. Still investigating. I still don't know what actually happens when it "times out", if it's killing DRT because it's hung, or what.

Ojan Vafai

Comment 7 2010-02-24 17:07:40 PST

It's unlikely the http server since plenty non-http tests timeout. It wouldn't be hard to modify the script to distinguish between cases where the test times out versus cases where DRT needs to be killed. All that code is in test_shell_thread.py if you want to go that route. One way to try and understand this a bit would be to print the output from DRT to stdout (or a file) so you can see if the output from DRT makes sense in the cases where timeouts happen.

Dirk Pranke

Comment 8 2010-09-23 14:36:34 PDT

interesting. dglazkov just found a bug in server_process that py that might be contributing to this. https://bugs.webkit.org/show_bug.cgi?id=46406

Dirk Pranke

Comment 9 2011-04-01 16:22:03 PDT

It might be a combination of the bug dimitri fixed, or the fact that our timeouts are lower than ORWT by default, or any number of other bugs fixed in the past year. I think things should be okay now; after marking a couple of tests as SLOW I'm no longer seeing timeouts on SL. Please reopen or file a new bug if we're still seeing issues.

Note You need to log in before you can comment on or make changes to this bug.