LUCI Scheduler :: chromium :: Comparison Mac (reclient)(CQ)

Status: FAILED

Triggered by: -

Duration: 14 hours

Actions:

Definition (rev 68a096aac7b38333e551951996639de0ed1220e9)

buildbucket: <
  server: "cr-buildbucket.appspot.com"
  bucket: "ci"
  builder: "Comparison Mac (reclient)(CQ)"
>

Incoming triggers

bc3b061b6b55b818b193886b320e760b634245f0 (1 month ago)
a4f9be41fdbc81defa8144a1893d23650bf181dd (1 month ago)
ca756e0b32cc8a537d56d3d04171e0e2690393d2 (1 month ago)
56bc3769f2c2d1dafcc512f6d00c804be46849dc (1 month ago)
5e99cd15b90ed728f1091dccf65b5cad0543d21c (1 month ago)
4f23dc6cd1fc1ed6e0a06305974697da72dcdb4e (1 month ago)
9fbb58a1eaf40c651256277b4981764981547470 (1 month ago)
86d72d0261b16de5ffcab9f1284fb7956766488e (1 month ago)

Incoming properties

{
  "branch": "refs/heads/main",
  "repository": "https://chromium.googlesource.com/chromium/src",
  "revision": "86d72d0261b16de5ffcab9f1284fb7956766488e"
}

Incoming tags

buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/86d72d0261b16de5ffcab9f1284fb7956766488e
gitiles_ref:refs/heads/main

Debug log (UTC)

[14:01:46.487] New invocation is queued and will start shortly
[14:01:47.753] Starting the invocation (attempt 1)
[14:01:47.777] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/86d72d0261b16de5ffcab9f1284fb7956766488e"
[14:01:47.777] Popped gitiles commit info from properties and tags
[14:01:47.777] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[14:01:47.777] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[14:01:47.777] Buildbucket request:
{
  "requestId": "8891915373882270336",
  "builder": {
    "project": "chromium",
    "bucket": "ci",
    "builder": "Comparison Mac (reclient)(CQ)"
  },
  "properties": {
    },
  "gitilesCommit": {
    "host": "chromium.googlesource.com",
    "project": "chromium/src",
    "id": "86d72d0261b16de5ffcab9f1284fb7956766488e",
    "ref": "refs/heads/main"
  },
  "tags": [
    {
      "key": "scheduler_invocation_id",
      "value": "8891915373882270336"
    },
    {
      "key": "scheduler_job_id",
      "value": "chromium/Comparison Mac (reclient)(CQ)"
    },
    {
      "key": "user_agent",
      "value": "luci-scheduler"
    }
  ],
  "notify": {
    "pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
  }
}
[14:01:48.290] Scheduled build:
{
  "id": "8726485311808503585",
  "builder": {
    "project": "chromium",
    "bucket": "ci",
    "builder": "Comparison Mac (reclient)(CQ)"
  },
  "number": 10703,
  "createdBy": "project:chromium",
  "createTime": "2025-01-06T14:01:47.839027969Z",
  "updateTime": "2025-01-06T14:01:47.839027969Z",
  "status": "SCHEDULED",
  "input": {
    "gitilesCommit": {
      "host": "chromium.googlesource.com",
      "project": "chromium/src",
      "id": "86d72d0261b16de5ffcab9f1284fb7956766488e",
      "ref": "refs/heads/main"
    }
  }
}
[14:01:48.290] Task URL: https://cr-buildbucket.appspot.com/build/8726485311808503585
[14:01:48.290] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:2:0) after 4m49s
[14:01:49.292] Received PubSub notification, asking Buildbucket for the build status
[14:01:49.312] Build status: SCHEDULED
[14:01:51.901] Received PubSub notification, asking Buildbucket for the build status
[14:01:51.988] Build status: STARTED
[14:06:37.279] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:2:0)
[14:06:37.300] Build status: STARTED
[14:06:37.300] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:5:0) after 2m3s
[14:08:40.330] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:5:0)
[14:08:40.347] Build status: STARTED
[14:08:40.348] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:6:0) after 3m5s
[14:11:45.369] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:6:0)
[14:11:45.405] Build status: STARTED
[14:11:45.405] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:7:0) after 6m6s
[14:17:51.558] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:7:0)
[14:17:51.582] Build status: STARTED
[14:17:51.582] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:8:0) after 7m43s
[14:25:34.604] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:8:0)
[14:25:34.635] Build status: STARTED
[14:25:34.635] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:9:0) after 4m32s
[14:30:06.657] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:9:0)
[14:30:06.689] Build status: STARTED
[14:30:06.689] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:10:0) after 3m5s
[14:33:11.712] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:10:0)
[14:33:11.739] Build status: STARTED
[14:33:11.739] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:11:0) after 1m59s
[14:35:10.815] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:11:0)
[14:35:10.838] Build status: STARTED
[14:35:10.838] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:12:0) after 6m57s
[14:42:07.862] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:12:0)
[14:42:07.879] Build status: STARTED
[14:42:07.879] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:13:0) after 9m58s
[14:52:05.933] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:13:0)
[14:52:05.949] Build status: STARTED
[14:52:05.949] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:14:0) after 8m45s
[15:00:50.968] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:14:0)
[15:00:51.046] Build status: STARTED
[15:00:51.046] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:15:0) after 2m19s
[15:03:10.071] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:15:0)
[15:03:10.099] Build status: STARTED
[15:03:10.099] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:16:0) after 3m33s
[15:06:43.107] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:16:0)
[15:06:43.125] Build status: STARTED
[15:06:43.125] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:17:0) after 5m2s
[15:11:45.356] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:17:0)
[15:11:45.391] Build status: STARTED
[15:11:45.391] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:18:0) after 7m0s
[15:18:45.413] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:18:0)
[15:18:45.431] Build status: STARTED
[15:18:45.431] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:19:0) after 5m23s
[15:24:08.527] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:19:0)
[15:24:08.546] Build status: STARTED
[15:24:08.546] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:20:0) after 8m38s
[15:32:46.575] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:20:0)
[15:32:46.596] Build status: STARTED
[15:32:46.596] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:21:0) after 7m48s
[15:40:34.626] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:21:0)
[15:40:34.646] Build status: STARTED
[15:40:34.646] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:22:0) after 5m50s
[15:46:24.733] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:22:0)
[15:46:24.751] Build status: STARTED
[15:46:24.751] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:23:0) after 5m14s
[15:51:38.769] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:23:0)
[15:51:38.790] Build status: STARTED
[15:51:38.790] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:24:0) after 5m50s
[15:57:28.838] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:24:0)
[15:57:28.946] Build status: STARTED
[15:57:28.946] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:25:0) after 1m49s
[15:59:17.970] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:25:0)
[15:59:17.985] Build status: STARTED
[15:59:17.985] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:26:0) after 7m9s
[16:06:27.034] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:26:0)
[16:06:27.050] Build status: STARTED
[16:06:27.050] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:27:0) after 9m33s
[16:16:00.171] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:27:0)
[16:16:00.192] Build status: STARTED
[16:16:00.192] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:28:0) after 8m39s
[16:24:39.240] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:28:0)
[16:24:39.264] Build status: STARTED
[16:24:39.264] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:29:0) after 5m33s
[16:30:12.338] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:29:0)
[16:30:12.355] Build status: STARTED
[16:30:12.355] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:30:0) after 1m38s
[16:31:50.523] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:30:0)
[16:31:50.543] Build status: STARTED
[16:31:50.543] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:31:0) after 5m4s
[16:36:54.571] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:31:0)
[16:36:54.591] Build status: STARTED
[16:36:54.591] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:32:0) after 4m22s
[16:41:16.806] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:32:0)
[16:41:16.824] Build status: STARTED
[16:41:16.824] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:33:0) after 6m56s
[16:48:12.901] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:33:0)
[16:48:12.917] Build status: STARTED
[16:48:12.917] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:34:0) after 9m8s
[16:57:20.935] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:34:0)
[16:57:20.951] Build status: STARTED
[16:57:20.951] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:35:0) after 5m56s
[17:03:16.972] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:35:0)
[17:03:16.993] Build status: STARTED
[17:03:16.993] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:36:0) after 8m6s
[17:11:23.062] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:36:0)
[17:11:23.079] Build status: STARTED
[17:11:23.079] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:37:0) after 5m50s
[17:17:13.097] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:37:0)
[17:17:13.121] Build status: STARTED
[17:17:13.121] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:38:0) after 1m57s
[17:19:10.143] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:38:0)
[17:19:10.325] Build status: STARTED
[17:19:10.325] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:39:0) after 9m22s
[17:28:32.347] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:39:0)
[17:28:32.511] Build status: STARTED
[17:28:32.511] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:40:0) after 2m42s
[17:31:14.625] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:40:0)
[17:31:14.642] Build status: STARTED
[17:31:14.642] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:41:0) after 6m7s
[17:37:21.701] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:41:0)
[17:37:21.717] Build status: STARTED
[17:37:21.717] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:42:0) after 8m22s
[17:45:43.732] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:42:0)
[17:45:43.756] Build status: STARTED
[17:45:43.756] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:43:0) after 7m28s
[17:53:11.816] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:43:0)
[17:53:11.837] Build status: STARTED
[17:53:11.837] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:44:0) after 2m21s
[17:55:32.863] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:44:0)
[17:55:32.885] Build status: STARTED
[17:55:32.885] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:45:0) after 6m40s
[18:02:12.914] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:45:0)
[18:02:12.939] Build status: STARTED
[18:02:12.939] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:46:0) after 6m37s
[18:08:50.063] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:46:0)
[18:08:50.096] Build status: STARTED
[18:08:50.096] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:47:0) after 3m54s
[18:12:44.193] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:47:0)
[18:12:44.212] Build status: STARTED
[18:12:44.212] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:48:0) after 1m24s
[18:14:08.332] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:48:0)
[18:14:08.350] Build status: STARTED
[18:14:08.350] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:49:0) after 6m13s
[18:20:21.402] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:49:0)
[18:20:21.477] Build status: STARTED
[18:20:21.477] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:50:0) after 6m43s
[18:27:04.526] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:50:0)
[18:27:04.564] Build status: STARTED
[18:27:04.564] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:51:0) after 4m31s
[18:31:35.630] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:51:0)
[18:31:35.710] Build status: STARTED
[18:31:35.710] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:52:0) after 8m22s
[18:39:57.735] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:52:0)
[18:39:57.756] Build status: STARTED
[18:39:57.756] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:53:0) after 1m39s
[18:41:36.774] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:53:0)
[18:41:36.794] Build status: STARTED
[18:41:36.795] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:54:0) after 8m18s
[18:49:54.822] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:54:0)
[18:49:54.839] Build status: STARTED
[18:49:54.839] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:55:0) after 2m6s
[18:52:00.917] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:55:0)
[18:52:00.936] Build status: STARTED
[18:52:00.936] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:56:0) after 7m11s
[18:59:11.968] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:56:0)
[18:59:12.085] Build status: STARTED
[18:59:12.085] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:57:0) after 6m0s
[19:05:12.117] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:57:0)
[19:05:12.136] Build status: STARTED
[19:05:12.136] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:58:0) after 9m16s
[19:14:28.157] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:58:0)
[19:14:28.175] Build status: STARTED
[19:14:28.175] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:59:0) after 9m40s
[19:24:08.213] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:59:0)
[19:24:08.231] Build status: STARTED
[19:24:08.231] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:60:0) after 8m3s
[19:32:11.308] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:60:0)
[19:32:11.349] Build status: STARTED
[19:32:11.349] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:61:0) after 5m5s
[19:37:16.461] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:61:0)
[19:37:16.479] Build status: STARTED
[19:37:16.479] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:62:0) after 4m58s
[19:42:14.497] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:62:0)
[19:42:14.545] Build status: STARTED
[19:42:14.545] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:63:0) after 3m22s
[19:45:36.600] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:63:0)
[19:45:36.621] Build status: STARTED
[19:45:36.621] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:64:0) after 1m12s
[19:46:48.683] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:64:0)
[19:46:48.701] Build status: STARTED
[19:46:48.701] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:65:0) after 1m49s
[19:48:37.718] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:65:0)
[19:48:37.736] Build status: STARTED
[19:48:37.736] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:66:0) after 3m46s
[19:52:23.767] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:66:0)
[19:52:23.812] Build status: STARTED
[19:52:23.812] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:67:0) after 9m22s
[20:01:45.795] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:67:0)
[20:01:45.818] Build status: STARTED
[20:01:45.818] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:68:0) after 1m44s
[20:03:29.828] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:68:0)
[20:03:29.848] Build status: STARTED
[20:03:29.848] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:69:0) after 9m42s
[20:13:11.869] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:69:0)
[20:13:11.886] Build status: STARTED
[20:13:11.886] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:70:0) after 4m42s
[20:17:53.914] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:70:0)
[20:17:53.936] Build status: STARTED
[20:17:53.936] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:71:0) after 3m44s
[20:21:38.043] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:71:0)
[20:21:38.114] Build status: STARTED
[20:21:38.114] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:72:0) after 9m46s
[20:31:24.152] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:72:0)
[20:31:24.171] Build status: STARTED
[20:31:24.171] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:73:0) after 1m15s
[20:32:39.201] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:73:0)
[20:32:39.220] Build status: STARTED
[20:32:39.220] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:74:0) after 8m12s
[20:40:51.239] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:74:0)
[20:40:51.261] Build status: STARTED
[20:40:51.261] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:75:0) after 9m51s
[20:50:42.300] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:75:0)
[20:50:42.322] Build status: STARTED
[20:50:42.323] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:76:0) after 8m56s
[20:59:38.352] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:76:0)
[20:59:38.373] Build status: STARTED
[20:59:38.374] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:77:0) after 8m42s
[21:08:20.501] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:77:0)
[21:08:20.518] Build status: STARTED
[21:08:20.518] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:78:0) after 8m11s
[21:16:31.577] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:78:0)
[21:16:31.634] Build status: STARTED
[21:16:31.634] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:79:0) after 2m4s
[21:18:35.656] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:79:0)
[21:18:35.685] Build status: STARTED
[21:18:35.685] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:80:0) after 2m15s
[21:20:50.707] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:80:0)
[21:20:50.724] Build status: STARTED
[21:20:50.724] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:81:0) after 3m25s
[21:24:15.758] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:81:0)
[21:24:15.777] Build status: STARTED
[21:24:15.777] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:82:0) after 5m17s
[21:29:32.805] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:82:0)
[21:29:32.823] Build status: STARTED
[21:29:32.823] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:83:0) after 7m0s
[21:36:32.843] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:83:0)
[21:36:32.900] Build status: STARTED
[21:36:32.900] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:84:0) after 6m44s
[21:43:16.916] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:84:0)
[21:43:16.981] Build status: STARTED
[21:43:16.981] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:85:0) after 3m20s
[21:46:37.002] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:85:0)
[21:46:37.027] Build status: STARTED
[21:46:37.027] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:86:0) after 4m10s
[21:50:47.058] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:86:0)
[21:50:47.079] Build status: STARTED
[21:50:47.079] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:87:0) after 5m40s
[21:56:27.105] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:87:0)
[21:56:27.154] Build status: STARTED
[21:56:27.154] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:88:0) after 5m3s
[22:01:30.178] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:88:0)
[22:01:30.193] Build status: STARTED
[22:01:30.193] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:89:0) after 4m57s
[22:06:27.269] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:89:0)
[22:06:27.295] Build status: STARTED
[22:06:27.295] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:90:0) after 4m6s
[22:10:33.322] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:90:0)
[22:10:33.347] Build status: STARTED
[22:10:33.347] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:91:0) after 9m25s
[22:19:58.418] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:91:0)
[22:19:58.444] Build status: STARTED
[22:19:58.444] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:92:0) after 3m42s
[22:23:40.503] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:92:0)
[22:23:40.517] Build status: STARTED
[22:23:40.517] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:93:0) after 9m14s
[22:32:54.547] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:93:0)
[22:32:54.576] Build status: STARTED
[22:32:54.576] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:94:0) after 5m24s
[22:38:18.591] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:94:0)
[22:38:18.610] Build status: STARTED
[22:38:18.610] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:95:0) after 6m48s
[22:45:06.602] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:95:0)
[22:45:06.625] Build status: STARTED
[22:45:06.625] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:96:0) after 4m48s
[22:49:54.653] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:96:0)
[22:49:54.727] Build status: STARTED
[22:49:54.727] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:97:0) after 3m1s
[22:52:55.802] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:97:0)
[22:52:55.822] Build status: STARTED
[22:52:55.822] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:98:0) after 5m7s
[22:58:02.814] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:98:0)
[22:58:02.832] Build status: STARTED
[22:58:02.833] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:99:0) after 1m21s
[22:59:23.851] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:99:0)
[22:59:23.873] Build status: STARTED
[22:59:23.873] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:100:0) after 3m8s
[23:02:31.898] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:100:0)
[23:02:31.916] Build status: STARTED
[23:02:31.916] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:101:0) after 4m43s
[23:07:15.018] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:101:0)
[23:07:15.034] Build status: STARTED
[23:07:15.034] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:102:0) after 8m20s
[23:15:35.100] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:102:0)
[23:15:35.116] Build status: STARTED
[23:15:35.116] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:103:0) after 4m57s
[23:20:32.225] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:103:0)
[23:20:32.285] Build status: STARTED
[23:20:32.285] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:104:0) after 5m45s
[23:26:17.304] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:104:0)
[23:26:17.323] Build status: STARTED
[23:26:17.323] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:105:0) after 2m5s
[23:28:22.350] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:105:0)
[23:28:22.369] Build status: STARTED
[23:28:22.369] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:106:0) after 6m3s
[23:34:25.390] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:106:0)
[23:34:25.407] Build status: STARTED
[23:34:25.407] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:107:0) after 1m22s
[23:35:47.489] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:107:0)
[23:35:47.509] Build status: STARTED
[23:35:47.509] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:108:0) after 4m0s
[23:39:47.542] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:108:0)
[23:39:47.565] Build status: STARTED
[23:39:47.565] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:109:0) after 2m9s
[23:41:56.655] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:109:0)
[23:41:56.674] Build status: STARTED
[23:41:56.674] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:110:0) after 1m34s
[23:43:30.768] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:110:0)
[23:43:30.783] Build status: STARTED
[23:43:30.783] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:111:0) after 3m58s
[23:47:28.860] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:111:0)
[23:47:28.874] Build status: STARTED
[23:47:28.874] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:112:0) after 3m34s
[23:51:02.956] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:112:0)
[23:51:03.116] Build status: STARTED
[23:51:03.116] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:113:0) after 4m56s
[23:55:59.176] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:113:0)
[23:55:59.198] Build status: STARTED
[23:55:59.198] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:114:0) after 8m53s
[00:04:52.227] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:114:0)
[00:04:52.245] Build status: STARTED
[00:04:52.245] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:115:0) after 4m31s
[00:09:23.523] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:115:0)
[00:09:23.548] Build status: STARTED
[00:09:23.548] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:116:0) after 7m6s
[00:16:29.575] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:116:0)
[00:16:29.592] Build status: STARTED
[00:16:29.592] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:117:0) after 3m50s
[00:20:19.618] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:117:0)
[00:20:19.662] Build status: STARTED
[00:20:19.662] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:118:0) after 9m24s
[00:29:43.709] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:118:0)
[00:29:43.737] Build status: STARTED
[00:29:43.737] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:119:0) after 3m23s
[00:33:06.759] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:119:0)
[00:33:06.787] Build status: STARTED
[00:33:06.787] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:120:0) after 8m4s
[00:41:10.838] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:120:0)
[00:41:11.010] Build status: STARTED
[00:41:11.010] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:121:0) after 7m52s
[00:49:03.068] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:121:0)
[00:49:03.093] Build status: STARTED
[00:49:03.093] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:122:0) after 1m9s
[00:50:12.118] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:122:0)
[00:50:12.155] Build status: STARTED
[00:50:12.155] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:123:0) after 5m53s
[00:56:05.231] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:123:0)
[00:56:05.258] Build status: STARTED
[00:56:05.258] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:124:0) after 2m48s
[00:58:53.246] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:124:0)
[00:58:53.273] Build status: STARTED
[00:58:53.273] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:125:0) after 3m55s
[01:02:48.375] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:125:0)
[01:02:48.398] Build status: STARTED
[01:02:48.398] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:126:0) after 7m54s
[01:10:42.463] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:126:0)
[01:10:42.481] Build status: STARTED
[01:10:42.481] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:127:0) after 2m6s
[01:12:48.499] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:127:0)
[01:12:48.581] Build status: STARTED
[01:12:48.581] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:128:0) after 9m7s
[01:21:55.653] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:128:0)
[01:21:55.928] Build status: STARTED
[01:21:55.928] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:129:0) after 7m46s
[01:29:42.718] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:129:0)
[01:29:42.737] Build status: STARTED
[01:29:42.737] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:130:0) after 9m37s
[01:39:19.966] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:130:0)
[01:39:20.041] Build status: STARTED
[01:39:20.041] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:131:0) after 1m29s
[01:40:49.287] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:131:0)
[01:40:49.307] Build status: STARTED
[01:40:49.307] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:132:0) after 6m52s
[01:47:41.344] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:132:0)
[01:47:41.395] Build status: STARTED
[01:47:41.395] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:133:0) after 7m52s
[01:55:33.518] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:133:0)
[01:55:33.542] Build status: STARTED
[01:55:33.542] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:134:0) after 1m13s
[01:56:46.565] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:134:0)
[01:56:46.581] Build status: STARTED
[01:56:46.581] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:135:0) after 4m22s
[02:01:08.637] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:135:0)
[02:01:08.661] Build status: STARTED
[02:01:08.661] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:136:0) after 5m37s
[02:06:45.688] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:136:0)
[02:06:45.711] Build status: STARTED
[02:06:45.711] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:137:0) after 3m10s
[02:09:56.490] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:137:0)
[02:09:56.509] Build status: STARTED
[02:09:56.510] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:138:0) after 6m50s
[02:16:46.695] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:138:0)
[02:16:46.713] Build status: STARTED
[02:16:46.713] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:139:0) after 2m10s
[02:18:57.249] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:139:0)
[02:18:57.268] Build status: STARTED
[02:18:57.268] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:140:0) after 8m48s
[02:27:45.316] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:140:0)
[02:27:45.335] Build status: STARTED
[02:27:45.335] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:141:0) after 9m30s
[02:37:15.403] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:141:0)
[02:37:15.419] Build status: STARTED
[02:37:15.419] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:142:0) after 9m7s
[02:46:22.632] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:142:0)
[02:46:22.647] Build status: STARTED
[02:46:22.647] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:143:0) after 4m15s
[02:50:37.750] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:143:0)
[02:50:37.771] Build status: STARTED
[02:50:37.771] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:144:0) after 2m8s
[02:52:45.791] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:144:0)
[02:52:45.857] Build status: STARTED
[02:52:45.857] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:145:0) after 9m14s
[03:01:59.926] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:145:0)
[03:01:59.950] Build status: STARTED
[03:01:59.950] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:146:0) after 9m41s
[03:11:40.974] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:146:0)
[03:11:40.998] Build status: STARTED
[03:11:40.998] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:147:0) after 9m33s
[03:21:14.026] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:147:0)
[03:21:14.068] Build status: STARTED
[03:21:14.068] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:148:0) after 4m50s
[03:26:04.120] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:148:0)
[03:26:04.141] Build status: STARTED
[03:26:04.141] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:149:0) after 7m2s
[03:33:06.328] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:149:0)
[03:33:06.356] Build status: STARTED
[03:33:06.356] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:150:0) after 3m15s
[03:36:21.425] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:150:0)
[03:36:21.444] Build status: STARTED
[03:36:21.444] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:151:0) after 9m19s
[03:45:40.446] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:151:0)
[03:45:40.465] Build status: STARTED
[03:45:40.465] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:152:0) after 6m56s
[03:52:36.491] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:152:0)
[03:52:36.506] Build status: STARTED
[03:52:36.506] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:153:0) after 1m14s
[03:53:50.825] Handling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:153:0)
[03:53:50.841] Build status: STARTED
[03:53:50.841] Scheduling timer "check-buildbucket-build-status" (chromium/Comparison Mac (reclient)(CQ):8891915373882270336:154:0) after 9m13s
[04:02:34.885] Received PubSub notification, asking Buildbucket for the build status
[04:02:34.903] Build:
{
  "id": "8726485311808503585",
  "builder": {
    "project": "chromium",
    "bucket": "ci",
    "builder": "Comparison Mac (reclient)(CQ)"
  },
  "number": 10703,
  "createdBy": "project:chromium",
  "createTime": "2025-01-06T14:01:47.839027969Z",
  "startTime": "2025-01-06T14:01:51.465207103Z",
  "endTime": "2025-01-07T04:02:34.555322136Z",
  "updateTime": "2025-01-07T04:02:34.555322136Z",
  "status": "INFRA_FAILURE",
  "statusDetails": {
    "timeout": {

    }
  },
  "input": {
    "gitilesCommit": {
      "host": "chromium.googlesource.com",
      "project": "chromium/src",
      "id": "86d72d0261b16de5ffcab9f1284fb7956766488e",
      "ref": "refs/heads/main"
    }
  }
}
[04:02:34.903] Invocation finished in 14h0m48.425178105s with status FAILED