[22:11:59.276] New invocation is queued and will start shortly
[22:12:00.649] Starting the invocation (attempt 1)
[22:12:00.684] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/7c9f63c0e77e8848467ecbb7d109f88ac609aaf2"
[22:12:00.684] Popped gitiles commit info from properties and tags
[22:12:00.684] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[22:12:00.685] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[22:12:00.685] Buildbucket request:
{
"requestId": "8995436864943173728",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacASan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "7c9f63c0e77e8848467ecbb7d109f88ac609aaf2",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8995436864943173728"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacASan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[22:12:01.168] Scheduled build:
{
"id": "8830006802714323825",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacASan"
},
"number": 26278,
"createdBy": "project:chromium",
"createTime": "2021-11-20T22:12:00.766896793Z",
"updateTime": "2021-11-20T22:12:00.766896793Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "7c9f63c0e77e8848467ecbb7d109f88ac609aaf2",
"ref": "refs/heads/main"
}
}
}
[22:12:01.168] Task URL: https://cr-buildbucket.appspot.com/build/8830006802714323825
[22:12:01.168] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:2:0) after 3m12s
[22:15:13.733] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:2:0)
[22:15:13.767] Build status: SCHEDULED
[22:15:13.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:3:0) after 9m7s
[22:24:20.811] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:3:0)
[22:24:20.840] Build status: SCHEDULED
[22:24:20.840] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:4:0) after 1m39s
[22:26:00.573] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:4:0)
[22:26:00.604] Build status: SCHEDULED
[22:26:00.604] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:5:0) after 5m13s
[22:31:13.637] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:5:0)
[22:31:13.670] Build status: SCHEDULED
[22:31:13.670] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:6:0) after 7m49s
[22:38:12.343] Received PubSub notification, asking Buildbucket for the build status
[22:38:12.404] Build status: STARTED
[22:39:02.689] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:6:0)
[22:39:02.726] Build status: STARTED
[22:39:02.726] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:8:0) after 6m31s
[22:45:34.054] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:8:0)
[22:45:34.097] Build status: STARTED
[22:45:34.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:9:0) after 4m12s
[22:49:46.266] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:9:0)
[22:49:46.299] Build status: STARTED
[22:49:46.299] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:10:0) after 5m51s
[22:55:37.725] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:10:0)
[22:55:37.772] Build status: STARTED
[22:55:37.772] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:11:0) after 4m49s
[23:00:27.195] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:11:0)
[23:00:27.227] Build status: STARTED
[23:00:27.227] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:12:0) after 6m14s
[23:06:41.251] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:12:0)
[23:06:41.346] Build status: STARTED
[23:06:41.346] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:13:0) after 6m13s
[23:12:54.898] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:13:0)
[23:12:54.943] Build status: STARTED
[23:12:54.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:14:0) after 7m4s
[23:19:58.969] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:14:0)
[23:19:59.017] Build status: STARTED
[23:19:59.017] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:15:0) after 3m24s
[23:23:23.039] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:15:0)
[23:23:23.072] Build status: STARTED
[23:23:23.072] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:16:0) after 7m20s
[23:30:43.083] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:16:0)
[23:30:43.244] Build status: STARTED
[23:30:43.244] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:17:0) after 4m44s
[23:35:27.657] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:17:0)
[23:35:27.685] Build status: STARTED
[23:35:27.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:18:0) after 2m49s
[23:38:17.476] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:18:0)
[23:38:17.506] Build status: STARTED
[23:38:17.506] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:19:0) after 6m35s
[23:44:52.655] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:19:0)
[23:44:52.684] Build status: STARTED
[23:44:52.684] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:20:0) after 5m3s
[23:49:55.781] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:20:0)
[23:49:55.878] Build status: STARTED
[23:49:55.878] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:21:0) after 4m49s
[23:54:44.912] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:21:0)
[23:54:45.020] Build status: STARTED
[23:54:45.020] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:22:0) after 9m10s
[00:03:55.073] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:22:0)
[00:03:55.206] Build status: STARTED
[00:03:55.206] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:23:0) after 4m24s
[00:08:19.384] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:23:0)
[00:08:19.424] Build status: STARTED
[00:08:19.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:24:0) after 4m10s
[00:12:29.445] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:24:0)
[00:12:29.475] Build status: STARTED
[00:12:29.475] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:25:0) after 7m53s
[00:20:22.502] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:25:0)
[00:20:22.534] Build status: STARTED
[00:20:22.534] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:26:0) after 7m59s
[00:28:21.557] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:26:0)
[00:28:21.588] Build status: STARTED
[00:28:21.588] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:27:0) after 5m11s
[00:33:32.648] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:27:0)
[00:33:32.683] Build status: STARTED
[00:33:32.683] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:28:0) after 1m59s
[00:35:31.703] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:28:0)
[00:35:31.741] Build status: STARTED
[00:35:31.741] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:29:0) after 1m14s
[00:36:45.760] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:29:0)
[00:36:45.802] Build status: STARTED
[00:36:45.802] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:30:0) after 6m28s
[00:43:14.110] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:30:0)
[00:43:14.143] Build status: STARTED
[00:43:14.143] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:31:0) after 2m31s
[00:45:46.662] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:31:0)
[00:45:46.768] Build status: STARTED
[00:45:46.768] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:32:0) after 1m41s
[00:47:28.327] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:32:0)
[00:47:28.368] Build status: STARTED
[00:47:28.368] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:33:0) after 2m54s
[00:50:22.844] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:33:0)
[00:50:22.912] Build status: STARTED
[00:50:22.912] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:34:0) after 9m43s
[01:00:05.935] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:34:0)
[01:00:05.975] Build status: STARTED
[01:00:05.975] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:35:0) after 1m53s
[01:01:58.998] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:35:0)
[01:01:59.032] Build status: STARTED
[01:01:59.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:36:0) after 5m49s
[01:07:48.074] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:36:0)
[01:07:48.105] Build status: STARTED
[01:07:48.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:37:0) after 6m57s
[01:14:45.123] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:37:0)
[01:14:45.159] Build status: STARTED
[01:14:45.159] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:38:0) after 1m35s
[01:16:20.195] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:38:0)
[01:16:20.222] Build status: STARTED
[01:16:20.222] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:39:0) after 2m30s
[01:18:50.511] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:39:0)
[01:18:50.566] Build status: STARTED
[01:18:50.566] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:40:0) after 5m51s
[01:24:41.635] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:40:0)
[01:24:41.664] Build status: STARTED
[01:24:41.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:41:0) after 6m53s
[01:31:34.766] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:41:0)
[01:31:34.795] Build status: STARTED
[01:31:34.795] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:42:0) after 8m28s
[01:40:02.816] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:42:0)
[01:40:02.844] Build status: STARTED
[01:40:02.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:43:0) after 3m0s
[01:43:02.962] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:43:0)
[01:43:03.013] Build status: STARTED
[01:43:03.013] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:44:0) after 8m19s
[01:51:22.054] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:44:0)
[01:51:22.084] Build status: STARTED
[01:51:22.084] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:45:0) after 6m56s
[01:58:18.152] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:45:0)
[01:58:18.179] Build status: STARTED
[01:58:18.179] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:46:0) after 1m7s
[01:59:25.396] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:46:0)
[01:59:25.424] Build status: STARTED
[01:59:25.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:47:0) after 1m32s
[02:00:57.470] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:47:0)
[02:00:57.511] Build status: STARTED
[02:00:57.511] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:48:0) after 7m41s
[02:08:38.641] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:48:0)
[02:08:38.671] Build status: STARTED
[02:08:38.671] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:49:0) after 4m26s
[02:13:04.704] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:49:0)
[02:13:04.742] Build status: STARTED
[02:13:04.742] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:50:0) after 8m51s
[02:21:55.865] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:50:0)
[02:21:55.893] Build status: STARTED
[02:21:55.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:51:0) after 7m29s
[02:29:24.994] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:51:0)
[02:29:25.028] Build status: STARTED
[02:29:25.028] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:52:0) after 4m30s
[02:33:55.068] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:52:0)
[02:33:55.099] Build status: STARTED
[02:33:55.099] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:53:0) after 2m20s
[02:36:15.122] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:53:0)
[02:36:15.157] Build status: STARTED
[02:36:15.157] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:54:0) after 8m1s
[02:44:16.149] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:54:0)
[02:44:16.186] Build status: STARTED
[02:44:16.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:55:0) after 5m33s
[02:49:49.550] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:55:0)
[02:49:49.633] Build status: STARTED
[02:49:49.633] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:56:0) after 3m11s
[02:53:00.706] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:56:0)
[02:53:00.736] Build status: STARTED
[02:53:00.736] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:57:0) after 3m2s
[02:56:02.772] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:57:0)
[02:56:02.807] Build status: STARTED
[02:56:02.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:58:0) after 6m33s
[03:02:35.855] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:58:0)
[03:02:35.890] Build status: STARTED
[03:02:35.890] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:59:0) after 7m39s
[03:10:14.909] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:59:0)
[03:10:14.937] Build status: STARTED
[03:10:14.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:60:0) after 4m50s
[03:15:04.988] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:60:0)
[03:15:05.191] Build status: STARTED
[03:15:05.191] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:61:0) after 9m50s
[03:24:55.180] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:61:0)
[03:24:55.209] Build status: STARTED
[03:24:55.209] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:62:0) after 5m56s
[03:30:51.284] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:62:0)
[03:30:51.317] Build status: STARTED
[03:30:51.317] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:63:0) after 1m47s
[03:32:38.346] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:63:0)
[03:32:38.379] Build status: STARTED
[03:32:38.380] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:64:0) after 1m27s
[03:34:05.403] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:64:0)
[03:34:05.434] Build status: STARTED
[03:34:05.434] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:65:0) after 9m54s
[03:43:59.542] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:65:0)
[03:43:59.579] Build status: STARTED
[03:43:59.579] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:66:0) after 7m24s
[03:51:23.864] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:66:0)
[03:51:23.945] Build status: STARTED
[03:51:23.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:67:0) after 5m46s
[03:57:09.976] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:67:0)
[03:57:10.005] Build status: STARTED
[03:57:10.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:68:0) after 7m39s
[04:04:49.028] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:68:0)
[04:04:49.094] Build status: STARTED
[04:04:49.094] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:69:0) after 2m44s
[04:07:33.121] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:69:0)
[04:07:33.151] Build status: STARTED
[04:07:33.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:70:0) after 9m39s
[04:17:12.179] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:70:0)
[04:17:12.232] Build status: STARTED
[04:17:12.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:71:0) after 5m34s
[04:22:46.808] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:71:0)
[04:22:46.851] Build status: STARTED
[04:22:46.851] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:72:0) after 6m35s
[04:29:21.977] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:72:0)
[04:29:22.081] Build status: STARTED
[04:29:22.081] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:73:0) after 1m26s
[04:30:48.305] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:73:0)
[04:30:48.346] Build status: STARTED
[04:30:48.346] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:74:0) after 3m21s
[04:34:09.494] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:74:0)
[04:34:09.525] Build status: STARTED
[04:34:09.525] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:75:0) after 3m20s
[04:37:29.546] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:75:0)
[04:37:29.580] Build status: STARTED
[04:37:29.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:76:0) after 3m5s
[04:40:34.605] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:76:0)
[04:40:34.637] Build status: STARTED
[04:40:34.637] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:77:0) after 2m47s
[04:43:21.945] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:77:0)
[04:43:21.977] Build status: STARTED
[04:43:21.977] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:78:0) after 2m28s
[04:45:50.177] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:78:0)
[04:45:50.212] Build status: STARTED
[04:45:50.212] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:79:0) after 4m37s
[04:50:27.237] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:79:0)
[04:50:27.269] Build status: STARTED
[04:50:27.269] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:80:0) after 2m18s
[04:52:45.316] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:80:0)
[04:52:45.561] Build status: STARTED
[04:52:45.561] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:81:0) after 1m50s
[04:54:35.588] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:81:0)
[04:54:35.629] Build status: STARTED
[04:54:35.629] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:82:0) after 3m13s
[04:57:48.918] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:82:0)
[04:57:48.948] Build status: STARTED
[04:57:48.948] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:83:0) after 6m36s
[05:04:25.056] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:83:0)
[05:04:25.095] Build status: STARTED
[05:04:25.095] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:84:0) after 3m39s
[05:08:04.466] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:84:0)
[05:08:04.559] Build status: STARTED
[05:08:04.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:85:0) after 1m52s
[05:09:57.571] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:85:0)
[05:09:57.600] Build status: STARTED
[05:09:57.600] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:86:0) after 4m2s
[05:13:59.637] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:86:0)
[05:13:59.673] Build status: STARTED
[05:13:59.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:87:0) after 4m13s
[05:18:12.729] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:87:0)
[05:18:12.774] Build status: STARTED
[05:18:12.774] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:88:0) after 6m32s
[05:24:45.718] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:88:0)
[05:24:45.750] Build status: STARTED
[05:24:45.750] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:89:0) after 1m49s
[05:26:36.726] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:89:0)
[05:26:36.774] Build status: STARTED
[05:26:36.774] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:90:0) after 9m7s
[05:35:43.804] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:90:0)
[05:35:43.831] Build status: STARTED
[05:35:43.831] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:91:0) after 8m16s
[05:43:59.965] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:91:0)
[05:43:59.996] Build status: STARTED
[05:43:59.996] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:92:0) after 2m34s
[05:46:34.119] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:92:0)
[05:46:34.188] Build status: STARTED
[05:46:34.188] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:93:0) after 5m54s
[05:52:28.226] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:93:0)
[05:52:28.258] Build status: STARTED
[05:52:28.258] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:94:0) after 8m30s
[06:00:58.514] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:94:0)
[06:00:58.631] Build status: STARTED
[06:00:58.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:95:0) after 2m6s
[06:03:05.074] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:95:0)
[06:03:05.103] Build status: STARTED
[06:03:05.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:96:0) after 7m56s
[06:11:01.510] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:96:0)
[06:11:01.783] Build status: STARTED
[06:11:01.783] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:97:0) after 8m54s
[06:19:55.981] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:97:0)
[06:19:56.019] Build status: STARTED
[06:19:56.019] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:98:0) after 6m47s
[06:26:43.281] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:98:0)
[06:26:43.308] Build status: STARTED
[06:26:43.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:99:0) after 1m43s
[06:28:26.374] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:99:0)
[06:28:26.410] Build status: STARTED
[06:28:26.410] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:100:0) after 4m16s
[06:32:42.585] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:100:0)
[06:32:42.628] Build status: STARTED
[06:32:42.628] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:101:0) after 2m15s
[06:34:57.850] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:101:0)
[06:34:57.886] Build status: STARTED
[06:34:57.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:102:0) after 3m35s
[06:38:32.910] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:102:0)
[06:38:32.941] Build status: STARTED
[06:38:32.941] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:103:0) after 3m1s
[06:41:34.125] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:103:0)
[06:41:34.151] Build status: STARTED
[06:41:34.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:104:0) after 6m53s
[06:48:27.946] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:104:0)
[06:48:27.980] Build status: STARTED
[06:48:27.980] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:105:0) after 2m24s
[06:50:52.144] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:105:0)
[06:50:52.180] Build status: STARTED
[06:50:52.180] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:106:0) after 5m29s
[06:56:21.340] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:106:0)
[06:56:21.378] Build status: STARTED
[06:56:21.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:107:0) after 3m8s
[06:59:29.817] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:107:0)
[06:59:29.849] Build status: STARTED
[06:59:29.849] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:108:0) after 7m36s
[07:07:05.993] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:108:0)
[07:07:06.031] Build status: STARTED
[07:07:06.031] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:109:0) after 7m12s
[07:14:18.611] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:109:0)
[07:14:18.642] Build status: STARTED
[07:14:18.642] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:110:0) after 1m47s
[07:16:05.674] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:110:0)
[07:16:05.753] Build status: STARTED
[07:16:05.753] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:111:0) after 4m42s
[07:20:47.982] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:111:0)
[07:20:48.018] Build status: STARTED
[07:20:48.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:112:0) after 4m29s
[07:25:17.892] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:112:0)
[07:25:17.963] Build status: STARTED
[07:25:17.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:113:0) after 6m46s
[07:32:04.412] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:113:0)
[07:32:04.441] Build status: STARTED
[07:32:04.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:114:0) after 3m27s
[07:35:31.542] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:114:0)
[07:35:31.572] Build status: STARTED
[07:35:31.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:115:0) after 6m48s
[07:42:19.616] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:115:0)
[07:42:19.663] Build status: STARTED
[07:42:19.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:116:0) after 4m9s
[07:46:28.688] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:116:0)
[07:46:28.724] Build status: STARTED
[07:46:28.724] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:117:0) after 1m14s
[07:47:42.754] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:117:0)
[07:47:42.954] Build status: STARTED
[07:47:42.954] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:118:0) after 2m44s
[07:50:27.002] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:118:0)
[07:50:27.035] Build status: STARTED
[07:50:27.035] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:119:0) after 2m12s
[07:52:39.306] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:119:0)
[07:52:39.335] Build status: STARTED
[07:52:39.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:120:0) after 7m4s
[07:59:43.414] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:120:0)
[07:59:43.446] Build status: STARTED
[07:59:43.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:121:0) after 4m42s
[08:04:26.370] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:121:0)
[08:04:26.416] Build status: STARTED
[08:04:26.416] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:122:0) after 1m46s
[08:06:12.579] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:122:0)
[08:06:12.655] Build status: STARTED
[08:06:12.655] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:123:0) after 8m25s
[08:14:37.648] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:123:0)
[08:14:37.682] Build status: STARTED
[08:14:37.682] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:124:0) after 7m19s
[08:21:56.778] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:124:0)
[08:21:56.812] Build status: STARTED
[08:21:56.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:125:0) after 8m41s
[08:30:37.843] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:125:0)
[08:30:37.879] Build status: STARTED
[08:30:37.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:126:0) after 1m24s
[08:32:02.746] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:126:0)
[08:32:02.784] Build status: STARTED
[08:32:02.784] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:127:0) after 3m57s
[08:36:00.556] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:127:0)
[08:36:00.590] Build status: STARTED
[08:36:00.590] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:128:0) after 7m20s
[08:43:20.572] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:128:0)
[08:43:20.615] Build status: STARTED
[08:43:20.615] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:129:0) after 2m27s
[08:45:47.838] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:129:0)
[08:45:47.869] Build status: STARTED
[08:45:47.869] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:130:0) after 5m14s
[08:51:01.899] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:130:0)
[08:51:01.955] Build status: STARTED
[08:51:01.955] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:131:0) after 5m15s
[08:56:17.028] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:131:0)
[08:56:17.065] Build status: STARTED
[08:56:17.065] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:132:0) after 9m4s
[09:05:21.113] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:132:0)
[09:05:21.202] Build status: STARTED
[09:05:21.202] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:133:0) after 1m13s
[09:06:34.236] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:133:0)
[09:06:34.282] Build status: STARTED
[09:06:34.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:134:0) after 7m15s
[09:13:49.319] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:134:0)
[09:13:49.350] Build status: STARTED
[09:13:49.350] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8995436864943173728:135:0) after 5m47s
[09:14:19.505] Received PubSub notification, asking Buildbucket for the build status
[09:14:19.538] Build:
{
"id": "8830006802714323825",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacASan"
},
"number": 26278,
"createdBy": "project:chromium",
"createTime": "2021-11-20T22:12:00.766896793Z",
"startTime": "2021-11-20T22:38:11.670243Z",
"endTime": "2021-11-21T09:14:19.079292979Z",
"updateTime": "2021-11-21T09:14:19.079292979Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "7c9f63c0e77e8848467ecbb7d109f88ac609aaf2",
"ref": "refs/heads/main"
}
}
}
[09:14:19.538] Invocation finished in 11h2m20.281377425s with status FAILED