[21:49:58.928] New invocation is queued and will start shortly
[21:50:00.009] Starting the invocation (attempt 1)
[21:50:00.038] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/f8a90a556ba3648b75ac49c14715540692a03c49"
[21:50:00.038] Popped gitiles commit info from properties and tags
[21:50:00.038] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:50:00.039] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:50:00.039] Buildbucket request:
{
"requestId": "8851660863736528400",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "f8a90a556ba3648b75ac49c14715540692a03c49",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851660863736528400"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMac"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:50:00.634] Scheduled build:
{
"id": "8686230801850484129",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 40092,
"createdBy": "project:chromium",
"createTime": "2026-03-26T21:50:00.106134459Z",
"updateTime": "2026-03-26T21:50:00.106134459Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "f8a90a556ba3648b75ac49c14715540692a03c49",
"ref": "refs/heads/main"
}
}
}
[21:50:00.634] Task URL: https://cr-buildbucket.appspot.com/build/8686230801850484129
[21:50:00.634] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:2:0) after 6m14s
[21:50:01.585] Received PubSub notification, asking Buildbucket for the build status
[21:50:01.604] Build status: SCHEDULED
[21:50:55.474] Received PubSub notification, asking Buildbucket for the build status
[21:50:55.499] Build status: STARTED
[21:56:14.667] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:2:0)
[21:56:14.690] Build status: STARTED
[21:56:14.690] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:5:0) after 1m38s
[21:57:52.839] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:5:0)
[21:57:52.855] Build status: STARTED
[21:57:52.855] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:6:0) after 8m59s
[22:06:51.987] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:6:0)
[22:06:52.007] Build status: STARTED
[22:06:52.007] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:7:0) after 4m3s
[22:10:55.043] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:7:0)
[22:10:55.062] Build status: STARTED
[22:10:55.062] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:8:0) after 3m46s
[22:14:41.093] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:8:0)
[22:14:41.120] Build status: STARTED
[22:14:41.120] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:9:0) after 9m19s
[22:24:00.153] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:9:0)
[22:24:00.176] Build status: STARTED
[22:24:00.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:10:0) after 8m33s
[22:32:33.192] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:10:0)
[22:32:33.207] Build status: STARTED
[22:32:33.208] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:11:0) after 8m46s
[22:41:19.369] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:11:0)
[22:41:19.390] Build status: STARTED
[22:41:19.390] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:12:0) after 3m55s
[22:45:14.446] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:12:0)
[22:45:14.499] Build status: STARTED
[22:45:14.500] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:13:0) after 9m8s
[22:54:22.537] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:13:0)
[22:54:22.553] Build status: STARTED
[22:54:22.553] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:14:0) after 5m27s
[22:59:49.600] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:14:0)
[22:59:49.621] Build status: STARTED
[22:59:49.621] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:15:0) after 9m13s
[23:09:02.790] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:15:0)
[23:09:02.813] Build status: STARTED
[23:09:02.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:16:0) after 1m30s
[23:10:32.876] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:16:0)
[23:10:32.903] Build status: STARTED
[23:10:32.903] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:17:0) after 4m38s
[23:15:10.885] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:17:0)
[23:15:10.902] Build status: STARTED
[23:15:10.902] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:18:0) after 3m43s
[23:18:53.974] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:18:0)
[23:18:54.014] Build status: STARTED
[23:18:54.014] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:19:0) after 8m44s
[23:27:38.061] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:19:0)
[23:27:38.180] Build status: STARTED
[23:27:38.180] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:20:0) after 9m35s
[23:37:13.198] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:20:0)
[23:37:13.217] Build status: STARTED
[23:37:13.217] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:21:0) after 5m33s
[23:42:46.207] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:21:0)
[23:42:46.226] Build status: STARTED
[23:42:46.226] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:22:0) after 7m19s
[23:50:05.215] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:22:0)
[23:50:05.302] Build status: STARTED
[23:50:05.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:23:0) after 4m0s
[23:54:05.315] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:23:0)
[23:54:05.339] Build status: STARTED
[23:54:05.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:24:0) after 5m37s
[23:59:42.358] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:24:0)
[23:59:42.378] Build status: STARTED
[23:59:42.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:25:0) after 9m26s
[00:09:08.390] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:25:0)
[00:09:08.455] Build status: STARTED
[00:09:08.455] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:26:0) after 8m17s
[00:17:25.569] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:26:0)
[00:17:25.584] Build status: STARTED
[00:17:25.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:27:0) after 8m42s
[00:26:07.818] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:27:0)
[00:26:07.835] Build status: STARTED
[00:26:07.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:28:0) after 3m28s
[00:29:35.958] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:28:0)
[00:29:35.974] Build status: STARTED
[00:29:35.974] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:29:0) after 9m13s
[00:38:49.113] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:29:0)
[00:38:49.140] Build status: STARTED
[00:38:49.140] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:30:0) after 6m56s
[00:45:45.136] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:30:0)
[00:45:45.160] Build status: STARTED
[00:45:45.160] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:31:0) after 4m38s
[00:50:23.179] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:31:0)
[00:50:23.200] Build status: STARTED
[00:50:23.200] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:32:0) after 9m15s
[00:59:38.244] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:32:0)
[00:59:38.267] Build status: STARTED
[00:59:38.267] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:33:0) after 3m28s
[01:03:06.249] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:33:0)
[01:03:06.269] Build status: STARTED
[01:03:06.270] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:34:0) after 6m12s
[01:09:18.570] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:34:0)
[01:09:18.586] Build status: STARTED
[01:09:18.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:35:0) after 4m37s
[01:13:55.767] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:35:0)
[01:13:55.820] Build status: STARTED
[01:13:55.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:36:0) after 8m31s
[01:22:26.937] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:36:0)
[01:22:26.950] Build status: STARTED
[01:22:26.951] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:37:0) after 5m6s
[01:27:33.020] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:37:0)
[01:27:33.046] Build status: STARTED
[01:27:33.046] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:38:0) after 2m31s
[01:30:04.082] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:38:0)
[01:30:04.112] Build status: STARTED
[01:30:04.112] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:39:0) after 7m47s
[01:37:51.177] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:39:0)
[01:37:51.194] Build status: STARTED
[01:37:51.194] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:40:0) after 1m48s
[01:39:39.215] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:40:0)
[01:39:39.238] Build status: STARTED
[01:39:39.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:41:0) after 6m27s
[01:46:06.251] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:41:0)
[01:46:06.268] Build status: STARTED
[01:46:06.268] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:42:0) after 8m59s
[01:55:05.432] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:42:0)
[01:55:05.447] Build status: STARTED
[01:55:05.447] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:43:0) after 8m59s
[02:04:04.527] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:43:0)
[02:04:04.543] Build status: STARTED
[02:04:04.543] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:44:0) after 5m19s
[02:09:23.580] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:44:0)
[02:09:23.599] Build status: STARTED
[02:09:23.599] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:45:0) after 6m13s
[02:15:36.737] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:45:0)
[02:15:36.809] Build status: STARTED
[02:15:36.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:46:0) after 7m12s
[02:22:48.876] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:46:0)
[02:22:48.899] Build status: STARTED
[02:22:48.899] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:47:0) after 7m2s
[02:29:50.936] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:47:0)
[02:29:51.078] Build status: STARTED
[02:29:51.078] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:48:0) after 8m18s
[02:38:09.062] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:48:0)
[02:38:09.084] Build status: STARTED
[02:38:09.084] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:49:0) after 9m30s
[02:47:39.149] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:49:0)
[02:47:39.168] Build status: STARTED
[02:47:39.168] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:50:0) after 8m6s
[02:55:45.207] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:50:0)
[02:55:45.224] Build status: STARTED
[02:55:45.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:51:0) after 1m48s
[02:57:33.202] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:51:0)
[02:57:33.254] Build status: STARTED
[02:57:33.254] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:52:0) after 9m48s
[03:07:21.269] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:52:0)
[03:07:21.289] Build status: STARTED
[03:07:21.289] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:53:0) after 4m51s
[03:12:12.322] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:53:0)
[03:12:12.340] Build status: STARTED
[03:12:12.340] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:54:0) after 3m15s
[03:15:27.457] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:54:0)
[03:15:27.479] Build status: STARTED
[03:15:27.479] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:55:0) after 8m21s
[03:23:48.507] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:55:0)
[03:23:48.528] Build status: STARTED
[03:23:48.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:56:0) after 6m8s
[03:29:56.551] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:56:0)
[03:29:56.573] Build status: STARTED
[03:29:56.573] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:57:0) after 7m18s
[03:37:14.741] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:57:0)
[03:37:14.758] Build status: STARTED
[03:37:14.758] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:58:0) after 5m53s
[03:43:07.777] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:58:0)
[03:43:07.794] Build status: STARTED
[03:43:07.795] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:59:0) after 1m31s
[03:44:38.819] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:59:0)
[03:44:38.844] Build status: STARTED
[03:44:38.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851660863736528400:60:0) after 7m58s
[03:48:18.004] Received PubSub notification, asking Buildbucket for the build status
[03:48:18.019] Build:
{
"id": "8686230801850484129",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 40092,
"createdBy": "project:chromium",
"createTime": "2026-03-26T21:50:00.106134459Z",
"startTime": "2026-03-26T21:50:55.287704173Z",
"endTime": "2026-03-27T03:48:17.737376314Z",
"updateTime": "2026-03-27T03:48:17.737376314Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "f8a90a556ba3648b75ac49c14715540692a03c49",
"ref": "refs/heads/main"
}
}
}
[03:48:18.019] Invocation finished in 5h58m19.09545716s with status FAILED