[20:09:34.825] New invocation is queued and will start shortly
[20:09:36.911] Starting the invocation (attempt 1)
[20:09:36.948] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/97eebf5444014e122f4b65b40c54af7c1513137b"
[20:09:36.948] Popped gitiles commit info from properties and tags
[20:09:36.948] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[20:09:36.948] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[20:09:36.948] Buildbucket request:
{
"requestId": "8851757777436598176",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "97eebf5444014e122f4b65b40c54af7c1513137b",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851757777436598176"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[20:09:37.521] Scheduled build:
{
"id": "8686327714488770625",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2302,
"createdBy": "project:chromium",
"createTime": "2026-03-25T20:09:37.012585865Z",
"updateTime": "2026-03-25T20:09:37.012585865Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "97eebf5444014e122f4b65b40c54af7c1513137b",
"ref": "refs/heads/main"
}
}
}
[20:09:37.521] Task URL: https://cr-buildbucket.appspot.com/build/8686327714488770625
[20:09:37.521] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:2:0) after 2m18s
[20:09:37.827] Received PubSub notification, asking Buildbucket for the build status
[20:09:37.844] Build status: SCHEDULED
[20:10:17.407] Received PubSub notification, asking Buildbucket for the build status
[20:10:17.427] Build status: STARTED
[20:11:55.538] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:2:0)
[20:11:55.588] Build status: STARTED
[20:11:55.588] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:5:0) after 2m15s
[20:14:10.678] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:5:0)
[20:14:10.696] Build status: STARTED
[20:14:10.696] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:6:0) after 1m10s
[20:15:20.771] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:6:0)
[20:15:20.837] Build status: STARTED
[20:15:20.837] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:7:0) after 1m12s
[20:16:32.858] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:7:0)
[20:16:32.878] Build status: STARTED
[20:16:32.878] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:8:0) after 6m51s
[20:23:24.059] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:8:0)
[20:23:24.137] Build status: STARTED
[20:23:24.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:9:0) after 6m8s
[20:29:32.240] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:9:0)
[20:29:32.257] Build status: STARTED
[20:29:32.257] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:10:0) after 9m9s
[20:38:41.395] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:10:0)
[20:38:41.414] Build status: STARTED
[20:38:41.414] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:11:0) after 7m19s
[20:46:00.527] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:11:0)
[20:46:00.544] Build status: STARTED
[20:46:00.544] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:12:0) after 8m52s
[20:54:52.563] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:12:0)
[20:54:52.582] Build status: STARTED
[20:54:52.582] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:13:0) after 8m24s
[21:03:16.620] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:13:0)
[21:03:16.634] Build status: STARTED
[21:03:16.634] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:14:0) after 3m19s
[21:06:35.661] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:14:0)
[21:06:35.688] Build status: STARTED
[21:06:35.688] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:15:0) after 9m4s
[21:15:39.728] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:15:0)
[21:15:39.745] Build status: STARTED
[21:15:39.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:16:0) after 9m48s
[21:25:27.856] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:16:0)
[21:25:27.876] Build status: STARTED
[21:25:27.876] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:17:0) after 5m59s
[21:31:26.911] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:17:0)
[21:31:26.964] Build status: STARTED
[21:31:26.965] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:18:0) after 3m45s
[21:35:11.989] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:18:0)
[21:35:12.017] Build status: STARTED
[21:35:12.017] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:19:0) after 5m0s
[21:40:12.060] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:19:0)
[21:40:12.079] Build status: STARTED
[21:40:12.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:20:0) after 2m3s
[21:42:15.161] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:20:0)
[21:42:15.182] Build status: STARTED
[21:42:15.183] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:21:0) after 4m59s
[21:47:14.315] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:21:0)
[21:47:14.330] Build status: STARTED
[21:47:14.330] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:22:0) after 7m16s
[21:54:30.349] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:22:0)
[21:54:30.370] Build status: STARTED
[21:54:30.370] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:23:0) after 4m25s
[21:58:55.389] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:23:0)
[21:58:55.408] Build status: STARTED
[21:58:55.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:24:0) after 5m37s
[22:04:32.565] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:24:0)
[22:04:32.584] Build status: STARTED
[22:04:32.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:25:0) after 5m55s
[22:10:27.620] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:25:0)
[22:10:27.635] Build status: STARTED
[22:10:27.635] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:26:0) after 1m42s
[22:12:09.659] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:26:0)
[22:12:09.674] Build status: STARTED
[22:12:09.674] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:27:0) after 9m22s
[22:21:31.792] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:27:0)
[22:21:31.810] Build status: STARTED
[22:21:31.810] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:28:0) after 9m29s
[22:31:00.828] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:28:0)
[22:31:00.844] Build status: STARTED
[22:31:00.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:29:0) after 8m4s
[22:39:04.871] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:29:0)
[22:39:04.886] Build status: STARTED
[22:39:04.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:30:0) after 5m23s
[22:44:27.904] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:30:0)
[22:44:28.028] Build status: STARTED
[22:44:28.028] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:31:0) after 4m50s
[22:49:18.040] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:31:0)
[22:49:18.060] Build status: STARTED
[22:49:18.060] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:32:0) after 5m29s
[22:54:47.044] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:32:0)
[22:54:47.063] Build status: STARTED
[22:54:47.063] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:33:0) after 6m50s
[23:01:37.123] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:33:0)
[23:01:37.141] Build status: STARTED
[23:01:37.141] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:34:0) after 7m41s
[23:09:18.163] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:34:0)
[23:09:18.177] Build status: STARTED
[23:09:18.178] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:35:0) after 4m42s
[23:14:00.181] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:35:0)
[23:14:00.203] Build status: STARTED
[23:14:00.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:36:0) after 4m14s
[23:18:14.279] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:36:0)
[23:18:14.301] Build status: STARTED
[23:18:14.301] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:37:0) after 2m54s
[23:21:08.305] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:37:0)
[23:21:08.322] Build status: STARTED
[23:21:08.322] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:38:0) after 7m44s
[23:28:52.342] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:38:0)
[23:28:52.417] Build status: STARTED
[23:28:52.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:39:0) after 9m36s
[23:38:28.466] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:39:0)
[23:38:28.832] Build status: STARTED
[23:38:28.832] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:40:0) after 1m16s
[23:39:44.911] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:40:0)
[23:39:44.968] Build status: STARTED
[23:39:44.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:41:0) after 6m5s
[23:45:50.099] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:41:0)
[23:45:51.283] Build status: STARTED
[23:45:51.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:42:0) after 9m50s
[23:55:41.346] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:42:0)
[23:55:41.386] Build status: STARTED
[23:55:41.386] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:43:0) after 5m16s
[00:00:57.422] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:43:0)
[00:00:57.442] Build status: STARTED
[00:00:57.443] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:44:0) after 1m53s
[00:02:50.586] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:44:0)
[00:02:50.603] Build status: STARTED
[00:02:50.603] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:45:0) after 3m49s
[00:06:39.630] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:45:0)
[00:06:39.720] Build status: STARTED
[00:06:39.720] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:46:0) after 2m0s
[00:08:39.763] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:46:0)
[00:08:39.779] Build status: STARTED
[00:08:39.779] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:47:0) after 1m19s
[00:09:58.849] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:47:0)
[00:09:58.873] Build status: STARTED
[00:09:58.874] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:48:0) after 6m21s
[00:16:19.945] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:48:0)
[00:16:19.962] Build status: STARTED
[00:16:19.962] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:49:0) after 1m49s
[00:18:09.489] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:49:0)
[00:18:09.531] Build status: STARTED
[00:18:09.531] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:50:0) after 8m39s
[00:26:48.604] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:50:0)
[00:26:48.621] Build status: STARTED
[00:26:48.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:51:0) after 6m7s
[00:32:55.645] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:51:0)
[00:32:55.660] Build status: STARTED
[00:32:55.660] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:52:0) after 7m49s
[00:40:44.710] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:52:0)
[00:40:44.725] Build status: STARTED
[00:40:44.725] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:53:0) after 6m35s
[00:47:19.821] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:53:0)
[00:47:19.839] Build status: STARTED
[00:47:19.839] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:54:0) after 3m43s
[00:51:02.859] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:54:0)
[00:51:02.872] Build status: STARTED
[00:51:02.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851757777436598176:55:0) after 7m22s
[00:53:30.652] Received PubSub notification, asking Buildbucket for the build status
[00:53:30.667] Build:
{
"id": "8686327714488770625",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2302,
"createdBy": "project:chromium",
"createTime": "2026-03-25T20:09:37.012585865Z",
"startTime": "2026-03-25T20:10:17.047445151Z",
"endTime": "2026-03-26T00:53:30.199617708Z",
"updateTime": "2026-03-26T00:53:30.199617708Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "97eebf5444014e122f4b65b40c54af7c1513137b",
"ref": "refs/heads/main"
}
}
}
[00:53:30.667] Invocation finished in 4h43m55.849748636s with status FAILED