[01:09:48.756] New invocation is queued and will start shortly
[01:09:49.809] Starting the invocation (attempt 1)
[01:09:49.838] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/bda78f7bcf06ae6366aeb8f95decfbbb57d99c27"
[01:09:49.838] Popped gitiles commit info from properties and tags
[01:09:49.838] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[01:09:49.838] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[01:09:49.838] Buildbucket request:
{
"requestId": "8851376500592172336",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bda78f7bcf06ae6366aeb8f95decfbbb57d99c27",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851376500592172336"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[01:09:50.472] Scheduled build:
{
"id": "8685946438684968785",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2364,
"createdBy": "project:chromium",
"createTime": "2026-03-30T01:09:49.953771680Z",
"updateTime": "2026-03-30T01:09:49.953771680Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bda78f7bcf06ae6366aeb8f95decfbbb57d99c27",
"ref": "refs/heads/main"
}
}
}
[01:09:50.472] Task URL: https://cr-buildbucket.appspot.com/build/8685946438684968785
[01:09:50.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:2:0) after 6m11s
[01:09:51.359] Received PubSub notification, asking Buildbucket for the build status
[01:09:51.378] Build status: SCHEDULED
[01:10:34.176] Received PubSub notification, asking Buildbucket for the build status
[01:10:34.194] Build status: STARTED
[01:16:01.551] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:2:0)
[01:16:01.567] Build status: STARTED
[01:16:01.567] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:5:0) after 3m51s
[01:19:52.737] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:5:0)
[01:19:52.750] Build status: STARTED
[01:19:52.750] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:6:0) after 7m40s
[01:27:32.995] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:6:0)
[01:27:33.018] Build status: STARTED
[01:27:33.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:7:0) after 7m23s
[01:34:56.036] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:7:0)
[01:34:56.051] Build status: STARTED
[01:34:56.051] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:8:0) after 4m12s
[01:39:08.164] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:8:0)
[01:39:08.179] Build status: STARTED
[01:39:08.179] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:9:0) after 5m46s
[01:44:54.195] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:9:0)
[01:44:54.211] Build status: STARTED
[01:44:54.212] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:10:0) after 8m15s
[01:53:09.256] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:10:0)
[01:53:09.273] Build status: STARTED
[01:53:09.273] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:11:0) after 2m0s
[01:55:09.250] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:11:0)
[01:55:09.314] Build status: STARTED
[01:55:09.314] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:12:0) after 1m0s
[01:56:09.531] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:12:0)
[01:56:09.555] Build status: STARTED
[01:56:09.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:13:0) after 3m1s
[01:59:10.668] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:13:0)
[01:59:10.694] Build status: STARTED
[01:59:10.694] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:14:0) after 7m58s
[02:07:09.358] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:14:0)
[02:07:09.383] Build status: STARTED
[02:07:09.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:15:0) after 8m16s
[02:15:25.403] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:15:0)
[02:15:25.436] Build status: STARTED
[02:15:25.436] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:16:0) after 8m26s
[02:23:51.463] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:16:0)
[02:23:51.485] Build status: STARTED
[02:23:51.485] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:17:0) after 3m20s
[02:27:11.541] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:17:0)
[02:27:11.626] Build status: STARTED
[02:27:11.626] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:18:0) after 7m21s
[02:34:32.644] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:18:0)
[02:34:32.662] Build status: STARTED
[02:34:32.662] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:19:0) after 4m49s
[02:39:21.675] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:19:0)
[02:39:21.694] Build status: STARTED
[02:39:21.694] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:20:0) after 7m58s
[02:47:19.712] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:20:0)
[02:47:19.728] Build status: STARTED
[02:47:19.728] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:21:0) after 3m47s
[02:51:06.830] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:21:0)
[02:51:06.852] Build status: STARTED
[02:51:06.852] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:22:0) after 5m47s
[02:56:53.829] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:22:0)
[02:56:53.848] Build status: STARTED
[02:56:53.848] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:23:0) after 1m24s
[02:58:17.946] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:23:0)
[02:58:17.962] Build status: STARTED
[02:58:17.962] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:24:0) after 4m57s
[03:03:14.983] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:24:0)
[03:03:15.213] Build status: STARTED
[03:03:15.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:25:0) after 9m33s
[03:12:48.272] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:25:0)
[03:12:48.286] Build status: STARTED
[03:12:48.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:26:0) after 4m19s
[03:17:07.303] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:26:0)
[03:17:07.386] Build status: STARTED
[03:17:07.386] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:27:0) after 5m15s
[03:22:22.403] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:27:0)
[03:22:22.425] Build status: STARTED
[03:22:22.425] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:28:0) after 2m14s
[03:24:36.446] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:28:0)
[03:24:36.466] Build status: STARTED
[03:24:36.466] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:29:0) after 5m29s
[03:30:05.558] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:29:0)
[03:30:05.585] Build status: STARTED
[03:30:05.585] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:30:0) after 5m43s
[03:35:48.620] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:30:0)
[03:35:48.678] Build status: STARTED
[03:35:48.679] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:31:0) after 9m14s
[03:45:03.348] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:31:0)
[03:45:03.380] Build status: STARTED
[03:45:03.380] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:32:0) after 9m58s
[03:55:01.360] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:32:0)
[03:55:01.377] Build status: STARTED
[03:55:01.377] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:33:0) after 9m27s
[04:04:28.754] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:33:0)
[04:04:28.814] Build status: STARTED
[04:04:28.814] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:34:0) after 8m37s
[04:13:05.986] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:34:0)
[04:13:06.006] Build status: STARTED
[04:13:06.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:35:0) after 4m0s
[04:17:06.029] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:35:0)
[04:17:06.043] Build status: STARTED
[04:17:06.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:36:0) after 8m12s
[04:25:18.143] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:36:0)
[04:25:18.160] Build status: STARTED
[04:25:18.160] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:37:0) after 6m12s
[04:31:30.178] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:37:0)
[04:31:30.244] Build status: STARTED
[04:31:30.244] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:38:0) after 3m1s
[04:34:31.263] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:38:0)
[04:34:31.282] Build status: STARTED
[04:34:31.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:39:0) after 7m39s
[04:42:10.390] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:39:0)
[04:42:10.407] Build status: STARTED
[04:42:10.407] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:40:0) after 5m49s
[04:48:00.031] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:40:0)
[04:48:00.051] Build status: STARTED
[04:48:00.051] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:41:0) after 3m20s
[04:51:20.072] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:41:0)
[04:51:20.093] Build status: STARTED
[04:51:20.094] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:42:0) after 5m2s
[04:56:22.109] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:42:0)
[04:56:22.129] Build status: STARTED
[04:56:22.129] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:43:0) after 8m0s
[05:04:22.114] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:43:0)
[05:04:22.131] Build status: STARTED
[05:04:22.131] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:44:0) after 3m47s
[05:08:09.197] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:44:0)
[05:08:09.212] Build status: STARTED
[05:08:09.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:45:0) after 3m53s
[05:12:02.231] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:45:0)
[05:12:02.307] Build status: STARTED
[05:12:02.307] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:46:0) after 2m51s
[05:14:53.713] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:46:0)
[05:14:53.730] Build status: STARTED
[05:14:53.730] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:47:0) after 3m10s
[05:18:03.796] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:47:0)
[05:18:03.817] Build status: STARTED
[05:18:03.817] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:48:0) after 3m15s
[05:21:18.835] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:48:0)
[05:21:18.860] Build status: STARTED
[05:21:18.860] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:49:0) after 4m50s
[05:26:08.959] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:49:0)
[05:26:08.995] Build status: STARTED
[05:26:08.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:50:0) after 1m48s
[05:27:57.012] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:50:0)
[05:27:57.028] Build status: STARTED
[05:27:57.028] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:51:0) after 3m17s
[05:31:14.041] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:51:0)
[05:31:14.076] Build status: STARTED
[05:31:14.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:52:0) after 5m18s
[05:36:32.138] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:52:0)
[05:36:32.155] Build status: STARTED
[05:36:32.155] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:53:0) after 8m47s
[05:45:19.207] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:53:0)
[05:45:19.226] Build status: STARTED
[05:45:19.226] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851376500592172336:54:0) after 8m35s
[05:49:54.395] Received PubSub notification, asking Buildbucket for the build status
[05:49:54.413] Build:
{
"id": "8685946438684968785",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2364,
"createdBy": "project:chromium",
"createTime": "2026-03-30T01:09:49.953771680Z",
"startTime": "2026-03-30T01:10:33.822340458Z",
"endTime": "2026-03-30T05:49:54.138190810Z",
"updateTime": "2026-03-30T05:49:54.138190810Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bda78f7bcf06ae6366aeb8f95decfbbb57d99c27",
"ref": "refs/heads/main"
}
}
}
[05:49:54.413] Invocation finished in 4h40m5.662112045s with status FAILED