[16:58:34.472] New invocation is queued and will start shortly
[16:58:35.603] Starting the invocation (attempt 1)
[16:58:35.630] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0bd7eed851895a2713144465f83d749a226253a0"
[16:58:35.630] Popped gitiles commit info from properties and tags
[16:58:35.630] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[16:58:35.630] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[16:58:35.630] Buildbucket request:
{
"requestId": "8850592033925156992",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0bd7eed851895a2713144465f83d749a226253a0",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850592033925156992"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[16:58:36.184] Scheduled build:
{
"id": "8685161971956331457",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2408,
"createdBy": "project:chromium",
"createTime": "2026-04-07T16:58:35.724228922Z",
"updateTime": "2026-04-07T16:58:35.724228922Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0bd7eed851895a2713144465f83d749a226253a0",
"ref": "refs/heads/main"
}
}
}
[16:58:36.184] Task URL: https://cr-buildbucket.appspot.com/build/8685161971956331457
[16:58:36.184] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:2:0) after 3m39s
[16:58:37.207] Received PubSub notification, asking Buildbucket for the build status
[16:58:37.233] Build status: SCHEDULED
[16:59:23.345] Received PubSub notification, asking Buildbucket for the build status
[16:59:23.365] Build status: STARTED
[17:02:15.223] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:2:0)
[17:02:15.243] Build status: STARTED
[17:02:15.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:5:0) after 7m2s
[17:09:17.288] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:5:0)
[17:09:17.315] Build status: STARTED
[17:09:17.315] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:6:0) after 3m22s
[17:12:39.439] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:6:0)
[17:12:39.552] Build status: STARTED
[17:12:39.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:7:0) after 4m7s
[17:16:46.530] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:7:0)
[17:16:46.545] Build status: STARTED
[17:16:46.545] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:8:0) after 2m1s
[17:18:47.568] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:8:0)
[17:18:47.591] Build status: STARTED
[17:18:47.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:9:0) after 3m7s
[17:21:54.604] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:9:0)
[17:21:54.619] Build status: STARTED
[17:21:54.619] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:10:0) after 7m55s
[17:29:49.640] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:10:0)
[17:29:49.665] Build status: STARTED
[17:29:49.665] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:11:0) after 3m48s
[17:33:37.847] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:11:0)
[17:33:37.873] Build status: STARTED
[17:33:37.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:12:0) after 8m54s
[17:42:32.022] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:12:0)
[17:42:32.086] Build status: STARTED
[17:42:32.086] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:13:0) after 9m36s
[17:52:08.122] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:13:0)
[17:52:08.144] Build status: STARTED
[17:52:08.144] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:14:0) after 8m31s
[18:00:39.176] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:14:0)
[18:00:39.194] Build status: STARTED
[18:00:39.194] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:15:0) after 5m1s
[18:05:40.220] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:15:0)
[18:05:40.240] Build status: STARTED
[18:05:40.240] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:16:0) after 1m6s
[18:06:46.337] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:16:0)
[18:06:46.386] Build status: STARTED
[18:06:46.386] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:17:0) after 6m25s
[18:13:11.406] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:17:0)
[18:13:11.482] Build status: STARTED
[18:13:11.482] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:18:0) after 8m53s
[18:22:04.567] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:18:0)
[18:22:04.583] Build status: STARTED
[18:22:04.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:19:0) after 2m20s
[18:24:24.677] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:19:0)
[18:24:24.698] Build status: STARTED
[18:24:24.699] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:20:0) after 4m43s
[18:29:07.800] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:20:0)
[18:29:07.821] Build status: STARTED
[18:29:07.821] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:21:0) after 9m52s
[18:38:59.971] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:21:0)
[18:39:00.000] Build status: STARTED
[18:39:00.000] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:22:0) after 4m59s
[18:43:59.083] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:22:0)
[18:43:59.109] Build status: STARTED
[18:43:59.109] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:23:0) after 8m21s
[18:52:20.183] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:23:0)
[18:52:20.229] Build status: STARTED
[18:52:20.229] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:24:0) after 6m3s
[18:58:23.349] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:24:0)
[18:58:23.364] Build status: STARTED
[18:58:23.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:25:0) after 1m27s
[18:59:50.450] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:25:0)
[18:59:50.576] Build status: STARTED
[18:59:50.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:26:0) after 4m32s
[19:04:22.601] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:26:0)
[19:04:22.621] Build status: STARTED
[19:04:22.621] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:27:0) after 6m14s
[19:10:36.814] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:27:0)
[19:10:37.040] Build status: STARTED
[19:10:37.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:28:0) after 5m37s
[19:16:14.377] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:28:0)
[19:16:14.395] Build status: STARTED
[19:16:14.395] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:29:0) after 4m48s
[19:21:02.473] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:29:0)
[19:21:02.528] Build status: STARTED
[19:21:02.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:30:0) after 9m6s
[19:30:08.586] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:30:0)
[19:30:08.625] Build status: STARTED
[19:30:08.625] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:31:0) after 1m13s
[19:31:21.689] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:31:0)
[19:31:21.707] Build status: STARTED
[19:31:21.708] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:32:0) after 1m38s
[19:32:59.732] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:32:0)
[19:32:59.752] Build status: STARTED
[19:32:59.752] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:33:0) after 4m58s
[19:37:57.769] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:33:0)
[19:37:57.786] Build status: STARTED
[19:37:57.786] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:34:0) after 5m21s
[19:43:18.769] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:34:0)
[19:43:18.796] Build status: STARTED
[19:43:18.796] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:35:0) after 1m23s
[19:44:41.814] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:35:0)
[19:44:41.834] Build status: STARTED
[19:44:41.834] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:36:0) after 2m34s
[19:47:15.889] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:36:0)
[19:47:15.905] Build status: STARTED
[19:47:15.906] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:37:0) after 2m17s
[19:49:32.986] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:37:0)
[19:49:33.042] Build status: STARTED
[19:49:33.042] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:38:0) after 5m56s
[19:55:29.098] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:38:0)
[19:55:29.121] Build status: STARTED
[19:55:29.121] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:39:0) after 2m15s
[19:57:44.140] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:39:0)
[19:57:44.162] Build status: STARTED
[19:57:44.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:40:0) after 6m17s
[20:04:01.372] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:40:0)
[20:04:01.391] Build status: STARTED
[20:04:01.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:41:0) after 5m5s
[20:09:06.465] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:41:0)
[20:09:06.485] Build status: STARTED
[20:09:06.485] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:42:0) after 2m13s
[20:11:19.506] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:42:0)
[20:11:19.536] Build status: STARTED
[20:11:19.536] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:43:0) after 2m34s
[20:13:53.554] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:43:0)
[20:13:53.580] Build status: STARTED
[20:13:53.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:44:0) after 9m13s
[20:23:06.620] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:44:0)
[20:23:06.639] Build status: STARTED
[20:23:06.639] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:45:0) after 3m3s
[20:26:09.666] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:45:0)
[20:26:09.691] Build status: STARTED
[20:26:09.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:46:0) after 4m9s
[20:30:18.809] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:46:0)
[20:30:18.829] Build status: STARTED
[20:30:18.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:47:0) after 5m31s
[20:35:49.857] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:47:0)
[20:35:49.873] Build status: STARTED
[20:35:49.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:48:0) after 7m4s
[20:42:53.850] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:48:0)
[20:42:53.876] Build status: STARTED
[20:42:53.876] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:49:0) after 3m48s
[20:46:41.895] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:49:0)
[20:46:41.916] Build status: STARTED
[20:46:41.916] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:50:0) after 9m40s
[20:56:21.972] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:50:0)
[20:56:21.996] Build status: STARTED
[20:56:21.996] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:51:0) after 6m13s
[21:02:35.142] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:51:0)
[21:02:35.166] Build status: STARTED
[21:02:35.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:52:0) after 6m11s
[21:08:46.236] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:52:0)
[21:08:46.261] Build status: STARTED
[21:08:46.261] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:53:0) after 1m35s
[21:10:21.278] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:53:0)
[21:10:21.334] Build status: STARTED
[21:10:21.334] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:54:0) after 1m14s
[21:11:35.372] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:54:0)
[21:11:35.392] Build status: STARTED
[21:11:35.392] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:55:0) after 9m14s
[21:20:49.430] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:55:0)
[21:20:49.447] Build status: STARTED
[21:20:49.447] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:56:0) after 4m29s
[21:25:18.822] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:56:0)
[21:25:18.841] Build status: STARTED
[21:25:18.841] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:57:0) after 7m24s
[21:32:43.027] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:57:0)
[21:32:43.043] Build status: STARTED
[21:32:43.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:58:0) after 1m21s
[21:34:04.068] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:58:0)
[21:34:04.086] Build status: STARTED
[21:34:04.086] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:59:0) after 6m50s
[21:40:54.143] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:59:0)
[21:40:54.194] Build status: STARTED
[21:40:54.194] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:60:0) after 4m15s
[21:45:09.321] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:60:0)
[21:45:09.341] Build status: STARTED
[21:45:09.341] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850592033925156992:61:0) after 5m33s
[21:45:59.516] Received PubSub notification, asking Buildbucket for the build status
[21:45:59.534] Build:
{
"id": "8685161971956331457",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2408,
"createdBy": "project:chromium",
"createTime": "2026-04-07T16:58:35.724228922Z",
"startTime": "2026-04-07T16:59:23.096401343Z",
"endTime": "2026-04-07T21:45:57.515802539Z",
"updateTime": "2026-04-07T21:45:57.515802539Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0bd7eed851895a2713144465f83d749a226253a0",
"ref": "refs/heads/main"
}
}
}
[21:45:59.534] Invocation finished in 4h47m25.070554416s with status FAILED