[11:09:28.482] New invocation is queued and will start shortly
[11:09:30.940] Starting the invocation (attempt 1)
[11:09:30.964] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/368d532865ac5cc5a91520285838addba73f6656"
[11:09:30.964] Popped gitiles commit info from properties and tags
[11:09:30.964] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[11:09:30.964] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[11:09:30.964] Buildbucket request:
{
"requestId": "8851882354914875392",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "368d532865ac5cc5a91520285838addba73f6656",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851882354914875392"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[11:09:31.629] Scheduled build:
{
"id": "8686452291518653761",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2295,
"createdBy": "project:chromium",
"createTime": "2026-03-24T11:09:31.098057657Z",
"updateTime": "2026-03-24T11:09:31.098057657Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "368d532865ac5cc5a91520285838addba73f6656",
"ref": "refs/heads/main"
}
}
}
[11:09:31.629] Task URL: https://cr-buildbucket.appspot.com/build/8686452291518653761
[11:09:31.629] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:2:0) after 6m39s
[11:09:32.626] Received PubSub notification, asking Buildbucket for the build status
[11:09:32.652] Build status: SCHEDULED
[11:10:15.812] Received PubSub notification, asking Buildbucket for the build status
[11:10:15.828] Build status: STARTED
[11:16:10.649] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:2:0)
[11:16:10.675] Build status: STARTED
[11:16:10.675] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:5:0) after 1m49s
[11:17:59.714] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:5:0)
[11:17:59.738] Build status: STARTED
[11:17:59.738] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:6:0) after 8m58s
[11:26:57.850] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:6:0)
[11:26:57.865] Build status: STARTED
[11:26:57.865] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:7:0) after 5m23s
[11:32:20.927] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:7:0)
[11:32:20.947] Build status: STARTED
[11:32:20.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:8:0) after 8m36s
[11:40:57.078] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:8:0)
[11:40:57.095] Build status: STARTED
[11:40:57.095] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:9:0) after 1m9s
[11:42:06.154] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:9:0)
[11:42:06.168] Build status: STARTED
[11:42:06.168] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:10:0) after 2m27s
[11:44:33.237] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:10:0)
[11:44:33.258] Build status: STARTED
[11:44:33.258] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:11:0) after 3m44s
[11:48:17.244] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:11:0)
[11:48:17.314] Build status: STARTED
[11:48:17.314] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:12:0) after 4m25s
[11:52:42.401] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:12:0)
[11:52:42.419] Build status: STARTED
[11:52:42.419] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:13:0) after 4m39s
[11:57:21.486] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:13:0)
[11:57:21.506] Build status: STARTED
[11:57:21.507] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:14:0) after 5m17s
[12:02:38.577] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:14:0)
[12:02:38.589] Build status: STARTED
[12:02:38.589] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:15:0) after 2m17s
[12:04:55.646] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:15:0)
[12:04:55.667] Build status: STARTED
[12:04:55.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:16:0) after 6m41s
[12:11:36.688] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:16:0)
[12:11:36.713] Build status: STARTED
[12:11:36.714] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:17:0) after 7m26s
[12:19:02.722] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:17:0)
[12:19:02.741] Build status: STARTED
[12:19:02.741] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:18:0) after 7m37s
[12:26:39.723] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:18:0)
[12:26:39.748] Build status: STARTED
[12:26:39.748] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:19:0) after 8m59s
[12:35:38.774] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:19:0)
[12:35:38.795] Build status: STARTED
[12:35:38.795] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:20:0) after 4m37s
[12:40:15.775] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:20:0)
[12:40:15.792] Build status: STARTED
[12:40:15.792] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:21:0) after 8m22s
[12:48:37.864] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:21:0)
[12:48:37.882] Build status: STARTED
[12:48:37.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:22:0) after 3m28s
[12:52:05.901] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:22:0)
[12:52:05.915] Build status: STARTED
[12:52:05.915] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:23:0) after 7m11s
[12:59:16.933] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:23:0)
[12:59:17.024] Build status: STARTED
[12:59:17.024] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:24:0) after 6m10s
[13:05:27.006] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:24:0)
[13:05:27.034] Build status: STARTED
[13:05:27.034] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:25:0) after 6m39s
[13:12:06.056] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:25:0)
[13:12:06.078] Build status: STARTED
[13:12:06.078] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:26:0) after 3m42s
[13:15:48.148] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:26:0)
[13:15:48.170] Build status: STARTED
[13:15:48.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:27:0) after 6m7s
[13:21:55.169] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:27:0)
[13:21:55.186] Build status: STARTED
[13:21:55.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:28:0) after 5m14s
[13:27:09.202] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:28:0)
[13:27:09.254] Build status: STARTED
[13:27:09.254] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:29:0) after 5m51s
[13:33:00.280] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:29:0)
[13:33:00.299] Build status: STARTED
[13:33:00.299] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:30:0) after 4m14s
[13:37:14.325] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:30:0)
[13:37:14.391] Build status: STARTED
[13:37:14.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:31:0) after 8m6s
[13:45:20.407] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:31:0)
[13:45:20.431] Build status: STARTED
[13:45:20.431] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:32:0) after 5m14s
[13:50:34.496] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:32:0)
[13:50:34.524] Build status: STARTED
[13:50:34.524] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:33:0) after 2m14s
[13:52:48.592] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:33:0)
[13:52:48.628] Build status: STARTED
[13:52:48.628] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:34:0) after 9m33s
[14:02:21.706] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:34:0)
[14:02:21.725] Build status: STARTED
[14:02:21.725] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:35:0) after 6m12s
[14:08:33.763] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:35:0)
[14:08:33.778] Build status: STARTED
[14:08:33.778] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:36:0) after 4m56s
[14:13:29.795] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:36:0)
[14:13:29.822] Build status: STARTED
[14:13:29.822] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:37:0) after 7m0s
[14:20:29.842] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:37:0)
[14:20:29.858] Build status: STARTED
[14:20:29.859] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:38:0) after 2m29s
[14:22:58.914] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:38:0)
[14:22:58.950] Build status: STARTED
[14:22:58.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:39:0) after 8m10s
[14:31:09.054] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:39:0)
[14:31:09.073] Build status: STARTED
[14:31:09.073] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:40:0) after 2m41s
[14:33:50.181] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:40:0)
[14:33:50.203] Build status: STARTED
[14:33:50.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:41:0) after 6m54s
[14:40:44.267] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:41:0)
[14:40:44.284] Build status: STARTED
[14:40:44.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:42:0) after 2m31s
[14:43:15.307] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:42:0)
[14:43:15.333] Build status: STARTED
[14:43:15.333] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:43:0) after 5m9s
[14:48:24.398] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:43:0)
[14:48:24.452] Build status: STARTED
[14:48:24.453] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:44:0) after 1m28s
[14:49:52.473] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:44:0)
[14:49:52.504] Build status: STARTED
[14:49:52.504] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:45:0) after 5m54s
[14:55:46.614] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:45:0)
[14:55:46.639] Build status: STARTED
[14:55:46.639] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:46:0) after 4m27s
[15:00:13.658] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:46:0)
[15:00:13.688] Build status: STARTED
[15:00:13.688] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:47:0) after 5m25s
[15:05:38.782] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:47:0)
[15:05:38.800] Build status: STARTED
[15:05:38.800] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:48:0) after 1m11s
[15:06:49.866] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:48:0)
[15:06:49.885] Build status: STARTED
[15:06:49.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:49:0) after 4m48s
[15:11:37.973] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:49:0)
[15:11:37.988] Build status: STARTED
[15:11:37.988] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:50:0) after 2m35s
[15:14:13.040] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:50:0)
[15:14:13.065] Build status: STARTED
[15:14:13.065] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:51:0) after 2m45s
[15:16:58.081] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:51:0)
[15:16:58.102] Build status: STARTED
[15:16:58.102] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:52:0) after 8m44s
[15:25:42.242] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:52:0)
[15:25:42.314] Build status: STARTED
[15:25:42.314] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:53:0) after 7m21s
[15:33:03.406] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:53:0)
[15:33:03.442] Build status: STARTED
[15:33:03.442] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:54:0) after 8m10s
[15:41:13.481] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:54:0)
[15:41:13.557] Build status: STARTED
[15:41:13.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:55:0) after 1m57s
[15:43:10.678] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:55:0)
[15:43:10.699] Build status: STARTED
[15:43:10.699] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:56:0) after 2m41s
[15:45:51.793] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:56:0)
[15:45:51.813] Build status: STARTED
[15:45:51.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851882354914875392:57:0) after 9m42s
[15:51:49.257] Received PubSub notification, asking Buildbucket for the build status
[15:51:49.273] Build:
{
"id": "8686452291518653761",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2295,
"createdBy": "project:chromium",
"createTime": "2026-03-24T11:09:31.098057657Z",
"startTime": "2026-03-24T11:10:15.074808962Z",
"endTime": "2026-03-24T15:51:48.637520604Z",
"updateTime": "2026-03-24T15:51:48.637520604Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "368d532865ac5cc5a91520285838addba73f6656",
"ref": "refs/heads/main"
}
}
}
[15:51:49.274] Invocation finished in 4h42m20.797347506s with status FAILED