[04:07:40.293] New invocation is queued and will start shortly
[04:07:41.408] Starting the invocation (attempt 1)
[04:07:41.432] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/c7e9116783f4036c2ccef5e4d992053cfc55efef"
[04:07:41.432] Popped gitiles commit info from properties and tags
[04:07:41.432] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:07:41.432] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:07:41.432] Buildbucket request:
{
"requestId": "8842849195831972528",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c7e9116783f4036c2ccef5e4d992053cfc55efef",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8842849195831972528"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:07:41.902] Scheduled build:
{
"id": "8677419133918980129",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 3978,
"createdBy": "project:chromium",
"createTime": "2026-07-02T04:07:41.497502156Z",
"updateTime": "2026-07-02T04:07:41.497502156Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c7e9116783f4036c2ccef5e4d992053cfc55efef",
"ref": "refs/heads/main"
}
}
}
[04:07:41.902] Task URL: https://cr-buildbucket.appspot.com/build/8677419133918980129
[04:07:41.902] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:2:0) after 8m42s
[04:07:42.913] Received PubSub notification, asking Buildbucket for the build status
[04:07:42.933] Build status: SCHEDULED
[04:07:46.391] Received PubSub notification, asking Buildbucket for the build status
[04:07:46.416] Build status: STARTED
[04:16:24.015] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:2:0)
[04:16:24.039] Build status: STARTED
[04:16:24.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:5:0) after 2m40s
[04:19:04.062] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:5:0)
[04:19:04.087] Build status: STARTED
[04:19:04.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:6:0) after 4m48s
[04:23:52.121] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:6:0)
[04:23:52.152] Build status: STARTED
[04:23:52.152] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:7:0) after 8m35s
[04:32:27.225] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:7:0)
[04:32:27.248] Build status: STARTED
[04:32:27.248] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:8:0) after 7m10s
[04:39:37.365] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:8:0)
[04:39:37.965] Build status: STARTED
[04:39:37.965] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:9:0) after 5m16s
[04:44:53.979] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:9:0)
[04:44:54.005] Build status: STARTED
[04:44:54.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:10:0) after 2m40s
[04:47:34.084] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:10:0)
[04:47:34.102] Build status: STARTED
[04:47:34.102] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:11:0) after 8m46s
[04:56:20.248] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:11:0)
[04:56:20.268] Build status: STARTED
[04:56:20.268] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:12:0) after 6m19s
[05:02:39.374] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:12:0)
[05:02:39.390] Build status: STARTED
[05:02:39.390] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:13:0) after 1m0s
[05:03:39.406] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:13:0)
[05:03:39.436] Build status: STARTED
[05:03:39.436] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:14:0) after 3m36s
[05:07:15.455] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:14:0)
[05:07:15.473] Build status: STARTED
[05:07:15.473] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:15:0) after 9m48s
[05:17:03.497] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:15:0)
[05:17:03.558] Build status: STARTED
[05:17:03.558] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:16:0) after 6m7s
[05:23:10.582] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:16:0)
[05:23:10.692] Build status: STARTED
[05:23:10.692] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:17:0) after 4m16s
[05:27:26.759] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:17:0)
[05:27:26.775] Build status: STARTED
[05:27:26.775] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:18:0) after 1m29s
[05:28:55.829] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:18:0)
[05:28:55.846] Build status: STARTED
[05:28:55.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:19:0) after 8m49s
[05:37:44.965] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:19:0)
[05:37:44.987] Build status: STARTED
[05:37:44.987] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:20:0) after 8m55s
[05:46:40.048] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:20:0)
[05:46:40.109] Build status: STARTED
[05:46:40.109] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:21:0) after 9m36s
[05:56:16.147] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:21:0)
[05:56:16.168] Build status: STARTED
[05:56:16.168] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:22:0) after 2m1s
[05:58:17.182] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:22:0)
[05:58:17.206] Build status: STARTED
[05:58:17.206] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:23:0) after 3m53s
[06:02:10.222] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:23:0)
[06:02:10.240] Build status: STARTED
[06:02:10.240] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:24:0) after 3m26s
[06:05:36.275] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:24:0)
[06:05:36.302] Build status: STARTED
[06:05:36.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:25:0) after 9m38s
[06:15:14.408] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:25:0)
[06:15:14.421] Build status: STARTED
[06:15:14.421] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:26:0) after 9m2s
[06:24:16.435] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:26:0)
[06:24:16.451] Build status: STARTED
[06:24:16.451] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:27:0) after 3m2s
[06:27:18.566] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:27:0)
[06:27:18.587] Build status: STARTED
[06:27:18.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:28:0) after 3m49s
[06:31:07.639] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:28:0)
[06:31:07.702] Build status: STARTED
[06:31:07.702] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:29:0) after 1m43s
[06:32:50.744] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:29:0)
[06:32:50.776] Build status: STARTED
[06:32:50.777] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:30:0) after 4m16s
[06:37:06.853] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:30:0)
[06:37:06.873] Build status: STARTED
[06:37:06.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:31:0) after 9m26s
[06:46:32.866] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:31:0)
[06:46:32.883] Build status: STARTED
[06:46:32.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:32:0) after 1m39s
[06:48:11.900] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:32:0)
[06:48:11.920] Build status: STARTED
[06:48:11.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:33:0) after 3m6s
[06:51:18.055] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:33:0)
[06:51:18.071] Build status: STARTED
[06:51:18.071] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:34:0) after 3m3s
[06:54:21.092] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:34:0)
[06:54:21.109] Build status: STARTED
[06:54:21.109] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:35:0) after 7m33s
[07:01:54.177] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:35:0)
[07:01:54.196] Build status: STARTED
[07:01:54.196] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:36:0) after 2m4s
[07:03:58.212] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:36:0)
[07:03:58.317] Build status: STARTED
[07:03:58.317] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:37:0) after 8m35s
[07:12:33.372] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:37:0)
[07:12:33.393] Build status: STARTED
[07:12:33.393] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:38:0) after 8m49s
[07:21:22.517] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:38:0)
[07:21:22.538] Build status: STARTED
[07:21:22.538] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:39:0) after 8m51s
[07:30:13.561] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:39:0)
[07:30:13.598] Build status: STARTED
[07:30:13.598] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:40:0) after 8m46s
[07:38:59.592] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:40:0)
[07:38:59.621] Build status: STARTED
[07:38:59.621] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:41:0) after 9m55s
[07:48:54.671] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:41:0)
[07:48:54.692] Build status: STARTED
[07:48:54.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:42:0) after 1m49s
[07:50:43.710] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:42:0)
[07:50:43.726] Build status: STARTED
[07:50:43.726] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:43:0) after 5m22s
[07:56:05.747] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:43:0)
[07:56:05.772] Build status: STARTED
[07:56:05.773] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:44:0) after 7m8s
[08:03:13.791] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:44:0)
[08:03:13.810] Build status: STARTED
[08:03:13.810] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:45:0) after 3m2s
[08:06:15.858] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:45:0)
[08:06:15.883] Build status: STARTED
[08:06:15.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:46:0) after 2m45s
[08:09:01.024] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:46:0)
[08:09:01.043] Build status: STARTED
[08:09:01.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:47:0) after 7m44s
[08:16:45.111] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:47:0)
[08:16:45.131] Build status: STARTED
[08:16:45.131] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8842849195831972528:48:0) after 4m49s
[08:21:26.336] Received PubSub notification, asking Buildbucket for the build status
[08:21:26.378] Build:
{
"id": "8677419133918980129",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 3978,
"createdBy": "project:chromium",
"createTime": "2026-07-02T04:07:41.497502156Z",
"startTime": "2026-07-02T04:07:45.917348680Z",
"endTime": "2026-07-02T08:21:25.917742722Z",
"updateTime": "2026-07-02T08:21:25.917742722Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c7e9116783f4036c2ccef5e4d992053cfc55efef",
"ref": "refs/heads/main"
}
}
}
[08:21:26.378] Invocation finished in 4h13m46.089254572s with status FAILED