[09:31:08.571] New invocation is queued and will start shortly
[09:31:10.108] Starting the invocation (attempt 1)
[09:31:10.135] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/ff9d0b8e17d3da1fb81df0e7c0afced05021be80"
[09:31:10.135] Popped gitiles commit info from properties and tags
[09:31:10.135] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[09:31:10.135] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[09:31:10.135] Buildbucket request:
{
"requestId": "8833406760271561408",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ff9d0b8e17d3da1fb81df0e7c0afced05021be80",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8833406760271561408"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[09:31:10.659] Scheduled build:
{
"id": "8667976697893810545",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 5856,
"createdBy": "project:chromium",
"createTime": "2026-10-14T09:31:10.218218824Z",
"updateTime": "2026-10-14T09:31:10.218218824Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ff9d0b8e17d3da1fb81df0e7c0afced05021be80",
"ref": "refs/heads/main"
}
}
}
[09:31:10.659] Task URL: https://cr-buildbucket.appspot.com/build/8667976697893810545
[09:31:10.659] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:2:0) after 8m57s
[09:31:44.648] Received PubSub notification, asking Buildbucket for the build status
[09:31:44.674] Build status: SCHEDULED
[09:33:09.979] Received PubSub notification, asking Buildbucket for the build status
[09:33:10.051] Build status: STARTED
[09:40:07.727] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:2:0)
[09:40:07.747] Build status: STARTED
[09:40:07.747] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:5:0) after 1m49s
[09:41:56.762] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:5:0)
[09:41:56.784] Build status: STARTED
[09:41:56.784] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:6:0) after 6m39s
[09:48:35.816] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:6:0)
[09:48:35.831] Build status: STARTED
[09:48:35.831] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:7:0) after 9m11s
[09:57:46.930] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:7:0)
[09:57:46.950] Build status: STARTED
[09:57:46.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:8:0) after 6m36s
[10:04:22.976] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:8:0)
[10:04:23.150] Build status: STARTED
[10:04:23.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:9:0) after 5m44s
[10:10:07.168] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:9:0)
[10:10:07.185] Build status: STARTED
[10:10:07.185] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:10:0) after 5m7s
[10:15:14.211] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:10:0)
[10:15:14.297] Build status: STARTED
[10:15:14.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:11:0) after 4m43s
[10:19:57.419] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:11:0)
[10:19:57.440] Build status: STARTED
[10:19:57.440] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:12:0) after 2m38s
[10:22:35.523] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:12:0)
[10:22:35.543] Build status: STARTED
[10:22:35.543] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:13:0) after 6m11s
[10:28:46.693] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:13:0)
[10:28:46.714] Build status: STARTED
[10:28:46.714] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:14:0) after 3m9s
[10:31:55.841] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:14:0)
[10:31:55.862] Build status: STARTED
[10:31:55.862] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:15:0) after 1m18s
[10:33:13.909] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:15:0)
[10:33:13.963] Build status: STARTED
[10:33:13.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:16:0) after 5m34s
[10:38:47.979] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:16:0)
[10:38:48.057] Build status: STARTED
[10:38:48.057] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:17:0) after 3m19s
[10:42:07.144] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:17:0)
[10:42:07.231] Build status: STARTED
[10:42:07.231] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:18:0) after 6m6s
[10:48:13.263] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:18:0)
[10:48:13.284] Build status: STARTED
[10:48:13.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:19:0) after 2m6s
[10:50:19.384] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:19:0)
[10:50:19.419] Build status: STARTED
[10:50:19.419] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:20:0) after 2m48s
[10:53:07.613] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:20:0)
[10:53:07.659] Build status: STARTED
[10:53:07.661] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:21:0) after 2m54s
[10:56:01.777] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:21:0)
[10:56:01.794] Build status: STARTED
[10:56:01.795] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:22:0) after 6m47s
[11:02:48.865] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:22:0)
[11:02:49.053] Build status: STARTED
[11:02:49.053] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:23:0) after 6m20s
[11:09:09.056] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:23:0)
[11:09:09.071] Build status: STARTED
[11:09:09.071] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:24:0) after 3m21s
[11:12:30.097] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:24:0)
[11:12:30.117] Build status: STARTED
[11:12:30.117] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:25:0) after 4m16s
[11:16:46.197] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:25:0)
[11:16:46.219] Build status: STARTED
[11:16:46.219] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:26:0) after 2m8s
[11:18:54.285] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:26:0)
[11:18:54.317] Build status: STARTED
[11:18:54.317] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:27:0) after 9m19s
[11:28:13.338] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:27:0)
[11:28:13.383] Build status: STARTED
[11:28:13.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:28:0) after 5m58s
[11:34:11.543] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:28:0)
[11:34:11.564] Build status: STARTED
[11:34:11.564] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:29:0) after 4m58s
[11:39:09.601] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:29:0)
[11:39:09.619] Build status: STARTED
[11:39:09.619] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:30:0) after 3m37s
[11:42:46.695] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:30:0)
[11:42:46.745] Build status: STARTED
[11:42:46.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:31:0) after 3m19s
[11:46:05.765] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:31:0)
[11:46:05.890] Build status: STARTED
[11:46:05.890] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:32:0) after 7m2s
[11:53:07.911] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:32:0)
[11:53:07.932] Build status: STARTED
[11:53:07.932] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:33:0) after 5m27s
[11:58:34.943] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:33:0)
[11:58:34.977] Build status: STARTED
[11:58:34.977] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:34:0) after 7m33s
[12:06:08.008] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:34:0)
[12:06:08.045] Build status: STARTED
[12:06:08.045] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:35:0) after 8m35s
[12:14:43.069] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:35:0)
[12:14:43.096] Build status: STARTED
[12:14:43.096] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:36:0) after 5m5s
[12:19:48.283] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:36:0)
[12:19:48.301] Build status: STARTED
[12:19:48.301] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:37:0) after 7m27s
[12:27:15.321] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:37:0)
[12:27:15.354] Build status: STARTED
[12:27:15.354] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:38:0) after 1m5s
[12:28:20.461] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:38:0)
[12:28:20.481] Build status: STARTED
[12:28:20.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:39:0) after 1m33s
[12:29:53.501] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:39:0)
[12:29:53.518] Build status: STARTED
[12:29:53.518] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:40:0) after 4m38s
[12:34:31.538] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:40:0)
[12:34:31.557] Build status: STARTED
[12:34:31.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:41:0) after 2m10s
[12:36:41.726] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:41:0)
[12:36:41.746] Build status: STARTED
[12:36:41.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:42:0) after 4m10s
[12:40:51.911] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:42:0)
[12:40:51.932] Build status: STARTED
[12:40:51.932] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:43:0) after 1m46s
[12:42:38.044] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:43:0)
[12:42:38.073] Build status: STARTED
[12:42:38.073] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:44:0) after 6m53s
[12:49:31.100] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:44:0)
[12:49:31.123] Build status: STARTED
[12:49:31.123] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8833406760271561408:45:0) after 4m48s
[12:51:50.866] Received PubSub notification, asking Buildbucket for the build status
[12:51:50.888] Build:
{
"id": "8667976697893810545",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 5856,
"createdBy": "project:chromium",
"createTime": "2026-10-14T09:31:10.218218824Z",
"startTime": "2026-10-14T09:32:50.963366667Z",
"endTime": "2026-10-14T12:51:49.948990866Z",
"updateTime": "2026-10-14T12:51:49.948990866Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ff9d0b8e17d3da1fb81df0e7c0afced05021be80",
"ref": "refs/heads/main"
}
}
}
[12:51:50.888] Invocation finished in 3h20m42.321251844s with status FAILED