[18:17:19.121] New invocation is queued and will start shortly
[18:17:20.252] Starting the invocation (attempt 1)
[18:17:20.284] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/5019f9bde3506d0e81781eeb5e7df5f01573ccee"
[18:17:20.284] Popped gitiles commit info from properties and tags
[18:17:20.284] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[18:17:20.284] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[18:17:20.284] Buildbucket request:
{
"requestId": "8841346189195238704",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5019f9bde3506d0e81781eeb5e7df5f01573ccee",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8841346189195238704"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[18:17:20.882] Scheduled build:
{
"id": "8675916127250437937",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3880,
"createdBy": "project:chromium",
"createTime": "2026-07-18T18:17:20.354411268Z",
"updateTime": "2026-07-18T18:17:20.354411268Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5019f9bde3506d0e81781eeb5e7df5f01573ccee",
"ref": "refs/heads/main"
}
}
}
[18:17:20.882] Task URL: https://cr-buildbucket.appspot.com/build/8675916127250437937
[18:17:20.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:2:0) after 8m54s
[18:17:21.763] Received PubSub notification, asking Buildbucket for the build status
[18:17:21.780] Build status: SCHEDULED
[18:17:25.723] Received PubSub notification, asking Buildbucket for the build status
[18:17:25.739] Build status: STARTED
[18:26:15.098] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:2:0)
[18:26:15.134] Build status: STARTED
[18:26:15.134] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:5:0) after 2m42s
[18:28:57.235] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:5:0)
[18:28:57.255] Build status: STARTED
[18:28:57.255] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:6:0) after 9m46s
[18:38:43.357] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:6:0)
[18:38:43.376] Build status: STARTED
[18:38:43.376] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:7:0) after 3m15s
[18:41:58.447] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:7:0)
[18:41:58.465] Build status: STARTED
[18:41:58.465] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:8:0) after 2m26s
[18:44:24.722] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:8:0)
[18:44:24.744] Build status: STARTED
[18:44:24.744] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:9:0) after 3m59s
[18:48:24.141] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:9:0)
[18:48:24.159] Build status: STARTED
[18:48:24.159] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:10:0) after 8m3s
[18:56:27.385] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:10:0)
[18:56:27.400] Build status: STARTED
[18:56:27.400] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:11:0) after 3m50s
[19:00:17.504] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:11:0)
[19:00:17.572] Build status: STARTED
[19:00:17.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:12:0) after 2m50s
[19:03:07.592] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:12:0)
[19:03:07.606] Build status: STARTED
[19:03:07.606] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:13:0) after 3m57s
[19:07:04.623] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:13:0)
[19:07:04.641] Build status: STARTED
[19:07:04.641] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:14:0) after 1m2s
[19:08:06.756] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:14:0)
[19:08:06.778] Build status: STARTED
[19:08:06.778] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:15:0) after 5m32s
[19:13:38.795] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:15:0)
[19:13:38.879] Build status: STARTED
[19:13:38.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:16:0) after 9m7s
[19:22:45.990] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:16:0)
[19:22:46.076] Build status: STARTED
[19:22:46.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:17:0) after 4m29s
[19:27:15.097] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:17:0)
[19:27:15.116] Build status: STARTED
[19:27:15.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:18:0) after 1m1s
[19:28:16.322] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:18:0)
[19:28:16.343] Build status: STARTED
[19:28:16.343] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:19:0) after 7m29s
[19:35:45.408] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:19:0)
[19:35:45.434] Build status: STARTED
[19:35:45.434] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:20:0) after 8m11s
[19:43:56.454] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:20:0)
[19:43:56.631] Build status: STARTED
[19:43:56.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:21:0) after 7m1s
[19:50:57.647] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:21:0)
[19:50:57.666] Build status: STARTED
[19:50:57.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:22:0) after 4m17s
[19:55:14.822] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:22:0)
[19:55:14.836] Build status: STARTED
[19:55:14.836] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:23:0) after 2m59s
[19:58:14.744] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:23:0)
[19:58:14.769] Build status: STARTED
[19:58:14.769] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:24:0) after 4m38s
[20:02:53.283] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:24:0)
[20:02:53.315] Build status: STARTED
[20:02:53.315] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:25:0) after 8m52s
[20:11:45.401] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:25:0)
[20:11:45.459] Build status: STARTED
[20:11:45.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:26:0) after 8m26s
[20:20:11.484] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:26:0)
[20:20:11.501] Build status: STARTED
[20:20:11.502] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:27:0) after 8m14s
[20:28:25.522] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:27:0)
[20:28:25.549] Build status: STARTED
[20:28:25.549] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:28:0) after 6m12s
[20:34:37.522] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:28:0)
[20:34:37.555] Build status: STARTED
[20:34:37.556] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:29:0) after 5m9s
[20:39:46.811] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:29:0)
[20:39:46.846] Build status: STARTED
[20:39:46.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:30:0) after 9m47s
[20:49:33.977] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:30:0)
[20:49:33.992] Build status: STARTED
[20:49:33.992] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:31:0) after 9m13s
[20:58:47.112] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:31:0)
[20:58:47.132] Build status: STARTED
[20:58:47.133] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:32:0) after 9m54s
[21:08:41.496] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:32:0)
[21:08:41.532] Build status: STARTED
[21:08:41.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:33:0) after 9m21s
[21:18:02.788] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:33:0)
[21:18:02.805] Build status: STARTED
[21:18:02.805] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:34:0) after 2m5s
[21:20:07.915] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:34:0)
[21:20:07.936] Build status: STARTED
[21:20:07.936] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841346189195238704:35:0) after 6m7s
[21:24:49.665] Received PubSub notification, asking Buildbucket for the build status
[21:24:49.684] Build:
{
"id": "8675916127250437937",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3880,
"createdBy": "project:chromium",
"createTime": "2026-07-18T18:17:20.354411268Z",
"startTime": "2026-07-18T18:17:24.334344783Z",
"endTime": "2026-07-18T21:24:46.645370523Z",
"updateTime": "2026-07-18T21:24:46.645370523Z",
"status": "INFRA_FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5019f9bde3506d0e81781eeb5e7df5f01573ccee",
"ref": "refs/heads/main"
}
}
}
[21:24:49.684] Invocation finished in 3h7m30.567574237s with status FAILED