[18:23:03.571] New invocation is queued and will start shortly
[18:23:04.677] Starting the invocation (attempt 1)
[18:23:04.702] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/30209cd277d10a9dec6e4554026bdaf7a934fdc2"
[18:23:04.702] Popped gitiles commit info from properties and tags
[18:23:04.702] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[18:23:04.702] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[18:23:04.702] Buildbucket request:
{
"requestId": "8832104937440147792",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "30209cd277d10a9dec6e4554026bdaf7a934fdc2",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8832104937440147792"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[18:23:05.243] Scheduled build:
{
"id": "8666674875506339793",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 6031,
"createdBy": "project:chromium",
"createTime": "2026-10-28T18:23:04.794101614Z",
"updateTime": "2026-10-28T18:23:04.794101614Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "30209cd277d10a9dec6e4554026bdaf7a934fdc2",
"ref": "refs/heads/main"
}
}
}
[18:23:05.243] Task URL: https://cr-buildbucket.appspot.com/build/8666674875506339793
[18:23:05.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:2:0) after 1m7s
[18:23:06.225] Received PubSub notification, asking Buildbucket for the build status
[18:23:06.261] Build status: SCHEDULED
[18:23:45.302] Received PubSub notification, asking Buildbucket for the build status
[18:23:45.365] Build status: STARTED
[18:24:12.263] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:2:0)
[18:24:12.371] Build status: STARTED
[18:24:12.371] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:5:0) after 1m22s
[18:25:34.401] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:5:0)
[18:25:34.435] Build status: STARTED
[18:25:34.435] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:6:0) after 1m25s
[18:26:59.433] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:6:0)
[18:26:59.456] Build status: STARTED
[18:26:59.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:7:0) after 8m47s
[18:35:46.508] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:7:0)
[18:35:46.534] Build status: STARTED
[18:35:46.534] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:8:0) after 8m50s
[18:44:36.948] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:8:0)
[18:44:36.982] Build status: STARTED
[18:44:36.982] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:9:0) after 8m29s
[18:53:06.074] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:9:0)
[18:53:06.143] Build status: STARTED
[18:53:06.143] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:10:0) after 6m55s
[19:00:01.563] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:10:0)
[19:00:01.584] Build status: STARTED
[19:00:01.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:11:0) after 6m38s
[19:06:39.662] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:11:0)
[19:06:39.688] Build status: STARTED
[19:06:39.688] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:12:0) after 7m7s
[19:13:46.713] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:12:0)
[19:13:46.731] Build status: STARTED
[19:13:46.731] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:13:0) after 7m45s
[19:21:31.786] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:13:0)
[19:21:31.805] Build status: STARTED
[19:21:31.805] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:14:0) after 1m12s
[19:22:43.878] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:14:0)
[19:22:43.951] Build status: STARTED
[19:22:43.951] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:15:0) after 7m38s
[19:30:22.066] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:15:0)
[19:30:22.089] Build status: STARTED
[19:30:22.089] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:16:0) after 2m53s
[19:33:15.143] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:16:0)
[19:33:15.196] Build status: STARTED
[19:33:15.196] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:17:0) after 4m59s
[19:38:14.309] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:17:0)
[19:38:14.326] Build status: STARTED
[19:38:14.326] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:18:0) after 6m9s
[19:44:23.350] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:18:0)
[19:44:23.376] Build status: STARTED
[19:44:23.376] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:19:0) after 5m32s
[19:49:55.396] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:19:0)
[19:49:55.415] Build status: STARTED
[19:49:55.415] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:20:0) after 7m48s
[19:57:43.483] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:20:0)
[19:57:43.507] Build status: STARTED
[19:57:43.508] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:21:0) after 9m37s
[20:07:20.525] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:21:0)
[20:07:20.598] Build status: STARTED
[20:07:20.598] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:22:0) after 7m52s
[20:15:12.664] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:22:0)
[20:15:12.689] Build status: STARTED
[20:15:12.689] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:23:0) after 6m18s
[20:21:30.741] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:23:0)
[20:21:30.802] Build status: STARTED
[20:21:30.802] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:24:0) after 1m22s
[20:22:52.857] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:24:0)
[20:22:52.889] Build status: STARTED
[20:22:52.890] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:25:0) after 9m49s
[20:32:42.080] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:25:0)
[20:32:42.100] Build status: STARTED
[20:32:42.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:26:0) after 4m59s
[20:37:41.140] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:26:0)
[20:37:41.163] Build status: STARTED
[20:37:41.163] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:27:0) after 6m47s
[20:44:28.225] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:27:0)
[20:44:28.244] Build status: STARTED
[20:44:28.244] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:28:0) after 4m34s
[20:49:02.236] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:28:0)
[20:49:02.251] Build status: STARTED
[20:49:02.251] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:29:0) after 6m4s
[20:55:06.279] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:29:0)
[20:55:06.302] Build status: STARTED
[20:55:06.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:30:0) after 6m33s
[21:01:39.368] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:30:0)
[21:01:39.388] Build status: STARTED
[21:01:39.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:31:0) after 4m31s
[21:06:10.488] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:31:0)
[21:06:10.517] Build status: STARTED
[21:06:10.517] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:32:0) after 1m57s
[21:08:07.578] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:32:0)
[21:08:07.603] Build status: STARTED
[21:08:07.603] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:33:0) after 1m42s
[21:09:49.747] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:33:0)
[21:09:49.772] Build status: STARTED
[21:09:49.772] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:34:0) after 3m44s
[21:13:33.901] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:34:0)
[21:13:33.918] Build status: STARTED
[21:13:33.918] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:35:0) after 1m46s
[21:15:19.991] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:35:0)
[21:15:20.012] Build status: STARTED
[21:15:20.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:36:0) after 3m37s
[21:18:57.262] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:36:0)
[21:18:57.287] Build status: STARTED
[21:18:57.287] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:37:0) after 5m42s
[21:24:39.308] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:37:0)
[21:24:39.328] Build status: STARTED
[21:24:39.328] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:38:0) after 2m6s
[21:26:45.392] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:38:0)
[21:26:45.416] Build status: STARTED
[21:26:45.416] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:39:0) after 9m51s
[21:36:36.463] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:39:0)
[21:36:36.486] Build status: STARTED
[21:36:36.486] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8832104937440147792:40:0) after 8m53s
[21:39:53.280] Received PubSub notification, asking Buildbucket for the build status
[21:39:53.296] Build:
{
"id": "8666674875506339793",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 6031,
"createdBy": "project:chromium",
"createTime": "2026-10-28T18:23:04.794101614Z",
"startTime": "2026-10-28T18:23:45.070650202Z",
"endTime": "2026-10-28T21:39:52.981974838Z",
"updateTime": "2026-10-28T21:39:52.981974838Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "30209cd277d10a9dec6e4554026bdaf7a934fdc2",
"ref": "refs/heads/main"
}
}
}
[21:39:53.296] Invocation finished in 3h16m49.730111585s with status FAILED