[20:28:00.981] New invocation is queued and will start shortly
[20:28:04.462] Starting the invocation (attempt 1)
[20:28:04.490] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/124906c276bf3693608658481eb2bb1bfd7f95a8"
[20:28:04.491] Popped gitiles commit info from properties and tags
[20:28:04.491] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[20:28:04.491] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[20:28:04.491] Buildbucket request:
{
"requestId": "8854383929577992144",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "124906c276bf3693608658481eb2bb1bfd7f95a8",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8854383929577992144"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[20:28:05.065] Scheduled build:
{
"id": "8688953865159269553",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1417,
"createdBy": "project:chromium",
"createTime": "2026-02-24T20:28:04.566712674Z",
"updateTime": "2026-02-24T20:28:04.566712674Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "124906c276bf3693608658481eb2bb1bfd7f95a8",
"ref": "refs/heads/main"
}
},
"canary": true
}
[20:28:05.065] Task URL: https://cr-buildbucket.appspot.com/build/8688953865159269553
[20:28:05.065] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:2:0) after 5m32s
[20:28:05.995] Received PubSub notification, asking Buildbucket for the build status
[20:28:06.016] Build status: SCHEDULED
[20:29:01.087] Received PubSub notification, asking Buildbucket for the build status
[20:29:01.121] Build status: STARTED
[20:33:37.134] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:2:0)
[20:33:37.159] Build status: STARTED
[20:33:37.160] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:5:0) after 7m58s
[20:41:35.187] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:5:0)
[20:41:35.201] Build status: STARTED
[20:41:35.201] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:6:0) after 8m55s
[20:50:30.278] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:6:0)
[20:50:30.297] Build status: STARTED
[20:50:30.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:7:0) after 4m54s
[20:55:24.539] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:7:0)
[20:55:24.560] Build status: STARTED
[20:55:24.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:8:0) after 9m3s
[21:04:27.600] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:8:0)
[21:04:27.617] Build status: STARTED
[21:04:27.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:9:0) after 5m15s
[21:09:42.631] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:9:0)
[21:09:42.664] Build status: STARTED
[21:09:42.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:10:0) after 5m10s
[21:14:52.640] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:10:0)
[21:14:52.663] Build status: STARTED
[21:14:52.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:11:0) after 4m33s
[21:19:25.698] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:11:0)
[21:19:25.721] Build status: STARTED
[21:19:25.721] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:12:0) after 6m7s
[21:25:32.711] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:12:0)
[21:25:32.725] Build status: STARTED
[21:25:32.725] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:13:0) after 1m1s
[21:26:33.829] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:13:0)
[21:26:33.842] Build status: STARTED
[21:26:33.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:14:0) after 8m33s
[21:35:06.838] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:14:0)
[21:35:06.925] Build status: STARTED
[21:35:06.925] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:15:0) after 1m53s
[21:36:59.968] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:15:0)
[21:37:00.012] Build status: STARTED
[21:37:00.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:16:0) after 1m40s
[21:38:40.068] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:16:0)
[21:38:40.127] Build status: STARTED
[21:38:40.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:17:0) after 1m7s
[21:39:47.147] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:17:0)
[21:39:47.166] Build status: STARTED
[21:39:47.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:18:0) after 9m35s
[21:49:22.233] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:18:0)
[21:49:22.250] Build status: STARTED
[21:49:22.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:19:0) after 6m38s
[21:56:00.263] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:19:0)
[21:56:00.287] Build status: STARTED
[21:56:00.287] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:20:0) after 1m37s
[21:57:37.377] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:20:0)
[21:57:37.406] Build status: STARTED
[21:57:37.406] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:21:0) after 8m9s
[22:05:46.424] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:21:0)
[22:05:46.450] Build status: STARTED
[22:05:46.450] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:22:0) after 1m11s
[22:06:57.468] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:22:0)
[22:06:57.484] Build status: STARTED
[22:06:57.484] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:23:0) after 5m3s
[22:12:00.571] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:23:0)
[22:12:00.584] Build status: STARTED
[22:12:00.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:24:0) after 7m37s
[22:19:37.653] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:24:0)
[22:19:37.667] Build status: STARTED
[22:19:37.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:25:0) after 7m54s
[22:27:31.831] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:25:0)
[22:27:31.847] Build status: STARTED
[22:27:31.847] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:26:0) after 6m38s
[22:34:09.872] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:26:0)
[22:34:09.895] Build status: STARTED
[22:34:09.895] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:27:0) after 5m10s
[22:39:19.912] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:27:0)
[22:39:19.925] Build status: STARTED
[22:39:19.925] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:28:0) after 8m2s
[22:47:21.961] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:28:0)
[22:47:21.988] Build status: STARTED
[22:47:21.988] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:29:0) after 6m44s
[22:54:06.019] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:29:0)
[22:54:06.040] Build status: STARTED
[22:54:06.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:30:0) after 9m54s
[23:04:00.062] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:30:0)
[23:04:00.078] Build status: STARTED
[23:04:00.078] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:31:0) after 7m44s
[23:11:44.094] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:31:0)
[23:11:44.107] Build status: STARTED
[23:11:44.107] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:32:0) after 3m54s
[23:15:38.230] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:32:0)
[23:15:38.253] Build status: STARTED
[23:15:38.253] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:33:0) after 7m25s
[23:23:03.262] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:33:0)
[23:23:03.280] Build status: STARTED
[23:23:03.280] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:34:0) after 4m11s
[23:27:14.447] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:34:0)
[23:27:14.564] Build status: STARTED
[23:27:14.564] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:35:0) after 9m28s
[23:36:42.614] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:35:0)
[23:36:42.630] Build status: STARTED
[23:36:42.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:36:0) after 3m2s
[23:39:44.807] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:36:0)
[23:39:44.823] Build status: STARTED
[23:39:44.823] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:37:0) after 9m29s
[23:49:13.892] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:37:0)
[23:49:13.905] Build status: STARTED
[23:49:13.905] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:38:0) after 2m5s
[23:51:19.053] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:38:0)
[23:51:19.067] Build status: STARTED
[23:51:19.067] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:39:0) after 5m21s
[23:56:40.112] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:39:0)
[23:56:40.135] Build status: STARTED
[23:56:40.135] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:40:0) after 7m36s
[00:04:16.229] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:40:0)
[00:04:16.249] Build status: STARTED
[00:04:16.249] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:41:0) after 8m51s
[00:13:07.303] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:41:0)
[00:13:07.323] Build status: STARTED
[00:13:07.323] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8854383929577992144:42:0) after 2m14s
[00:15:12.395] Received PubSub notification, asking Buildbucket for the build status
[00:15:12.411] Build:
{
"id": "8688953865159269553",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1417,
"createdBy": "project:chromium",
"createTime": "2026-02-24T20:28:04.566712674Z",
"startTime": "2026-02-24T20:29:00.888145375Z",
"endTime": "2026-02-25T00:15:12.097849647Z",
"updateTime": "2026-02-25T00:15:12.097849647Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "124906c276bf3693608658481eb2bb1bfd7f95a8",
"ref": "refs/heads/main"
}
},
"canary": true
}
[00:15:12.411] Invocation finished in 3h47m11.442059194s with status FAILED