[22:08:54.389] New invocation is queued and will start shortly
[22:08:55.708] Starting the invocation (attempt 1)
[22:08:55.736] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/81108c38fb0f7baeae60d103c11c431af20cb27a"
[22:08:55.736] Popped gitiles commit info from properties and tags
[22:08:55.736] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[22:08:55.736] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[22:08:55.736] Buildbucket request:
{
"requestId": "8842962364349876336",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "81108c38fb0f7baeae60d103c11c431af20cb27a",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8842962364349876336"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[22:08:56.462] Scheduled build:
{
"id": "8677532302213709889",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3590,
"createdBy": "project:chromium",
"createTime": "2026-06-30T22:08:55.801858602Z",
"updateTime": "2026-06-30T22:08:55.801858602Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "81108c38fb0f7baeae60d103c11c431af20cb27a",
"ref": "refs/heads/main"
}
}
}
[22:08:56.462] Task URL: https://cr-buildbucket.appspot.com/build/8677532302213709889
[22:08:56.462] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:2:0) after 3m37s
[22:08:57.380] Received PubSub notification, asking Buildbucket for the build status
[22:08:57.404] Build status: SCHEDULED
[22:09:37.923] Received PubSub notification, asking Buildbucket for the build status
[22:09:37.943] Build status: STARTED
[22:12:33.657] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:2:0)
[22:12:33.683] Build status: STARTED
[22:12:33.683] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:5:0) after 9m29s
[22:22:02.729] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:5:0)
[22:22:02.750] Build status: STARTED
[22:22:02.750] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:6:0) after 3m49s
[22:25:51.769] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:6:0)
[22:25:51.793] Build status: STARTED
[22:25:51.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:7:0) after 3m59s
[22:29:50.823] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:7:0)
[22:29:50.901] Build status: STARTED
[22:29:50.901] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:8:0) after 7m24s
[22:37:14.923] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:8:0)
[22:37:14.939] Build status: STARTED
[22:37:14.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:9:0) after 5m25s
[22:42:39.939] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:9:0)
[22:42:39.966] Build status: STARTED
[22:42:39.966] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:10:0) after 9m5s
[22:51:44.987] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:10:0)
[22:51:45.007] Build status: STARTED
[22:51:45.008] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:11:0) after 3m1s
[22:54:46.034] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:11:0)
[22:54:46.058] Build status: STARTED
[22:54:46.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:12:0) after 4m6s
[22:58:52.171] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:12:0)
[22:58:52.226] Build status: STARTED
[22:58:52.226] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:13:0) after 3m5s
[23:01:57.258] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:13:0)
[23:01:57.383] Build status: STARTED
[23:01:57.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:14:0) after 1m17s
[23:03:14.443] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:14:0)
[23:03:14.459] Build status: STARTED
[23:03:14.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:15:0) after 4m7s
[23:07:21.755] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:15:0)
[23:07:21.811] Build status: STARTED
[23:07:21.811] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:16:0) after 5m43s
[23:13:04.937] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:16:0)
[23:13:04.960] Build status: STARTED
[23:13:04.960] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:17:0) after 9m22s
[23:22:27.017] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:17:0)
[23:22:27.103] Build status: STARTED
[23:22:27.104] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:18:0) after 8m53s
[23:31:20.123] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:18:0)
[23:31:20.151] Build status: STARTED
[23:31:20.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:19:0) after 2m12s
[23:33:32.201] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:19:0)
[23:33:32.224] Build status: STARTED
[23:33:32.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:20:0) after 8m2s
[23:41:34.222] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:20:0)
[23:41:34.239] Build status: STARTED
[23:41:34.239] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:21:0) after 1m51s
[23:43:25.257] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:21:0)
[23:43:25.278] Build status: STARTED
[23:43:25.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:22:0) after 1m35s
[23:45:00.381] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:22:0)
[23:45:00.449] Build status: STARTED
[23:45:00.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:23:0) after 1m2s
[23:46:02.471] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:23:0)
[23:46:02.488] Build status: STARTED
[23:46:02.488] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:24:0) after 1m21s
[23:47:23.576] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:24:0)
[23:47:23.597] Build status: STARTED
[23:47:23.597] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:25:0) after 3m42s
[23:51:05.674] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:25:0)
[23:51:05.691] Build status: STARTED
[23:51:05.692] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:26:0) after 3m4s
[23:54:09.714] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:26:0)
[23:54:09.740] Build status: STARTED
[23:54:09.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:27:0) after 6m54s
[00:01:03.889] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:27:0)
[00:01:03.920] Build status: STARTED
[00:01:03.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:28:0) after 6m21s
[00:07:24.941] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:28:0)
[00:07:24.959] Build status: STARTED
[00:07:24.959] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:29:0) after 9m21s
[00:16:46.016] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:29:0)
[00:16:46.205] Build status: STARTED
[00:16:46.205] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:30:0) after 3m5s
[00:19:51.195] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:30:0)
[00:19:51.212] Build status: STARTED
[00:19:51.212] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:31:0) after 7m31s
[00:27:22.324] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:31:0)
[00:27:22.349] Build status: STARTED
[00:27:22.349] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:32:0) after 6m26s
[00:33:48.417] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:32:0)
[00:33:48.461] Build status: STARTED
[00:33:48.461] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:33:0) after 2m45s
[00:36:33.687] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:33:0)
[00:36:33.704] Build status: STARTED
[00:36:33.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:34:0) after 2m35s
[00:39:08.724] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:34:0)
[00:39:08.743] Build status: STARTED
[00:39:08.743] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:35:0) after 3m40s
[00:42:48.771] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:35:0)
[00:42:48.883] Build status: STARTED
[00:42:48.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:36:0) after 3m34s
[00:46:22.949] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:36:0)
[00:46:22.973] Build status: STARTED
[00:46:22.973] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:37:0) after 1m30s
[00:47:53.008] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:37:0)
[00:47:53.025] Build status: STARTED
[00:47:53.025] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:38:0) after 3m13s
[00:51:06.123] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:38:0)
[00:51:06.142] Build status: STARTED
[00:51:06.142] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:39:0) after 1m43s
[00:52:49.290] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:39:0)
[00:52:49.309] Build status: STARTED
[00:52:49.310] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:40:0) after 9m26s
[01:02:15.349] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:40:0)
[01:02:15.373] Build status: STARTED
[01:02:15.373] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:41:0) after 6m52s
[01:09:07.441] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:41:0)
[01:09:07.508] Build status: STARTED
[01:09:07.508] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:42:0) after 1m20s
[01:10:27.525] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:42:0)
[01:10:27.552] Build status: STARTED
[01:10:27.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8842962364349876336:43:0) after 3m42s
[01:14:05.625] Received PubSub notification, asking Buildbucket for the build status
[01:14:05.643] Build:
{
"id": "8677532302213709889",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3590,
"createdBy": "project:chromium",
"createTime": "2026-06-30T22:08:55.801858602Z",
"startTime": "2026-06-30T22:09:37.639873941Z",
"endTime": "2026-07-01T01:14:05.049187987Z",
"updateTime": "2026-07-01T01:14:05.049187987Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "81108c38fb0f7baeae60d103c11c431af20cb27a",
"ref": "refs/heads/main"
}
}
}
[01:14:05.643] Invocation finished in 3h5m11.264711717s with status FAILED