[15:09:24.731] New invocation is queued and will start shortly
[15:09:25.794] Starting the invocation (attempt 1)
[15:09:25.829] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/c4970b5e0900bbd4a365ca6762fd07b703109037"
[15:09:25.829] Popped gitiles commit info from properties and tags
[15:09:25.829] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:09:25.830] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:09:25.830] Buildbucket request:
{
"requestId": "8841176817320020880",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c4970b5e0900bbd4a365ca6762fd07b703109037",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8841176817320020880"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:09:26.376] Scheduled build:
{
"id": "8675746755432302049",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3892,
"createdBy": "project:chromium",
"createTime": "2026-07-20T15:09:25.907654251Z",
"updateTime": "2026-07-20T15:09:25.907654251Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c4970b5e0900bbd4a365ca6762fd07b703109037",
"ref": "refs/heads/main"
}
}
}
[15:09:26.376] Task URL: https://cr-buildbucket.appspot.com/build/8675746755432302049
[15:09:26.376] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:2:0) after 6m42s
[15:09:27.311] Received PubSub notification, asking Buildbucket for the build status
[15:09:27.335] Build status: SCHEDULED
[15:09:32.476] Received PubSub notification, asking Buildbucket for the build status
[15:09:32.489] Build status: STARTED
[15:16:08.398] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:2:0)
[15:16:08.431] Build status: STARTED
[15:16:08.431] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:5:0) after 4m8s
[15:20:16.701] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:5:0)
[15:20:16.793] Build status: STARTED
[15:20:16.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:6:0) after 4m17s
[15:24:33.816] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:6:0)
[15:24:33.844] Build status: STARTED
[15:24:33.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:7:0) after 5m57s
[15:30:30.997] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:7:0)
[15:30:31.021] Build status: STARTED
[15:30:31.021] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:8:0) after 3m46s
[15:34:17.135] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:8:0)
[15:34:17.156] Build status: STARTED
[15:34:17.156] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:9:0) after 9m2s
[15:43:19.195] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:9:0)
[15:43:19.226] Build status: STARTED
[15:43:19.227] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:10:0) after 2m25s
[15:45:44.226] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:10:0)
[15:45:44.279] Build status: STARTED
[15:45:44.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:11:0) after 3m43s
[15:49:27.298] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:11:0)
[15:49:27.495] Build status: STARTED
[15:49:27.495] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:12:0) after 6m10s
[15:55:37.587] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:12:0)
[15:55:37.603] Build status: STARTED
[15:55:37.603] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:13:0) after 9m18s
[16:04:55.639] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:13:0)
[16:04:55.680] Build status: STARTED
[16:04:55.680] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:14:0) after 8m4s
[16:12:59.702] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:14:0)
[16:12:59.722] Build status: STARTED
[16:12:59.722] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:15:0) after 2m37s
[16:15:36.885] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:15:0)
[16:15:36.916] Build status: STARTED
[16:15:36.916] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:16:0) after 1m53s
[16:17:29.934] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:16:0)
[16:17:29.963] Build status: STARTED
[16:17:29.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:17:0) after 8m24s
[16:25:54.072] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:17:0)
[16:25:54.163] Build status: STARTED
[16:25:54.163] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:18:0) after 8m57s
[16:34:51.307] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:18:0)
[16:34:51.335] Build status: STARTED
[16:34:51.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:19:0) after 1m45s
[16:36:36.345] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:19:0)
[16:36:36.378] Build status: STARTED
[16:36:36.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:20:0) after 1m9s
[16:37:45.534] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:20:0)
[16:37:45.558] Build status: STARTED
[16:37:45.558] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:21:0) after 4m4s
[16:41:49.675] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:21:0)
[16:41:49.697] Build status: STARTED
[16:41:49.697] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:22:0) after 6m51s
[16:48:40.831] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:22:0)
[16:48:40.854] Build status: STARTED
[16:48:40.854] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:23:0) after 3m33s
[16:52:13.932] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:23:0)
[16:52:13.958] Build status: STARTED
[16:52:13.958] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:24:0) after 1m37s
[16:53:51.027] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:24:0)
[16:53:51.072] Build status: STARTED
[16:53:51.072] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:25:0) after 8m23s
[17:02:14.092] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:25:0)
[17:02:14.115] Build status: STARTED
[17:02:14.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:26:0) after 5m26s
[17:07:40.196] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:26:0)
[17:07:40.212] Build status: STARTED
[17:07:40.212] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:27:0) after 3m51s
[17:11:31.234] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:27:0)
[17:11:31.257] Build status: STARTED
[17:11:31.257] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:28:0) after 8m37s
[17:20:08.298] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:28:0)
[17:20:08.316] Build status: STARTED
[17:20:08.316] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:29:0) after 2m4s
[17:22:12.335] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:29:0)
[17:22:12.379] Build status: STARTED
[17:22:12.380] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:30:0) after 5m36s
[17:27:48.489] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:30:0)
[17:27:48.522] Build status: STARTED
[17:27:48.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:31:0) after 6m35s
[17:34:23.538] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:31:0)
[17:34:23.558] Build status: STARTED
[17:34:23.558] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:32:0) after 8m36s
[17:42:59.611] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:32:0)
[17:42:59.633] Build status: STARTED
[17:42:59.633] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:33:0) after 9m32s
[17:52:31.630] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:33:0)
[17:52:31.645] Build status: STARTED
[17:52:31.645] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:34:0) after 2m28s
[17:54:59.714] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:34:0)
[17:54:59.809] Build status: STARTED
[17:54:59.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:35:0) after 6m46s
[18:01:45.827] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:35:0)
[18:01:45.846] Build status: STARTED
[18:01:45.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:36:0) after 8m15s
[18:10:00.866] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:36:0)
[18:10:00.959] Build status: STARTED
[18:10:00.959] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:37:0) after 6m4s
[18:16:05.025] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:37:0)
[18:16:05.079] Build status: STARTED
[18:16:05.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8841176817320020880:38:0) after 9m25s
[18:19:31.588] Received PubSub notification, asking Buildbucket for the build status
[18:19:31.607] Build:
{
"id": "8675746755432302049",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3892,
"createdBy": "project:chromium",
"createTime": "2026-07-20T15:09:25.907654251Z",
"startTime": "2026-07-20T15:09:32.278568393Z",
"endTime": "2026-07-20T18:19:31.186423573Z",
"updateTime": "2026-07-20T18:19:31.186423573Z",
"status": "INFRA_FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c4970b5e0900bbd4a365ca6762fd07b703109037",
"ref": "refs/heads/main"
}
}
}
[18:19:31.607] Invocation finished in 3h10m6.884450984s with status FAILED