[22:25:04.050] New invocation is queued and will start shortly
[22:25:05.103] Starting the invocation (attempt 1)
[22:25:05.126] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/a909ab25fe3c735f780118a60443c37dc421e53b"
[22:25:05.126] Popped gitiles commit info from properties and tags
[22:25:05.126] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[22:25:05.126] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[22:25:05.126] Buildbucket request:
{
"requestId": "8846766420175289600",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a909ab25fe3c735f780118a60443c37dc421e53b",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8846766420175289600"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[22:25:05.731] Scheduled build:
{
"id": "8681336358336884161",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 2792,
"createdBy": "project:chromium",
"createTime": "2026-05-19T22:25:05.178417228Z",
"updateTime": "2026-05-19T22:25:05.178417228Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a909ab25fe3c735f780118a60443c37dc421e53b",
"ref": "refs/heads/main"
}
},
"canary": true
}
[22:25:05.731] Task URL: https://cr-buildbucket.appspot.com/build/8681336358336884161
[22:25:05.731] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:2:0) after 8m48s
[22:25:06.271] Received PubSub notification, asking Buildbucket for the build status
[22:25:06.294] Build status: SCHEDULED
[22:25:11.995] Received PubSub notification, asking Buildbucket for the build status
[22:25:12.081] Build status: STARTED
[22:33:53.748] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:2:0)
[22:33:53.768] Build status: STARTED
[22:33:53.768] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:5:0) after 4m46s
[22:38:39.799] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:5:0)
[22:38:39.816] Build status: STARTED
[22:38:39.817] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:6:0) after 2m49s
[22:41:28.972] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:6:0)
[22:41:29.036] Build status: STARTED
[22:41:29.036] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:7:0) after 6m4s
[22:47:33.051] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:7:0)
[22:47:33.074] Build status: STARTED
[22:47:33.074] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:8:0) after 5m52s
[22:53:25.092] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:8:0)
[22:53:25.110] Build status: STARTED
[22:53:25.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:9:0) after 6m20s
[22:59:45.115] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:9:0)
[22:59:45.787] Build status: STARTED
[22:59:45.787] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:10:0) after 8m55s
[23:08:40.816] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:10:0)
[23:08:40.836] Build status: STARTED
[23:08:40.836] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:11:0) after 5m51s
[23:14:31.984] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:11:0)
[23:14:32.005] Build status: STARTED
[23:14:32.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:12:0) after 7m25s
[23:21:57.071] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:12:0)
[23:21:57.090] Build status: STARTED
[23:21:57.090] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:13:0) after 1m25s
[23:23:22.100] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:13:0)
[23:23:22.118] Build status: STARTED
[23:23:22.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:14:0) after 2m36s
[23:25:58.247] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:14:0)
[23:25:58.305] Build status: STARTED
[23:25:58.306] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:15:0) after 2m41s
[23:28:39.319] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:15:0)
[23:28:39.345] Build status: STARTED
[23:28:39.345] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:16:0) after 4m0s
[23:32:39.371] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:16:0)
[23:32:39.394] Build status: STARTED
[23:32:39.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:17:0) after 3m19s
[23:35:58.444] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:17:0)
[23:35:58.504] Build status: STARTED
[23:35:58.504] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:18:0) after 2m33s
[23:38:31.488] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:18:0)
[23:38:31.513] Build status: STARTED
[23:38:31.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:19:0) after 1m9s
[23:39:40.528] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:19:0)
[23:39:40.543] Build status: STARTED
[23:39:40.543] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:20:0) after 9m50s
[23:49:30.700] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:20:0)
[23:49:30.716] Build status: STARTED
[23:49:30.716] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:21:0) after 6m4s
[23:55:34.731] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:21:0)
[23:55:34.748] Build status: STARTED
[23:55:34.748] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:22:0) after 6m23s
[00:01:57.814] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:22:0)
[00:01:57.855] Build status: STARTED
[00:01:57.855] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:23:0) after 6m54s
[00:08:51.879] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:23:0)
[00:08:51.934] Build status: STARTED
[00:08:51.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:24:0) after 8m18s
[00:17:10.019] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:24:0)
[00:17:10.032] Build status: STARTED
[00:17:10.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:25:0) after 2m25s
[00:19:35.097] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:25:0)
[00:19:35.149] Build status: STARTED
[00:19:35.149] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:26:0) after 6m59s
[00:26:34.179] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:26:0)
[00:26:34.194] Build status: STARTED
[00:26:34.194] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:27:0) after 2m33s
[00:29:07.236] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:27:0)
[00:29:07.252] Build status: STARTED
[00:29:07.252] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:28:0) after 1m9s
[00:30:16.267] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:28:0)
[00:30:16.305] Build status: STARTED
[00:30:16.305] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:29:0) after 5m25s
[00:35:41.297] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:29:0)
[00:35:41.315] Build status: STARTED
[00:35:41.315] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:30:0) after 8m7s
[00:43:48.372] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:30:0)
[00:43:48.394] Build status: STARTED
[00:43:48.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:31:0) after 5m36s
[00:49:24.412] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:31:0)
[00:49:24.466] Build status: STARTED
[00:49:24.466] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:32:0) after 1m51s
[00:51:15.597] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:32:0)
[00:51:15.615] Build status: STARTED
[00:51:15.615] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:33:0) after 1m52s
[00:53:07.634] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:33:0)
[00:53:07.664] Build status: STARTED
[00:53:07.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8846766420175289600:34:0) after 8m54s
[00:58:32.091] Received PubSub notification, asking Buildbucket for the build status
[00:58:32.106] Build:
{
"id": "8681336358336884161",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 2792,
"createdBy": "project:chromium",
"createTime": "2026-05-19T22:25:05.178417228Z",
"startTime": "2026-05-19T22:25:11.706704683Z",
"endTime": "2026-05-20T00:58:31.662629633Z",
"updateTime": "2026-05-20T00:58:31.662629633Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a909ab25fe3c735f780118a60443c37dc421e53b",
"ref": "refs/heads/main"
}
},
"canary": true
}
[00:58:32.106] Invocation finished in 2h33m28.06519182s with status FAILED