[20:36:56.828] New invocation is queued and will start shortly
[20:36:58.202] Starting the invocation (attempt 1)
[20:36:58.229] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/87d71150fea39e70372a0f5c3c5edffc5a271e91"
[20:36:58.229] Popped gitiles commit info from properties and tags
[20:36:58.230] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[20:36:58.230] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[20:36:58.230] Buildbucket request:
{
"requestId": "8845414268074297072",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "87d71150fea39e70372a0f5c3c5edffc5a271e91",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8845414268074297072"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[20:36:58.739] Scheduled build:
{
"id": "8679984205831319473",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3004,
"createdBy": "project:chromium",
"createTime": "2026-06-03T20:36:58.294550635Z",
"updateTime": "2026-06-03T20:36:58.294550635Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "87d71150fea39e70372a0f5c3c5edffc5a271e91",
"ref": "refs/heads/main"
}
}
}
[20:36:58.739] Task URL: https://cr-buildbucket.appspot.com/build/8679984205831319473
[20:36:58.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:2:0) after 8m49s
[20:36:59.699] Received PubSub notification, asking Buildbucket for the build status
[20:36:59.741] Build status: SCHEDULED
[20:37:05.029] Received PubSub notification, asking Buildbucket for the build status
[20:37:05.097] Build status: STARTED
[20:45:47.807] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:2:0)
[20:45:47.835] Build status: STARTED
[20:45:47.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:5:0) after 1m23s
[20:47:10.924] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:5:0)
[20:47:10.947] Build status: STARTED
[20:47:10.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:6:0) after 3m26s
[20:50:36.994] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:6:0)
[20:50:37.027] Build status: STARTED
[20:50:37.027] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:7:0) after 7m2s
[20:57:39.115] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:7:0)
[20:57:39.136] Build status: STARTED
[20:57:39.136] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:8:0) after 4m40s
[21:02:19.181] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:8:0)
[21:02:19.214] Build status: STARTED
[21:02:19.214] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:9:0) after 8m15s
[21:10:34.317] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:9:0)
[21:10:34.339] Build status: STARTED
[21:10:34.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:10:0) after 2m49s
[21:13:23.314] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:10:0)
[21:13:23.333] Build status: STARTED
[21:13:23.333] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:11:0) after 3m3s
[21:16:26.373] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:11:0)
[21:16:26.396] Build status: STARTED
[21:16:26.396] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:12:0) after 4m57s
[21:21:23.441] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:12:0)
[21:21:23.466] Build status: STARTED
[21:21:23.467] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:13:0) after 4m23s
[21:25:46.548] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:13:0)
[21:25:46.571] Build status: STARTED
[21:25:46.571] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:14:0) after 4m41s
[21:30:27.684] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:14:0)
[21:30:27.702] Build status: STARTED
[21:30:27.702] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:15:0) after 5m22s
[21:35:49.866] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:15:0)
[21:35:49.885] Build status: STARTED
[21:35:49.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:16:0) after 6m51s
[21:42:40.958] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:16:0)
[21:42:41.029] Build status: STARTED
[21:42:41.029] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:17:0) after 4m37s
[21:47:18.058] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:17:0)
[21:47:18.081] Build status: STARTED
[21:47:18.081] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:18:0) after 1m54s
[21:49:12.099] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:18:0)
[21:49:12.118] Build status: STARTED
[21:49:12.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:19:0) after 8m59s
[21:58:11.161] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:19:0)
[21:58:11.178] Build status: STARTED
[21:58:11.178] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:20:0) after 2m1s
[22:00:12.198] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:20:0)
[22:00:12.230] Build status: STARTED
[22:00:12.230] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:21:0) after 5m57s
[22:06:09.321] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:21:0)
[22:06:09.514] Build status: STARTED
[22:06:09.514] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:22:0) after 9m51s
[22:16:00.534] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:22:0)
[22:16:00.555] Build status: STARTED
[22:16:00.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:23:0) after 2m23s
[22:18:23.590] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:23:0)
[22:18:23.663] Build status: STARTED
[22:18:23.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:24:0) after 2m12s
[22:20:35.689] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:24:0)
[22:20:35.714] Build status: STARTED
[22:20:35.714] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:25:0) after 2m21s
[22:22:56.733] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:25:0)
[22:22:56.749] Build status: STARTED
[22:22:56.749] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:26:0) after 5m1s
[22:27:57.745] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:26:0)
[22:27:57.765] Build status: STARTED
[22:27:57.765] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:27:0) after 7m1s
[22:34:58.911] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:27:0)
[22:34:58.930] Build status: STARTED
[22:34:58.930] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:28:0) after 2m16s
[22:37:15.232] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:28:0)
[22:37:15.275] Build status: STARTED
[22:37:15.275] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:29:0) after 1m23s
[22:38:38.386] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:29:0)
[22:38:38.407] Build status: STARTED
[22:38:38.407] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:30:0) after 1m12s
[22:39:50.449] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:30:0)
[22:39:50.470] Build status: STARTED
[22:39:50.470] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:31:0) after 1m56s
[22:41:46.491] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:31:0)
[22:41:46.509] Build status: STARTED
[22:41:46.509] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:32:0) after 9m32s
[22:51:18.530] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:32:0)
[22:51:18.545] Build status: STARTED
[22:51:18.545] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:33:0) after 1m16s
[22:52:34.607] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:33:0)
[22:52:34.672] Build status: STARTED
[22:52:34.672] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:34:0) after 7m37s
[23:00:11.693] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:34:0)
[23:00:11.733] Build status: STARTED
[23:00:11.733] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:35:0) after 7m56s
[23:08:07.793] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:35:0)
[23:08:07.812] Build status: STARTED
[23:08:07.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:36:0) after 4m7s
[23:12:14.832] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:36:0)
[23:12:14.853] Build status: STARTED
[23:12:14.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:37:0) after 8m59s
[23:21:13.861] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:37:0)
[23:21:13.885] Build status: STARTED
[23:21:13.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:38:0) after 7m8s
[23:28:21.947] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:38:0)
[23:28:21.991] Build status: STARTED
[23:28:21.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:39:0) after 4m11s
[23:32:33.072] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:39:0)
[23:32:33.091] Build status: STARTED
[23:32:33.091] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:40:0) after 4m51s
[23:37:24.108] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:40:0)
[23:37:24.123] Build status: STARTED
[23:37:24.123] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:41:0) after 4m27s
[23:41:51.141] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:41:0)
[23:41:51.351] Build status: STARTED
[23:41:51.351] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845414268074297072:42:0) after 7m3s
[23:47:48.639] Received PubSub notification, asking Buildbucket for the build status
[23:47:48.655] Build:
{
"id": "8679984205831319473",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3004,
"createdBy": "project:chromium",
"createTime": "2026-06-03T20:36:58.294550635Z",
"startTime": "2026-06-03T20:37:04.831146737Z",
"endTime": "2026-06-03T23:47:48.357563053Z",
"updateTime": "2026-06-03T23:47:48.357563053Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "87d71150fea39e70372a0f5c3c5edffc5a271e91",
"ref": "refs/heads/main"
}
}
}
[23:47:48.655] Invocation finished in 3h10m51.882797311s with status FAILED