[18:00:18.985] New invocation is queued and will start shortly
[18:00:22.561] Starting the invocation (attempt 1)
[18:00:22.589] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/67bd0dd250a1781c6e6ad77636a8709ded9a621c"
[18:00:22.589] Popped gitiles commit info from properties and tags
[18:00:22.589] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[18:00:22.589] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[18:00:22.589] Buildbucket request:
{
"requestId": "8843068601249442912",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "67bd0dd250a1781c6e6ad77636a8709ded9a621c",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8843068601249442912"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[18:00:23.941] Scheduled build:
{
"id": "8677638536552469985",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3582,
"createdBy": "project:chromium",
"createTime": "2026-06-29T18:00:22.840081136Z",
"updateTime": "2026-06-29T18:00:22.840081136Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "67bd0dd250a1781c6e6ad77636a8709ded9a621c",
"ref": "refs/heads/main"
}
}
}
[18:00:23.941] Task URL: https://cr-buildbucket.appspot.com/build/8677638536552469985
[18:00:23.941] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:2:0) after 7m46s
[18:00:24.865] Received PubSub notification, asking Buildbucket for the build status
[18:00:24.913] Build status: SCHEDULED
[18:00:36.032] Received PubSub notification, asking Buildbucket for the build status
[18:00:36.049] Build status: STARTED
[18:08:10.017] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:2:0)
[18:08:10.038] Build status: STARTED
[18:08:10.038] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:5:0) after 4m40s
[18:12:50.056] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:5:0)
[18:12:50.077] Build status: STARTED
[18:12:50.078] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:6:0) after 6m46s
[18:19:36.098] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:6:0)
[18:19:36.121] Build status: STARTED
[18:19:36.121] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:7:0) after 6m22s
[18:25:58.097] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:7:0)
[18:25:58.116] Build status: STARTED
[18:25:58.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:8:0) after 4m45s
[18:30:43.134] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:8:0)
[18:30:43.171] Build status: STARTED
[18:30:43.171] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:9:0) after 5m57s
[18:36:40.197] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:9:0)
[18:36:40.224] Build status: STARTED
[18:36:40.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:10:0) after 5m5s
[18:41:45.342] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:10:0)
[18:41:45.361] Build status: STARTED
[18:41:45.361] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:11:0) after 5m4s
[18:46:49.412] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:11:0)
[18:46:49.444] Build status: STARTED
[18:46:49.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:12:0) after 4m50s
[18:51:39.428] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:12:0)
[18:51:39.525] Build status: STARTED
[18:51:39.525] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:13:0) after 1m33s
[18:53:12.545] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:13:0)
[18:53:12.559] Build status: STARTED
[18:53:12.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:14:0) after 6m24s
[18:59:36.723] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:14:0)
[18:59:36.747] Build status: STARTED
[18:59:36.747] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:15:0) after 4m45s
[19:04:21.855] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:15:0)
[19:04:21.880] Build status: STARTED
[19:04:21.880] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:16:0) after 8m49s
[19:13:10.949] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:16:0)
[19:13:10.969] Build status: STARTED
[19:13:10.969] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:17:0) after 5m1s
[19:18:11.988] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:17:0)
[19:18:12.009] Build status: STARTED
[19:18:12.009] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:18:0) after 3m49s
[19:22:01.193] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:18:0)
[19:22:01.213] Build status: STARTED
[19:22:01.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:19:0) after 6m38s
[19:28:39.325] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:19:0)
[19:28:39.346] Build status: STARTED
[19:28:39.346] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:20:0) after 4m3s
[19:32:42.491] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:20:0)
[19:32:42.552] Build status: STARTED
[19:32:42.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:21:0) after 3m46s
[19:36:28.585] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:21:0)
[19:36:28.599] Build status: STARTED
[19:36:28.599] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:22:0) after 7m54s
[19:44:22.683] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:22:0)
[19:44:22.730] Build status: STARTED
[19:44:22.730] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:23:0) after 5m40s
[19:50:02.804] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:23:0)
[19:50:02.821] Build status: STARTED
[19:50:02.821] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:24:0) after 1m39s
[19:51:41.849] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:24:0)
[19:51:41.868] Build status: STARTED
[19:51:41.868] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:25:0) after 5m49s
[19:57:30.885] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:25:0)
[19:57:30.900] Build status: STARTED
[19:57:30.900] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:26:0) after 1m18s
[19:58:49.037] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:26:0)
[19:58:49.055] Build status: STARTED
[19:58:49.055] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:27:0) after 7m13s
[20:06:02.143] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:27:0)
[20:06:02.170] Build status: STARTED
[20:06:02.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:28:0) after 3m53s
[20:09:55.287] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:28:0)
[20:09:55.308] Build status: STARTED
[20:09:55.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:29:0) after 2m18s
[20:12:13.330] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:29:0)
[20:12:13.350] Build status: STARTED
[20:12:13.350] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:30:0) after 3m29s
[20:15:42.371] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:30:0)
[20:15:42.404] Build status: STARTED
[20:15:42.404] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:31:0) after 7m49s
[20:23:31.423] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:31:0)
[20:23:31.522] Build status: STARTED
[20:23:31.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:32:0) after 8m17s
[20:31:48.593] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:32:0)
[20:31:48.611] Build status: STARTED
[20:31:48.611] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:33:0) after 1m0s
[20:32:48.681] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:33:0)
[20:32:48.742] Build status: STARTED
[20:32:48.742] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:34:0) after 8m56s
[20:41:44.761] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:34:0)
[20:41:44.782] Build status: STARTED
[20:41:44.782] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:35:0) after 8m56s
[20:50:40.886] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:35:0)
[20:50:40.905] Build status: STARTED
[20:50:40.905] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:36:0) after 3m22s
[20:54:02.976] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:36:0)
[20:54:02.995] Build status: STARTED
[20:54:02.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:37:0) after 1m6s
[20:55:09.063] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:37:0)
[20:55:09.081] Build status: STARTED
[20:55:09.081] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:38:0) after 1m18s
[20:56:27.268] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:38:0)
[20:56:27.285] Build status: STARTED
[20:56:27.285] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:39:0) after 7m24s
[21:03:51.306] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:39:0)
[21:03:51.332] Build status: STARTED
[21:03:51.332] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843068601249442912:40:0) after 6m27s
[21:04:42.050] Received PubSub notification, asking Buildbucket for the build status
[21:04:42.069] Build:
{
"id": "8677638536552469985",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3582,
"createdBy": "project:chromium",
"createTime": "2026-06-29T18:00:22.840081136Z",
"startTime": "2026-06-29T18:00:34.063401513Z",
"endTime": "2026-06-29T21:04:41.769756011Z",
"updateTime": "2026-06-29T21:04:41.769756011Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "67bd0dd250a1781c6e6ad77636a8709ded9a621c",
"ref": "refs/heads/main"
}
}
}
[21:04:42.069] Invocation finished in 3h4m23.091517794s with status FAILED