[04:37:31.051] New invocation is queued and will start shortly
[04:37:32.144] Starting the invocation (attempt 1)
[04:37:32.169] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/1277786c8db5614181d8f414607f818bfe7bbdd9"
[04:37:32.169] Popped gitiles commit info from properties and tags
[04:37:32.169] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:37:32.170] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:37:32.171] Buildbucket request:
{
"requestId": "8835508963808439328",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1277786c8db5614181d8f414607f818bfe7bbdd9",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8835508963808439328"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:37:32.929] Scheduled build:
{
"id": "8670078901828761121",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 4678,
"createdBy": "project:chromium",
"createTime": "2026-09-21T04:37:32.318625278Z",
"updateTime": "2026-09-21T04:37:32.318625278Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1277786c8db5614181d8f414607f818bfe7bbdd9",
"ref": "refs/heads/main"
}
}
}
[04:37:32.930] Task URL: https://cr-buildbucket.appspot.com/build/8670078901828761121
[04:37:32.930] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:2:0) after 1m3s
[04:37:33.763] Received PubSub notification, asking Buildbucket for the build status
[04:37:33.789] Build status: SCHEDULED
[04:37:36.177] Received PubSub notification, asking Buildbucket for the build status
[04:37:36.198] Build status: STARTED
[04:38:35.900] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:2:0)
[04:38:35.928] Build status: STARTED
[04:38:35.928] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:5:0) after 9m28s
[04:48:03.942] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:5:0)
[04:48:03.971] Build status: STARTED
[04:48:03.972] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:6:0) after 7m38s
[04:55:41.987] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:6:0)
[04:55:42.005] Build status: STARTED
[04:55:42.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:7:0) after 1m15s
[04:56:57.051] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:7:0)
[04:56:57.069] Build status: STARTED
[04:56:57.069] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:8:0) after 7m2s
[05:03:59.174] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:8:0)
[05:03:59.194] Build status: STARTED
[05:03:59.194] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:9:0) after 5m34s
[05:09:33.694] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:9:0)
[05:09:33.714] Build status: STARTED
[05:09:33.714] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:10:0) after 6m24s
[05:15:57.744] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:10:0)
[05:15:57.776] Build status: STARTED
[05:15:57.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:11:0) after 3m38s
[05:19:35.991] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:11:0)
[05:19:36.080] Build status: STARTED
[05:19:36.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:12:0) after 2m4s
[05:21:40.096] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:12:0)
[05:21:40.115] Build status: STARTED
[05:21:40.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:13:0) after 6m30s
[05:28:10.182] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:13:0)
[05:28:10.205] Build status: STARTED
[05:28:10.205] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:14:0) after 3m26s
[05:31:36.223] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:14:0)
[05:31:36.250] Build status: STARTED
[05:31:36.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:15:0) after 9m43s
[05:41:19.266] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:15:0)
[05:41:19.293] Build status: STARTED
[05:41:19.293] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:16:0) after 3m7s
[05:44:26.312] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:16:0)
[05:44:26.396] Build status: STARTED
[05:44:26.396] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:17:0) after 4m22s
[05:48:48.600] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:17:0)
[05:48:48.624] Build status: STARTED
[05:48:48.624] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:18:0) after 7m5s
[05:55:53.723] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:18:0)
[05:55:53.747] Build status: STARTED
[05:55:53.747] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:19:0) after 8m3s
[06:03:56.882] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:19:0)
[06:03:56.975] Build status: STARTED
[06:03:56.975] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:20:0) after 2m4s
[06:06:01.151] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:20:0)
[06:06:01.320] Build status: STARTED
[06:06:01.320] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:21:0) after 5m42s
[06:11:43.336] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:21:0)
[06:11:43.353] Build status: STARTED
[06:11:43.353] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:22:0) after 5m10s
[06:16:53.841] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:22:0)
[06:16:53.864] Build status: STARTED
[06:16:53.864] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:23:0) after 3m29s
[06:20:23.213] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:23:0)
[06:20:23.231] Build status: STARTED
[06:20:23.231] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:24:0) after 4m27s
[06:24:50.477] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:24:0)
[06:24:50.585] Build status: STARTED
[06:24:50.585] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:25:0) after 8m44s
[06:33:34.703] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:25:0)
[06:33:34.724] Build status: STARTED
[06:33:34.724] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:26:0) after 3m25s
[06:36:59.828] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:26:0)
[06:36:59.861] Build status: STARTED
[06:36:59.861] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:27:0) after 8m5s
[06:45:05.172] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:27:0)
[06:45:05.217] Build status: STARTED
[06:45:05.217] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:28:0) after 7m53s
[06:52:58.478] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:28:0)
[06:52:58.495] Build status: STARTED
[06:52:58.495] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:29:0) after 4m13s
[06:57:11.743] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:29:0)
[06:57:11.765] Build status: STARTED
[06:57:11.765] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:30:0) after 9m49s
[07:07:00.942] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:30:0)
[07:07:01.036] Build status: STARTED
[07:07:01.036] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:31:0) after 7m33s
[07:14:34.251] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:31:0)
[07:14:34.273] Build status: STARTED
[07:14:34.273] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:32:0) after 1m7s
[07:15:41.331] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:32:0)
[07:15:41.359] Build status: STARTED
[07:15:41.359] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:33:0) after 1m45s
[07:17:26.425] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:33:0)
[07:17:26.446] Build status: STARTED
[07:17:26.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:34:0) after 9m30s
[07:26:56.541] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:34:0)
[07:26:56.561] Build status: STARTED
[07:26:56.561] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:35:0) after 4m35s
[07:31:31.636] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:35:0)
[07:31:31.665] Build status: STARTED
[07:31:31.665] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:36:0) after 7m19s
[07:38:50.752] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:36:0)
[07:38:50.770] Build status: STARTED
[07:38:50.770] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:37:0) after 8m28s
[07:47:18.791] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:37:0)
[07:47:18.860] Build status: STARTED
[07:47:18.860] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:38:0) after 4m0s
[07:51:18.965] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:38:0)
[07:51:18.994] Build status: STARTED
[07:51:18.994] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:39:0) after 6m27s
[07:57:46.107] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:39:0)
[07:57:46.151] Build status: STARTED
[07:57:46.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:40:0) after 1m35s
[07:59:21.764] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:40:0)
[07:59:21.792] Build status: STARTED
[07:59:21.792] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:41:0) after 5m41s
[08:05:02.810] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:41:0)
[08:05:02.827] Build status: STARTED
[08:05:02.827] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:42:0) after 6m29s
[08:11:31.868] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:42:0)
[08:11:31.910] Build status: STARTED
[08:11:31.910] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:43:0) after 6m13s
[08:17:44.923] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:43:0)
[08:17:44.946] Build status: STARTED
[08:17:44.946] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8835508963808439328:44:0) after 8m5s
[08:25:26.633] Received PubSub notification, asking Buildbucket for the build status
[08:25:26.695] Build:
{
"id": "8670078901828761121",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 4678,
"createdBy": "project:chromium",
"createTime": "2026-09-21T04:37:32.318625278Z",
"startTime": "2026-09-21T04:37:35.986000481Z",
"endTime": "2026-09-21T08:25:26.359752367Z",
"updateTime": "2026-09-21T08:25:26.359752367Z",
"status": "INFRA_FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1277786c8db5614181d8f414607f818bfe7bbdd9",
"ref": "refs/heads/main"
}
}
}
[08:25:26.696] Invocation finished in 3h47m55.649856527s with status FAILED