[15:57:12.198] New invocation is queued and will start shortly
[15:57:13.292] Starting the invocation (attempt 1)
[15:57:13.323] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/37da889422e1fe1228648a7176a22a5879d08d09"
[15:57:13.323] Popped gitiles commit info from properties and tags
[15:57:13.323] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:57:13.323] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:57:13.323] Buildbucket request:
{
"requestId": "8833835456281446080",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "37da889422e1fe1228648a7176a22a5879d08d09",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8833835456281446080"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:57:13.971] Scheduled build:
{
"id": "8668405394382968001",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 23255,
"createdBy": "project:chromium",
"createTime": "2026-10-09T15:57:13.387749446Z",
"updateTime": "2026-10-09T15:57:13.387749446Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "37da889422e1fe1228648a7176a22a5879d08d09",
"ref": "refs/heads/main"
}
}
}
[15:57:13.971] Task URL: https://cr-buildbucket.appspot.com/build/8668405394382968001
[15:57:13.971] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:2:0) after 6m53s
[15:57:15.004] Received PubSub notification, asking Buildbucket for the build status
[15:57:15.029] Build status: SCHEDULED
[15:58:08.993] Received PubSub notification, asking Buildbucket for the build status
[15:58:09.029] Build status: STARTED
[16:04:07.059] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:2:0)
[16:04:07.078] Build status: STARTED
[16:04:07.078] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:5:0) after 9m18s
[16:13:25.152] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:5:0)
[16:13:25.212] Build status: STARTED
[16:13:25.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:6:0) after 3m8s
[16:16:33.364] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:6:0)
[16:16:33.395] Build status: STARTED
[16:16:33.395] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:7:0) after 6m13s
[16:22:46.568] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:7:0)
[16:22:46.589] Build status: STARTED
[16:22:46.590] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:8:0) after 1m17s
[16:24:03.809] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:8:0)
[16:24:03.826] Build status: STARTED
[16:24:03.826] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:9:0) after 4m36s
[16:28:39.847] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:9:0)
[16:28:39.869] Build status: STARTED
[16:28:39.870] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:10:0) after 5m55s
[16:34:34.925] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:10:0)
[16:34:34.945] Build status: STARTED
[16:34:34.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:11:0) after 5m57s
[16:40:31.995] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:11:0)
[16:40:32.015] Build status: STARTED
[16:40:32.015] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:12:0) after 3m44s
[16:44:16.071] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:12:0)
[16:44:16.087] Build status: STARTED
[16:44:16.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:13:0) after 7m57s
[16:52:13.105] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:13:0)
[16:52:13.137] Build status: STARTED
[16:52:13.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:14:0) after 3m52s
[16:56:05.191] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:14:0)
[16:56:05.210] Build status: STARTED
[16:56:05.210] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:15:0) after 6m7s
[17:02:12.195] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:15:0)
[17:02:12.225] Build status: STARTED
[17:02:12.225] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:16:0) after 7m21s
[17:09:33.284] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:16:0)
[17:09:33.304] Build status: STARTED
[17:09:33.305] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:17:0) after 7m12s
[17:16:45.370] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:17:0)
[17:16:45.389] Build status: STARTED
[17:16:45.389] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:18:0) after 1m18s
[17:18:03.405] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:18:0)
[17:18:03.424] Build status: STARTED
[17:18:03.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:19:0) after 8m41s
[17:26:44.534] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:19:0)
[17:26:44.549] Build status: STARTED
[17:26:44.549] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:20:0) after 8m13s
[17:34:57.570] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:20:0)
[17:34:57.596] Build status: STARTED
[17:34:57.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:21:0) after 6m38s
[17:41:35.655] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:21:0)
[17:41:35.713] Build status: STARTED
[17:41:35.713] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:22:0) after 9m46s
[17:51:21.734] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:22:0)
[17:51:21.757] Build status: STARTED
[17:51:21.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:23:0) after 5m18s
[17:56:39.773] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:23:0)
[17:56:39.844] Build status: STARTED
[17:56:39.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:24:0) after 8m47s
[18:05:26.950] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:24:0)
[18:05:27.026] Build status: STARTED
[18:05:27.026] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:25:0) after 5m7s
[18:10:34.019] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:25:0)
[18:10:34.040] Build status: STARTED
[18:10:34.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:26:0) after 1m28s
[18:12:02.063] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:26:0)
[18:12:02.087] Build status: STARTED
[18:12:02.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:27:0) after 7m10s
[18:19:12.121] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:27:0)
[18:19:12.148] Build status: STARTED
[18:19:12.148] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:28:0) after 2m34s
[18:21:46.170] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:28:0)
[18:21:46.186] Build status: STARTED
[18:21:46.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:29:0) after 5m51s
[18:27:37.268] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:29:0)
[18:27:37.288] Build status: STARTED
[18:27:37.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:30:0) after 9m36s
[18:37:13.357] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:30:0)
[18:37:13.374] Build status: STARTED
[18:37:13.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:31:0) after 6m52s
[18:44:05.394] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:31:0)
[18:44:05.459] Build status: STARTED
[18:44:05.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:32:0) after 4m34s
[18:48:39.494] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:32:0)
[18:48:39.514] Build status: STARTED
[18:48:39.514] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:33:0) after 6m13s
[18:54:52.540] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:33:0)
[18:54:52.558] Build status: STARTED
[18:54:52.558] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:34:0) after 6m24s
[19:01:16.578] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:34:0)
[19:01:16.606] Build status: STARTED
[19:01:16.606] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:35:0) after 1m46s
[19:03:02.705] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:35:0)
[19:03:02.725] Build status: STARTED
[19:03:02.725] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:36:0) after 3m43s
[19:06:45.835] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:36:0)
[19:06:45.866] Build status: STARTED
[19:06:45.867] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:37:0) after 7m2s
[19:13:48.055] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:37:0)
[19:13:48.074] Build status: STARTED
[19:13:48.074] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:38:0) after 2m38s
[19:16:26.196] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:38:0)
[19:16:26.218] Build status: STARTED
[19:16:26.218] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:39:0) after 1m3s
[19:17:29.214] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:39:0)
[19:17:29.238] Build status: STARTED
[19:17:29.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:40:0) after 1m27s
[19:18:56.309] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:40:0)
[19:18:56.326] Build status: STARTED
[19:18:56.326] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:41:0) after 2m54s
[19:21:50.349] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:41:0)
[19:21:50.391] Build status: STARTED
[19:21:50.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:42:0) after 2m43s
[19:24:33.417] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:42:0)
[19:24:33.482] Build status: STARTED
[19:24:33.482] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:43:0) after 6m30s
[19:31:03.454] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:43:0)
[19:31:03.496] Build status: STARTED
[19:31:03.496] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:44:0) after 6m28s
[19:37:31.515] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:44:0)
[19:37:31.538] Build status: STARTED
[19:37:31.538] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:45:0) after 2m52s
[19:40:23.559] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:45:0)
[19:40:23.596] Build status: STARTED
[19:40:23.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:46:0) after 3m51s
[19:44:14.612] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:46:0)
[19:44:14.633] Build status: STARTED
[19:44:14.633] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:47:0) after 6m33s
[19:50:47.668] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:47:0)
[19:50:47.686] Build status: STARTED
[19:50:47.686] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:48:0) after 3m44s
[19:54:31.711] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:48:0)
[19:54:31.731] Build status: STARTED
[19:54:31.731] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:49:0) after 1m44s
[19:56:15.821] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:49:0)
[19:56:15.842] Build status: STARTED
[19:56:15.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:50:0) after 9m44s
[20:05:59.870] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:50:0)
[20:05:59.889] Build status: STARTED
[20:05:59.889] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:51:0) after 5m7s
[20:11:07.046] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:51:0)
[20:11:07.098] Build status: STARTED
[20:11:07.098] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:52:0) after 2m33s
[20:13:40.216] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:52:0)
[20:13:40.244] Build status: STARTED
[20:13:40.244] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:53:0) after 2m57s
[20:16:37.711] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:53:0)
[20:16:37.767] Build status: STARTED
[20:16:37.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833835456281446080:54:0) after 5m31s
[20:20:34.247] Received PubSub notification, asking Buildbucket for the build status
[20:20:34.279] Build:
{
"id": "8668405394382968001",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 23255,
"createdBy": "project:chromium",
"createTime": "2026-10-09T15:57:13.387749446Z",
"startTime": "2026-10-09T15:58:02.731480862Z",
"endTime": "2026-10-09T20:20:33.915000588Z",
"updateTime": "2026-10-09T20:20:33.915000588Z",
"status": "INFRA_FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "37da889422e1fe1228648a7176a22a5879d08d09",
"ref": "refs/heads/main"
}
}
}
[20:20:34.279] Invocation finished in 4h23m22.08600154s with status FAILED