[17:36:21.196] New invocation is queued and will start shortly
[17:36:22.400] Starting the invocation (attempt 1)
[17:36:22.422] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/178776c6e4e72c6bbd94099c9fe12d2c04b21cee"
[17:36:22.422] Popped gitiles commit info from properties and tags
[17:36:22.422] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[17:36:22.422] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[17:36:22.422] Buildbucket request:
{
"requestId": "8842526527079279856",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "178776c6e4e72c6bbd94099c9fe12d2c04b21cee",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8842526527079279856"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[17:36:23.326] Scheduled build:
{
"id": "8677096464711779425",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 21503,
"createdBy": "project:chromium",
"createTime": "2026-07-05T17:36:22.832425282Z",
"updateTime": "2026-07-05T17:36:22.832425282Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "178776c6e4e72c6bbd94099c9fe12d2c04b21cee",
"ref": "refs/heads/main"
}
},
"canary": true
}
[17:36:23.326] Task URL: https://cr-buildbucket.appspot.com/build/8677096464711779425
[17:36:23.326] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:2:0) after 6m44s
[17:36:24.313] Received PubSub notification, asking Buildbucket for the build status
[17:36:24.341] Build status: SCHEDULED
[17:37:42.757] Received PubSub notification, asking Buildbucket for the build status
[17:37:42.776] Build status: STARTED
[17:43:07.444] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:2:0)
[17:43:07.468] Build status: STARTED
[17:43:07.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:5:0) after 7m34s
[17:50:41.520] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:5:0)
[17:50:41.545] Build status: STARTED
[17:50:41.545] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:6:0) after 3m56s
[17:54:37.647] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:6:0)
[17:54:37.664] Build status: STARTED
[17:54:37.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:7:0) after 5m32s
[18:00:09.949] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:7:0)
[18:00:09.981] Build status: STARTED
[18:00:09.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:8:0) after 9m35s
[18:09:45.099] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:8:0)
[18:09:45.132] Build status: STARTED
[18:09:45.132] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:9:0) after 1m34s
[18:11:19.231] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:9:0)
[18:11:19.275] Build status: STARTED
[18:11:19.275] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:10:0) after 4m46s
[18:16:05.339] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:10:0)
[18:16:05.355] Build status: STARTED
[18:16:05.355] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:11:0) after 4m59s
[18:21:04.338] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:11:0)
[18:21:04.367] Build status: STARTED
[18:21:04.367] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:12:0) after 3m56s
[18:25:00.457] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:12:0)
[18:25:00.477] Build status: STARTED
[18:25:00.477] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:13:0) after 1m27s
[18:26:27.493] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:13:0)
[18:26:27.510] Build status: STARTED
[18:26:27.510] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:14:0) after 6m51s
[18:33:18.638] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:14:0)
[18:33:18.661] Build status: STARTED
[18:33:18.661] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:15:0) after 3m0s
[18:36:18.678] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:15:0)
[18:36:18.705] Build status: STARTED
[18:36:18.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:16:0) after 9m6s
[18:45:24.719] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:16:0)
[18:45:24.745] Build status: STARTED
[18:45:24.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:17:0) after 3m57s
[18:49:21.890] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:17:0)
[18:49:21.910] Build status: STARTED
[18:49:21.910] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:18:0) after 6m20s
[18:55:41.927] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:18:0)
[18:55:41.946] Build status: STARTED
[18:55:41.946] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:19:0) after 7m9s
[19:02:51.054] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:19:0)
[19:02:51.077] Build status: STARTED
[19:02:51.077] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:20:0) after 4m15s
[19:07:06.231] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:20:0)
[19:07:06.251] Build status: STARTED
[19:07:06.252] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:21:0) after 5m12s
[19:12:18.572] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:21:0)
[19:12:18.672] Build status: STARTED
[19:12:18.672] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:22:0) after 5m59s
[19:18:17.698] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:22:0)
[19:18:17.716] Build status: STARTED
[19:18:17.716] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:23:0) after 7m24s
[19:25:41.826] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:23:0)
[19:25:41.846] Build status: STARTED
[19:25:41.847] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:24:0) after 7m32s
[19:33:13.923] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:24:0)
[19:33:13.952] Build status: STARTED
[19:33:13.953] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:25:0) after 2m30s
[19:35:44.027] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:25:0)
[19:35:44.047] Build status: STARTED
[19:35:44.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:26:0) after 6m25s
[19:42:09.109] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:26:0)
[19:42:09.156] Build status: STARTED
[19:42:09.157] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:27:0) after 2m55s
[19:45:04.175] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:27:0)
[19:45:04.200] Build status: STARTED
[19:45:04.200] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:28:0) after 7m0s
[19:52:04.220] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:28:0)
[19:52:04.238] Build status: STARTED
[19:52:04.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:29:0) after 2m18s
[19:54:22.265] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:29:0)
[19:54:22.284] Build status: STARTED
[19:54:22.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:30:0) after 3m38s
[19:58:00.300] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:30:0)
[19:58:00.350] Build status: STARTED
[19:58:00.350] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:31:0) after 1m55s
[19:59:55.421] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:31:0)
[19:59:55.438] Build status: STARTED
[19:59:55.438] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:32:0) after 3m42s
[20:03:37.461] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:32:0)
[20:03:37.499] Build status: STARTED
[20:03:37.499] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:33:0) after 2m48s
[20:06:25.613] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:33:0)
[20:06:25.631] Build status: STARTED
[20:06:25.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:34:0) after 4m32s
[20:10:57.728] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:34:0)
[20:10:57.769] Build status: STARTED
[20:10:57.769] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:35:0) after 3m22s
[20:14:19.784] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:35:0)
[20:14:19.808] Build status: STARTED
[20:14:19.808] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:36:0) after 7m30s
[20:21:49.851] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:36:0)
[20:21:50.079] Build status: STARTED
[20:21:50.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:37:0) after 1m36s
[20:23:26.099] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:37:0)
[20:23:26.124] Build status: STARTED
[20:23:26.125] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:38:0) after 5m59s
[20:29:25.372] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:38:0)
[20:29:25.394] Build status: STARTED
[20:29:25.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:39:0) after 8m11s
[20:37:36.642] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:39:0)
[20:37:36.691] Build status: STARTED
[20:37:36.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:40:0) after 7m23s
[20:44:59.707] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:40:0)
[20:44:59.731] Build status: STARTED
[20:44:59.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:41:0) after 2m44s
[20:47:43.808] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:41:0)
[20:47:43.825] Build status: STARTED
[20:47:43.825] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:42:0) after 4m8s
[20:51:51.940] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:42:0)
[20:51:51.961] Build status: STARTED
[20:51:51.962] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:43:0) after 4m13s
[20:56:05.664] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:43:0)
[20:56:05.680] Build status: STARTED
[20:56:05.680] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:44:0) after 1m59s
[20:58:04.782] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:44:0)
[20:58:04.809] Build status: STARTED
[20:58:04.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:45:0) after 8m4s
[21:06:08.918] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:45:0)
[21:06:08.942] Build status: STARTED
[21:06:08.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:46:0) after 2m51s
[21:09:00.350] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:46:0)
[21:09:00.369] Build status: STARTED
[21:09:00.369] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:47:0) after 5m45s
[21:14:45.586] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:47:0)
[21:14:45.674] Build status: STARTED
[21:14:45.674] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:48:0) after 7m33s
[21:22:19.100] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:48:0)
[21:22:19.135] Build status: STARTED
[21:22:19.135] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:49:0) after 6m25s
[21:28:44.138] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:49:0)
[21:28:44.170] Build status: STARTED
[21:28:44.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:50:0) after 8m33s
[21:37:17.187] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:50:0)
[21:37:17.205] Build status: STARTED
[21:37:17.205] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:51:0) after 2m2s
[21:39:19.335] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:51:0)
[21:39:19.401] Build status: STARTED
[21:39:19.401] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:52:0) after 7m31s
[21:46:50.451] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:52:0)
[21:46:50.473] Build status: STARTED
[21:46:50.473] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:53:0) after 9m42s
[21:56:32.666] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:53:0)
[21:56:32.683] Build status: STARTED
[21:56:32.683] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:54:0) after 3m39s
[22:00:11.835] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:54:0)
[22:00:11.856] Build status: STARTED
[22:00:11.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:55:0) after 5m55s
[22:06:07.061] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:55:0)
[22:06:07.081] Build status: STARTED
[22:06:07.081] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:56:0) after 3m39s
[22:09:46.162] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:56:0)
[22:09:46.203] Build status: STARTED
[22:09:46.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:57:0) after 1m7s
[22:10:53.288] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:57:0)
[22:10:53.305] Build status: STARTED
[22:10:53.305] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:58:0) after 4m17s
[22:15:10.428] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:58:0)
[22:15:10.443] Build status: STARTED
[22:15:10.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:59:0) after 4m57s
[22:20:07.460] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:59:0)
[22:20:07.498] Build status: STARTED
[22:20:07.498] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842526527079279856:60:0) after 5m55s
[22:24:23.438] Received PubSub notification, asking Buildbucket for the build status
[22:24:23.460] Build:
{
"id": "8677096464711779425",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 21503,
"createdBy": "project:chromium",
"createTime": "2026-07-05T17:36:22.832425282Z",
"startTime": "2026-07-05T17:37:42.584955564Z",
"endTime": "2026-07-05T22:24:23.143469428Z",
"updateTime": "2026-07-05T22:24:23.143469428Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "178776c6e4e72c6bbd94099c9fe12d2c04b21cee",
"ref": "refs/heads/main"
}
},
"canary": true
}
[22:24:23.460] Invocation finished in 4h48m2.268606527s with status FAILED