[19:47:18.459] New invocation is queued and will start shortly
[19:47:19.528] Starting the invocation (attempt 1)
[19:47:19.562] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0fce3b89a7f81d0180de0efb44081c3775f1cbad"
[19:47:19.562] Popped gitiles commit info from properties and tags
[19:47:19.562] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[19:47:19.562] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[19:47:19.562] Buildbucket request:
{
"requestId": "8833549188468196464",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWin64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0fce3b89a7f81d0180de0efb44081c3775f1cbad",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8833549188468196464"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTWin64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[19:47:20.516] Scheduled build:
{
"id": "8668119126402374657",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWin64PGO"
},
"number": 16485,
"createdBy": "project:chromium",
"createTime": "2026-10-12T19:47:19.808343019Z",
"updateTime": "2026-10-12T19:47:19.808343019Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0fce3b89a7f81d0180de0efb44081c3775f1cbad",
"ref": "refs/heads/main"
}
}
}
[19:47:20.516] Task URL: https://cr-buildbucket.appspot.com/build/8668119126402374657
[19:47:20.517] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:2:0) after 6m54s
[19:47:21.436] Received PubSub notification, asking Buildbucket for the build status
[19:47:21.458] Build status: SCHEDULED
[19:47:27.551] Received PubSub notification, asking Buildbucket for the build status
[19:47:27.591] Build status: STARTED
[19:54:14.596] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:2:0)
[19:54:14.618] Build status: STARTED
[19:54:14.618] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:5:0) after 3m1s
[19:57:15.640] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:5:0)
[19:57:15.659] Build status: STARTED
[19:57:15.659] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:6:0) after 9m26s
[20:06:41.718] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:6:0)
[20:06:41.779] Build status: STARTED
[20:06:41.779] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:7:0) after 3m34s
[20:10:15.846] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:7:0)
[20:10:15.888] Build status: STARTED
[20:10:15.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:8:0) after 6m44s
[20:16:59.985] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:8:0)
[20:17:00.006] Build status: STARTED
[20:17:00.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:9:0) after 8m8s
[20:25:08.029] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:9:0)
[20:25:08.056] Build status: STARTED
[20:25:08.056] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:10:0) after 3m56s
[20:29:04.117] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:10:0)
[20:29:04.140] Build status: STARTED
[20:29:04.140] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:11:0) after 3m39s
[20:32:43.210] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:11:0)
[20:32:43.232] Build status: STARTED
[20:32:43.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:12:0) after 3m50s
[20:36:33.254] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:12:0)
[20:36:33.274] Build status: STARTED
[20:36:33.274] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:13:0) after 3m15s
[20:39:48.501] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:13:0)
[20:39:48.522] Build status: STARTED
[20:39:48.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:14:0) after 6m16s
[20:46:04.661] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:14:0)
[20:46:04.711] Build status: STARTED
[20:46:04.711] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:15:0) after 1m15s
[20:47:19.706] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:15:0)
[20:47:19.724] Build status: STARTED
[20:47:19.725] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:16:0) after 7m24s
[20:54:43.750] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:16:0)
[20:54:43.773] Build status: STARTED
[20:54:43.773] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:17:0) after 4m24s
[20:59:07.865] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:17:0)
[20:59:07.890] Build status: STARTED
[20:59:07.890] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:18:0) after 5m11s
[21:04:19.097] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:18:0)
[21:04:19.117] Build status: STARTED
[21:04:19.117] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:19:0) after 3m30s
[21:07:49.322] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:19:0)
[21:07:49.344] Build status: STARTED
[21:07:49.344] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:20:0) after 7m39s
[21:15:28.467] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:20:0)
[21:15:28.500] Build status: STARTED
[21:15:28.500] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:21:0) after 1m32s
[21:17:00.565] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:21:0)
[21:17:00.624] Build status: STARTED
[21:17:00.624] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:22:0) after 5m1s
[21:22:01.759] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:22:0)
[21:22:01.784] Build status: STARTED
[21:22:01.784] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:23:0) after 5m8s
[21:27:09.778] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:23:0)
[21:27:09.815] Build status: STARTED
[21:27:09.815] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:24:0) after 9m6s
[21:36:15.799] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:24:0)
[21:36:15.817] Build status: STARTED
[21:36:15.817] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:25:0) after 5m21s
[21:41:36.868] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:25:0)
[21:41:36.894] Build status: STARTED
[21:41:36.894] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:26:0) after 2m10s
[21:43:46.921] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:26:0)
[21:43:46.946] Build status: STARTED
[21:43:46.946] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:27:0) after 4m28s
[21:48:15.024] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:27:0)
[21:48:15.055] Build status: STARTED
[21:48:15.055] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:28:0) after 6m45s
[21:55:00.232] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:28:0)
[21:55:00.251] Build status: STARTED
[21:55:00.251] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:29:0) after 7m5s
[22:02:05.275] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:29:0)
[22:02:05.294] Build status: STARTED
[22:02:05.294] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:30:0) after 5m19s
[22:07:24.368] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:30:0)
[22:07:24.390] Build status: STARTED
[22:07:24.390] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:31:0) after 5m58s
[22:13:22.460] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:31:0)
[22:13:22.482] Build status: STARTED
[22:13:22.482] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:32:0) after 8m13s
[22:21:35.509] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:32:0)
[22:21:35.524] Build status: STARTED
[22:21:35.524] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:33:0) after 4m49s
[22:26:24.557] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:33:0)
[22:26:24.596] Build status: STARTED
[22:26:24.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:34:0) after 2m54s
[22:29:18.777] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:34:0)
[22:29:18.799] Build status: STARTED
[22:29:18.799] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:35:0) after 1m7s
[22:30:25.847] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:35:0)
[22:30:25.915] Build status: STARTED
[22:30:25.915] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:36:0) after 8m5s
[22:38:30.984] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:36:0)
[22:38:31.023] Build status: STARTED
[22:38:31.023] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:37:0) after 2m57s
[22:41:28.208] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:37:0)
[22:41:28.236] Build status: STARTED
[22:41:28.236] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:38:0) after 7m52s
[22:49:20.257] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:38:0)
[22:49:20.276] Build status: STARTED
[22:49:20.276] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:39:0) after 1m49s
[22:51:09.311] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:39:0)
[22:51:09.340] Build status: STARTED
[22:51:09.340] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:40:0) after 8m27s
[22:59:36.418] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:40:0)
[22:59:36.441] Build status: STARTED
[22:59:36.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:41:0) after 3m33s
[23:03:09.458] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:41:0)
[23:03:09.532] Build status: STARTED
[23:03:09.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:42:0) after 5m21s
[23:08:30.687] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:42:0)
[23:08:30.705] Build status: STARTED
[23:08:30.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:43:0) after 1m50s
[23:10:20.916] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:43:0)
[23:10:20.934] Build status: STARTED
[23:10:20.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:44:0) after 5m4s
[23:15:24.962] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:44:0)
[23:15:24.987] Build status: STARTED
[23:15:24.987] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:45:0) after 8m14s
[23:23:39.031] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:45:0)
[23:23:39.062] Build status: STARTED
[23:23:39.063] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:46:0) after 8m37s
[23:32:16.247] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:46:0)
[23:32:16.286] Build status: STARTED
[23:32:16.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:47:0) after 7m39s
[23:39:55.317] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:47:0)
[23:39:55.336] Build status: STARTED
[23:39:55.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:48:0) after 2m18s
[23:42:13.512] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:48:0)
[23:42:13.541] Build status: STARTED
[23:42:13.541] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:49:0) after 2m40s
[23:44:53.534] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:49:0)
[23:44:53.554] Build status: STARTED
[23:44:53.554] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:50:0) after 8m52s
[23:53:45.575] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:50:0)
[23:53:45.600] Build status: STARTED
[23:53:45.600] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:51:0) after 6m33s
[00:00:18.644] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:51:0)
[00:00:18.672] Build status: STARTED
[00:00:18.672] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:52:0) after 7m43s
[00:08:01.842] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:52:0)
[00:08:01.864] Build status: STARTED
[00:08:01.864] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:53:0) after 5m55s
[00:13:56.946] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:53:0)
[00:13:56.973] Build status: STARTED
[00:13:56.973] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:54:0) after 2m18s
[00:16:15.074] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:54:0)
[00:16:15.090] Build status: STARTED
[00:16:15.090] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:55:0) after 4m5s
[00:20:20.124] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:55:0)
[00:20:20.183] Build status: STARTED
[00:20:20.183] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:56:0) after 7m24s
[00:27:44.229] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:56:0)
[00:27:44.245] Build status: STARTED
[00:27:44.245] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:57:0) after 4m8s
[00:31:52.321] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:57:0)
[00:31:52.345] Build status: STARTED
[00:31:52.345] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:58:0) after 1m22s
[00:33:14.436] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:58:0)
[00:33:14.466] Build status: STARTED
[00:33:14.466] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:59:0) after 2m22s
[00:35:36.515] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:59:0)
[00:35:36.534] Build status: STARTED
[00:35:36.534] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:60:0) after 7m10s
[00:42:46.613] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:60:0)
[00:42:46.633] Build status: STARTED
[00:42:46.633] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:61:0) after 8m41s
[00:51:27.658] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:61:0)
[00:51:27.676] Build status: STARTED
[00:51:27.677] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:62:0) after 7m18s
[00:58:45.911] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:62:0)
[00:58:45.932] Build status: STARTED
[00:58:45.932] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:63:0) after 1m44s
[01:00:30.002] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:63:0)
[01:00:30.024] Build status: STARTED
[01:00:30.025] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:64:0) after 4m5s
[01:04:35.054] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:64:0)
[01:04:35.083] Build status: STARTED
[01:04:35.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:65:0) after 5m33s
[01:10:08.214] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:65:0)
[01:10:08.235] Build status: STARTED
[01:10:08.235] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:66:0) after 9m20s
[01:19:28.406] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:66:0)
[01:19:28.427] Build status: STARTED
[01:19:28.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:67:0) after 5m6s
[01:24:34.462] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:67:0)
[01:24:34.479] Build status: STARTED
[01:24:34.479] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833549188468196464:68:0) after 9m52s
[01:25:48.877] Received PubSub notification, asking Buildbucket for the build status
[01:25:48.901] Build:
{
"id": "8668119126402374657",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWin64PGO"
},
"number": 16485,
"createdBy": "project:chromium",
"createTime": "2026-10-12T19:47:19.808343019Z",
"startTime": "2026-10-12T19:47:27.371344884Z",
"endTime": "2026-10-13T01:25:48.364471815Z",
"updateTime": "2026-10-13T01:25:48.364471815Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0fce3b89a7f81d0180de0efb44081c3775f1cbad",
"ref": "refs/heads/main"
}
}
}
[01:25:48.901] Invocation finished in 5h38m30.447284218s with status FAILED