[08:34:59.505] New invocation is queued and will start shortly
[08:35:00.809] Starting the invocation (attempt 1)
[08:35:00.855] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/a70c0e3a7224b9ca12eb2c71b299174a25b17ec9"
[08:35:00.855] Popped gitiles commit info from properties and tags
[08:35:00.855] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[08:35:00.855] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[08:35:00.855] Buildbucket request:
{
"requestId": "8833410293018060144",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWin64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a70c0e3a7224b9ca12eb2c71b299174a25b17ec9",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8833410293018060144"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTWin64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[08:35:01.560] Scheduled build:
{
"id": "8667980230847761473",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWin64PGO"
},
"number": 16492,
"createdBy": "project:chromium",
"createTime": "2026-10-14T08:35:00.931395801Z",
"updateTime": "2026-10-14T08:35:00.931395801Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a70c0e3a7224b9ca12eb2c71b299174a25b17ec9",
"ref": "refs/heads/main"
}
}
}
[08:35:01.560] Task URL: https://cr-buildbucket.appspot.com/build/8667980230847761473
[08:35:01.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:2:0) after 7m19s
[08:35:02.552] Received PubSub notification, asking Buildbucket for the build status
[08:35:02.571] Build status: SCHEDULED
[08:35:08.634] Received PubSub notification, asking Buildbucket for the build status
[08:35:08.656] Build status: STARTED
[08:42:20.631] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:2:0)
[08:42:20.682] Build status: STARTED
[08:42:20.682] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:5:0) after 6m28s
[08:48:48.672] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:5:0)
[08:48:48.694] Build status: STARTED
[08:48:48.694] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:6:0) after 5m50s
[08:54:38.726] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:6:0)
[08:54:38.789] Build status: STARTED
[08:54:38.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:7:0) after 7m17s
[09:01:55.856] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:7:0)
[09:01:55.873] Build status: STARTED
[09:01:55.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:8:0) after 9m12s
[09:11:07.999] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:8:0)
[09:11:08.023] Build status: STARTED
[09:11:08.023] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:9:0) after 3m17s
[09:14:25.106] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:9:0)
[09:14:25.127] Build status: STARTED
[09:14:25.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:10:0) after 8m19s
[09:22:44.247] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:10:0)
[09:22:44.276] Build status: STARTED
[09:22:44.276] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:11:0) after 5m53s
[09:28:37.403] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:11:0)
[09:28:37.424] Build status: STARTED
[09:28:37.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:12:0) after 4m17s
[09:32:54.454] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:12:0)
[09:32:54.472] Build status: STARTED
[09:32:54.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:13:0) after 4m45s
[09:37:39.504] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:13:0)
[09:37:39.526] Build status: STARTED
[09:37:39.526] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:14:0) after 6m54s
[09:44:33.671] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:14:0)
[09:44:33.698] Build status: STARTED
[09:44:33.698] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:15:0) after 5m12s
[09:49:45.739] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:15:0)
[09:49:45.773] Build status: STARTED
[09:49:45.774] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:16:0) after 6m26s
[09:56:11.795] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:16:0)
[09:56:11.825] Build status: STARTED
[09:56:11.825] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:17:0) after 3m49s
[10:00:00.862] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:17:0)
[10:00:00.898] Build status: STARTED
[10:00:00.898] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:18:0) after 9m13s
[10:09:13.960] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:18:0)
[10:09:13.981] Build status: STARTED
[10:09:13.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:19:0) after 9m30s
[10:18:44.035] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:19:0)
[10:18:44.057] Build status: STARTED
[10:18:44.057] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:20:0) after 4m18s
[10:23:02.128] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:20:0)
[10:23:02.145] Build status: STARTED
[10:23:02.145] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:21:0) after 6m1s
[10:29:03.302] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:21:0)
[10:29:03.323] Build status: STARTED
[10:29:03.323] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:22:0) after 3m27s
[10:32:30.344] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:22:0)
[10:32:30.385] Build status: STARTED
[10:32:30.385] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:23:0) after 7m27s
[10:39:57.461] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:23:0)
[10:39:57.483] Build status: STARTED
[10:39:57.483] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:24:0) after 1m26s
[10:41:23.557] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:24:0)
[10:41:23.600] Build status: STARTED
[10:41:23.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:25:0) after 1m11s
[10:42:34.622] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:25:0)
[10:42:34.650] Build status: STARTED
[10:42:34.650] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:26:0) after 2m41s
[10:45:15.831] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:26:0)
[10:45:15.854] Build status: STARTED
[10:45:15.854] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:27:0) after 3m29s
[10:48:44.878] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:27:0)
[10:48:44.898] Build status: STARTED
[10:48:44.898] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:28:0) after 2m41s
[10:51:25.946] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:28:0)
[10:51:25.963] Build status: STARTED
[10:51:25.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:29:0) after 1m55s
[10:53:20.987] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:29:0)
[10:53:21.016] Build status: STARTED
[10:53:21.016] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:30:0) after 7m53s
[11:01:14.045] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:30:0)
[11:01:14.067] Build status: STARTED
[11:01:14.067] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:31:0) after 4m36s
[11:05:50.188] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:31:0)
[11:05:50.210] Build status: STARTED
[11:05:50.210] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:32:0) after 4m24s
[11:10:14.258] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:32:0)
[11:10:14.290] Build status: STARTED
[11:10:14.290] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:33:0) after 2m0s
[11:12:14.310] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:33:0)
[11:12:14.357] Build status: STARTED
[11:12:14.357] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:34:0) after 4m7s
[11:16:21.541] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:34:0)
[11:16:21.565] Build status: STARTED
[11:16:21.565] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:35:0) after 8m9s
[11:24:30.553] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:35:0)
[11:24:30.622] Build status: STARTED
[11:24:30.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:36:0) after 2m35s
[11:27:05.731] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:36:0)
[11:27:05.751] Build status: STARTED
[11:27:05.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:37:0) after 4m17s
[11:31:22.798] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:37:0)
[11:31:22.852] Build status: STARTED
[11:31:22.852] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:38:0) after 9m2s
[11:40:25.067] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:38:0)
[11:40:25.152] Build status: STARTED
[11:40:25.152] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:39:0) after 9m7s
[11:49:32.136] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:39:0)
[11:49:32.164] Build status: STARTED
[11:49:32.164] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:40:0) after 6m5s
[11:55:37.206] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:40:0)
[11:55:37.223] Build status: STARTED
[11:55:37.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:41:0) after 3m39s
[11:59:16.293] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:41:0)
[11:59:16.314] Build status: STARTED
[11:59:16.314] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:42:0) after 7m6s
[12:06:22.364] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:42:0)
[12:06:22.449] Build status: STARTED
[12:06:22.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:43:0) after 2m14s
[12:08:36.484] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:43:0)
[12:08:36.506] Build status: STARTED
[12:08:36.506] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:44:0) after 9m7s
[12:17:43.821] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:44:0)
[12:17:43.853] Build status: STARTED
[12:17:43.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:45:0) after 9m27s
[12:27:10.888] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:45:0)
[12:27:10.907] Build status: STARTED
[12:27:10.907] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:46:0) after 5m47s
[12:32:57.979] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:46:0)
[12:32:58.012] Build status: STARTED
[12:32:58.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:47:0) after 5m30s
[12:38:28.138] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:47:0)
[12:38:28.153] Build status: STARTED
[12:38:28.153] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:48:0) after 2m6s
[12:40:34.445] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:48:0)
[12:40:34.463] Build status: STARTED
[12:40:34.464] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:49:0) after 2m4s
[12:42:38.508] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:49:0)
[12:42:38.547] Build status: STARTED
[12:42:38.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:50:0) after 1m15s
[12:43:53.590] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:50:0)
[12:43:53.623] Build status: STARTED
[12:43:53.623] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:51:0) after 5m56s
[12:49:49.704] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:51:0)
[12:49:49.724] Build status: STARTED
[12:49:49.724] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:52:0) after 4m37s
[12:54:26.750] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:52:0)
[12:54:26.768] Build status: STARTED
[12:54:26.768] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:53:0) after 9m23s
[13:03:49.845] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:53:0)
[13:03:49.870] Build status: STARTED
[13:03:49.870] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:54:0) after 1m36s
[13:05:25.987] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:54:0)
[13:05:26.013] Build status: STARTED
[13:05:26.013] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:55:0) after 7m5s
[13:12:31.033] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:55:0)
[13:12:31.091] Build status: STARTED
[13:12:31.091] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:56:0) after 3m19s
[13:15:50.208] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:56:0)
[13:15:50.226] Build status: STARTED
[13:15:50.226] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:57:0) after 2m33s
[13:18:23.304] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:57:0)
[13:18:23.324] Build status: STARTED
[13:18:23.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:58:0) after 7m33s
[13:25:56.369] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:58:0)
[13:25:56.404] Build status: STARTED
[13:25:56.404] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:59:0) after 2m16s
[13:28:12.439] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:59:0)
[13:28:12.495] Build status: STARTED
[13:28:12.495] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:60:0) after 8m15s
[13:36:27.565] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:60:0)
[13:36:27.585] Build status: STARTED
[13:36:27.585] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:61:0) after 4m26s
[13:40:53.603] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:61:0)
[13:40:53.636] Build status: STARTED
[13:40:53.636] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:62:0) after 4m4s
[13:44:57.659] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:62:0)
[13:44:57.691] Build status: STARTED
[13:44:57.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:63:0) after 9m46s
[13:54:43.775] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:63:0)
[13:54:43.801] Build status: STARTED
[13:54:43.801] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:64:0) after 4m24s
[13:59:07.921] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:64:0)
[13:59:07.941] Build status: STARTED
[13:59:07.941] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:65:0) after 9m54s
[14:09:02.019] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:65:0)
[14:09:02.036] Build status: STARTED
[14:09:02.036] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8833410293018060144:66:0) after 8m8s
[14:10:27.179] Received PubSub notification, asking Buildbucket for the build status
[14:10:27.203] Build:
{
"id": "8667980230847761473",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWin64PGO"
},
"number": 16492,
"createdBy": "project:chromium",
"createTime": "2026-10-14T08:35:00.931395801Z",
"startTime": "2026-10-14T08:35:07.908209007Z",
"endTime": "2026-10-14T14:10:26.130909295Z",
"updateTime": "2026-10-14T14:10:26.130909295Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a70c0e3a7224b9ca12eb2c71b299174a25b17ec9",
"ref": "refs/heads/main"
}
}
}
[14:10:27.203] Invocation finished in 5h35m27.725242326s with status FAILED