[06:01:17.170] New invocation is queued and will start shortly
[06:01:19.033] Starting the invocation (attempt 1)
[06:01:19.062] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/ed404a0335cfd0bab79554a46891d56f9640b640"
[06:01:19.062] Popped gitiles commit info from properties and tags
[06:01:19.062] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[06:01:19.063] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[06:01:19.063] Buildbucket request:
{
"requestId": "8868843377176685920",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWin64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ed404a0335cfd0bab79554a46891d56f9640b640",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8868843377176685920"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTWin64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[06:01:19.712] Scheduled build:
{
"id": "8703413314481755121",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWin64PGO"
},
"number": 12456,
"createdBy": "project:chromium",
"createTime": "2025-09-18T06:01:19.118980091Z",
"updateTime": "2025-09-18T06:01:19.118980091Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ed404a0335cfd0bab79554a46891d56f9640b640",
"ref": "refs/heads/main"
}
}
}
[06:01:19.712] Task URL: https://cr-buildbucket.appspot.com/build/8703413314481755121
[06:01:19.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:2:0) after 6m12s
[06:01:20.598] Received PubSub notification, asking Buildbucket for the build status
[06:01:20.615] Build status: SCHEDULED
[06:07:31.790] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:2:0)
[06:07:31.805] Build status: SCHEDULED
[06:07:31.805] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:4:0) after 1m0s
[06:08:32.635] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:4:0)
[06:08:32.667] Build status: SCHEDULED
[06:08:32.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:5:0) after 3m5s
[06:11:38.095] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:5:0)
[06:11:38.116] Build status: SCHEDULED
[06:11:38.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:6:0) after 9m47s
[06:21:25.247] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:6:0)
[06:21:25.348] Build status: SCHEDULED
[06:21:25.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:7:0) after 7m46s
[06:29:11.372] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:7:0)
[06:29:11.395] Build status: SCHEDULED
[06:29:11.395] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:8:0) after 4m36s
[06:33:47.430] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:8:0)
[06:33:47.474] Build status: SCHEDULED
[06:33:47.474] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:9:0) after 6m25s
[06:40:12.705] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:9:0)
[06:40:12.733] Build status: SCHEDULED
[06:40:12.733] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:10:0) after 6m22s
[06:46:34.816] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:10:0)
[06:46:34.831] Build status: SCHEDULED
[06:46:34.831] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:11:0) after 7m56s
[06:54:30.945] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:11:0)
[06:54:31.059] Build status: SCHEDULED
[06:54:31.059] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:12:0) after 4m0s
[06:58:31.110] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:12:0)
[06:58:31.126] Build status: SCHEDULED
[06:58:31.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:13:0) after 4m21s
[07:02:52.152] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:13:0)
[07:02:52.217] Build status: SCHEDULED
[07:02:52.217] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:14:0) after 8m10s
[07:11:02.242] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:14:0)
[07:11:02.258] Build status: SCHEDULED
[07:11:02.258] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:15:0) after 1m9s
[07:12:11.354] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:15:0)
[07:12:11.374] Build status: SCHEDULED
[07:12:11.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:16:0) after 1m58s
[07:14:09.422] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:16:0)
[07:14:09.508] Build status: SCHEDULED
[07:14:09.508] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:17:0) after 1m20s
[07:15:29.597] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:17:0)
[07:15:29.618] Build status: SCHEDULED
[07:15:29.618] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:18:0) after 9m42s
[07:25:11.634] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:18:0)
[07:25:11.654] Build status: SCHEDULED
[07:25:11.654] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:19:0) after 6m30s
[07:31:41.683] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:19:0)
[07:31:41.717] Build status: SCHEDULED
[07:31:41.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:20:0) after 5m57s
[07:37:38.806] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:20:0)
[07:37:38.935] Build status: SCHEDULED
[07:37:38.935] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:21:0) after 2m42s
[07:40:20.955] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:21:0)
[07:40:20.971] Build status: SCHEDULED
[07:40:20.971] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:22:0) after 9m8s
[07:49:28.993] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:22:0)
[07:49:29.055] Build status: SCHEDULED
[07:49:29.055] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:23:0) after 3m55s
[07:53:24.164] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:23:0)
[07:53:24.185] Build status: SCHEDULED
[07:53:24.185] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:24:0) after 5m53s
[07:59:17.210] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:24:0)
[07:59:17.228] Build status: SCHEDULED
[07:59:17.228] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:25:0) after 4m48s
[08:04:05.288] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:25:0)
[08:04:05.338] Build status: SCHEDULED
[08:04:05.338] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:26:0) after 8m53s
[08:10:11.781] Received PubSub notification, asking Buildbucket for the build status
[08:10:11.797] Build status: STARTED
[08:12:58.394] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:26:0)
[08:12:58.411] Build status: STARTED
[08:12:58.411] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:28:0) after 6m19s
[08:19:17.509] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:28:0)
[08:19:17.537] Build status: STARTED
[08:19:17.537] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:29:0) after 8m58s
[08:28:15.561] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:29:0)
[08:28:15.582] Build status: STARTED
[08:28:15.582] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:30:0) after 7m13s
[08:35:28.611] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:30:0)
[08:35:28.641] Build status: STARTED
[08:35:28.641] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:31:0) after 3m40s
[08:39:08.745] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:31:0)
[08:39:08.903] Build status: STARTED
[08:39:08.903] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:32:0) after 7m48s
[08:46:56.931] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:32:0)
[08:46:56.949] Build status: STARTED
[08:46:56.949] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:33:0) after 3m53s
[08:50:50.002] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:33:0)
[08:50:50.036] Build status: STARTED
[08:50:50.036] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:34:0) after 7m50s
[08:58:40.100] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:34:0)
[08:58:40.115] Build status: STARTED
[08:58:40.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:35:0) after 3m25s
[09:02:05.148] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:35:0)
[09:02:05.165] Build status: STARTED
[09:02:05.165] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:36:0) after 8m11s
[09:10:16.193] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:36:0)
[09:10:16.212] Build status: STARTED
[09:10:16.212] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:37:0) after 3m13s
[09:13:29.239] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:37:0)
[09:13:29.263] Build status: STARTED
[09:13:29.264] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:38:0) after 3m28s
[09:16:57.334] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:38:0)
[09:16:57.349] Build status: STARTED
[09:16:57.349] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:39:0) after 3m23s
[09:20:20.378] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:39:0)
[09:20:20.394] Build status: STARTED
[09:20:20.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:40:0) after 8m1s
[09:28:21.411] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:40:0)
[09:28:21.428] Build status: STARTED
[09:28:21.428] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:41:0) after 8m20s
[09:36:41.447] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:41:0)
[09:36:41.592] Build status: STARTED
[09:36:41.592] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:42:0) after 7m37s
[09:44:18.646] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:42:0)
[09:44:18.670] Build status: STARTED
[09:44:18.670] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:43:0) after 8m12s
[09:52:30.729] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:43:0)
[09:52:30.744] Build status: STARTED
[09:52:30.744] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:44:0) after 2m8s
[09:54:38.772] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:44:0)
[09:54:38.798] Build status: STARTED
[09:54:38.798] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:45:0) after 1m9s
[09:55:47.829] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:45:0)
[09:55:47.950] Build status: STARTED
[09:55:47.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:46:0) after 2m56s
[09:58:43.975] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:46:0)
[09:58:43.994] Build status: STARTED
[09:58:43.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:47:0) after 3m14s
[10:01:58.027] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:47:0)
[10:01:58.080] Build status: STARTED
[10:01:58.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:48:0) after 8m24s
[10:10:22.111] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:48:0)
[10:10:22.176] Build status: STARTED
[10:10:22.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:49:0) after 4m48s
[10:15:10.293] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:49:0)
[10:15:10.321] Build status: STARTED
[10:15:10.321] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:50:0) after 9m2s
[10:24:12.452] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:50:0)
[10:24:12.493] Build status: STARTED
[10:24:12.493] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:51:0) after 7m13s
[10:31:25.530] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:51:0)
[10:31:25.558] Build status: STARTED
[10:31:25.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:52:0) after 2m16s
[10:33:41.578] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:52:0)
[10:33:41.679] Build status: STARTED
[10:33:41.679] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:53:0) after 7m41s
[10:41:22.710] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:53:0)
[10:41:22.729] Build status: STARTED
[10:41:22.729] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:54:0) after 5m58s
[10:47:20.837] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:54:0)
[10:47:20.855] Build status: STARTED
[10:47:20.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:55:0) after 8m25s
[10:55:45.991] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:55:0)
[10:55:46.009] Build status: STARTED
[10:55:46.009] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:56:0) after 4m17s
[11:00:03.154] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:56:0)
[11:00:03.238] Build status: STARTED
[11:00:03.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:57:0) after 4m7s
[11:04:10.446] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:57:0)
[11:04:10.460] Build status: STARTED
[11:04:10.460] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:58:0) after 9m27s
[11:13:37.485] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:58:0)
[11:13:37.504] Build status: STARTED
[11:13:37.504] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:59:0) after 6m11s
[11:19:48.650] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:59:0)
[11:19:48.691] Build status: STARTED
[11:19:48.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:60:0) after 2m11s
[11:21:59.791] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:60:0)
[11:21:59.811] Build status: STARTED
[11:21:59.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:61:0) after 3m1s
[11:25:01.473] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:61:0)
[11:25:01.522] Build status: STARTED
[11:25:01.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:62:0) after 5m31s
[11:30:32.643] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:62:0)
[11:30:32.657] Build status: STARTED
[11:30:32.658] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:63:0) after 4m44s
[11:35:16.937] Handling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:63:0)
[11:35:16.956] Build status: STARTED
[11:35:16.956] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWin64PGO:8868843377176685920:64:0) after 3m52s
[11:37:40.712] Received PubSub notification, asking Buildbucket for the build status
[11:37:40.733] Build:
{
"id": "8703413314481755121",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWin64PGO"
},
"number": 12456,
"createdBy": "project:chromium",
"createTime": "2025-09-18T06:01:19.118980091Z",
"startTime": "2025-09-18T08:10:11.611418090Z",
"endTime": "2025-09-18T11:37:40.105454765Z",
"updateTime": "2025-09-18T11:37:40.105454765Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ed404a0335cfd0bab79554a46891d56f9640b640",
"ref": "refs/heads/main"
}
}
}
[11:37:40.733] Invocation finished in 5h36m23.569467371s with status FAILED