[11:25:15.514] New invocation is queued and will start shortly
[11:25:16.649] Starting the invocation (attempt 1)
[11:25:16.698] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/3e4c084b9c0cef43fda6eb3dc16d6bdfa937c9d8"
[11:25:16.698] Popped gitiles commit info from properties and tags
[11:25:16.698] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[11:25:16.698] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[11:25:16.699] Buildbucket request:
{
"requestId": "8852153152775515152",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWinArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3e4c084b9c0cef43fda6eb3dc16d6bdfa937c9d8",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8852153152775515152"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTWinArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[11:25:17.220] Scheduled build:
{
"id": "8686723090837597377",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWinArm64PGO"
},
"number": 7746,
"createdBy": "project:chromium",
"createTime": "2026-03-21T11:25:16.743601259Z",
"updateTime": "2026-03-21T11:25:16.743601259Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3e4c084b9c0cef43fda6eb3dc16d6bdfa937c9d8",
"ref": "refs/heads/main"
}
}
}
[11:25:17.220] Task URL: https://cr-buildbucket.appspot.com/build/8686723090837597377
[11:25:17.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:2:0) after 5m8s
[11:25:18.167] Received PubSub notification, asking Buildbucket for the build status
[11:25:18.209] Build status: SCHEDULED
[11:26:52.187] Received PubSub notification, asking Buildbucket for the build status
[11:26:52.207] Build status: STARTED
[11:30:25.236] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:2:0)
[11:30:25.257] Build status: STARTED
[11:30:25.257] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:5:0) after 6m57s
[11:37:22.271] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:5:0)
[11:37:22.288] Build status: STARTED
[11:37:22.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:6:0) after 5m40s
[11:43:02.405] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:6:0)
[11:43:02.424] Build status: STARTED
[11:43:02.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:7:0) after 4m59s
[11:48:01.665] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:7:0)
[11:48:01.685] Build status: STARTED
[11:48:01.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:8:0) after 5m37s
[11:53:38.814] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:8:0)
[11:53:38.832] Build status: STARTED
[11:53:38.832] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:9:0) after 7m26s
[12:01:04.980] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:9:0)
[12:01:05.032] Build status: STARTED
[12:01:05.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:10:0) after 6m3s
[12:07:08.064] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:10:0)
[12:07:08.128] Build status: STARTED
[12:07:08.128] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:11:0) after 2m53s
[12:10:01.220] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:11:0)
[12:10:01.271] Build status: STARTED
[12:10:01.271] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:12:0) after 6m43s
[12:16:44.286] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:12:0)
[12:16:44.362] Build status: STARTED
[12:16:44.362] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:13:0) after 5m53s
[12:22:37.703] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:13:0)
[12:22:37.717] Build status: STARTED
[12:22:37.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:14:0) after 6m39s
[12:29:16.765] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:14:0)
[12:29:16.781] Build status: STARTED
[12:29:16.781] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:15:0) after 8m32s
[12:37:48.823] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:15:0)
[12:37:48.886] Build status: STARTED
[12:37:48.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:16:0) after 9m55s
[12:47:43.918] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:16:0)
[12:47:43.933] Build status: STARTED
[12:47:43.933] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:17:0) after 4m37s
[12:52:21.170] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:17:0)
[12:52:21.219] Build status: STARTED
[12:52:21.219] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:18:0) after 4m19s
[12:56:40.311] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:18:0)
[12:56:40.348] Build status: STARTED
[12:56:40.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:19:0) after 1m18s
[12:57:58.414] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:19:0)
[12:57:58.498] Build status: STARTED
[12:57:58.498] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:20:0) after 9m50s
[13:07:48.512] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:20:0)
[13:07:48.583] Build status: STARTED
[13:07:48.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:21:0) after 1m49s
[13:09:37.600] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:21:0)
[13:09:37.613] Build status: STARTED
[13:09:37.614] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:22:0) after 3m46s
[13:13:23.630] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:22:0)
[13:13:23.652] Build status: STARTED
[13:13:23.652] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:23:0) after 9m38s
[13:23:02.309] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:23:0)
[13:23:02.333] Build status: STARTED
[13:23:02.333] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:24:0) after 2m32s
[13:25:34.423] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:24:0)
[13:25:34.440] Build status: STARTED
[13:25:34.440] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:25:0) after 8m18s
[13:33:52.453] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:25:0)
[13:33:52.479] Build status: STARTED
[13:33:52.479] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:26:0) after 2m9s
[13:36:01.491] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:26:0)
[13:36:01.547] Build status: STARTED
[13:36:01.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:27:0) after 9m7s
[13:45:08.835] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:27:0)
[13:45:08.853] Build status: STARTED
[13:45:08.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:28:0) after 8m52s
[13:54:00.960] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:28:0)
[13:54:01.018] Build status: STARTED
[13:54:01.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:29:0) after 5m49s
[13:59:50.383] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:29:0)
[13:59:50.407] Build status: STARTED
[13:59:50.407] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:30:0) after 5m27s
[14:05:17.425] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:30:0)
[14:05:17.479] Build status: STARTED
[14:05:17.479] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:31:0) after 4m56s
[14:10:13.549] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:31:0)
[14:10:13.574] Build status: STARTED
[14:10:13.574] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:32:0) after 5m32s
[14:15:45.918] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:32:0)
[14:15:46.102] Build status: STARTED
[14:15:46.102] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:33:0) after 6m53s
[14:22:39.111] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:33:0)
[14:22:39.127] Build status: STARTED
[14:22:39.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:34:0) after 7m7s
[14:29:46.130] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:34:0)
[14:29:46.149] Build status: STARTED
[14:29:46.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:35:0) after 1m33s
[14:31:19.162] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:35:0)
[14:31:19.181] Build status: STARTED
[14:31:19.181] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:36:0) after 3m34s
[14:34:53.186] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:36:0)
[14:34:53.202] Build status: STARTED
[14:34:53.202] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:37:0) after 1m25s
[14:36:18.217] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:37:0)
[14:36:18.263] Build status: STARTED
[14:36:18.263] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:38:0) after 6m52s
[14:43:10.278] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:38:0)
[14:43:10.298] Build status: STARTED
[14:43:10.298] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:39:0) after 3m38s
[14:46:48.317] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:39:0)
[14:46:48.340] Build status: STARTED
[14:46:48.340] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:40:0) after 8m11s
[14:54:59.395] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:40:0)
[14:54:59.430] Build status: STARTED
[14:54:59.430] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:41:0) after 7m18s
[15:02:17.646] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:41:0)
[15:02:17.667] Build status: STARTED
[15:02:17.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:42:0) after 3m48s
[15:06:05.684] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:42:0)
[15:06:05.770] Build status: STARTED
[15:06:05.770] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:43:0) after 4m10s
[15:10:15.860] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:43:0)
[15:10:15.885] Build status: STARTED
[15:10:15.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:44:0) after 3m35s
[15:13:51.090] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:44:0)
[15:13:51.111] Build status: STARTED
[15:13:51.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:45:0) after 1m45s
[15:15:36.161] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:45:0)
[15:15:36.181] Build status: STARTED
[15:15:36.181] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:46:0) after 8m2s
[15:23:38.219] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:46:0)
[15:23:38.235] Build status: STARTED
[15:23:38.235] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:47:0) after 8m44s
[15:32:22.252] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:47:0)
[15:32:22.270] Build status: STARTED
[15:32:22.270] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:48:0) after 6m34s
[15:38:57.051] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:48:0)
[15:38:57.069] Build status: STARTED
[15:38:57.069] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:49:0) after 7m19s
[15:46:16.323] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:49:0)
[15:46:16.343] Build status: STARTED
[15:46:16.343] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:50:0) after 7m45s
[15:54:01.555] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:50:0)
[15:54:01.579] Build status: STARTED
[15:54:01.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:51:0) after 6m38s
[16:00:39.705] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:51:0)
[16:00:39.730] Build status: STARTED
[16:00:39.730] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:52:0) after 7m49s
[16:08:28.901] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:52:0)
[16:08:28.918] Build status: STARTED
[16:08:28.918] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:53:0) after 1m12s
[16:09:41.341] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:53:0)
[16:09:41.368] Build status: STARTED
[16:09:41.368] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:54:0) after 6m54s
[16:16:35.583] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:54:0)
[16:16:35.601] Build status: STARTED
[16:16:35.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:55:0) after 9m56s
[16:26:31.637] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:55:0)
[16:26:31.660] Build status: STARTED
[16:26:31.660] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:56:0) after 3m27s
[16:29:58.754] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:56:0)
[16:29:58.802] Build status: STARTED
[16:29:58.802] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:57:0) after 5m15s
[16:35:14.059] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:57:0)
[16:35:14.103] Build status: STARTED
[16:35:14.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:58:0) after 9m28s
[16:44:42.294] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:58:0)
[16:44:42.358] Build status: STARTED
[16:44:42.358] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:59:0) after 5m39s
[16:50:21.426] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:59:0)
[16:50:21.446] Build status: STARTED
[16:50:21.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:60:0) after 2m41s
[16:53:02.465] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:60:0)
[16:53:02.478] Build status: STARTED
[16:53:02.478] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:61:0) after 6m46s
[16:59:48.459] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:61:0)
[16:59:48.478] Build status: STARTED
[16:59:48.478] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852153152775515152:62:0) after 3m47s
[16:59:58.738] Received PubSub notification, asking Buildbucket for the build status
[16:59:58.816] Build:
{
"id": "8686723090837597377",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWinArm64PGO"
},
"number": 7746,
"createdBy": "project:chromium",
"createTime": "2026-03-21T11:25:16.743601259Z",
"startTime": "2026-03-21T11:26:49.784193651Z",
"endTime": "2026-03-21T16:59:58.420061934Z",
"updateTime": "2026-03-21T16:59:58.420061934Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3e4c084b9c0cef43fda6eb3dc16d6bdfa937c9d8",
"ref": "refs/heads/main"
}
}
}
[16:59:58.816] Invocation finished in 5h34m43.30542122s with status FAILED