[17:00:01.046] New invocation is queued and will start shortly
[17:00:02.816] Starting the invocation (attempt 1)
[17:00:02.842] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/e0216542e52b691df8ec0c71508a49d344173b25"
[17:00:02.842] Popped gitiles commit info from properties and tags
[17:00:02.842] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[17:00:02.843] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[17:00:02.843] Buildbucket request:
{
"requestId": "8852132091572786640",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWinArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e0216542e52b691df8ec0c71508a49d344173b25",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8852132091572786640"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTWinArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[17:00:03.468] Scheduled build:
{
"id": "8686702028989384305",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWinArm64PGO"
},
"number": 7747,
"createdBy": "project:chromium",
"createTime": "2026-03-21T17:00:02.886154524Z",
"updateTime": "2026-03-21T17:00:02.886154524Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e0216542e52b691df8ec0c71508a49d344173b25",
"ref": "refs/heads/main"
}
}
}
[17:00:03.468] Task URL: https://cr-buildbucket.appspot.com/build/8686702028989384305
[17:00:03.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:2:0) after 1m25s
[17:00:04.405] Received PubSub notification, asking Buildbucket for the build status
[17:00:04.423] Build status: SCHEDULED
[17:00:08.615] Received PubSub notification, asking Buildbucket for the build status
[17:00:08.632] Build status: STARTED
[17:01:28.604] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:2:0)
[17:01:28.621] Build status: STARTED
[17:01:28.621] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:5:0) after 4m53s
[17:06:21.881] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:5:0)
[17:06:21.923] Build status: STARTED
[17:06:21.923] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:6:0) after 6m56s
[17:13:17.951] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:6:0)
[17:13:17.974] Build status: STARTED
[17:13:17.974] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:7:0) after 4m49s
[17:18:07.101] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:7:0)
[17:18:07.119] Build status: STARTED
[17:18:07.119] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:8:0) after 3m22s
[17:21:29.153] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:8:0)
[17:21:29.175] Build status: STARTED
[17:21:29.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:9:0) after 3m22s
[17:24:51.289] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:9:0)
[17:24:51.309] Build status: STARTED
[17:24:51.309] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:10:0) after 6m28s
[17:31:19.380] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:10:0)
[17:31:19.403] Build status: STARTED
[17:31:19.403] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:11:0) after 7m41s
[17:39:00.696] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:11:0)
[17:39:00.716] Build status: STARTED
[17:39:00.716] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:12:0) after 1m7s
[17:40:07.774] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:12:0)
[17:40:07.794] Build status: STARTED
[17:40:07.794] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:13:0) after 1m50s
[17:41:57.832] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:13:0)
[17:41:57.876] Build status: STARTED
[17:41:57.876] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:14:0) after 7m50s
[17:49:47.922] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:14:0)
[17:49:47.939] Build status: STARTED
[17:49:47.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:15:0) after 9m12s
[17:58:59.968] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:15:0)
[17:58:59.985] Build status: STARTED
[17:58:59.985] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:16:0) after 2m19s
[18:01:19.004] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:16:0)
[18:01:19.087] Build status: STARTED
[18:01:19.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:17:0) after 8m11s
[18:09:30.087] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:17:0)
[18:09:30.100] Build status: STARTED
[18:09:30.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:18:0) after 7m19s
[18:16:49.112] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:18:0)
[18:16:49.128] Build status: STARTED
[18:16:49.128] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:19:0) after 2m54s
[18:19:43.193] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:19:0)
[18:19:43.207] Build status: STARTED
[18:19:43.207] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:20:0) after 6m49s
[18:26:32.225] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:20:0)
[18:26:32.248] Build status: STARTED
[18:26:32.248] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:21:0) after 4m50s
[18:31:22.297] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:21:0)
[18:31:22.313] Build status: STARTED
[18:31:22.313] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:22:0) after 7m1s
[18:38:23.427] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:22:0)
[18:38:23.444] Build status: STARTED
[18:38:23.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:23:0) after 3m30s
[18:41:53.514] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:23:0)
[18:41:53.531] Build status: STARTED
[18:41:53.531] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:24:0) after 9m51s
[18:51:44.608] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:24:0)
[18:51:44.640] Build status: STARTED
[18:51:44.641] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:25:0) after 9m50s
[19:01:34.850] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:25:0)
[19:01:34.870] Build status: STARTED
[19:01:34.870] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:26:0) after 2m49s
[19:04:24.038] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:26:0)
[19:04:24.055] Build status: STARTED
[19:04:24.055] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:27:0) after 1m38s
[19:06:02.430] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:27:0)
[19:06:02.473] Build status: STARTED
[19:06:02.474] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:28:0) after 6m38s
[19:12:40.521] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:28:0)
[19:12:40.545] Build status: STARTED
[19:12:40.546] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:29:0) after 1m38s
[19:14:19.104] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:29:0)
[19:14:19.123] Build status: STARTED
[19:14:19.123] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:30:0) after 4m31s
[19:18:50.194] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:30:0)
[19:18:50.210] Build status: STARTED
[19:18:50.210] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:31:0) after 4m0s
[19:22:50.246] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:31:0)
[19:22:50.362] Build status: STARTED
[19:22:50.362] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:32:0) after 6m30s
[19:29:20.825] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:32:0)
[19:29:20.841] Build status: STARTED
[19:29:20.841] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:33:0) after 2m52s
[19:32:12.908] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:33:0)
[19:32:12.932] Build status: STARTED
[19:32:12.932] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:34:0) after 8m37s
[19:40:50.074] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:34:0)
[19:40:50.117] Build status: STARTED
[19:40:50.117] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:35:0) after 4m32s
[19:45:22.133] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:35:0)
[19:45:22.156] Build status: STARTED
[19:45:22.156] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:36:0) after 8m36s
[19:53:58.208] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:36:0)
[19:53:58.225] Build status: STARTED
[19:53:58.225] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:37:0) after 1m11s
[19:55:09.260] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:37:0)
[19:55:09.277] Build status: STARTED
[19:55:09.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:38:0) after 9m17s
[20:04:26.489] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:38:0)
[20:04:26.517] Build status: STARTED
[20:04:26.518] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:39:0) after 3m14s
[20:07:40.491] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:39:0)
[20:07:40.509] Build status: STARTED
[20:07:40.509] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:40:0) after 2m26s
[20:10:06.626] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:40:0)
[20:10:06.678] Build status: STARTED
[20:10:06.678] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:41:0) after 3m53s
[20:13:59.757] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:41:0)
[20:13:59.777] Build status: STARTED
[20:13:59.777] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:42:0) after 9m32s
[20:23:31.809] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:42:0)
[20:23:31.829] Build status: STARTED
[20:23:31.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:43:0) after 9m50s
[20:33:22.355] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:43:0)
[20:33:22.375] Build status: STARTED
[20:33:22.375] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:44:0) after 8m26s
[20:41:48.401] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:44:0)
[20:41:48.419] Build status: STARTED
[20:41:48.419] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:45:0) after 1m22s
[20:43:10.554] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:45:0)
[20:43:10.572] Build status: STARTED
[20:43:10.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:46:0) after 7m6s
[20:50:16.598] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:46:0)
[20:50:16.614] Build status: STARTED
[20:50:16.614] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:47:0) after 8m22s
[20:58:38.957] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:47:0)
[20:58:38.978] Build status: STARTED
[20:58:38.978] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:48:0) after 7m36s
[21:06:14.996] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:48:0)
[21:06:15.080] Build status: STARTED
[21:06:15.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:49:0) after 7m31s
[21:13:46.165] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:49:0)
[21:13:46.185] Build status: STARTED
[21:13:46.185] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:50:0) after 8m36s
[21:22:22.396] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:50:0)
[21:22:22.412] Build status: STARTED
[21:22:22.412] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:51:0) after 8m57s
[21:31:19.565] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:51:0)
[21:31:19.620] Build status: STARTED
[21:31:19.620] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:52:0) after 4m52s
[21:36:12.059] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:52:0)
[21:36:12.094] Build status: STARTED
[21:36:12.094] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:53:0) after 4m30s
[21:40:42.465] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:53:0)
[21:40:42.483] Build status: STARTED
[21:40:42.483] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:54:0) after 2m56s
[21:43:38.499] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:54:0)
[21:43:38.516] Build status: STARTED
[21:43:38.516] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:55:0) after 5m40s
[21:49:18.521] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:55:0)
[21:49:18.541] Build status: STARTED
[21:49:18.541] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:56:0) after 6m24s
[21:55:42.648] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:56:0)
[21:55:42.697] Build status: STARTED
[21:55:42.697] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:57:0) after 2m11s
[21:57:53.734] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:57:0)
[21:57:53.753] Build status: STARTED
[21:57:53.753] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:58:0) after 1m51s
[21:59:44.887] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:58:0)
[21:59:44.914] Build status: STARTED
[21:59:44.914] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:59:0) after 6m54s
[22:06:39.176] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:59:0)
[22:06:39.202] Build status: STARTED
[22:06:39.202] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:60:0) after 6m57s
[22:13:36.315] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:60:0)
[22:13:36.352] Build status: STARTED
[22:13:36.352] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852132091572786640:61:0) after 8m18s
[22:18:35.822] Received PubSub notification, asking Buildbucket for the build status
[22:18:35.876] Build:
{
"id": "8686702028989384305",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWinArm64PGO"
},
"number": 7747,
"createdBy": "project:chromium",
"createTime": "2026-03-21T17:00:02.886154524Z",
"startTime": "2026-03-21T17:00:08.345937319Z",
"endTime": "2026-03-21T22:18:30.515011908Z",
"updateTime": "2026-03-21T22:18:30.515011908Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e0216542e52b691df8ec0c71508a49d344173b25",
"ref": "refs/heads/main"
}
}
}
[22:18:35.876] Invocation finished in 5h18m34.837085201s with status FAILED