[17:35:52.610] New invocation is queued and will start shortly
[17:35:53.919] Starting the invocation (attempt 1)
[17:35:53.947] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/3d66375b5502340d3f62d396dc55eaa543b757c2"
[17:35:53.947] Popped gitiles commit info from properties and tags
[17:35:53.947] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[17:35:53.948] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[17:35:53.948] Buildbucket request:
{
"requestId": "8850680284037138816",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3d66375b5502340d3f62d396dc55eaa543b757c2",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850680284037138816"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[17:35:54.795] Scheduled build:
{
"id": "8685250221851881537",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 20224,
"createdBy": "project:chromium",
"createTime": "2026-04-06T17:35:54.066044099Z",
"updateTime": "2026-04-06T17:35:54.066044099Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3d66375b5502340d3f62d396dc55eaa543b757c2",
"ref": "refs/heads/main"
}
}
}
[17:35:54.795] Task URL: https://cr-buildbucket.appspot.com/build/8685250221851881537
[17:35:54.795] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:2:0) after 8m16s
[17:36:14.499] Received PubSub notification, asking Buildbucket for the build status
[17:36:14.528] Build status: SCHEDULED
[17:36:45.434] Received PubSub notification, asking Buildbucket for the build status
[17:36:45.493] Build status: STARTED
[17:44:10.774] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:2:0)
[17:44:10.801] Build status: STARTED
[17:44:10.801] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:5:0) after 2m9s
[17:46:19.867] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:5:0)
[17:46:19.888] Build status: STARTED
[17:46:19.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:6:0) after 5m0s
[17:51:20.009] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:6:0)
[17:51:20.032] Build status: STARTED
[17:51:20.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:7:0) after 9m39s
[18:00:59.253] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:7:0)
[18:00:59.281] Build status: STARTED
[18:00:59.281] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:8:0) after 7m10s
[18:08:09.310] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:8:0)
[18:08:09.338] Build status: STARTED
[18:08:09.338] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:9:0) after 7m41s
[18:15:50.357] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:9:0)
[18:15:50.376] Build status: STARTED
[18:15:50.376] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:10:0) after 1m57s
[18:17:47.426] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:10:0)
[18:17:47.461] Build status: STARTED
[18:17:47.461] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:11:0) after 3m43s
[18:21:30.510] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:11:0)
[18:21:30.547] Build status: STARTED
[18:21:30.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:12:0) after 3m8s
[18:24:38.568] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:12:0)
[18:24:38.587] Build status: STARTED
[18:24:38.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:13:0) after 8m35s
[18:33:13.694] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:13:0)
[18:33:13.739] Build status: STARTED
[18:33:13.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:14:0) after 4m19s
[18:37:32.803] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:14:0)
[18:37:32.818] Build status: STARTED
[18:37:32.818] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:15:0) after 6m23s
[18:43:55.861] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:15:0)
[18:43:55.884] Build status: STARTED
[18:43:55.884] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:16:0) after 4m26s
[18:48:21.970] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:16:0)
[18:48:21.991] Build status: STARTED
[18:48:21.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:17:0) after 4m17s
[18:52:38.965] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:17:0)
[18:52:38.980] Build status: STARTED
[18:52:38.980] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:18:0) after 7m31s
[19:00:10.074] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:18:0)
[19:00:10.121] Build status: STARTED
[19:00:10.121] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:19:0) after 9m44s
[19:09:54.216] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:19:0)
[19:09:54.242] Build status: STARTED
[19:09:54.242] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:20:0) after 9m30s
[19:19:24.305] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:20:0)
[19:19:24.328] Build status: STARTED
[19:19:24.328] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:21:0) after 6m17s
[19:25:41.345] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:21:0)
[19:25:41.367] Build status: STARTED
[19:25:41.367] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:22:0) after 8m34s
[19:34:15.471] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:22:0)
[19:34:15.491] Build status: STARTED
[19:34:15.491] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:23:0) after 7m7s
[19:41:22.510] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:23:0)
[19:41:22.527] Build status: STARTED
[19:41:22.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:24:0) after 7m26s
[19:48:48.598] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:24:0)
[19:48:48.622] Build status: STARTED
[19:48:48.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:25:0) after 2m9s
[19:50:57.724] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:25:0)
[19:50:57.743] Build status: STARTED
[19:50:57.743] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:26:0) after 9m18s
[20:00:15.767] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:26:0)
[20:00:15.788] Build status: STARTED
[20:00:15.788] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:27:0) after 2m45s
[20:03:00.805] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:27:0)
[20:03:00.888] Build status: STARTED
[20:03:00.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:28:0) after 9m2s
[20:12:02.938] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:28:0)
[20:12:02.958] Build status: STARTED
[20:12:02.958] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:29:0) after 1m31s
[20:13:33.976] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:29:0)
[20:13:33.996] Build status: STARTED
[20:13:33.996] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:30:0) after 9m22s
[20:22:56.067] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:30:0)
[20:22:56.090] Build status: STARTED
[20:22:56.090] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:31:0) after 4m16s
[20:27:12.158] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:31:0)
[20:27:12.232] Build status: STARTED
[20:27:12.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:32:0) after 5m22s
[20:32:34.254] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:32:0)
[20:32:34.270] Build status: STARTED
[20:32:34.270] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:33:0) after 5m17s
[20:37:51.366] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:33:0)
[20:37:51.385] Build status: STARTED
[20:37:51.385] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:34:0) after 5m6s
[20:42:57.413] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:34:0)
[20:42:57.549] Build status: STARTED
[20:42:57.549] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:35:0) after 3m25s
[20:46:22.564] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:35:0)
[20:46:22.586] Build status: STARTED
[20:46:22.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:36:0) after 9m28s
[20:55:50.718] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:36:0)
[20:55:50.737] Build status: STARTED
[20:55:50.737] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:37:0) after 6m50s
[21:02:40.757] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:37:0)
[21:02:40.776] Build status: STARTED
[21:02:40.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:38:0) after 6m28s
[21:09:08.858] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:38:0)
[21:09:08.887] Build status: STARTED
[21:09:08.887] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:39:0) after 2m28s
[21:11:36.909] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:39:0)
[21:11:36.927] Build status: STARTED
[21:11:36.927] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:40:0) after 2m18s
[21:13:54.943] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:40:0)
[21:13:55.048] Build status: STARTED
[21:13:55.049] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:41:0) after 9m50s
[21:23:45.027] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:41:0)
[21:23:45.046] Build status: STARTED
[21:23:45.046] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:42:0) after 5m31s
[21:29:16.114] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:42:0)
[21:29:16.130] Build status: STARTED
[21:29:16.130] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:43:0) after 3m52s
[21:33:08.183] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:43:0)
[21:33:08.234] Build status: STARTED
[21:33:08.234] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:44:0) after 3m31s
[21:36:39.325] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:44:0)
[21:36:39.348] Build status: STARTED
[21:36:39.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:45:0) after 1m9s
[21:37:48.420] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:45:0)
[21:37:48.457] Build status: STARTED
[21:37:48.457] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:46:0) after 8m58s
[21:46:46.474] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:46:0)
[21:46:46.490] Build status: STARTED
[21:46:46.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:47:0) after 4m48s
[21:51:34.557] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:47:0)
[21:51:34.575] Build status: STARTED
[21:51:34.575] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:48:0) after 5m0s
[21:56:34.644] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:48:0)
[21:56:34.690] Build status: STARTED
[21:56:34.690] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:49:0) after 4m59s
[22:01:33.708] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:49:0)
[22:01:33.727] Build status: STARTED
[22:01:33.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:50:0) after 3m35s
[22:05:08.703] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:50:0)
[22:05:08.722] Build status: STARTED
[22:05:08.722] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:51:0) after 2m38s
[22:07:46.846] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:51:0)
[22:07:46.899] Build status: STARTED
[22:07:46.899] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:52:0) after 3m13s
[22:10:59.953] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:52:0)
[22:11:00.072] Build status: STARTED
[22:11:00.072] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:53:0) after 1m21s
[22:12:21.149] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:53:0)
[22:12:21.166] Build status: STARTED
[22:12:21.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:54:0) after 9m37s
[22:21:58.239] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:54:0)
[22:21:58.341] Build status: STARTED
[22:21:58.341] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:55:0) after 5m28s
[22:27:26.412] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:55:0)
[22:27:26.433] Build status: STARTED
[22:27:26.433] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:56:0) after 4m46s
[22:32:12.450] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:56:0)
[22:32:12.478] Build status: STARTED
[22:32:12.478] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:57:0) after 3m51s
[22:36:03.494] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:57:0)
[22:36:03.515] Build status: STARTED
[22:36:03.516] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:58:0) after 4m39s
[22:40:42.571] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:58:0)
[22:40:42.587] Build status: STARTED
[22:40:42.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:59:0) after 9m26s
[22:50:08.604] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:59:0)
[22:50:08.707] Build status: STARTED
[22:50:08.707] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:60:0) after 8m4s
[02:06:26.787] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:60:0)
[02:06:26.820] Build status: STARTED
[02:06:26.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:61:0) after 8m35s
[02:15:01.943] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:61:0)
[02:15:01.958] Build status: STARTED
[02:15:01.958] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850680284037138816:62:0) after 1m33s
[02:16:30.415] Received PubSub notification, asking Buildbucket for the build status
[02:16:30.436] Build:
{
"id": "8685250221851881537",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 20224,
"createdBy": "project:chromium",
"createTime": "2026-04-06T17:35:54.066044099Z",
"startTime": "2026-04-06T17:36:45.266813058Z",
"endTime": "2026-04-07T02:16:29.950872995Z",
"updateTime": "2026-04-07T02:16:29.950872995Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3d66375b5502340d3f62d396dc55eaa543b757c2",
"ref": "refs/heads/main"
}
}
}
[02:16:30.436] Invocation finished in 8h40m37.83745399s with status FAILED