[08:11:44.049] New invocation is queued and will start shortly
[08:11:45.113] Starting the invocation (attempt 1)
[08:11:45.143] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/c52ae89ece584a8ee0bcb57eb703ae450592811d"
[08:11:45.143] Popped gitiles commit info from properties and tags
[08:11:45.143] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[08:11:45.143] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[08:11:45.143] Buildbucket request:
{
"requestId": "8850715776818117040",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c52ae89ece584a8ee0bcb57eb703ae450592811d",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850715776818117040"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[08:11:45.738] Scheduled build:
{
"id": "8685285714904346449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 20222,
"createdBy": "project:chromium",
"createTime": "2026-04-06T08:11:45.253198868Z",
"updateTime": "2026-04-06T08:11:45.253198868Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c52ae89ece584a8ee0bcb57eb703ae450592811d",
"ref": "refs/heads/main"
}
},
"canary": true
}
[08:11:45.738] Task URL: https://cr-buildbucket.appspot.com/build/8685285714904346449
[08:11:45.738] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:2:0) after 9m1s
[08:11:46.705] Received PubSub notification, asking Buildbucket for the build status
[08:11:46.722] Build status: SCHEDULED
[08:11:51.275] Received PubSub notification, asking Buildbucket for the build status
[08:11:51.484] Build status: STARTED
[08:20:47.013] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:2:0)
[08:20:47.082] Build status: STARTED
[08:20:47.082] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:5:0) after 9m6s
[08:29:53.276] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:5:0)
[08:29:53.297] Build status: STARTED
[08:29:53.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:6:0) after 9m4s
[08:38:57.412] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:6:0)
[08:38:57.429] Build status: STARTED
[08:38:57.429] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:7:0) after 8m12s
[08:47:09.459] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:7:0)
[08:47:09.560] Build status: STARTED
[08:47:09.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:8:0) after 9m5s
[08:56:14.575] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:8:0)
[08:56:14.590] Build status: STARTED
[08:56:14.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:9:0) after 2m53s
[08:59:07.661] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:9:0)
[08:59:07.747] Build status: STARTED
[08:59:07.748] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:10:0) after 3m13s
[09:02:20.761] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:10:0)
[09:02:20.786] Build status: STARTED
[09:02:20.786] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:11:0) after 7m35s
[09:09:55.799] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:11:0)
[09:09:55.820] Build status: STARTED
[09:09:55.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:12:0) after 2m25s
[09:12:20.864] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:12:0)
[09:12:21.040] Build status: STARTED
[09:12:21.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:13:0) after 7m58s
[09:20:19.057] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:13:0)
[09:20:19.076] Build status: STARTED
[09:20:19.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:14:0) after 1m2s
[09:21:21.559] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:14:0)
[09:21:21.577] Build status: STARTED
[09:21:21.577] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:15:0) after 4m43s
[09:26:04.597] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:15:0)
[09:26:04.619] Build status: STARTED
[09:26:04.619] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:16:0) after 4m16s
[09:30:20.776] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:16:0)
[09:30:20.818] Build status: STARTED
[09:30:20.818] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:17:0) after 1m12s
[09:31:33.065] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:17:0)
[09:31:33.117] Build status: STARTED
[09:31:33.117] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:18:0) after 4m25s
[09:35:58.393] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:18:0)
[09:35:58.440] Build status: STARTED
[09:35:58.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:19:0) after 8m29s
[09:44:27.647] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:19:0)
[09:44:27.676] Build status: STARTED
[09:44:27.676] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:20:0) after 9m30s
[09:53:57.690] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:20:0)
[09:53:57.714] Build status: STARTED
[09:53:57.715] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:21:0) after 6m43s
[10:00:40.830] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:21:0)
[10:00:40.848] Build status: STARTED
[10:00:40.848] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:22:0) after 3m14s
[10:03:55.130] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:22:0)
[10:03:55.256] Build status: STARTED
[10:03:55.257] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:23:0) after 7m31s
[10:11:26.415] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:23:0)
[10:11:26.429] Build status: STARTED
[10:11:26.429] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:24:0) after 8m43s
[10:20:09.459] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:24:0)
[10:20:09.475] Build status: STARTED
[10:20:09.475] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:25:0) after 2m44s
[10:22:53.573] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:25:0)
[10:22:53.641] Build status: STARTED
[10:22:53.641] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:26:0) after 8m22s
[10:31:15.849] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:26:0)
[10:31:15.867] Build status: STARTED
[10:31:15.867] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:27:0) after 9m50s
[10:41:05.979] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:27:0)
[10:41:05.997] Build status: STARTED
[10:41:05.997] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:28:0) after 8m3s
[10:49:09.016] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:28:0)
[10:49:09.038] Build status: STARTED
[10:49:09.038] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:29:0) after 2m28s
[10:51:37.123] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:29:0)
[10:51:37.141] Build status: STARTED
[10:51:37.141] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:30:0) after 8m9s
[10:59:46.156] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:30:0)
[10:59:46.170] Build status: STARTED
[10:59:46.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:31:0) after 9m11s
[11:08:57.591] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:31:0)
[11:08:57.693] Build status: STARTED
[11:08:57.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:32:0) after 2m13s
[11:11:10.801] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:32:0)
[11:11:10.883] Build status: STARTED
[11:11:10.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:33:0) after 7m25s
[11:18:35.999] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:33:0)
[11:18:36.087] Build status: STARTED
[11:18:36.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:34:0) after 7m20s
[11:25:56.188] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:34:0)
[11:25:56.203] Build status: STARTED
[11:25:56.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:35:0) after 9m57s
[11:35:53.480] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:35:0)
[11:35:53.509] Build status: STARTED
[11:35:53.509] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:36:0) after 1m15s
[11:37:08.712] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:36:0)
[11:37:08.788] Build status: STARTED
[11:37:08.788] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:37:0) after 3m29s
[11:40:37.849] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:37:0)
[11:40:37.877] Build status: STARTED
[11:40:37.877] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:38:0) after 3m12s
[11:43:49.916] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:38:0)
[11:43:49.938] Build status: STARTED
[11:43:49.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:39:0) after 5m2s
[11:48:51.963] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:39:0)
[11:48:51.976] Build status: STARTED
[11:48:51.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:40:0) after 7m2s
[11:55:54.049] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:40:0)
[11:55:54.152] Build status: STARTED
[11:55:54.152] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:41:0) after 2m40s
[11:58:34.168] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:41:0)
[11:58:34.208] Build status: STARTED
[11:58:34.208] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:42:0) after 6m44s
[12:05:18.258] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:42:0)
[12:05:18.308] Build status: STARTED
[12:05:18.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:43:0) after 7m53s
[12:13:11.335] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:43:0)
[12:13:11.449] Build status: STARTED
[12:13:11.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:44:0) after 2m41s
[12:15:52.473] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:44:0)
[12:15:52.491] Build status: STARTED
[12:15:52.491] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:45:0) after 1m6s
[12:16:58.486] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:45:0)
[12:16:58.505] Build status: STARTED
[12:16:58.505] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:46:0) after 2m45s
[12:19:43.755] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:46:0)
[12:19:43.772] Build status: STARTED
[12:19:43.772] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:47:0) after 3m51s
[12:23:34.789] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:47:0)
[12:23:34.833] Build status: STARTED
[12:23:34.833] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:48:0) after 2m50s
[12:26:24.857] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:48:0)
[12:26:24.881] Build status: STARTED
[12:26:24.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:49:0) after 1m59s
[12:28:23.961] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:49:0)
[12:28:23.979] Build status: STARTED
[12:28:23.979] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:50:0) after 8m43s
[12:37:07.095] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:50:0)
[12:37:07.114] Build status: STARTED
[12:37:07.114] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:51:0) after 5m15s
[12:42:22.112] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:51:0)
[12:42:22.130] Build status: STARTED
[12:42:22.130] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:52:0) after 1m28s
[12:43:50.146] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:52:0)
[12:43:50.176] Build status: STARTED
[12:43:50.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:53:0) after 1m36s
[12:45:26.194] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:53:0)
[12:45:26.219] Build status: STARTED
[12:45:26.219] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:54:0) after 5m4s
[12:50:30.337] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:54:0)
[12:50:30.357] Build status: STARTED
[12:50:30.357] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:55:0) after 2m32s
[12:53:02.386] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:55:0)
[12:53:02.401] Build status: STARTED
[12:53:02.401] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:56:0) after 4m58s
[12:58:00.423] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:56:0)
[12:58:00.469] Build status: STARTED
[12:58:00.469] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850715776818117040:57:0) after 5m47s
[12:58:11.149] Received PubSub notification, asking Buildbucket for the build status
[12:58:11.166] Build:
{
"id": "8685285714904346449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 20222,
"createdBy": "project:chromium",
"createTime": "2026-04-06T08:11:45.253198868Z",
"startTime": "2026-04-06T08:11:51.057304737Z",
"endTime": "2026-04-06T12:58:10.642776249Z",
"updateTime": "2026-04-06T12:58:10.642776249Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c52ae89ece584a8ee0bcb57eb703ae450592811d",
"ref": "refs/heads/main"
}
},
"canary": true
}
[12:58:11.166] Invocation finished in 4h46m27.122867426s with status FAILED