[13:46:46.345] New invocation is queued and will start shortly
[13:46:47.432] Starting the invocation (attempt 1)
[13:46:47.453] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0ae4bdbe1b8c6ed74d0acbb5a4455800746eb644"
[13:46:47.453] Popped gitiles commit info from properties and tags
[13:46:47.453] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[13:46:47.453] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[13:46:47.454] Buildbucket request:
{
"requestId": "8850875891968989088",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0ae4bdbe1b8c6ed74d0acbb5a4455800746eb644",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850875891968989088"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[13:46:47.957] Scheduled build:
{
"id": "8685445830093300193",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 20213,
"createdBy": "project:chromium",
"createTime": "2026-04-04T13:46:47.510881339Z",
"updateTime": "2026-04-04T13:46:47.510881339Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0ae4bdbe1b8c6ed74d0acbb5a4455800746eb644",
"ref": "refs/heads/main"
}
}
}
[13:46:47.957] Task URL: https://cr-buildbucket.appspot.com/build/8685445830093300193
[13:46:47.957] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:2:0) after 2m31s
[13:47:10.275] Received PubSub notification, asking Buildbucket for the build status
[13:47:10.295] Build status: SCHEDULED
[13:47:33.713] Received PubSub notification, asking Buildbucket for the build status
[13:47:33.738] Build status: STARTED
[13:49:18.968] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:2:0)
[13:49:18.983] Build status: STARTED
[13:49:18.983] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:5:0) after 2m25s
[13:51:44.019] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:5:0)
[13:51:44.057] Build status: STARTED
[13:51:44.057] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:6:0) after 4m27s
[13:56:11.070] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:6:0)
[13:56:11.145] Build status: STARTED
[13:56:11.145] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:7:0) after 5m47s
[14:01:58.161] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:7:0)
[14:01:58.222] Build status: STARTED
[14:01:58.222] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:8:0) after 7m25s
[14:09:23.557] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:8:0)
[14:09:23.576] Build status: STARTED
[14:09:23.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:9:0) after 4m51s
[14:14:14.881] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:9:0)
[14:14:14.909] Build status: STARTED
[14:14:14.909] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:10:0) after 4m39s
[14:18:54.027] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:10:0)
[14:18:54.042] Build status: STARTED
[14:18:54.042] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:11:0) after 1m17s
[14:20:11.164] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:11:0)
[14:20:11.247] Build status: STARTED
[14:20:11.247] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:12:0) after 4m2s
[14:24:13.277] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:12:0)
[14:24:13.292] Build status: STARTED
[14:24:13.292] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:13:0) after 4m47s
[14:29:00.292] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:13:0)
[14:29:00.329] Build status: STARTED
[14:29:00.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:14:0) after 9m3s
[14:38:03.349] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:14:0)
[14:38:03.370] Build status: STARTED
[14:38:03.370] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:15:0) after 1m42s
[14:39:45.488] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:15:0)
[14:39:45.512] Build status: STARTED
[14:39:45.512] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:16:0) after 7m37s
[14:47:22.785] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:16:0)
[14:47:22.875] Build status: STARTED
[14:47:22.875] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:17:0) after 8m7s
[14:55:30.051] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:17:0)
[14:55:30.067] Build status: STARTED
[14:55:30.067] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:18:0) after 1m28s
[14:56:58.141] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:18:0)
[14:56:58.158] Build status: STARTED
[14:56:58.158] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:19:0) after 1m48s
[14:58:46.174] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:19:0)
[14:58:46.222] Build status: STARTED
[14:58:46.222] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:20:0) after 7m12s
[15:05:58.267] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:20:0)
[15:05:58.281] Build status: STARTED
[15:05:58.281] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:21:0) after 6m9s
[15:12:07.662] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:21:0)
[15:12:07.675] Build status: STARTED
[15:12:07.675] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:22:0) after 9m20s
[15:21:27.701] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:22:0)
[15:21:27.774] Build status: STARTED
[15:21:27.774] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:23:0) after 8m55s
[15:30:23.262] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:23:0)
[15:30:23.282] Build status: STARTED
[15:30:23.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:24:0) after 6m31s
[15:36:54.388] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:24:0)
[15:36:54.475] Build status: STARTED
[15:36:54.475] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:25:0) after 3m8s
[15:40:02.520] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:25:0)
[15:40:02.536] Build status: STARTED
[15:40:02.536] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:26:0) after 1m36s
[15:41:38.550] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:26:0)
[15:41:38.563] Build status: STARTED
[15:41:38.563] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:27:0) after 1m28s
[15:43:06.590] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:27:0)
[15:43:06.622] Build status: STARTED
[15:43:06.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:28:0) after 8m13s
[15:51:19.696] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:28:0)
[15:51:19.714] Build status: STARTED
[15:51:19.714] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:29:0) after 9m53s
[16:01:12.782] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:29:0)
[16:01:12.801] Build status: STARTED
[16:01:12.801] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:30:0) after 5m26s
[16:06:38.831] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:30:0)
[16:06:38.859] Build status: STARTED
[16:06:38.859] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:31:0) after 8m22s
[16:15:00.874] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:31:0)
[16:15:00.911] Build status: STARTED
[16:15:00.911] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:32:0) after 7m39s
[16:22:39.934] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:32:0)
[16:22:40.007] Build status: STARTED
[16:22:40.007] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:33:0) after 7m0s
[16:29:40.062] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:33:0)
[16:29:40.093] Build status: STARTED
[16:29:40.093] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:34:0) after 9m21s
[16:39:01.207] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:34:0)
[16:39:01.224] Build status: STARTED
[16:39:01.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:35:0) after 2m55s
[16:41:56.240] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:35:0)
[16:41:56.262] Build status: STARTED
[16:41:56.262] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:36:0) after 4m22s
[16:46:18.423] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:36:0)
[16:46:18.454] Build status: STARTED
[16:46:18.454] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:37:0) after 8m7s
[16:54:25.469] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:37:0)
[16:54:25.484] Build status: STARTED
[16:54:25.484] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:38:0) after 1m36s
[16:56:01.502] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:38:0)
[16:56:01.530] Build status: STARTED
[16:56:01.530] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:39:0) after 6m5s
[17:02:06.547] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:39:0)
[17:02:06.564] Build status: STARTED
[17:02:06.564] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:40:0) after 4m5s
[17:06:11.684] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:40:0)
[17:06:11.702] Build status: STARTED
[17:06:11.702] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:41:0) after 2m32s
[17:08:43.786] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:41:0)
[17:08:43.807] Build status: STARTED
[17:08:43.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:42:0) after 7m16s
[17:15:59.823] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:42:0)
[17:15:59.846] Build status: STARTED
[17:15:59.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:43:0) after 6m42s
[17:22:41.864] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:43:0)
[17:22:41.941] Build status: STARTED
[17:22:41.941] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:44:0) after 2m55s
[17:25:37.069] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:44:0)
[17:25:37.134] Build status: STARTED
[17:25:37.134] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:45:0) after 6m21s
[17:31:58.149] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:45:0)
[17:31:58.167] Build status: STARTED
[17:31:58.167] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:46:0) after 6m0s
[17:37:58.285] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:46:0)
[17:37:58.301] Build status: STARTED
[17:37:58.301] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:47:0) after 7m49s
[17:45:47.320] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:47:0)
[17:45:47.342] Build status: STARTED
[17:45:47.342] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:48:0) after 7m40s
[17:53:27.359] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:48:0)
[17:53:27.440] Build status: STARTED
[17:53:27.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:49:0) after 8m12s
[18:01:39.459] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:49:0)
[18:01:39.480] Build status: STARTED
[18:01:39.480] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:50:0) after 1m35s
[18:03:14.526] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:50:0)
[18:03:14.546] Build status: STARTED
[18:03:14.546] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:51:0) after 3m54s
[18:07:08.688] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:51:0)
[18:07:08.704] Build status: STARTED
[18:07:08.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:52:0) after 2m57s
[18:10:05.852] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:52:0)
[18:10:05.876] Build status: STARTED
[18:10:05.876] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:53:0) after 1m13s
[18:11:18.892] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:53:0)
[18:11:18.910] Build status: STARTED
[18:11:18.910] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:54:0) after 9m27s
[18:20:46.141] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:54:0)
[18:20:46.158] Build status: STARTED
[18:20:46.158] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850875891968989088:55:0) after 8m36s
[18:24:51.532] Received PubSub notification, asking Buildbucket for the build status
[18:24:51.557] Build:
{
"id": "8685445830093300193",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 20213,
"createdBy": "project:chromium",
"createTime": "2026-04-04T13:46:47.510881339Z",
"startTime": "2026-04-04T13:47:20.602670517Z",
"endTime": "2026-04-04T18:24:51.240863188Z",
"updateTime": "2026-04-04T18:24:51.240863188Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0ae4bdbe1b8c6ed74d0acbb5a4455800746eb644",
"ref": "refs/heads/main"
}
}
}
[18:24:51.557] Invocation finished in 4h38m5.22041209s with status FAILED