[22:29:36.794] New invocation is queued and will start shortly
[22:29:37.909] Starting the invocation (attempt 1)
[22:29:37.934] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d4e77e375b211d179f0e3fed6b867ac9e13dd22e"
[22:29:37.934] Popped gitiles commit info from properties and tags
[22:29:37.934] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[22:29:37.937] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[22:29:37.937] Buildbucket request:
{
"requestId": "8850752400698991776",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d4e77e375b211d179f0e3fed6b867ac9e13dd22e",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850752400698991776"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[22:29:38.486] Scheduled build:
{
"id": "8685322338783404737",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 20220,
"createdBy": "project:chromium",
"createTime": "2026-04-05T22:29:38.000538747Z",
"updateTime": "2026-04-05T22:29:38.000538747Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d4e77e375b211d179f0e3fed6b867ac9e13dd22e",
"ref": "refs/heads/main"
}
}
}
[22:29:38.486] Task URL: https://cr-buildbucket.appspot.com/build/8685322338783404737
[22:29:38.486] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:2:0) after 8m43s
[22:29:39.429] Received PubSub notification, asking Buildbucket for the build status
[22:29:39.449] Build status: SCHEDULED
[22:29:43.301] Received PubSub notification, asking Buildbucket for the build status
[22:29:43.317] Build status: STARTED
[22:38:21.499] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:2:0)
[22:38:21.536] Build status: STARTED
[22:38:21.536] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:5:0) after 1m55s
[22:40:16.530] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:5:0)
[22:40:16.557] Build status: STARTED
[22:40:16.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:6:0) after 5m4s
[22:45:20.571] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:6:0)
[22:45:20.597] Build status: STARTED
[22:45:20.597] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:7:0) after 4m53s
[22:50:13.721] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:7:0)
[22:50:13.752] Build status: STARTED
[22:50:13.752] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:8:0) after 9m31s
[22:59:44.931] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:8:0)
[22:59:44.949] Build status: STARTED
[22:59:44.949] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:9:0) after 3m29s
[23:03:13.964] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:9:0)
[23:03:13.987] Build status: STARTED
[23:03:13.987] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:10:0) after 6m37s
[23:09:51.037] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:10:0)
[23:09:51.091] Build status: STARTED
[23:09:51.091] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:11:0) after 1m28s
[23:11:19.103] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:11:0)
[23:11:19.122] Build status: STARTED
[23:11:19.122] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:12:0) after 3m38s
[23:14:57.139] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:12:0)
[23:14:57.154] Build status: STARTED
[23:14:57.155] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:13:0) after 4m11s
[23:19:08.254] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:13:0)
[23:19:08.276] Build status: STARTED
[23:19:08.276] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:14:0) after 3m44s
[23:22:52.428] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:14:0)
[23:22:52.446] Build status: STARTED
[23:22:52.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:15:0) after 7m43s
[23:30:35.460] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:15:0)
[23:30:35.481] Build status: STARTED
[23:30:35.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:16:0) after 1m47s
[23:32:22.554] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:16:0)
[23:32:22.579] Build status: STARTED
[23:32:22.579] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:17:0) after 6m8s
[23:38:30.596] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:17:0)
[23:38:30.617] Build status: STARTED
[23:38:30.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:18:0) after 6m33s
[23:45:03.632] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:18:0)
[23:45:03.680] Build status: STARTED
[23:45:03.680] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:19:0) after 1m5s
[23:46:09.020] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:19:0)
[23:46:09.047] Build status: STARTED
[23:46:09.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:20:0) after 8m41s
[23:54:50.063] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:20:0)
[23:54:50.106] Build status: STARTED
[23:54:50.106] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:21:0) after 7m46s
[00:02:36.220] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:21:0)
[00:02:36.243] Build status: STARTED
[00:02:36.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:22:0) after 1m46s
[00:04:22.258] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:22:0)
[00:04:22.276] Build status: STARTED
[00:04:22.276] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:23:0) after 9m52s
[00:14:14.645] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:23:0)
[00:14:14.673] Build status: STARTED
[00:14:14.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:24:0) after 1m55s
[00:16:09.690] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:24:0)
[00:16:09.719] Build status: STARTED
[00:16:09.719] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:25:0) after 6m23s
[00:22:32.734] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:25:0)
[00:22:32.853] Build status: STARTED
[00:22:32.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:26:0) after 1m2s
[00:23:34.951] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:26:0)
[00:23:35.046] Build status: STARTED
[00:23:35.046] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:27:0) after 8m46s
[00:32:21.247] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:27:0)
[00:32:21.267] Build status: STARTED
[00:32:21.267] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:28:0) after 6m46s
[00:39:07.289] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:28:0)
[00:39:07.322] Build status: STARTED
[00:39:07.322] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:29:0) after 8m25s
[00:47:32.412] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:29:0)
[00:47:32.432] Build status: STARTED
[00:47:32.432] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:30:0) after 9m4s
[00:56:36.450] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:30:0)
[00:56:36.472] Build status: STARTED
[00:56:36.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:31:0) after 1m33s
[00:58:09.448] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:31:0)
[00:58:09.585] Build status: STARTED
[00:58:09.585] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:32:0) after 2m36s
[01:00:45.704] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:32:0)
[01:00:45.723] Build status: STARTED
[01:00:45.723] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:33:0) after 4m14s
[01:04:59.785] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:33:0)
[01:04:59.806] Build status: STARTED
[01:04:59.806] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:34:0) after 9m1s
[01:14:00.888] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:34:0)
[01:14:00.906] Build status: STARTED
[01:14:00.906] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:35:0) after 7m18s
[01:21:19.202] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:35:0)
[01:21:19.222] Build status: STARTED
[01:21:19.222] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:36:0) after 5m5s
[01:26:24.412] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:36:0)
[01:26:24.437] Build status: STARTED
[01:26:24.438] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:37:0) after 4m45s
[01:31:09.644] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:37:0)
[01:31:09.664] Build status: STARTED
[01:31:09.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:38:0) after 1m45s
[01:32:54.880] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:38:0)
[01:32:54.920] Build status: STARTED
[01:32:54.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:39:0) after 8m45s
[01:41:39.942] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:39:0)
[01:41:39.966] Build status: STARTED
[01:41:39.967] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:40:0) after 7m47s
[01:49:26.982] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:40:0)
[01:49:27.065] Build status: STARTED
[01:49:27.066] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:41:0) after 3m54s
[01:53:21.314] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:41:0)
[01:53:21.334] Build status: STARTED
[01:53:21.334] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:42:0) after 1m58s
[01:55:19.472] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:42:0)
[01:55:19.490] Build status: STARTED
[01:55:19.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:43:0) after 5m14s
[02:00:33.831] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:43:0)
[02:00:33.848] Build status: STARTED
[02:00:33.848] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:44:0) after 2m12s
[02:02:45.960] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:44:0)
[02:02:45.980] Build status: STARTED
[02:02:45.980] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:45:0) after 6m56s
[02:09:42.100] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:45:0)
[02:09:42.183] Build status: STARTED
[02:09:42.183] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:46:0) after 8m53s
[02:18:35.301] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:46:0)
[02:18:35.322] Build status: STARTED
[02:18:35.322] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:47:0) after 8m10s
[02:26:45.339] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:47:0)
[02:26:45.427] Build status: STARTED
[02:26:45.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:48:0) after 8m54s
[02:35:39.661] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:48:0)
[02:35:39.677] Build status: STARTED
[02:35:39.677] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:49:0) after 3m18s
[02:38:57.756] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:49:0)
[02:38:57.778] Build status: STARTED
[02:38:57.778] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:50:0) after 6m19s
[02:45:16.838] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:50:0)
[02:45:16.860] Build status: STARTED
[02:45:16.860] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:51:0) after 1m15s
[02:46:32.042] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:51:0)
[02:46:32.062] Build status: STARTED
[02:46:32.063] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:52:0) after 9m16s
[02:55:48.082] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:52:0)
[02:55:48.099] Build status: STARTED
[02:55:48.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:53:0) after 4m45s
[03:00:33.265] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:53:0)
[03:00:33.285] Build status: STARTED
[03:00:33.285] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:54:0) after 9m58s
[03:10:31.436] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:54:0)
[03:10:31.512] Build status: STARTED
[03:10:31.512] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:55:0) after 6m31s
[03:17:02.730] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:55:0)
[03:17:02.745] Build status: STARTED
[03:17:02.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850752400698991776:56:0) after 9m7s
[03:22:07.337] Received PubSub notification, asking Buildbucket for the build status
[03:22:07.354] Build:
{
"id": "8685322338783404737",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 20220,
"createdBy": "project:chromium",
"createTime": "2026-04-05T22:29:38.000538747Z",
"startTime": "2026-04-05T22:29:43.102335576Z",
"endTime": "2026-04-06T03:22:07.013297101Z",
"updateTime": "2026-04-06T03:22:07.013297101Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d4e77e375b211d179f0e3fed6b867ac9e13dd22e",
"ref": "refs/heads/main"
}
}
}
[03:22:07.354] Invocation finished in 4h52m30.564508003s with status FAILED