[13:07:36.597] New invocation is queued and will start shortly
[13:07:37.903] Starting the invocation (attempt 1)
[13:07:37.925] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/91f53891c5f2846be9f3629ffff7c5934029538b"
[13:07:37.926] Popped gitiles commit info from properties and tags
[13:07:37.926] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[13:07:37.926] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[13:07:37.926] Buildbucket request:
{
"requestId": "8863924319029345008",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "91f53891c5f2846be9f3629ffff7c5934029538b",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8863924319029345008"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[13:07:38.416] Scheduled build:
{
"id": "8698494256917720465",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 18395,
"createdBy": "project:chromium",
"createTime": "2025-11-11T13:07:37.978716799Z",
"updateTime": "2025-11-11T13:07:37.978716799Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "91f53891c5f2846be9f3629ffff7c5934029538b",
"ref": "refs/heads/main"
}
}
}
[13:07:38.416] Task URL: https://cr-buildbucket.appspot.com/build/8698494256917720465
[13:07:38.416] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:2:0) after 1m52s
[13:07:38.722] Received PubSub notification, asking Buildbucket for the build status
[13:07:38.755] Build status: SCHEDULED
[13:08:36.203] Received PubSub notification, asking Buildbucket for the build status
[13:08:36.217] Build status: STARTED
[13:09:30.432] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:2:0)
[13:09:30.472] Build status: STARTED
[13:09:30.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:5:0) after 9m32s
[13:19:02.486] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:5:0)
[13:19:02.513] Build status: STARTED
[13:19:02.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:6:0) after 9m51s
[13:28:53.526] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:6:0)
[13:28:53.542] Build status: STARTED
[13:28:53.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:7:0) after 6m34s
[13:35:27.555] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:7:0)
[13:35:27.579] Build status: STARTED
[13:35:27.579] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:8:0) after 3m14s
[13:38:41.564] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:8:0)
[13:38:41.592] Build status: STARTED
[13:38:41.593] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:9:0) after 4m8s
[13:42:49.655] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:9:0)
[13:42:49.691] Build status: STARTED
[13:42:49.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:10:0) after 1m39s
[13:44:28.730] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:10:0)
[13:44:28.748] Build status: STARTED
[13:44:28.748] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:11:0) after 1m11s
[13:45:39.764] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:11:0)
[13:45:39.783] Build status: STARTED
[13:45:39.783] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:12:0) after 3m31s
[13:49:10.799] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:12:0)
[13:49:10.817] Build status: STARTED
[13:49:10.817] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:13:0) after 8m25s
[13:57:35.833] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:13:0)
[13:57:35.847] Build status: STARTED
[13:57:35.847] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:14:0) after 5m49s
[14:03:24.832] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:14:0)
[14:03:24.850] Build status: STARTED
[14:03:24.850] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:15:0) after 7m8s
[14:10:32.838] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:15:0)
[14:10:32.853] Build status: STARTED
[14:10:32.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:16:0) after 2m26s
[14:12:58.869] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:16:0)
[14:12:58.889] Build status: STARTED
[14:12:58.889] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:17:0) after 7m20s
[14:20:18.904] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:17:0)
[14:20:18.926] Build status: STARTED
[14:20:18.926] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:18:0) after 7m27s
[14:27:45.906] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:18:0)
[14:27:45.924] Build status: STARTED
[14:27:45.924] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:19:0) after 8m24s
[14:36:09.943] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:19:0)
[14:36:09.957] Build status: STARTED
[14:36:09.957] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:20:0) after 2m59s
[14:39:09.029] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:20:0)
[14:39:09.081] Build status: STARTED
[14:39:09.081] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:21:0) after 4m47s
[14:43:56.100] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:21:0)
[14:43:56.122] Build status: STARTED
[14:43:56.122] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:22:0) after 7m34s
[14:51:30.176] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:22:0)
[14:51:30.208] Build status: STARTED
[14:51:30.209] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:23:0) after 9m58s
[15:01:28.325] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:23:0)
[15:01:28.342] Build status: STARTED
[15:01:28.343] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:24:0) after 5m19s
[15:06:47.405] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:24:0)
[15:06:47.422] Build status: STARTED
[15:06:47.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:25:0) after 2m51s
[15:09:38.442] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:25:0)
[15:09:38.463] Build status: STARTED
[15:09:38.463] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:26:0) after 9m43s
[15:19:21.552] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:26:0)
[15:19:21.565] Build status: STARTED
[15:19:21.565] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:27:0) after 3m17s
[15:22:38.590] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:27:0)
[15:22:38.630] Build status: STARTED
[15:22:38.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:28:0) after 4m9s
[15:26:47.678] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:28:0)
[15:26:47.866] Build status: STARTED
[15:26:47.866] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:29:0) after 3m47s
[15:30:34.968] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:29:0)
[15:30:34.992] Build status: STARTED
[15:30:34.992] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:30:0) after 8m12s
[15:38:47.055] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:30:0)
[15:38:47.352] Build status: STARTED
[15:38:47.352] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:31:0) after 5m55s
[15:44:42.376] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:31:0)
[15:44:42.403] Build status: STARTED
[15:44:42.403] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:32:0) after 9m21s
[15:54:03.599] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:32:0)
[15:54:03.666] Build status: STARTED
[15:54:03.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:33:0) after 8m56s
[16:02:59.686] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:33:0)
[16:02:59.715] Build status: STARTED
[16:02:59.715] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:34:0) after 1m31s
[16:04:30.725] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:34:0)
[16:04:30.758] Build status: STARTED
[16:04:30.758] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:35:0) after 8m53s
[16:13:23.776] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:35:0)
[16:13:23.791] Build status: STARTED
[16:13:23.791] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:36:0) after 9m25s
[16:22:48.862] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:36:0)
[16:22:48.881] Build status: STARTED
[16:22:48.881] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:37:0) after 8m10s
[16:30:58.949] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:37:0)
[16:30:58.963] Build status: STARTED
[16:30:58.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:38:0) after 4m44s
[16:35:42.988] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:38:0)
[16:35:43.011] Build status: STARTED
[16:35:43.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:39:0) after 8m56s
[16:44:39.010] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:39:0)
[16:44:39.025] Build status: STARTED
[16:44:39.025] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:40:0) after 6m21s
[16:51:00.043] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:40:0)
[16:51:00.056] Build status: STARTED
[16:51:00.056] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:41:0) after 9m5s
[17:00:05.079] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:41:0)
[17:00:05.096] Build status: STARTED
[17:00:05.096] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:42:0) after 7m1s
[17:07:06.123] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:42:0)
[17:07:06.140] Build status: STARTED
[17:07:06.140] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:43:0) after 5m33s
[17:12:39.199] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:43:0)
[17:12:39.221] Build status: STARTED
[17:12:39.221] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:44:0) after 3m22s
[17:16:01.274] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:44:0)
[17:16:01.297] Build status: STARTED
[17:16:01.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:45:0) after 5m59s
[17:22:00.368] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:45:0)
[17:22:00.384] Build status: STARTED
[17:22:00.384] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:46:0) after 9m1s
[17:31:01.407] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:46:0)
[17:31:01.439] Build status: STARTED
[17:31:01.439] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:47:0) after 5m54s
[17:36:55.506] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:47:0)
[17:36:55.545] Build status: STARTED
[17:36:55.545] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8863924319029345008:48:0) after 7m43s
[17:40:58.859] Received PubSub notification, asking Buildbucket for the build status
[17:40:58.872] Build:
{
"id": "8698494256917720465",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 18395,
"createdBy": "project:chromium",
"createTime": "2025-11-11T13:07:37.978716799Z",
"startTime": "2025-11-11T13:08:35.805281891Z",
"endTime": "2025-11-11T17:40:58.484131565Z",
"updateTime": "2025-11-11T17:40:58.484131565Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "91f53891c5f2846be9f3629ffff7c5934029538b",
"ref": "refs/heads/main"
}
}
}
[17:40:58.872] Invocation finished in 4h33m22.29221887s with status FAILED