[19:00:27.147] New invocation is queued and will start shortly
[19:00:28.397] Starting the invocation (attempt 1)
[19:00:28.424] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/908886263178aab22f731d12a3ef5bcc726043a0"
[19:00:28.424] Popped gitiles commit info from properties and tags
[19:00:28.424] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[19:00:28.425] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[19:00:28.425] Buildbucket request:
{
"requestId": "8864717492793291952",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "908886263178aab22f731d12a3ef5bcc726043a0",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8864717492793291952"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[19:00:29.083] Scheduled build:
{
"id": "8699287430707879185",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 18350,
"createdBy": "project:chromium",
"createTime": "2025-11-02T19:00:28.506855653Z",
"updateTime": "2025-11-02T19:00:28.506855653Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "908886263178aab22f731d12a3ef5bcc726043a0",
"ref": "refs/heads/main"
}
}
}
[19:00:29.083] Task URL: https://cr-buildbucket.appspot.com/build/8699287430707879185
[19:00:29.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:2:0) after 2m56s
[19:00:30.405] Received PubSub notification, asking Buildbucket for the build status
[19:00:30.420] Build status: SCHEDULED
[19:00:34.855] Received PubSub notification, asking Buildbucket for the build status
[19:00:34.877] Build status: STARTED
[19:03:25.097] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:2:0)
[19:03:25.111] Build status: STARTED
[19:03:25.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:5:0) after 3m27s
[19:06:52.168] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:5:0)
[19:06:52.182] Build status: STARTED
[19:06:52.182] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:6:0) after 6m51s
[19:13:43.200] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:6:0)
[19:13:43.223] Build status: STARTED
[19:13:43.223] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:7:0) after 5m3s
[19:18:46.274] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:7:0)
[19:18:46.295] Build status: STARTED
[19:18:46.295] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:8:0) after 2m34s
[19:21:20.311] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:8:0)
[19:21:20.329] Build status: STARTED
[19:21:20.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:9:0) after 8m33s
[19:29:53.351] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:9:0)
[19:29:53.381] Build status: STARTED
[19:29:53.381] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:10:0) after 4m50s
[19:34:43.427] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:10:0)
[19:34:43.441] Build status: STARTED
[19:34:43.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:11:0) after 6m32s
[19:41:15.679] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:11:0)
[19:41:15.699] Build status: STARTED
[19:41:15.699] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:12:0) after 7m14s
[19:48:29.850] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:12:0)
[19:48:29.884] Build status: STARTED
[19:48:29.884] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:13:0) after 9m16s
[19:57:45.900] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:13:0)
[19:57:45.916] Build status: STARTED
[19:57:45.916] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:14:0) after 3m32s
[20:01:17.929] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:14:0)
[20:01:17.956] Build status: STARTED
[20:01:17.956] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:15:0) after 4m48s
[20:06:06.082] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:15:0)
[20:06:06.095] Build status: STARTED
[20:06:06.095] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:16:0) after 6m51s
[20:12:57.109] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:16:0)
[20:12:57.190] Build status: STARTED
[20:12:57.190] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:17:0) after 4m38s
[20:17:35.240] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:17:0)
[20:17:35.265] Build status: STARTED
[20:17:35.265] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:18:0) after 3m26s
[20:21:01.300] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:18:0)
[20:21:01.355] Build status: STARTED
[20:21:01.355] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:19:0) after 5m2s
[20:26:03.373] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:19:0)
[20:26:03.391] Build status: STARTED
[20:26:03.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:20:0) after 8m20s
[20:34:23.447] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:20:0)
[20:34:23.541] Build status: STARTED
[20:34:23.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:21:0) after 3m25s
[20:37:48.655] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:21:0)
[20:37:48.668] Build status: STARTED
[20:37:48.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:22:0) after 9m36s
[20:47:24.877] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:22:0)
[20:47:24.895] Build status: STARTED
[20:47:24.895] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:23:0) after 6m5s
[20:53:30.238] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:23:0)
[20:53:30.268] Build status: STARTED
[20:53:30.268] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:24:0) after 4m10s
[20:57:40.384] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:24:0)
[20:57:40.401] Build status: STARTED
[20:57:40.401] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:25:0) after 3m20s
[21:01:00.447] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:25:0)
[21:01:00.462] Build status: STARTED
[21:01:00.462] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:26:0) after 1m17s
[21:02:17.800] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:26:0)
[21:02:17.813] Build status: STARTED
[21:02:17.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:27:0) after 4m16s
[21:06:33.831] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:27:0)
[21:06:33.854] Build status: STARTED
[21:06:33.854] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:28:0) after 8m9s
[21:14:42.871] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:28:0)
[21:14:42.898] Build status: STARTED
[21:14:42.898] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:29:0) after 9m25s
[21:24:07.992] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:29:0)
[21:24:08.059] Build status: STARTED
[21:24:08.059] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:30:0) after 6m20s
[21:30:28.153] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:30:0)
[21:30:28.170] Build status: STARTED
[21:30:28.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:31:0) after 7m51s
[21:38:19.184] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:31:0)
[21:38:19.208] Build status: STARTED
[21:38:19.208] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:32:0) after 2m25s
[21:40:44.311] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:32:0)
[21:40:44.326] Build status: STARTED
[21:40:44.326] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:33:0) after 9m30s
[21:50:14.469] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:33:0)
[21:50:14.484] Build status: STARTED
[21:50:14.484] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:34:0) after 3m28s
[21:53:42.604] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:34:0)
[21:53:42.620] Build status: STARTED
[21:53:42.620] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:35:0) after 9m24s
[22:03:06.636] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:35:0)
[22:03:06.650] Build status: STARTED
[22:03:06.650] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:36:0) after 2m57s
[22:06:03.670] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:36:0)
[22:06:03.693] Build status: STARTED
[22:06:03.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:37:0) after 8m31s
[22:14:34.799] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:37:0)
[22:14:34.820] Build status: STARTED
[22:14:34.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:38:0) after 1m33s
[22:16:07.834] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:38:0)
[22:16:07.851] Build status: STARTED
[22:16:07.851] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:39:0) after 9m23s
[22:25:30.864] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:39:0)
[22:25:30.879] Build status: STARTED
[22:25:30.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:40:0) after 2m14s
[22:27:44.895] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:40:0)
[22:27:44.934] Build status: STARTED
[22:27:44.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:41:0) after 1m57s
[22:29:41.950] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:41:0)
[22:29:42.022] Build status: STARTED
[22:29:42.022] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:42:0) after 5m50s
[22:35:32.042] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:42:0)
[22:35:32.064] Build status: STARTED
[22:35:32.064] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:43:0) after 1m17s
[22:36:49.152] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:43:0)
[22:36:49.176] Build status: STARTED
[22:36:49.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:44:0) after 2m24s
[22:39:13.228] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:44:0)
[22:39:13.267] Build status: STARTED
[22:39:13.267] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:45:0) after 7m19s
[22:46:32.238] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:45:0)
[22:46:32.254] Build status: STARTED
[22:46:32.254] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:46:0) after 4m56s
[22:51:28.482] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:46:0)
[22:51:28.501] Build status: STARTED
[22:51:28.501] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:47:0) after 1m20s
[22:52:48.488] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:47:0)
[22:52:48.527] Build status: STARTED
[22:52:48.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:48:0) after 1m1s
[22:53:49.633] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:48:0)
[22:53:49.749] Build status: STARTED
[22:53:49.749] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:49:0) after 1m55s
[22:55:45.100] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:49:0)
[22:55:45.128] Build status: STARTED
[22:55:45.128] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:50:0) after 2m13s
[22:57:58.193] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:50:0)
[22:57:58.235] Build status: STARTED
[22:57:58.235] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:51:0) after 1m16s
[22:59:14.668] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:51:0)
[22:59:14.681] Build status: STARTED
[22:59:14.681] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:52:0) after 8m49s
[23:08:03.699] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:52:0)
[23:08:03.720] Build status: STARTED
[23:08:03.720] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:53:0) after 4m47s
[23:12:50.737] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:53:0)
[23:12:50.752] Build status: STARTED
[23:12:50.752] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:54:0) after 2m48s
[23:15:38.860] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:54:0)
[23:15:38.875] Build status: STARTED
[23:15:38.875] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8864717492793291952:55:0) after 8m7s
[23:23:23.519] Received PubSub notification, asking Buildbucket for the build status
[23:23:23.621] Build:
{
"id": "8699287430707879185",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 18350,
"createdBy": "project:chromium",
"createTime": "2025-11-02T19:00:28.506855653Z",
"startTime": "2025-11-02T19:00:33.437794315Z",
"endTime": "2025-11-02T23:23:21.985532405Z",
"updateTime": "2025-11-02T23:23:21.985532405Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "908886263178aab22f731d12a3ef5bcc726043a0",
"ref": "refs/heads/main"
}
}
}
[23:23:23.621] Invocation finished in 4h22m56.487710736s with status FAILED