[12:08:33.540] New invocation is queued and will start shortly
[12:08:34.927] Starting the invocation (attempt 1)
[12:08:34.954] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0e50def3f714106fbbd45e4769484979439aae7f"
[12:08:34.954] Popped gitiles commit info from properties and tags
[12:08:34.954] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[12:08:34.954] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[12:08:34.955] Buildbucket request:
{
"requestId": "8842637747075995824",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0e50def3f714106fbbd45e4769484979439aae7f",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8842637747075995824"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[12:08:35.545] Scheduled build:
{
"id": "8677207684853602753",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 21496,
"createdBy": "project:chromium",
"createTime": "2026-07-04T12:08:35.040365966Z",
"updateTime": "2026-07-04T12:08:35.040365966Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0e50def3f714106fbbd45e4769484979439aae7f",
"ref": "refs/heads/main"
}
}
}
[12:08:35.545] Task URL: https://cr-buildbucket.appspot.com/build/8677207684853602753
[12:08:35.545] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:2:0) after 9m18s
[12:08:36.616] Received PubSub notification, asking Buildbucket for the build status
[12:08:36.639] Build status: SCHEDULED
[12:09:56.482] Received PubSub notification, asking Buildbucket for the build status
[12:09:56.499] Build status: STARTED
[12:17:53.576] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:2:0)
[12:17:53.615] Build status: STARTED
[12:17:53.615] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:5:0) after 8m17s
[12:26:10.927] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:5:0)
[12:26:10.944] Build status: STARTED
[12:26:10.944] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:6:0) after 4m12s
[12:30:22.967] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:6:0)
[12:30:22.986] Build status: STARTED
[12:30:22.986] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:7:0) after 5m26s
[12:35:49.027] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:7:0)
[12:35:49.046] Build status: STARTED
[12:35:49.046] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:8:0) after 6m27s
[12:42:16.545] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:8:0)
[12:42:16.561] Build status: STARTED
[12:42:16.561] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:9:0) after 8m48s
[12:51:04.575] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:9:0)
[12:51:04.625] Build status: STARTED
[12:51:04.625] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:10:0) after 9m11s
[13:00:15.667] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:10:0)
[13:00:15.705] Build status: STARTED
[13:00:15.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:11:0) after 2m2s
[13:02:17.720] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:11:0)
[13:02:17.737] Build status: STARTED
[13:02:17.737] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:12:0) after 5m5s
[13:07:23.062] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:12:0)
[13:07:23.080] Build status: STARTED
[13:07:23.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:13:0) after 5m44s
[13:13:07.099] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:13:0)
[13:13:07.116] Build status: STARTED
[13:13:07.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:14:0) after 6m17s
[13:19:24.134] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:14:0)
[13:19:24.153] Build status: STARTED
[13:19:24.153] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:15:0) after 6m7s
[13:25:31.170] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:15:0)
[13:25:31.189] Build status: STARTED
[13:25:31.190] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:16:0) after 2m46s
[13:28:17.355] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:16:0)
[13:28:17.377] Build status: STARTED
[13:28:17.377] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:17:0) after 4m15s
[13:32:32.465] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:17:0)
[13:32:32.480] Build status: STARTED
[13:32:32.480] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:18:0) after 7m16s
[13:39:48.557] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:18:0)
[13:39:48.607] Build status: STARTED
[13:39:48.607] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:19:0) after 5m21s
[13:45:09.745] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:19:0)
[13:45:09.769] Build status: STARTED
[13:45:09.769] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:20:0) after 1m51s
[13:47:00.783] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:20:0)
[13:47:00.800] Build status: STARTED
[13:47:00.800] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:21:0) after 8m14s
[13:55:14.814] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:21:0)
[13:55:14.829] Build status: STARTED
[13:55:14.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:22:0) after 4m55s
[14:00:09.852] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:22:0)
[14:00:09.868] Build status: STARTED
[14:00:09.868] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:23:0) after 7m55s
[14:08:05.159] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:23:0)
[14:08:05.175] Build status: STARTED
[14:08:05.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:24:0) after 9m34s
[14:17:39.479] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:24:0)
[14:17:39.495] Build status: STARTED
[14:17:39.495] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:25:0) after 4m36s
[14:22:15.769] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:25:0)
[14:22:15.789] Build status: STARTED
[14:22:15.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:26:0) after 2m18s
[14:24:33.989] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:26:0)
[14:24:34.014] Build status: STARTED
[14:24:34.014] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:27:0) after 8m14s
[14:32:48.034] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:27:0)
[14:32:48.054] Build status: STARTED
[14:32:48.054] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:28:0) after 8m39s
[14:41:27.149] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:28:0)
[14:41:27.165] Build status: STARTED
[14:41:27.165] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:29:0) after 9m17s
[14:50:44.463] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:29:0)
[14:50:44.482] Build status: STARTED
[14:50:44.482] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:30:0) after 2m21s
[14:53:05.662] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:30:0)
[14:53:05.678] Build status: STARTED
[14:53:05.679] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:31:0) after 5m47s
[14:58:52.699] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:31:0)
[14:58:52.758] Build status: STARTED
[14:58:52.758] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:32:0) after 8m51s
[15:07:43.847] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:32:0)
[15:07:43.879] Build status: STARTED
[15:07:43.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:33:0) after 9m19s
[15:17:02.947] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:33:0)
[15:17:02.966] Build status: STARTED
[15:17:02.966] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:34:0) after 3m28s
[15:20:30.986] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:34:0)
[15:20:31.022] Build status: STARTED
[15:20:31.022] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:35:0) after 8m32s
[15:29:03.047] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:35:0)
[15:29:03.066] Build status: STARTED
[15:29:03.066] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:36:0) after 7m12s
[15:36:15.165] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:36:0)
[15:36:15.186] Build status: STARTED
[15:36:15.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:37:0) after 8m2s
[15:44:17.219] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:37:0)
[15:44:17.236] Build status: STARTED
[15:44:17.236] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:38:0) after 9m54s
[15:54:11.356] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:38:0)
[15:54:11.378] Build status: STARTED
[15:54:11.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:39:0) after 2m1s
[15:56:12.637] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:39:0)
[15:56:12.656] Build status: STARTED
[15:56:12.656] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:40:0) after 1m4s
[15:57:16.673] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:40:0)
[15:57:16.691] Build status: STARTED
[15:57:16.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:41:0) after 2m49s
[16:00:06.462] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:41:0)
[16:00:06.524] Build status: STARTED
[16:00:06.525] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:42:0) after 4m33s
[16:04:39.779] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:42:0)
[16:04:39.803] Build status: STARTED
[16:04:39.803] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:43:0) after 7m58s
[16:12:37.836] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:43:0)
[16:12:37.858] Build status: STARTED
[16:12:37.858] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:44:0) after 8m28s
[16:21:05.940] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:44:0)
[16:21:05.962] Build status: STARTED
[16:21:05.962] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:45:0) after 6m29s
[16:27:35.321] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:45:0)
[16:27:35.342] Build status: STARTED
[16:27:35.342] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:46:0) after 4m6s
[16:31:41.404] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:46:0)
[16:31:41.423] Build status: STARTED
[16:31:41.423] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:47:0) after 2m50s
[16:34:31.471] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:47:0)
[16:34:31.493] Build status: STARTED
[16:34:31.493] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:48:0) after 7m2s
[16:41:33.741] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:48:0)
[16:41:33.770] Build status: STARTED
[16:41:33.770] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:49:0) after 8m46s
[16:50:20.071] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:49:0)
[16:50:20.086] Build status: STARTED
[16:50:20.086] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:50:0) after 2m54s
[16:53:14.370] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:50:0)
[16:53:14.386] Build status: STARTED
[16:53:14.386] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:51:0) after 5m57s
[16:59:11.546] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:51:0)
[16:59:11.568] Build status: STARTED
[16:59:11.568] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:52:0) after 1m43s
[17:00:54.686] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:52:0)
[17:00:54.703] Build status: STARTED
[17:00:54.703] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:53:0) after 2m0s
[17:02:54.907] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:53:0)
[17:02:54.929] Build status: STARTED
[17:02:54.929] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:54:0) after 3m17s
[17:06:11.946] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:54:0)
[17:06:11.965] Build status: STARTED
[17:06:11.965] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:55:0) after 4m34s
[17:10:46.082] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:55:0)
[17:10:46.116] Build status: STARTED
[17:10:46.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:56:0) after 8m26s
[17:19:12.134] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:56:0)
[17:19:12.164] Build status: STARTED
[17:19:12.164] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:57:0) after 8m14s
[17:27:26.276] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:57:0)
[17:27:26.294] Build status: STARTED
[17:27:26.294] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:58:0) after 3m8s
[17:30:34.332] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:58:0)
[17:30:34.460] Build status: STARTED
[17:30:34.460] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:59:0) after 7m30s
[17:38:04.530] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:59:0)
[17:38:04.549] Build status: STARTED
[17:38:04.549] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8842637747075995824:60:0) after 7m14s
[17:39:34.188] Received PubSub notification, asking Buildbucket for the build status
[17:39:34.204] Build:
{
"id": "8677207684853602753",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 21496,
"createdBy": "project:chromium",
"createTime": "2026-07-04T12:08:35.040365966Z",
"startTime": "2026-07-04T12:09:56.286504626Z",
"endTime": "2026-07-04T17:39:33.744004847Z",
"updateTime": "2026-07-04T17:39:33.744004847Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0e50def3f714106fbbd45e4769484979439aae7f",
"ref": "refs/heads/main"
}
}
}
[17:39:34.204] Invocation finished in 5h31m0.667035675s with status FAILED