[12:17:16.491] New invocation is queued and will start shortly
[12:17:17.676] Starting the invocation (attempt 1)
[12:17:17.702] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/e93273dd453321f736b52dbc1516f4b9efe7f916"
[12:17:17.702] Popped gitiles commit info from properties and tags
[12:17:17.702] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[12:17:17.702] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[12:17:17.702] Buildbucket request:
{
"requestId": "8867188976621205296",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e93273dd453321f736b52dbc1516f4b9efe7f916",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8867188976621205296"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[12:17:18.201] Scheduled build:
{
"id": "8701758914639557073",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 18113,
"createdBy": "project:chromium",
"createTime": "2025-10-06T12:17:17.757694638Z",
"updateTime": "2025-10-06T12:17:17.757694638Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e93273dd453321f736b52dbc1516f4b9efe7f916",
"ref": "refs/heads/main"
}
}
}
[12:17:18.201] Task URL: https://cr-buildbucket.appspot.com/build/8701758914639557073
[12:17:18.201] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:2:0) after 8m54s
[12:17:18.350] Received PubSub notification, asking Buildbucket for the build status
[12:17:18.371] Build status: SCHEDULED
[12:18:19.302] Received PubSub notification, asking Buildbucket for the build status
[12:18:19.366] Build status: STARTED
[12:26:12.546] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:2:0)
[12:26:12.610] Build status: STARTED
[12:26:12.610] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:5:0) after 9m25s
[12:35:38.028] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:5:0)
[12:35:38.049] Build status: STARTED
[12:35:38.049] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:6:0) after 9m2s
[12:44:40.140] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:6:0)
[12:44:40.163] Build status: STARTED
[12:44:40.163] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:7:0) after 7m41s
[12:52:21.181] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:7:0)
[12:52:21.200] Build status: STARTED
[12:52:21.200] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:8:0) after 4m21s
[12:56:42.215] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:8:0)
[12:56:42.430] Build status: STARTED
[12:56:42.430] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:9:0) after 7m49s
[13:04:31.431] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:9:0)
[13:04:31.447] Build status: STARTED
[13:04:31.447] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:10:0) after 1m37s
[13:06:08.509] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:10:0)
[13:06:08.532] Build status: STARTED
[13:06:08.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:11:0) after 9m7s
[13:15:15.550] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:11:0)
[13:15:15.567] Build status: STARTED
[13:15:15.568] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:12:0) after 2m30s
[13:17:45.590] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:12:0)
[13:17:45.605] Build status: STARTED
[13:17:45.605] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:13:0) after 9m23s
[13:27:08.624] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:13:0)
[13:27:08.666] Build status: STARTED
[13:27:08.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:14:0) after 8m32s
[13:35:40.694] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:14:0)
[13:35:40.711] Build status: STARTED
[13:35:40.711] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:15:0) after 8m0s
[13:43:40.728] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:15:0)
[13:43:40.780] Build status: STARTED
[13:43:40.780] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:16:0) after 1m46s
[13:45:27.022] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:16:0)
[13:45:27.040] Build status: STARTED
[13:45:27.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:17:0) after 7m28s
[13:52:55.107] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:17:0)
[13:52:55.123] Build status: STARTED
[13:52:55.123] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:18:0) after 9m31s
[14:02:26.178] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:18:0)
[14:02:26.288] Build status: STARTED
[14:02:26.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:19:0) after 9m11s
[14:11:37.342] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:19:0)
[14:11:37.394] Build status: STARTED
[14:11:37.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:20:0) after 8m22s
[14:19:59.460] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:20:0)
[14:19:59.478] Build status: STARTED
[14:19:59.478] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:21:0) after 5m5s
[14:25:04.544] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:21:0)
[14:25:04.561] Build status: STARTED
[14:25:04.561] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:22:0) after 5m30s
[14:30:34.577] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:22:0)
[14:30:34.592] Build status: STARTED
[14:30:34.592] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:23:0) after 2m56s
[14:33:30.607] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:23:0)
[14:33:30.622] Build status: STARTED
[14:33:30.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:24:0) after 6m19s
[14:39:49.675] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:24:0)
[14:39:49.700] Build status: STARTED
[14:39:49.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:25:0) after 5m3s
[14:44:52.719] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:25:0)
[14:44:52.731] Build status: STARTED
[14:44:52.731] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:26:0) after 2m33s
[14:47:25.807] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:26:0)
[14:47:25.826] Build status: STARTED
[14:47:25.826] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:27:0) after 2m7s
[14:49:32.893] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:27:0)
[14:49:32.912] Build status: STARTED
[14:49:32.912] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:28:0) after 9m22s
[14:58:54.925] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:28:0)
[14:58:55.007] Build status: STARTED
[14:58:55.007] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:29:0) after 5m41s
[15:04:36.026] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:29:0)
[15:04:36.048] Build status: STARTED
[15:04:36.048] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:30:0) after 7m8s
[15:11:44.069] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:30:0)
[15:11:44.090] Build status: STARTED
[15:11:44.090] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:31:0) after 3m48s
[15:15:32.236] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:31:0)
[15:15:32.321] Build status: STARTED
[15:15:32.321] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:32:0) after 7m59s
[15:23:31.417] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:32:0)
[15:23:31.448] Build status: STARTED
[15:23:31.448] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:33:0) after 2m40s
[15:26:11.545] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:33:0)
[15:26:11.561] Build status: STARTED
[15:26:11.561] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:34:0) after 7m4s
[15:33:15.584] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:34:0)
[15:33:15.601] Build status: STARTED
[15:33:15.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:35:0) after 4m48s
[15:38:03.623] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:35:0)
[15:38:03.638] Build status: STARTED
[15:38:03.638] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:36:0) after 8m57s
[15:47:00.659] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:36:0)
[15:47:00.710] Build status: STARTED
[15:47:00.710] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:37:0) after 4m41s
[15:51:41.766] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:37:0)
[15:51:41.783] Build status: STARTED
[15:51:41.783] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:38:0) after 4m13s
[15:55:54.959] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:38:0)
[15:55:54.983] Build status: STARTED
[15:55:54.983] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:39:0) after 2m26s
[15:58:21.079] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:39:0)
[15:58:21.105] Build status: STARTED
[15:58:21.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:40:0) after 1m54s
[16:00:15.170] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:40:0)
[16:00:15.237] Build status: STARTED
[16:00:15.237] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:41:0) after 6m31s
[16:06:46.263] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:41:0)
[16:06:46.281] Build status: STARTED
[16:06:46.281] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:42:0) after 2m49s
[16:09:35.298] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:42:0)
[16:09:35.313] Build status: STARTED
[16:09:35.313] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:43:0) after 5m19s
[16:14:54.331] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:43:0)
[16:14:54.347] Build status: STARTED
[16:14:54.347] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:44:0) after 3m25s
[16:18:19.365] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:44:0)
[16:18:19.449] Build status: STARTED
[16:18:19.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:45:0) after 4m30s
[16:22:49.470] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:45:0)
[16:22:49.496] Build status: STARTED
[16:22:49.496] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:46:0) after 5m58s
[16:28:47.512] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:46:0)
[16:28:47.532] Build status: STARTED
[16:28:47.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:47:0) after 9m48s
[16:38:35.547] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:47:0)
[16:38:35.564] Build status: STARTED
[16:38:35.565] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:48:0) after 6m52s
[16:45:27.581] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:48:0)
[16:45:27.599] Build status: STARTED
[16:45:27.600] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:49:0) after 8m53s
[16:54:20.621] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:49:0)
[16:54:20.639] Build status: STARTED
[16:54:20.639] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:50:0) after 8m27s
[17:02:47.788] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:50:0)
[17:02:47.812] Build status: STARTED
[17:02:47.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:51:0) after 8m48s
[17:11:35.873] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:51:0)
[17:11:35.889] Build status: STARTED
[17:11:35.889] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:52:0) after 2m57s
[17:14:32.960] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:52:0)
[17:14:33.033] Build status: STARTED
[17:14:33.033] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:53:0) after 3m37s
[17:18:10.095] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:53:0)
[17:18:10.112] Build status: STARTED
[17:18:10.112] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:54:0) after 5m41s
[17:23:51.159] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:54:0)
[17:23:51.192] Build status: STARTED
[17:23:51.192] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:55:0) after 4m12s
[17:28:03.210] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:55:0)
[17:28:03.232] Build status: STARTED
[17:28:03.233] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:56:0) after 1m19s
[17:29:22.301] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:56:0)
[17:29:22.322] Build status: STARTED
[17:29:22.322] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8867188976621205296:57:0) after 8m5s
[17:31:24.863] Received PubSub notification, asking Buildbucket for the build status
[17:31:24.879] Build:
{
"id": "8701758914639557073",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 18113,
"createdBy": "project:chromium",
"createTime": "2025-10-06T12:17:17.757694638Z",
"startTime": "2025-10-06T12:18:09.566387905Z",
"endTime": "2025-10-06T17:31:20.863548459Z",
"updateTime": "2025-10-06T17:31:20.863548459Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e93273dd453321f736b52dbc1516f4b9efe7f916",
"ref": "refs/heads/main"
}
}
}
[17:31:24.879] Invocation finished in 5h14m8.395125977s with status FAILED