[16:08:52.446] New invocation is queued and will start shortly
[16:08:53.593] Starting the invocation (attempt 1)
[16:08:53.630] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/fb8889a91996b0b1f018320db7c7270d05a8d229"
[16:08:53.630] Popped gitiles commit info from properties and tags
[16:08:53.630] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[16:08:53.630] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[16:08:53.630] Buildbucket request:
{
"requestId": "8868352166222902768",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "fb8889a91996b0b1f018320db7c7270d05a8d229",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8868352166222902768"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[16:08:54.136] Scheduled build:
{
"id": "8702922104262399649",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 17978,
"createdBy": "project:chromium",
"createTime": "2025-09-23T16:08:53.688576422Z",
"updateTime": "2025-09-23T16:08:53.688576422Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "fb8889a91996b0b1f018320db7c7270d05a8d229",
"ref": "refs/heads/main"
}
}
}
[16:08:54.136] Task URL: https://cr-buildbucket.appspot.com/build/8702922104262399649
[16:08:54.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:2:0) after 3m52s
[16:08:55.139] Received PubSub notification, asking Buildbucket for the build status
[16:08:55.157] Build status: SCHEDULED
[16:09:47.503] Received PubSub notification, asking Buildbucket for the build status
[16:09:47.531] Build status: STARTED
[16:12:46.182] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:2:0)
[16:12:46.200] Build status: STARTED
[16:12:46.200] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:5:0) after 3m30s
[16:16:16.216] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:5:0)
[16:16:16.232] Build status: STARTED
[16:16:16.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:6:0) after 7m39s
[16:23:55.252] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:6:0)
[16:23:55.317] Build status: STARTED
[16:23:55.317] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:7:0) after 7m52s
[16:31:47.341] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:7:0)
[16:31:47.412] Build status: STARTED
[16:31:47.412] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:8:0) after 8m31s
[16:40:18.520] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:8:0)
[16:40:18.542] Build status: STARTED
[16:40:18.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:9:0) after 2m15s
[16:42:33.654] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:9:0)
[16:42:33.684] Build status: STARTED
[16:42:33.684] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:10:0) after 7m59s
[16:50:32.755] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:10:0)
[16:50:32.789] Build status: STARTED
[16:50:32.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:11:0) after 5m36s
[16:56:08.951] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:11:0)
[16:56:09.025] Build status: STARTED
[16:56:09.026] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:12:0) after 2m5s
[16:58:14.142] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:12:0)
[16:58:14.173] Build status: STARTED
[16:58:14.173] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:13:0) after 7m34s
[17:05:48.486] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:13:0)
[17:05:48.552] Build status: STARTED
[17:05:48.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:14:0) after 4m23s
[17:10:11.592] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:14:0)
[17:10:11.631] Build status: STARTED
[17:10:11.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:15:0) after 9m3s
[17:19:14.702] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:15:0)
[17:19:14.720] Build status: STARTED
[17:19:14.720] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:16:0) after 2m23s
[17:21:37.843] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:16:0)
[17:21:37.867] Build status: STARTED
[17:21:37.867] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:17:0) after 4m42s
[17:26:19.873] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:17:0)
[17:26:19.894] Build status: STARTED
[17:26:19.894] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:18:0) after 9m58s
[17:36:18.025] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:18:0)
[17:36:18.042] Build status: STARTED
[17:36:18.042] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:19:0) after 8m18s
[17:44:36.091] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:19:0)
[17:44:36.111] Build status: STARTED
[17:44:36.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:20:0) after 8m46s
[17:53:22.195] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:20:0)
[17:53:22.238] Build status: STARTED
[17:53:22.239] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:21:0) after 7m14s
[18:00:36.282] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:21:0)
[18:00:36.304] Build status: STARTED
[18:00:36.304] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:22:0) after 9m44s
[18:10:20.326] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:22:0)
[18:10:20.345] Build status: STARTED
[18:10:20.345] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:23:0) after 2m44s
[18:13:04.409] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:23:0)
[18:13:04.425] Build status: STARTED
[18:13:04.425] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:24:0) after 4m8s
[18:17:12.468] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:24:0)
[18:17:12.523] Build status: STARTED
[18:17:12.523] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:25:0) after 8m20s
[18:25:32.548] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:25:0)
[18:25:32.566] Build status: STARTED
[18:25:32.566] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:26:0) after 5m29s
[18:31:01.556] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:26:0)
[18:31:01.575] Build status: STARTED
[18:31:01.575] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:27:0) after 6m16s
[18:37:17.598] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:27:0)
[18:37:17.622] Build status: STARTED
[18:37:17.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:28:0) after 9m7s
[18:46:24.661] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:28:0)
[18:46:24.679] Build status: STARTED
[18:46:24.679] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:29:0) after 6m16s
[18:52:40.659] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:29:0)
[18:52:40.675] Build status: STARTED
[18:52:40.675] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:30:0) after 4m13s
[18:56:53.740] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:30:0)
[18:56:53.788] Build status: STARTED
[18:56:53.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:31:0) after 1m8s
[18:58:01.808] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:31:0)
[18:58:01.824] Build status: STARTED
[18:58:01.824] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:32:0) after 3m6s
[19:01:07.853] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:32:0)
[19:01:07.873] Build status: STARTED
[19:01:07.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:33:0) after 4m6s
[19:05:13.903] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:33:0)
[19:05:13.927] Build status: STARTED
[19:05:13.927] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:34:0) after 2m39s
[19:07:52.977] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:34:0)
[19:07:52.994] Build status: STARTED
[19:07:52.994] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:35:0) after 5m4s
[19:12:57.016] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:35:0)
[19:12:57.040] Build status: STARTED
[19:12:57.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:36:0) after 8m28s
[19:21:25.034] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:36:0)
[19:21:25.076] Build status: STARTED
[19:21:25.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:37:0) after 9m59s
[19:31:24.204] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:37:0)
[19:31:24.262] Build status: STARTED
[19:31:24.262] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:38:0) after 5m23s
[19:36:47.369] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:38:0)
[19:36:47.395] Build status: STARTED
[19:36:47.396] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:39:0) after 5m48s
[19:42:35.484] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:39:0)
[19:42:35.510] Build status: STARTED
[19:42:35.510] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:40:0) after 3m27s
[19:46:02.563] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:40:0)
[19:46:02.600] Build status: STARTED
[19:46:02.600] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:41:0) after 9m23s
[19:55:25.674] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:41:0)
[19:55:25.746] Build status: STARTED
[19:55:25.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:42:0) after 1m37s
[19:57:02.960] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:42:0)
[19:57:03.012] Build status: STARTED
[19:57:03.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:43:0) after 4m19s
[20:01:22.123] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:43:0)
[20:01:22.239] Build status: STARTED
[20:01:22.239] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:44:0) after 6m4s
[20:07:26.310] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:44:0)
[20:07:26.325] Build status: STARTED
[20:07:26.325] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:45:0) after 1m26s
[20:08:52.447] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:45:0)
[20:08:52.463] Build status: STARTED
[20:08:52.463] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:46:0) after 7m37s
[20:16:29.641] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:46:0)
[20:16:29.660] Build status: STARTED
[20:16:29.660] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:47:0) after 1m13s
[20:17:42.855] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:47:0)
[20:17:42.875] Build status: STARTED
[20:17:42.875] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:48:0) after 4m3s
[20:21:45.989] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:48:0)
[20:21:46.007] Build status: STARTED
[20:21:46.007] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:49:0) after 3m34s
[20:25:20.027] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:49:0)
[20:25:20.047] Build status: STARTED
[20:25:20.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:50:0) after 7m49s
[20:33:09.073] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:50:0)
[20:33:09.096] Build status: STARTED
[20:33:09.096] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:51:0) after 2m55s
[20:36:04.190] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:51:0)
[20:36:04.214] Build status: STARTED
[20:36:04.214] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:52:0) after 8m6s
[20:44:10.302] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:52:0)
[20:44:10.317] Build status: STARTED
[20:44:10.317] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:53:0) after 5m58s
[20:50:08.327] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:53:0)
[20:50:08.355] Build status: STARTED
[20:50:08.355] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868352166222902768:54:0) after 3m59s
[20:50:19.424] Received PubSub notification, asking Buildbucket for the build status
[20:50:19.441] Build:
{
"id": "8702922104262399649",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 17978,
"createdBy": "project:chromium",
"createTime": "2025-09-23T16:08:53.688576422Z",
"startTime": "2025-09-23T16:09:47.316699237Z",
"endTime": "2025-09-23T20:50:19.171883112Z",
"updateTime": "2025-09-23T20:50:19.171883112Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "fb8889a91996b0b1f018320db7c7270d05a8d229",
"ref": "refs/heads/main"
}
}
}
[20:50:19.441] Invocation finished in 4h41m27.006511367s with status FAILED