[23:28:22.274] New invocation is queued and will start shortly
[23:28:23.377] Starting the invocation (attempt 1)
[23:28:23.398] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/72fc355890a420f3756d47a74a7bf8b6c8048209"
[23:28:23.398] Popped gitiles commit info from properties and tags
[23:28:23.399] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[23:28:23.399] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[23:28:23.399] Buildbucket request:
{
"requestId": "8868233918484386224",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "72fc355890a420f3756d47a74a7bf8b6c8048209",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8868233918484386224"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[23:28:23.847] Scheduled build:
{
"id": "8702803856600809681",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 17987,
"createdBy": "project:chromium",
"createTime": "2025-09-24T23:28:23.446279236Z",
"updateTime": "2025-09-24T23:28:23.446279236Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "72fc355890a420f3756d47a74a7bf8b6c8048209",
"ref": "refs/heads/main"
}
},
"canary": true
}
[23:28:23.847] Task URL: https://cr-buildbucket.appspot.com/build/8702803856600809681
[23:28:23.847] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:2:0) after 1m17s
[23:28:24.818] Received PubSub notification, asking Buildbucket for the build status
[23:28:24.839] Build status: SCHEDULED
[23:29:10.940] Received PubSub notification, asking Buildbucket for the build status
[23:29:10.956] Build status: STARTED
[23:29:40.964] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:2:0)
[23:29:41.039] Build status: STARTED
[23:29:41.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:5:0) after 6m27s
[23:36:08.187] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:5:0)
[23:36:08.214] Build status: STARTED
[23:36:08.214] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:6:0) after 3m57s
[23:40:05.346] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:6:0)
[23:40:05.378] Build status: STARTED
[23:40:05.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:7:0) after 8m52s
[23:48:58.632] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:7:0)
[23:48:58.647] Build status: STARTED
[23:48:58.647] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:8:0) after 2m32s
[23:51:30.840] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:8:0)
[23:51:30.869] Build status: STARTED
[23:51:30.869] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:9:0) after 8m40s
[00:00:11.181] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:9:0)
[00:00:11.231] Build status: STARTED
[00:00:11.231] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:10:0) after 7m31s
[00:07:42.478] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:10:0)
[00:07:42.517] Build status: STARTED
[00:07:42.517] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:11:0) after 4m25s
[00:12:07.546] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:11:0)
[00:12:07.562] Build status: STARTED
[00:12:07.562] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:12:0) after 7m28s
[00:19:35.557] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:12:0)
[00:19:35.573] Build status: STARTED
[00:19:35.573] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:13:0) after 9m45s
[00:29:20.589] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:13:0)
[00:29:20.607] Build status: STARTED
[00:29:20.607] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:14:0) after 8m14s
[00:37:35.374] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:14:0)
[00:37:35.403] Build status: STARTED
[00:37:35.403] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:15:0) after 7m52s
[00:45:27.951] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:15:0)
[00:45:27.970] Build status: STARTED
[00:45:27.970] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:16:0) after 1m43s
[00:47:11.463] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:16:0)
[00:47:11.518] Build status: STARTED
[00:47:11.518] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:17:0) after 3m26s
[00:50:37.711] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:17:0)
[00:50:37.729] Build status: STARTED
[00:50:37.729] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:18:0) after 1m7s
[00:51:45.031] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:18:0)
[00:51:45.047] Build status: STARTED
[00:51:45.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:19:0) after 3m38s
[00:55:23.323] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:19:0)
[00:55:23.341] Build status: STARTED
[00:55:23.341] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:20:0) after 2m11s
[00:57:34.484] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:20:0)
[00:57:34.499] Build status: STARTED
[00:57:34.499] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:21:0) after 2m6s
[00:59:40.561] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:21:0)
[00:59:40.575] Build status: STARTED
[00:59:40.575] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:22:0) after 4m55s
[01:04:35.589] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:22:0)
[01:04:35.603] Build status: STARTED
[01:04:35.603] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:23:0) after 7m13s
[01:11:48.618] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:23:0)
[01:11:48.654] Build status: STARTED
[01:11:48.654] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:24:0) after 4m53s
[01:16:41.802] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:24:0)
[01:16:41.819] Build status: STARTED
[01:16:41.819] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:25:0) after 5m32s
[01:22:13.867] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:25:0)
[01:22:13.885] Build status: STARTED
[01:22:13.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:26:0) after 3m1s
[01:25:15.196] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:26:0)
[01:25:15.220] Build status: STARTED
[01:25:15.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:27:0) after 2m49s
[01:28:04.303] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:27:0)
[01:28:04.325] Build status: STARTED
[01:28:04.325] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:28:0) after 4m38s
[01:32:42.785] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:28:0)
[01:32:42.847] Build status: STARTED
[01:32:42.847] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:29:0) after 1m26s
[01:34:08.866] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:29:0)
[01:34:08.882] Build status: STARTED
[01:34:08.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:30:0) after 7m55s
[01:42:03.895] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:30:0)
[01:42:03.920] Build status: STARTED
[01:42:03.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:31:0) after 6m53s
[01:48:56.932] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:31:0)
[01:48:56.947] Build status: STARTED
[01:48:56.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:32:0) after 2m56s
[01:51:52.973] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:32:0)
[01:51:52.989] Build status: STARTED
[01:51:52.989] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:33:0) after 6m34s
[01:58:26.957] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:33:0)
[01:58:27.007] Build status: STARTED
[01:58:27.007] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:34:0) after 2m32s
[02:00:59.126] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:34:0)
[02:00:59.151] Build status: STARTED
[02:00:59.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:35:0) after 5m25s
[02:06:24.257] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:35:0)
[02:06:24.270] Build status: STARTED
[02:06:24.270] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:36:0) after 5m59s
[02:12:23.288] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:36:0)
[02:12:23.304] Build status: STARTED
[02:12:23.304] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:37:0) after 3m58s
[02:16:21.333] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:37:0)
[02:16:21.351] Build status: STARTED
[02:16:21.351] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:38:0) after 9m37s
[02:25:58.381] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:38:0)
[02:25:58.396] Build status: STARTED
[02:25:58.396] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:39:0) after 5m55s
[02:31:53.506] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:39:0)
[02:31:53.531] Build status: STARTED
[02:31:53.531] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:40:0) after 8m47s
[02:40:40.546] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:40:0)
[02:40:40.606] Build status: STARTED
[02:40:40.606] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:41:0) after 8m26s
[02:49:06.576] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:41:0)
[02:49:06.591] Build status: STARTED
[02:49:06.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:42:0) after 1m33s
[02:50:39.606] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:42:0)
[02:50:39.661] Build status: STARTED
[02:50:39.661] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:43:0) after 8m58s
[02:59:37.671] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:43:0)
[02:59:37.685] Build status: STARTED
[02:59:37.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:44:0) after 6m59s
[03:06:36.935] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:44:0)
[03:06:36.951] Build status: STARTED
[03:06:36.951] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:45:0) after 7m42s
[03:14:18.966] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:45:0)
[03:14:18.986] Build status: STARTED
[03:14:18.986] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:46:0) after 7m8s
[03:21:27.110] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:46:0)
[03:21:27.189] Build status: STARTED
[03:21:27.189] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:47:0) after 7m8s
[03:28:35.317] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:47:0)
[03:28:35.343] Build status: STARTED
[03:28:35.343] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:48:0) after 6m44s
[03:35:19.654] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:48:0)
[03:35:19.673] Build status: STARTED
[03:35:19.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:49:0) after 9m17s
[03:44:36.924] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:49:0)
[03:44:36.940] Build status: STARTED
[03:44:36.940] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:50:0) after 4m29s
[03:49:06.083] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:50:0)
[03:49:06.100] Build status: STARTED
[03:49:06.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:51:0) after 7m25s
[03:56:31.118] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:51:0)
[03:56:31.172] Build status: STARTED
[03:56:31.172] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8868233918484386224:52:0) after 9m26s
[03:57:42.988] Received PubSub notification, asking Buildbucket for the build status
[03:57:43.022] Build:
{
"id": "8702803856600809681",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 17987,
"createdBy": "project:chromium",
"createTime": "2025-09-24T23:28:23.446279236Z",
"startTime": "2025-09-24T23:29:08.756845071Z",
"endTime": "2025-09-25T03:57:42.722219649Z",
"updateTime": "2025-09-25T03:57:42.722219649Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "72fc355890a420f3756d47a74a7bf8b6c8048209",
"ref": "refs/heads/main"
}
},
"canary": true
}
[03:57:43.022] Invocation finished in 4h29m20.755293002s with status FAILED