[06:13:54.783] New invocation is queued and will start shortly
[06:13:55.916] Starting the invocation (attempt 1)
[06:13:55.947] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/7e886ccab2be2e2e8a63fd13519db81e8bfbd6c4"
[06:13:55.947] Popped gitiles commit info from properties and tags
[06:13:55.947] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[06:13:55.947] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[06:13:55.947] Buildbucket request:
{
"requestId": "8858242737693846480",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "7e886ccab2be2e2e8a63fd13519db81e8bfbd6c4",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8858242737693846480"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[06:13:56.586] Scheduled build:
{
"id": "8692812675596580209",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 19356,
"createdBy": "project:chromium",
"createTime": "2026-01-13T06:13:56.160030554Z",
"updateTime": "2026-01-13T06:13:56.160030554Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "7e886ccab2be2e2e8a63fd13519db81e8bfbd6c4",
"ref": "refs/heads/main"
}
}
}
[06:13:56.586] Task URL: https://cr-buildbucket.appspot.com/build/8692812675596580209
[06:13:56.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:2:0) after 5m35s
[06:13:57.551] Received PubSub notification, asking Buildbucket for the build status
[06:13:57.570] Build status: SCHEDULED
[06:14:55.863] Received PubSub notification, asking Buildbucket for the build status
[06:14:55.880] Build status: STARTED
[06:19:31.620] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:2:0)
[06:19:31.637] Build status: STARTED
[06:19:31.637] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:5:0) after 8m53s
[06:28:24.655] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:5:0)
[06:28:24.667] Build status: STARTED
[06:28:24.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:6:0) after 2m15s
[06:30:39.795] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:6:0)
[06:30:39.822] Build status: STARTED
[06:30:39.822] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:7:0) after 7m4s
[06:37:43.869] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:7:0)
[06:37:43.892] Build status: STARTED
[06:37:43.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:8:0) after 3m2s
[06:40:45.986] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:8:0)
[06:40:46.009] Build status: STARTED
[06:40:46.009] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:9:0) after 6m42s
[06:47:28.058] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:9:0)
[06:47:28.099] Build status: STARTED
[06:47:28.099] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:10:0) after 7m45s
[06:55:13.133] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:10:0)
[06:55:13.185] Build status: STARTED
[06:55:13.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:11:0) after 1m5s
[06:56:18.204] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:11:0)
[06:56:18.233] Build status: STARTED
[06:56:18.233] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:12:0) after 9m31s
[07:05:49.261] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:12:0)
[07:05:49.283] Build status: STARTED
[07:05:49.283] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:13:0) after 4m56s
[07:10:45.301] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:13:0)
[07:10:45.383] Build status: STARTED
[07:10:45.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:14:0) after 1m48s
[07:12:33.451] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:14:0)
[07:12:33.468] Build status: STARTED
[07:12:33.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:15:0) after 9m55s
[07:22:28.485] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:15:0)
[07:22:28.502] Build status: STARTED
[07:22:28.502] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:16:0) after 8m37s
[07:31:05.533] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:16:0)
[07:31:05.578] Build status: STARTED
[07:31:05.578] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:17:0) after 6m23s
[07:37:28.682] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:17:0)
[07:37:28.698] Build status: STARTED
[07:37:28.698] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:18:0) after 9m50s
[07:47:18.807] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:18:0)
[07:47:18.820] Build status: STARTED
[07:47:18.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:19:0) after 6m36s
[07:53:54.846] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:19:0)
[07:53:54.861] Build status: STARTED
[07:53:54.861] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:20:0) after 2m22s
[07:56:16.880] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:20:0)
[07:56:16.928] Build status: STARTED
[07:56:16.928] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:21:0) after 3m23s
[07:59:40.143] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:21:0)
[07:59:40.262] Build status: STARTED
[07:59:40.262] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:22:0) after 5m27s
[08:05:07.482] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:22:0)
[08:05:07.496] Build status: STARTED
[08:05:07.496] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:23:0) after 4m27s
[08:09:34.593] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:23:0)
[08:09:34.646] Build status: STARTED
[08:09:34.646] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:24:0) after 2m4s
[08:11:38.812] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:24:0)
[08:11:38.829] Build status: STARTED
[08:11:38.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:25:0) after 2m27s
[08:14:05.872] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:25:0)
[08:14:05.887] Build status: STARTED
[08:14:05.887] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:26:0) after 8m28s
[08:22:34.056] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:26:0)
[08:22:34.104] Build status: STARTED
[08:22:34.104] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:27:0) after 9m45s
[08:32:19.196] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:27:0)
[08:32:19.209] Build status: STARTED
[08:32:19.209] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:28:0) after 6m35s
[08:38:54.336] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:28:0)
[08:38:54.354] Build status: STARTED
[08:38:54.354] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:29:0) after 5m34s
[08:44:28.491] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:29:0)
[08:44:28.511] Build status: STARTED
[08:44:28.511] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:30:0) after 3m12s
[08:47:40.575] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:30:0)
[08:47:40.608] Build status: STARTED
[08:47:40.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:31:0) after 7m19s
[08:54:59.650] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:31:0)
[08:54:59.666] Build status: STARTED
[08:54:59.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:32:0) after 8m16s
[09:03:15.737] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:32:0)
[09:03:15.826] Build status: STARTED
[09:03:15.826] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:33:0) after 1m15s
[09:04:30.988] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:33:0)
[09:04:31.012] Build status: STARTED
[09:04:31.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:34:0) after 9m59s
[09:14:30.031] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:34:0)
[09:14:30.047] Build status: STARTED
[09:14:30.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:35:0) after 5m11s
[09:19:41.036] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:35:0)
[09:19:41.064] Build status: STARTED
[09:19:41.064] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:36:0) after 4m2s
[09:23:43.120] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:36:0)
[09:23:43.139] Build status: STARTED
[09:23:43.139] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:37:0) after 1m53s
[09:25:36.353] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:37:0)
[09:25:36.374] Build status: STARTED
[09:25:36.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:38:0) after 6m32s
[09:32:08.387] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:38:0)
[09:32:08.431] Build status: STARTED
[09:32:08.431] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:39:0) after 6m55s
[09:39:03.591] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:39:0)
[09:39:03.631] Build status: STARTED
[09:39:03.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:40:0) after 1m30s
[09:40:33.796] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:40:0)
[09:40:33.816] Build status: STARTED
[09:40:33.816] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:41:0) after 8m32s
[09:49:05.901] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:41:0)
[09:49:05.916] Build status: STARTED
[09:49:05.916] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:42:0) after 2m53s
[09:51:59.040] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:42:0)
[09:51:59.058] Build status: STARTED
[09:51:59.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:43:0) after 7m16s
[09:59:15.339] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:43:0)
[09:59:15.359] Build status: STARTED
[09:59:15.359] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:44:0) after 1m15s
[10:00:30.388] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:44:0)
[10:00:30.407] Build status: STARTED
[10:00:30.407] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:45:0) after 4m13s
[10:04:43.425] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:45:0)
[10:04:43.441] Build status: STARTED
[10:04:43.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:46:0) after 1m21s
[10:06:04.472] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:46:0)
[10:06:04.485] Build status: STARTED
[10:06:04.485] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:47:0) after 3m26s
[10:09:30.533] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:47:0)
[10:09:30.636] Build status: STARTED
[10:09:30.637] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:48:0) after 4m11s
[10:13:41.700] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:48:0)
[10:13:41.750] Build status: STARTED
[10:13:41.750] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:49:0) after 2m53s
[10:16:34.768] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:49:0)
[10:16:34.782] Build status: STARTED
[10:16:34.782] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:50:0) after 5m4s
[10:21:38.797] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:50:0)
[10:21:38.815] Build status: STARTED
[10:21:38.815] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:51:0) after 4m25s
[10:26:03.922] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:51:0)
[10:26:03.943] Build status: STARTED
[10:26:03.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:52:0) after 3m24s
[10:29:27.969] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:52:0)
[10:29:28.065] Build status: STARTED
[10:29:28.066] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:53:0) after 5m33s
[10:35:01.084] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:53:0)
[10:35:01.107] Build status: STARTED
[10:35:01.107] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:54:0) after 7m40s
[10:42:41.124] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:54:0)
[10:42:41.199] Build status: STARTED
[10:42:41.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:55:0) after 7m52s
[10:50:33.215] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:55:0)
[10:50:33.284] Build status: STARTED
[10:50:33.285] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:56:0) after 1m47s
[10:52:20.578] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:56:0)
[10:52:20.600] Build status: STARTED
[10:52:20.600] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:57:0) after 2m13s
[10:54:34.436] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:57:0)
[10:54:34.451] Build status: STARTED
[10:54:34.451] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:58:0) after 6m29s
[11:01:03.439] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:58:0)
[11:01:03.454] Build status: STARTED
[11:01:03.454] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858242737693846480:59:0) after 9m15s
[11:06:04.329] Received PubSub notification, asking Buildbucket for the build status
[11:06:04.350] Build:
{
"id": "8692812675596580209",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 19356,
"createdBy": "project:chromium",
"createTime": "2026-01-13T06:13:56.160030554Z",
"startTime": "2026-01-13T06:14:55.683559288Z",
"endTime": "2026-01-13T11:06:03.782999347Z",
"updateTime": "2026-01-13T11:06:03.782999347Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "7e886ccab2be2e2e8a63fd13519db81e8bfbd6c4",
"ref": "refs/heads/main"
}
}
}
[11:06:04.350] Invocation finished in 4h52m9.572719858s with status FAILED