[23:23:36.309] New invocation is queued and will start shortly
[23:23:37.380] Starting the invocation (attempt 1)
[23:23:37.405] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/41bdc53fbb8337418365de6837ea5b94d00e7a3d"
[23:23:37.405] Popped gitiles commit info from properties and tags
[23:23:37.405] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[23:23:37.405] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[23:23:37.405] Buildbucket request:
{
"requestId": "8857634373268798016",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "41bdc53fbb8337418365de6837ea5b94d00e7a3d",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8857634373268798016"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[23:23:37.970] Scheduled build:
{
"id": "8692204311403780257",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 19397,
"createdBy": "project:chromium",
"createTime": "2026-01-19T23:23:37.466127066Z",
"updateTime": "2026-01-19T23:23:37.466127066Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "41bdc53fbb8337418365de6837ea5b94d00e7a3d",
"ref": "refs/heads/main"
}
}
}
[23:23:37.970] Task URL: https://cr-buildbucket.appspot.com/build/8692204311403780257
[23:23:37.970] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:2:0) after 3m12s
[23:23:38.256] Received PubSub notification, asking Buildbucket for the build status
[23:23:38.327] Build status: SCHEDULED
[23:24:27.546] Received PubSub notification, asking Buildbucket for the build status
[23:24:27.568] Build status: STARTED
[23:26:49.990] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:2:0)
[23:26:50.043] Build status: STARTED
[23:26:50.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:5:0) after 7m8s
[23:33:58.218] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:5:0)
[23:33:58.243] Build status: STARTED
[23:33:58.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:6:0) after 6m47s
[23:40:45.265] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:6:0)
[23:40:45.279] Build status: STARTED
[23:40:45.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:7:0) after 2m34s
[23:43:19.351] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:7:0)
[23:43:19.396] Build status: STARTED
[23:43:19.396] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:8:0) after 7m15s
[23:50:34.427] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:8:0)
[23:50:34.443] Build status: STARTED
[23:50:34.443] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:9:0) after 7m19s
[23:57:53.462] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:9:0)
[23:57:53.486] Build status: STARTED
[23:57:53.486] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:10:0) after 1m40s
[23:59:33.506] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:10:0)
[23:59:33.522] Build status: STARTED
[23:59:33.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:11:0) after 1m0s
[00:00:33.546] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:11:0)
[00:00:33.560] Build status: STARTED
[00:00:33.561] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:12:0) after 9m42s
[00:10:15.604] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:12:0)
[00:10:15.628] Build status: STARTED
[00:10:15.628] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:13:0) after 7m2s
[00:17:17.893] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:13:0)
[00:17:17.920] Build status: STARTED
[00:17:17.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:14:0) after 3m28s
[00:20:46.263] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:14:0)
[00:20:46.279] Build status: STARTED
[00:20:46.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:15:0) after 5m58s
[00:26:44.596] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:15:0)
[00:26:44.610] Build status: STARTED
[00:26:44.610] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:16:0) after 2m51s
[00:29:35.629] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:16:0)
[00:29:35.666] Build status: STARTED
[00:29:35.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:17:0) after 7m30s
[00:37:05.766] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:17:0)
[00:37:05.783] Build status: STARTED
[00:37:05.783] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:18:0) after 7m14s
[00:44:20.765] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:18:0)
[00:44:20.780] Build status: STARTED
[00:44:20.780] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:19:0) after 5m37s
[00:49:57.818] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:19:0)
[00:49:57.834] Build status: STARTED
[00:49:57.834] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:20:0) after 9m50s
[00:59:47.807] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:20:0)
[00:59:47.949] Build status: STARTED
[00:59:47.949] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:21:0) after 9m40s
[01:09:27.974] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:21:0)
[01:09:27.990] Build status: STARTED
[01:09:27.990] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:22:0) after 3m5s
[01:12:33.008] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:22:0)
[01:12:33.022] Build status: STARTED
[01:12:33.022] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:23:0) after 1m20s
[01:13:53.183] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:23:0)
[01:13:53.197] Build status: STARTED
[01:13:53.197] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:24:0) after 6m39s
[01:20:32.226] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:24:0)
[01:20:32.278] Build status: STARTED
[01:20:32.278] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:25:0) after 9m47s
[01:30:19.298] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:25:0)
[01:30:19.313] Build status: STARTED
[01:30:19.313] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:26:0) after 2m13s
[01:32:32.331] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:26:0)
[01:32:32.551] Build status: STARTED
[01:32:32.551] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:27:0) after 2m5s
[01:34:37.573] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:27:0)
[01:34:37.591] Build status: STARTED
[01:34:37.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:28:0) after 4m14s
[01:38:51.615] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:28:0)
[01:38:51.691] Build status: STARTED
[01:38:51.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:29:0) after 8m30s
[01:47:21.706] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:29:0)
[01:47:21.723] Build status: STARTED
[01:47:21.723] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:30:0) after 3m3s
[01:50:24.741] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:30:0)
[01:50:24.757] Build status: STARTED
[01:50:24.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:31:0) after 4m18s
[01:54:42.810] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:31:0)
[01:54:42.835] Build status: STARTED
[01:54:42.836] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:32:0) after 3m29s
[01:58:11.853] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:32:0)
[01:58:11.867] Build status: STARTED
[01:58:11.867] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:33:0) after 5m22s
[02:03:33.893] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:33:0)
[02:03:33.914] Build status: STARTED
[02:03:33.914] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:34:0) after 1m22s
[02:04:55.941] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:34:0)
[02:04:56.034] Build status: STARTED
[02:04:56.035] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:35:0) after 1m56s
[02:06:52.055] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:35:0)
[02:06:52.069] Build status: STARTED
[02:06:52.069] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:36:0) after 2m44s
[02:09:36.195] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:36:0)
[02:09:36.214] Build status: STARTED
[02:09:36.215] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:37:0) after 8m49s
[02:18:25.216] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:37:0)
[02:18:25.273] Build status: STARTED
[02:18:25.273] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:38:0) after 1m22s
[02:19:47.294] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:38:0)
[02:19:47.321] Build status: STARTED
[02:19:47.321] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:39:0) after 1m10s
[02:20:57.389] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:39:0)
[02:20:57.420] Build status: STARTED
[02:20:57.420] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:40:0) after 3m18s
[02:24:15.489] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:40:0)
[02:24:15.504] Build status: STARTED
[02:24:15.504] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:41:0) after 7m25s
[02:31:40.535] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:41:0)
[02:31:40.553] Build status: STARTED
[02:31:40.553] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:42:0) after 4m26s
[02:36:06.923] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:42:0)
[02:36:06.936] Build status: STARTED
[02:36:06.936] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:43:0) after 3m30s
[02:39:37.015] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:43:0)
[02:39:37.031] Build status: STARTED
[02:39:37.031] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:44:0) after 1m22s
[02:40:59.051] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:44:0)
[02:40:59.068] Build status: STARTED
[02:40:59.068] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:45:0) after 8m42s
[02:49:41.395] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:45:0)
[02:49:41.410] Build status: STARTED
[02:49:41.411] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:46:0) after 2m52s
[02:52:33.987] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:46:0)
[02:52:34.001] Build status: STARTED
[02:52:34.001] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:47:0) after 9m14s
[03:01:48.018] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:47:0)
[03:01:48.108] Build status: STARTED
[03:01:48.108] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:48:0) after 9m43s
[03:11:31.260] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:48:0)
[03:11:31.277] Build status: STARTED
[03:11:31.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:49:0) after 5m29s
[03:17:00.463] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:49:0)
[03:17:00.523] Build status: STARTED
[03:17:00.523] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:50:0) after 8m48s
[03:25:48.673] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:50:0)
[03:25:48.692] Build status: STARTED
[03:25:48.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:51:0) after 2m20s
[03:28:08.822] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:51:0)
[03:28:08.839] Build status: STARTED
[03:28:08.839] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:52:0) after 1m59s
[03:30:08.215] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:52:0)
[03:30:08.270] Build status: STARTED
[03:30:08.270] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:53:0) after 7m5s
[03:37:13.341] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:53:0)
[03:37:13.358] Build status: STARTED
[03:37:13.358] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:54:0) after 3m50s
[03:41:03.433] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:54:0)
[03:41:03.506] Build status: STARTED
[03:41:03.506] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:55:0) after 3m32s
[03:44:35.531] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:55:0)
[03:44:35.546] Build status: STARTED
[03:44:35.546] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:56:0) after 4m47s
[03:49:22.564] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:56:0)
[03:49:22.587] Build status: STARTED
[03:49:22.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:57:0) after 2m11s
[03:51:33.653] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:57:0)
[03:51:33.666] Build status: STARTED
[03:51:33.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:58:0) after 3m20s
[03:54:53.923] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:58:0)
[03:54:53.941] Build status: STARTED
[03:54:53.941] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:59:0) after 6m34s
[04:01:27.955] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:59:0)
[04:01:27.971] Build status: STARTED
[04:01:27.971] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8857634373268798016:60:0) after 3m36s
[04:02:42.226] Received PubSub notification, asking Buildbucket for the build status
[04:02:42.281] Build:
{
"id": "8692204311403780257",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 19397,
"createdBy": "project:chromium",
"createTime": "2026-01-19T23:23:37.466127066Z",
"startTime": "2026-01-19T23:24:27.344843821Z",
"endTime": "2026-01-20T04:02:41.976654843Z",
"updateTime": "2026-01-20T04:02:41.976654843Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "41bdc53fbb8337418365de6837ea5b94d00e7a3d",
"ref": "refs/heads/main"
}
}
}
[04:02:42.281] Invocation finished in 4h39m5.977566447s with status FAILED