[12:43:24.782] New invocation is queued and will start shortly
[12:43:27.062] Starting the invocation (attempt 1)
[12:43:27.090] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/2b3ee807d7a91d71a9e06256e0f0904ff9145281"
[12:43:27.090] Popped gitiles commit info from properties and tags
[12:43:27.090] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[12:43:27.090] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[12:43:27.090] Buildbucket request:
{
"requestId": "8850608087295275616",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2b3ee807d7a91d71a9e06256e0f0904ff9145281",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850608087295275616"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[12:43:28.013] Scheduled build:
{
"id": "8685178024143245425",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 20227,
"createdBy": "project:chromium",
"createTime": "2026-04-07T12:43:27.166303584Z",
"updateTime": "2026-04-07T12:43:27.166303584Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2b3ee807d7a91d71a9e06256e0f0904ff9145281",
"ref": "refs/heads/main"
}
}
}
[12:43:28.013] Task URL: https://cr-buildbucket.appspot.com/build/8685178024143245425
[12:43:28.013] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:2:0) after 8m4s
[12:43:31.302] Received PubSub notification, asking Buildbucket for the build status
[12:43:31.337] Build status: SCHEDULED
[12:44:36.740] Received PubSub notification, asking Buildbucket for the build status
[12:44:36.762] Build status: STARTED
[12:51:32.037] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:2:0)
[12:51:32.061] Build status: STARTED
[12:51:32.061] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:5:0) after 3m21s
[12:54:53.133] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:5:0)
[12:54:53.157] Build status: STARTED
[12:54:53.157] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:6:0) after 6m3s
[13:00:56.200] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:6:0)
[13:00:56.233] Build status: STARTED
[13:00:56.233] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:7:0) after 6m45s
[13:07:41.345] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:7:0)
[13:07:41.380] Build status: STARTED
[13:07:41.380] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:8:0) after 9m17s
[13:16:58.450] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:8:0)
[13:16:58.463] Build status: STARTED
[13:16:58.464] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:9:0) after 9m40s
[13:26:38.483] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:9:0)
[13:26:38.510] Build status: STARTED
[13:26:38.510] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:10:0) after 1m20s
[13:27:58.529] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:10:0)
[13:27:58.557] Build status: STARTED
[13:27:58.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:11:0) after 5m48s
[13:33:46.578] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:11:0)
[13:33:46.620] Build status: STARTED
[13:33:46.620] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:12:0) after 9m41s
[13:43:27.612] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:12:0)
[13:43:27.640] Build status: STARTED
[13:43:27.640] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:13:0) after 3m8s
[13:46:35.659] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:13:0)
[13:46:35.680] Build status: STARTED
[13:46:35.680] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:14:0) after 3m7s
[13:49:42.700] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:14:0)
[13:49:42.721] Build status: STARTED
[13:49:42.721] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:15:0) after 8m55s
[13:58:37.788] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:15:0)
[13:58:37.807] Build status: STARTED
[13:58:37.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:16:0) after 5m43s
[14:04:20.830] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:16:0)
[14:04:20.848] Build status: STARTED
[14:04:20.848] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:17:0) after 5m48s
[14:10:08.887] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:17:0)
[14:10:08.908] Build status: STARTED
[14:10:08.908] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:18:0) after 8m41s
[14:18:49.930] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:18:0)
[14:18:49.947] Build status: STARTED
[14:18:49.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:19:0) after 4m34s
[14:23:23.969] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:19:0)
[14:23:24.031] Build status: STARTED
[14:23:24.031] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:20:0) after 4m23s
[14:27:47.311] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:20:0)
[14:27:47.334] Build status: STARTED
[14:27:47.334] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:21:0) after 9m50s
[14:37:37.511] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:21:0)
[14:37:37.591] Build status: STARTED
[14:37:37.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:22:0) after 8m48s
[14:46:25.660] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:22:0)
[14:46:25.685] Build status: STARTED
[14:46:25.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:23:0) after 4m56s
[14:51:21.706] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:23:0)
[14:51:21.755] Build status: STARTED
[14:51:21.755] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:24:0) after 1m3s
[14:52:24.820] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:24:0)
[14:52:24.864] Build status: STARTED
[14:52:24.864] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:25:0) after 4m22s
[14:56:46.884] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:25:0)
[14:56:46.900] Build status: STARTED
[14:56:46.900] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:26:0) after 8m8s
[15:04:54.969] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:26:0)
[15:04:54.983] Build status: STARTED
[15:04:54.983] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:27:0) after 6m2s
[15:10:57.124] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:27:0)
[15:10:57.142] Build status: STARTED
[15:10:57.142] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:28:0) after 4m0s
[15:14:57.162] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:28:0)
[15:14:57.176] Build status: STARTED
[15:14:57.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:29:0) after 7m59s
[15:22:56.198] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:29:0)
[15:22:56.218] Build status: STARTED
[15:22:56.218] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:30:0) after 2m16s
[15:25:12.322] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:30:0)
[15:25:12.342] Build status: STARTED
[15:25:12.342] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:31:0) after 2m15s
[15:27:27.460] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:31:0)
[15:27:27.484] Build status: STARTED
[15:27:27.484] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:32:0) after 5m26s
[15:32:53.528] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:32:0)
[15:32:53.555] Build status: STARTED
[15:32:53.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:33:0) after 5m6s
[15:37:59.579] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:33:0)
[15:37:59.597] Build status: STARTED
[15:37:59.597] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:34:0) after 4m34s
[15:42:33.619] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:34:0)
[15:42:33.639] Build status: STARTED
[15:42:33.639] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:35:0) after 8m53s
[15:51:26.682] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:35:0)
[15:51:26.700] Build status: STARTED
[15:51:26.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:36:0) after 9m0s
[16:00:26.718] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:36:0)
[16:00:26.737] Build status: STARTED
[16:00:26.737] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:37:0) after 3m52s
[16:04:18.791] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:37:0)
[16:04:18.811] Build status: STARTED
[16:04:18.811] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:38:0) after 1m20s
[16:05:38.827] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:38:0)
[16:05:38.853] Build status: STARTED
[16:05:38.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:39:0) after 8m50s
[16:14:28.872] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:39:0)
[16:14:28.991] Build status: STARTED
[16:14:28.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:40:0) after 8m10s
[16:22:39.009] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:40:0)
[16:22:39.026] Build status: STARTED
[16:22:39.026] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:41:0) after 6m10s
[16:28:49.019] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:41:0)
[16:28:49.035] Build status: STARTED
[16:28:49.035] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:42:0) after 5m9s
[16:33:58.123] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:42:0)
[16:33:58.147] Build status: STARTED
[16:33:58.147] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:43:0) after 2m57s
[16:36:55.212] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:43:0)
[16:36:55.235] Build status: STARTED
[16:36:55.235] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:44:0) after 8m16s
[16:45:11.308] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:44:0)
[16:45:11.336] Build status: STARTED
[16:45:11.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:45:0) after 5m37s
[16:50:48.388] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:45:0)
[16:50:48.408] Build status: STARTED
[16:50:48.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:46:0) after 4m26s
[16:55:14.452] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:46:0)
[16:55:14.483] Build status: STARTED
[16:55:14.483] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:47:0) after 1m24s
[16:56:38.550] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:47:0)
[16:56:38.571] Build status: STARTED
[16:56:38.571] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:48:0) after 4m19s
[17:00:57.566] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:48:0)
[17:01:00.426] Build status: STARTED
[17:01:00.426] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:49:0) after 6m0s
[17:07:00.624] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:49:0)
[17:07:00.639] Build status: STARTED
[17:07:00.640] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:50:0) after 8m30s
[17:15:30.659] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:50:0)
[17:15:30.698] Build status: STARTED
[17:15:30.698] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:51:0) after 1m23s
[17:16:53.719] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:51:0)
[17:16:53.744] Build status: STARTED
[17:16:53.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:52:0) after 1m27s
[17:18:20.813] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:52:0)
[17:18:20.829] Build status: STARTED
[17:18:20.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:53:0) after 2m25s
[17:20:45.810] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:53:0)
[17:20:45.829] Build status: STARTED
[17:20:45.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:54:0) after 7m12s
[17:27:57.992] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:54:0)
[17:27:58.009] Build status: STARTED
[17:27:58.009] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:55:0) after 5m28s
[17:33:26.306] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:55:0)
[17:33:26.365] Build status: STARTED
[17:33:26.365] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:56:0) after 1m52s
[17:35:18.407] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:56:0)
[17:35:18.467] Build status: STARTED
[17:35:18.467] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:57:0) after 9m2s
[17:44:20.669] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:57:0)
[17:44:20.686] Build status: STARTED
[17:44:20.686] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:58:0) after 9m59s
[17:54:19.784] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:58:0)
[17:54:19.804] Build status: STARTED
[17:54:19.804] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:59:0) after 6m7s
[18:00:26.808] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:59:0)
[18:00:26.825] Build status: STARTED
[18:00:26.825] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:60:0) after 8m55s
[18:09:21.850] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:60:0)
[18:09:21.945] Build status: STARTED
[18:09:21.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8850608087295275616:61:0) after 5m38s
[18:12:54.391] Received PubSub notification, asking Buildbucket for the build status
[18:12:54.408] Build:
{
"id": "8685178024143245425",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 20227,
"createdBy": "project:chromium",
"createTime": "2026-04-07T12:43:27.166303584Z",
"startTime": "2026-04-07T12:44:30.602568519Z",
"endTime": "2026-04-07T18:12:54.055037459Z",
"updateTime": "2026-04-07T18:12:54.055037459Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2b3ee807d7a91d71a9e06256e0f0904ff9145281",
"ref": "refs/heads/main"
}
}
}
[18:12:54.408] Invocation finished in 5h29m29.630369289s with status FAILED