[12:07:14.587] New invocation is queued and will start shortly
[12:07:15.874] Starting the invocation (attempt 1)
[12:07:15.902] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/e78530e9709f91805a09988f0ddccb2ad53d1bcb"
[12:07:15.902] Popped gitiles commit info from properties and tags
[12:07:15.902] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[12:07:15.902] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[12:07:15.902] Buildbucket request:
{
"requestId": "8833578133228213456",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e78530e9709f91805a09988f0ddccb2ad53d1bcb",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8833578133228213456"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[12:07:16.519] Scheduled build:
{
"id": "8668148071126048113",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 23267,
"createdBy": "project:chromium",
"createTime": "2026-10-12T12:07:15.968323233Z",
"updateTime": "2026-10-12T12:07:15.968323233Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e78530e9709f91805a09988f0ddccb2ad53d1bcb",
"ref": "refs/heads/main"
}
}
}
[12:07:16.519] Task URL: https://cr-buildbucket.appspot.com/build/8668148071126048113
[12:07:16.519] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:2:0) after 1m57s
[12:07:17.702] Received PubSub notification, asking Buildbucket for the build status
[12:07:17.727] Build status: SCHEDULED
[12:08:52.854] Received PubSub notification, asking Buildbucket for the build status
[12:08:52.880] Build status: STARTED
[12:09:13.543] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:2:0)
[12:09:13.574] Build status: STARTED
[12:09:13.574] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:5:0) after 5m44s
[12:14:57.571] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:5:0)
[12:14:57.592] Build status: STARTED
[12:14:57.592] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:6:0) after 7m47s
[12:22:44.793] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:6:0)
[12:22:44.815] Build status: STARTED
[12:22:44.815] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:7:0) after 5m0s
[12:27:44.920] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:7:0)
[12:27:44.939] Build status: STARTED
[12:27:44.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:8:0) after 5m36s
[12:33:20.999] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:8:0)
[12:33:21.207] Build status: STARTED
[12:33:21.207] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:9:0) after 7m6s
[12:40:27.224] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:9:0)
[12:40:27.245] Build status: STARTED
[12:40:27.245] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:10:0) after 3m25s
[12:43:52.396] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:10:0)
[12:43:52.417] Build status: STARTED
[12:43:52.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:11:0) after 3m24s
[12:47:16.421] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:11:0)
[12:47:16.449] Build status: STARTED
[12:47:16.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:12:0) after 3m32s
[12:50:48.443] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:12:0)
[12:50:48.459] Build status: STARTED
[12:50:48.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:13:0) after 8m44s
[12:59:32.455] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:13:0)
[12:59:32.476] Build status: STARTED
[12:59:32.476] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:14:0) after 6m6s
[13:05:38.596] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:14:0)
[13:05:38.617] Build status: STARTED
[13:05:38.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:15:0) after 3m7s
[13:08:45.624] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:15:0)
[13:08:45.657] Build status: STARTED
[13:08:45.658] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:16:0) after 9m19s
[13:18:04.729] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:16:0)
[13:18:04.753] Build status: STARTED
[13:18:04.753] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:17:0) after 9m14s
[13:27:18.730] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:17:0)
[13:27:18.764] Build status: STARTED
[13:27:18.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:18:0) after 7m23s
[13:34:41.802] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:18:0)
[13:34:41.821] Build status: STARTED
[13:34:41.821] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:19:0) after 2m31s
[13:37:12.877] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:19:0)
[13:37:12.903] Build status: STARTED
[13:37:12.904] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:20:0) after 8m37s
[13:45:49.983] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:20:0)
[13:45:50.002] Build status: STARTED
[13:45:50.002] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:21:0) after 3m47s
[13:49:37.027] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:21:0)
[13:49:37.056] Build status: STARTED
[13:49:37.056] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:22:0) after 7m52s
[13:57:29.096] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:22:0)
[13:57:29.124] Build status: STARTED
[13:57:29.125] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:23:0) after 5m29s
[14:02:58.243] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:23:0)
[14:02:58.380] Build status: STARTED
[14:02:58.380] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:24:0) after 4m8s
[14:07:06.404] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:24:0)
[14:07:06.425] Build status: STARTED
[14:07:06.425] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:25:0) after 1m56s
[14:09:02.406] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:25:0)
[14:09:02.430] Build status: STARTED
[14:09:02.430] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:26:0) after 7m13s
[14:16:15.514] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:26:0)
[14:16:15.538] Build status: STARTED
[14:16:15.538] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:27:0) after 3m59s
[14:20:14.693] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:27:0)
[14:20:14.764] Build status: STARTED
[14:20:14.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:28:0) after 7m47s
[14:28:01.842] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:28:0)
[14:28:01.860] Build status: STARTED
[14:28:01.860] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:29:0) after 4m4s
[14:32:06.047] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:29:0)
[14:32:06.076] Build status: STARTED
[14:32:06.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:30:0) after 7m29s
[14:39:35.110] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:30:0)
[14:39:35.128] Build status: STARTED
[14:39:35.128] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:31:0) after 2m58s
[14:42:33.155] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:31:0)
[14:42:33.205] Build status: STARTED
[14:42:33.205] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:32:0) after 2m36s
[14:45:09.293] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:32:0)
[14:45:09.312] Build status: STARTED
[14:45:09.312] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:33:0) after 3m50s
[14:48:59.338] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:33:0)
[14:48:59.361] Build status: STARTED
[14:48:59.361] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:34:0) after 6m15s
[14:55:14.389] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:34:0)
[14:55:14.415] Build status: STARTED
[14:55:14.415] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:35:0) after 2m25s
[14:57:39.469] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:35:0)
[14:57:39.498] Build status: STARTED
[14:57:39.498] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:36:0) after 1m42s
[14:59:21.540] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:36:0)
[14:59:21.563] Build status: STARTED
[14:59:21.563] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:37:0) after 5m34s
[15:04:55.546] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:37:0)
[15:04:55.562] Build status: STARTED
[15:04:55.563] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:38:0) after 6m49s
[15:11:44.666] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:38:0)
[15:11:44.715] Build status: STARTED
[15:11:44.715] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:39:0) after 6m35s
[15:18:19.749] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:39:0)
[15:18:19.768] Build status: STARTED
[15:18:19.769] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:40:0) after 5m19s
[15:23:38.768] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:40:0)
[15:23:38.787] Build status: STARTED
[15:23:38.787] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:41:0) after 2m13s
[15:25:51.821] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:41:0)
[15:25:51.855] Build status: STARTED
[15:25:51.855] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:42:0) after 9m20s
[15:35:11.872] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:42:0)
[15:35:11.979] Build status: STARTED
[15:35:11.979] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:43:0) after 3m10s
[15:38:22.002] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:43:0)
[15:38:22.073] Build status: STARTED
[15:38:22.073] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:44:0) after 7m0s
[15:45:22.140] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:44:0)
[15:45:22.159] Build status: STARTED
[15:45:22.160] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:45:0) after 9m33s
[15:54:55.185] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:45:0)
[15:54:55.207] Build status: STARTED
[15:54:55.208] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:46:0) after 3m41s
[15:58:36.239] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:46:0)
[15:58:36.276] Build status: STARTED
[15:58:36.276] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8833578133228213456:47:0) after 9m29s
[16:02:32.902] Received PubSub notification, asking Buildbucket for the build status
[16:02:32.923] Build:
{
"id": "8668148071126048113",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 23267,
"createdBy": "project:chromium",
"createTime": "2026-10-12T12:07:15.968323233Z",
"startTime": "2026-10-12T12:08:52.542091278Z",
"endTime": "2026-10-12T16:02:32.591654870Z",
"updateTime": "2026-10-12T16:02:32.591654870Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e78530e9709f91805a09988f0ddccb2ad53d1bcb",
"ref": "refs/heads/main"
}
}
}
[16:02:32.923] Invocation finished in 3h55m18.342624596s with status FAILED