[00:28:23.272] New invocation is queued and will start shortly
[00:28:24.344] Starting the invocation (attempt 1)
[00:28:24.377] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d3b6dc6dcb6e4a034d83ccba7436ed562b6f56e5"
[00:28:24.377] Popped gitiles commit info from properties and tags
[00:28:24.377] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[00:28:24.378] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[00:28:24.378] Buildbucket request:
{
"requestId": "8843044185903017776",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d3b6dc6dcb6e4a034d83ccba7436ed562b6f56e5",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8843044185903017776"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[00:28:24.893] Scheduled build:
{
"id": "8677614124003706481",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 3559,
"createdBy": "project:chromium",
"createTime": "2026-06-30T00:28:24.461313722Z",
"updateTime": "2026-06-30T00:28:24.461313722Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d3b6dc6dcb6e4a034d83ccba7436ed562b6f56e5",
"ref": "refs/heads/main"
}
}
}
[00:28:24.893] Task URL: https://cr-buildbucket.appspot.com/build/8677614124003706481
[00:28:24.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:2:0) after 1m11s
[00:28:25.939] Received PubSub notification, asking Buildbucket for the build status
[00:28:25.953] Build status: SCHEDULED
[00:28:28.481] Received PubSub notification, asking Buildbucket for the build status
[00:28:28.506] Build status: STARTED
[00:29:35.922] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:2:0)
[00:29:35.941] Build status: STARTED
[00:29:35.941] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:5:0) after 1m25s
[00:31:01.094] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:5:0)
[00:31:01.170] Build status: STARTED
[00:31:01.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:6:0) after 9m51s
[00:40:52.198] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:6:0)
[00:40:52.228] Build status: STARTED
[00:40:52.228] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:7:0) after 8m41s
[00:49:33.281] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:7:0)
[00:49:33.300] Build status: STARTED
[00:49:33.304] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:8:0) after 1m36s
[00:51:09.291] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:8:0)
[00:51:09.308] Build status: STARTED
[00:51:09.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:9:0) after 9m38s
[01:00:47.447] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:9:0)
[01:00:47.504] Build status: STARTED
[01:00:47.505] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:10:0) after 2m14s
[01:03:01.515] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:10:0)
[01:03:01.535] Build status: STARTED
[01:03:01.535] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:11:0) after 2m27s
[01:05:28.572] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:11:0)
[01:05:28.586] Build status: STARTED
[01:05:28.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:12:0) after 3m1s
[01:08:29.706] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:12:0)
[01:08:29.724] Build status: STARTED
[01:08:29.724] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:13:0) after 2m10s
[01:10:39.838] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:13:0)
[01:10:39.859] Build status: STARTED
[01:10:39.866] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:14:0) after 8m46s
[01:19:25.924] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:14:0)
[01:19:25.988] Build status: STARTED
[01:19:25.988] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:15:0) after 2m32s
[01:21:58.015] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:15:0)
[01:21:58.052] Build status: STARTED
[01:21:58.052] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:16:0) after 2m58s
[01:24:56.049] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:16:0)
[01:24:56.072] Build status: STARTED
[01:24:56.072] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:17:0) after 2m58s
[01:27:54.122] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:17:0)
[01:27:54.148] Build status: STARTED
[01:27:54.148] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:18:0) after 3m35s
[01:31:29.169] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:18:0)
[01:31:29.232] Build status: STARTED
[01:31:29.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:19:0) after 2m50s
[01:34:19.313] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:19:0)
[01:34:19.393] Build status: STARTED
[01:34:19.393] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:20:0) after 9m27s
[01:43:46.472] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:20:0)
[01:43:46.496] Build status: STARTED
[01:43:46.496] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:21:0) after 4m55s
[01:48:41.526] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:21:0)
[01:48:41.563] Build status: STARTED
[01:48:41.564] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:22:0) after 7m2s
[01:55:43.585] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:22:0)
[01:55:43.605] Build status: STARTED
[01:55:43.605] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:23:0) after 1m45s
[01:57:28.626] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:23:0)
[01:57:28.652] Build status: STARTED
[01:57:28.652] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:24:0) after 8m13s
[02:05:41.704] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:24:0)
[02:05:41.761] Build status: STARTED
[02:05:41.761] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:25:0) after 5m51s
[02:11:32.851] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:25:0)
[02:11:32.866] Build status: STARTED
[02:11:32.866] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:26:0) after 7m17s
[02:18:49.936] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:26:0)
[02:18:49.956] Build status: STARTED
[02:18:49.956] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:27:0) after 9m59s
[02:28:48.958] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:27:0)
[02:28:49.120] Build status: STARTED
[02:28:49.120] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:28:0) after 5m40s
[02:34:29.154] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:28:0)
[02:34:29.172] Build status: STARTED
[02:34:29.172] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:29:0) after 3m55s
[02:38:24.198] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:29:0)
[02:38:24.218] Build status: STARTED
[02:38:24.218] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:30:0) after 9m57s
[02:48:21.286] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:30:0)
[02:48:21.304] Build status: STARTED
[02:48:21.304] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:31:0) after 2m30s
[02:50:51.323] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:31:0)
[02:50:51.341] Build status: STARTED
[02:50:51.341] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:32:0) after 2m18s
[02:53:09.362] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:32:0)
[02:53:09.380] Build status: STARTED
[02:53:09.380] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:33:0) after 1m24s
[02:54:33.399] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:33:0)
[02:54:33.417] Build status: STARTED
[02:54:33.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:34:0) after 3m19s
[02:57:52.517] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:34:0)
[02:57:52.541] Build status: STARTED
[02:57:52.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:35:0) after 7m30s
[03:05:22.572] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:35:0)
[03:05:22.596] Build status: STARTED
[03:05:22.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:36:0) after 8m3s
[03:13:25.614] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:36:0)
[03:13:25.633] Build status: STARTED
[03:13:25.633] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:37:0) after 1m27s
[03:14:52.678] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:37:0)
[03:14:52.695] Build status: STARTED
[03:14:52.695] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:38:0) after 6m9s
[03:21:01.714] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:38:0)
[03:21:01.732] Build status: STARTED
[03:21:01.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:39:0) after 9m1s
[03:30:02.774] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:39:0)
[03:30:02.842] Build status: STARTED
[03:30:02.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:40:0) after 8m53s
[03:38:55.831] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:40:0)
[03:38:55.864] Build status: STARTED
[03:38:55.864] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:41:0) after 4m20s
[03:43:15.897] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:41:0)
[03:43:15.926] Build status: STARTED
[03:43:15.926] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:42:0) after 7m55s
[03:51:10.951] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:42:0)
[03:51:10.966] Build status: STARTED
[03:51:10.966] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:43:0) after 4m59s
[03:56:09.988] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:43:0)
[03:56:10.009] Build status: STARTED
[03:56:10.009] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:44:0) after 4m48s
[04:00:58.130] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:44:0)
[04:00:58.163] Build status: STARTED
[04:00:58.163] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:45:0) after 3m5s
[04:04:03.279] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:45:0)
[04:04:03.307] Build status: STARTED
[04:04:03.307] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:46:0) after 7m6s
[04:11:09.345] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:46:0)
[04:11:09.389] Build status: STARTED
[04:11:09.389] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:47:0) after 8m31s
[04:19:40.407] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:47:0)
[04:19:40.446] Build status: STARTED
[04:19:40.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:48:0) after 1m9s
[04:20:49.470] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:48:0)
[04:20:49.486] Build status: STARTED
[04:20:49.486] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:49:0) after 8m8s
[04:28:57.670] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:49:0)
[04:28:57.699] Build status: STARTED
[04:28:57.699] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:50:0) after 3m25s
[04:32:22.717] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:50:0)
[04:32:22.733] Build status: STARTED
[04:32:22.733] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:51:0) after 4m38s
[04:37:00.771] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:51:0)
[04:37:00.788] Build status: STARTED
[04:37:00.788] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:52:0) after 7m18s
[04:44:18.815] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:52:0)
[04:44:18.831] Build status: STARTED
[04:44:18.832] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:53:0) after 5m20s
[04:49:38.831] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:53:0)
[04:49:38.856] Build status: STARTED
[04:49:38.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:54:0) after 7m30s
[04:57:08.923] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:54:0)
[04:57:08.941] Build status: STARTED
[04:57:08.941] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:55:0) after 4m26s
[05:01:34.984] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:55:0)
[05:01:35.008] Build status: STARTED
[05:01:35.008] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:56:0) after 3m48s
[05:05:23.035] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:56:0)
[05:05:23.062] Build status: STARTED
[05:05:23.062] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:57:0) after 8m51s
[05:14:14.047] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:57:0)
[05:14:14.124] Build status: STARTED
[05:14:14.124] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:58:0) after 1m27s
[05:15:41.154] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:58:0)
[05:15:41.171] Build status: STARTED
[05:15:41.171] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:59:0) after 9m29s
[05:25:10.243] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:59:0)
[05:25:10.260] Build status: STARTED
[05:25:10.260] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8843044185903017776:60:0) after 7m40s
[05:25:51.607] Received PubSub notification, asking Buildbucket for the build status
[05:25:51.640] Build:
{
"id": "8677614124003706481",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 3559,
"createdBy": "project:chromium",
"createTime": "2026-06-30T00:28:24.461313722Z",
"startTime": "2026-06-30T00:28:28.286367489Z",
"endTime": "2026-06-30T05:25:51.240450737Z",
"updateTime": "2026-06-30T05:25:51.240450737Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d3b6dc6dcb6e4a034d83ccba7436ed562b6f56e5",
"ref": "refs/heads/main"
}
}
}
[05:25:51.640] Invocation finished in 4h57m28.37487398s with status FAILED