[04:58:22.237] New invocation is queued and will start shortly
[04:58:24.120] Starting the invocation (attempt 1)
[04:58:24.143] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/8271028eb2ce709c94530d877c3833a26c59ea06"
[04:58:24.143] Popped gitiles commit info from properties and tags
[04:58:24.144] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:58:24.144] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:58:24.144] Buildbucket request:
{
"requestId": "8848916002871618496",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "8271028eb2ce709c94530d877c3833a26c59ea06",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8848916002871618496"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:58:24.606] Scheduled build:
{
"id": "8683485940162714737",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 2476,
"createdBy": "project:chromium",
"createTime": "2026-04-26T04:58:24.200343645Z",
"updateTime": "2026-04-26T04:58:24.200343645Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "8271028eb2ce709c94530d877c3833a26c59ea06",
"ref": "refs/heads/main"
}
}
}
[04:58:24.606] Task URL: https://cr-buildbucket.appspot.com/build/8683485940162714737
[04:58:24.606] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:2:0) after 1m1s
[04:58:26.465] Received PubSub notification, asking Buildbucket for the build status
[04:58:26.491] Build status: SCHEDULED
[04:58:55.830] Received PubSub notification, asking Buildbucket for the build status
[04:58:55.849] Build status: STARTED
[04:59:25.703] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:2:0)
[04:59:25.761] Build status: STARTED
[04:59:25.761] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:5:0) after 6m54s
[05:06:19.875] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:5:0)
[05:06:19.891] Build status: STARTED
[05:06:19.891] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:6:0) after 4m13s
[05:10:32.885] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:6:0)
[05:10:32.908] Build status: STARTED
[05:10:32.908] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:7:0) after 1m25s
[05:11:58.135] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:7:0)
[05:11:58.154] Build status: STARTED
[05:11:58.155] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:8:0) after 7m17s
[05:19:15.174] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:8:0)
[05:19:15.233] Build status: STARTED
[05:19:15.233] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:9:0) after 4m36s
[05:23:51.342] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:9:0)
[05:23:51.539] Build status: STARTED
[05:23:51.539] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:10:0) after 3m5s
[05:26:56.656] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:10:0)
[05:26:56.696] Build status: STARTED
[05:26:56.696] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:11:0) after 9m26s
[05:36:22.719] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:11:0)
[05:36:22.788] Build status: STARTED
[05:36:22.788] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:12:0) after 8m32s
[05:44:54.826] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:12:0)
[05:44:54.895] Build status: STARTED
[05:44:54.895] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:13:0) after 8m28s
[05:53:23.038] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:13:0)
[05:53:23.057] Build status: STARTED
[05:53:23.057] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:14:0) after 6m4s
[05:59:27.070] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:14:0)
[05:59:27.088] Build status: STARTED
[05:59:27.089] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:15:0) after 4m39s
[06:04:06.129] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:15:0)
[06:04:06.191] Build status: STARTED
[06:04:06.191] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:16:0) after 2m30s
[06:06:36.282] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:16:0)
[06:06:36.310] Build status: STARTED
[06:06:36.310] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:17:0) after 5m31s
[06:12:07.413] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:17:0)
[06:12:07.431] Build status: STARTED
[06:12:07.431] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:18:0) after 4m53s
[06:17:00.451] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:18:0)
[06:17:00.475] Build status: STARTED
[06:17:00.475] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:19:0) after 6m10s
[06:23:10.717] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:19:0)
[06:23:10.733] Build status: STARTED
[06:23:10.733] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:20:0) after 3m30s
[06:26:40.748] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:20:0)
[06:26:40.787] Build status: STARTED
[06:26:40.787] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:21:0) after 5m11s
[06:31:51.909] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:21:0)
[06:31:51.924] Build status: STARTED
[06:31:51.924] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:22:0) after 1m23s
[06:33:15.046] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:22:0)
[06:33:15.075] Build status: STARTED
[06:33:15.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:23:0) after 5m34s
[06:38:49.093] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:23:0)
[06:38:49.110] Build status: STARTED
[06:38:49.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:24:0) after 1m45s
[06:40:34.131] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:24:0)
[06:40:34.149] Build status: STARTED
[06:40:34.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:25:0) after 7m53s
[06:48:27.184] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:25:0)
[06:48:27.203] Build status: STARTED
[06:48:27.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:26:0) after 7m24s
[06:55:51.319] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:26:0)
[06:55:51.335] Build status: STARTED
[06:55:51.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:27:0) after 9m1s
[07:04:52.309] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:27:0)
[07:04:52.324] Build status: STARTED
[07:04:52.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:28:0) after 4m8s
[07:09:00.410] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:28:0)
[07:09:00.427] Build status: STARTED
[07:09:00.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:29:0) after 2m23s
[07:11:23.443] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:29:0)
[07:11:23.463] Build status: STARTED
[07:11:23.463] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:30:0) after 5m19s
[07:16:42.488] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:30:0)
[07:16:42.537] Build status: STARTED
[07:16:42.537] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:31:0) after 9m1s
[07:25:43.630] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:31:0)
[07:25:43.651] Build status: STARTED
[07:25:43.651] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:32:0) after 2m36s
[07:28:19.702] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:32:0)
[07:28:19.882] Build status: STARTED
[07:28:19.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:33:0) after 7m41s
[07:36:00.899] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:33:0)
[07:36:01.266] Build status: STARTED
[07:36:01.266] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:34:0) after 2m3s
[07:38:04.290] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:34:0)
[07:38:04.332] Build status: STARTED
[07:38:04.332] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:35:0) after 5m59s
[07:44:03.349] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:35:0)
[07:44:03.365] Build status: STARTED
[07:44:03.365] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:36:0) after 7m23s
[07:51:26.385] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:36:0)
[07:51:26.403] Build status: STARTED
[07:51:26.403] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:37:0) after 1m5s
[07:52:31.540] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:37:0)
[07:52:31.559] Build status: STARTED
[07:52:31.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:38:0) after 1m12s
[07:53:43.579] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:38:0)
[07:53:43.602] Build status: STARTED
[07:53:43.602] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:39:0) after 8m20s
[08:02:03.791] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:39:0)
[08:02:03.812] Build status: STARTED
[08:02:03.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:40:0) after 9m59s
[08:12:02.797] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:40:0)
[08:12:02.831] Build status: STARTED
[08:12:02.832] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:41:0) after 6m15s
[08:18:18.047] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:41:0)
[08:18:18.064] Build status: STARTED
[08:18:18.064] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:42:0) after 5m44s
[08:24:02.216] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:42:0)
[08:24:02.238] Build status: STARTED
[08:24:02.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:43:0) after 9m41s
[08:33:43.568] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:43:0)
[08:33:43.586] Build status: STARTED
[08:33:43.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:44:0) after 7m4s
[08:40:48.014] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:44:0)
[08:40:48.039] Build status: STARTED
[08:40:48.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:45:0) after 6m51s
[08:47:39.081] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:45:0)
[08:47:39.101] Build status: STARTED
[08:47:39.101] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:46:0) after 1m18s
[08:48:57.114] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:46:0)
[08:48:57.132] Build status: STARTED
[08:48:57.132] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:47:0) after 8m24s
[08:57:21.169] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:47:0)
[08:57:21.182] Build status: STARTED
[08:57:21.182] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:48:0) after 9m28s
[09:06:49.172] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:48:0)
[09:06:49.195] Build status: STARTED
[09:06:49.195] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:49:0) after 9m21s
[09:16:10.405] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:49:0)
[09:16:10.427] Build status: STARTED
[09:16:10.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:50:0) after 3m5s
[09:19:15.493] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:50:0)
[09:19:15.519] Build status: STARTED
[09:19:15.519] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:51:0) after 9m26s
[09:28:41.676] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:51:0)
[09:28:41.734] Build status: STARTED
[09:28:41.734] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:52:0) after 2m22s
[09:31:04.002] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:52:0)
[09:31:04.105] Build status: STARTED
[09:31:04.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:53:0) after 2m46s
[09:33:50.300] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:53:0)
[09:33:50.324] Build status: STARTED
[09:33:50.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:54:0) after 9m52s
[09:43:42.539] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:54:0)
[09:43:42.560] Build status: STARTED
[09:43:42.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:55:0) after 6m18s
[09:50:00.833] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:55:0)
[09:50:00.851] Build status: STARTED
[09:50:00.851] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8848916002871618496:56:0) after 9m55s
[09:55:08.546] Received PubSub notification, asking Buildbucket for the build status
[09:55:08.562] Build:
{
"id": "8683485940162714737",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 2476,
"createdBy": "project:chromium",
"createTime": "2026-04-26T04:58:24.200343645Z",
"startTime": "2026-04-26T04:58:49.166101446Z",
"endTime": "2026-04-26T09:55:08.140229516Z",
"updateTime": "2026-04-26T09:55:08.140229516Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "8271028eb2ce709c94530d877c3833a26c59ea06",
"ref": "refs/heads/main"
}
}
}
[09:55:08.562] Invocation finished in 4h56m46.329312751s with status FAILED