[00:30:00.529] New invocation is queued and will start shortly
[00:30:04.322] Starting the invocation (attempt 1)
[00:30:04.358] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/1a2964f4a7e6ce62845ce76aa53d3fb1ff05ecda"
[00:30:04.358] Popped gitiles commit info from properties and tags
[00:30:04.359] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[00:30:04.359] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[00:30:04.359] Buildbucket request:
{
"requestId": "8850563632132585120",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1a2964f4a7e6ce62845ce76aa53d3fb1ff05ecda",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850563632132585120"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[00:30:05.486] Scheduled build:
{
"id": "8685133567254774561",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 2107,
"createdBy": "project:chromium",
"createTime": "2026-04-08T00:30:04.559418373Z",
"updateTime": "2026-04-08T00:30:04.559418373Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1a2964f4a7e6ce62845ce76aa53d3fb1ff05ecda",
"ref": "refs/heads/main"
}
}
}
[00:30:05.486] Task URL: https://cr-buildbucket.appspot.com/build/8685133567254774561
[00:30:05.486] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:2:0) after 4m3s
[00:32:53.171] Received PubSub notification, asking Buildbucket for the build status
[00:32:53.200] Build status: SCHEDULED
[00:34:08.801] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:2:0)
[00:34:08.839] Build status: SCHEDULED
[00:34:08.839] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:4:0) after 8m32s
[00:35:54.005] Received PubSub notification, asking Buildbucket for the build status
[00:35:54.063] Build status: STARTED
[00:42:41.058] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:4:0)
[00:42:41.076] Build status: STARTED
[00:42:41.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:6:0) after 4m41s
[00:47:22.150] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:6:0)
[00:47:22.215] Build status: STARTED
[00:47:22.215] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:7:0) after 9m5s
[00:56:27.254] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:7:0)
[00:56:27.310] Build status: STARTED
[00:56:27.310] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:8:0) after 5m33s
[01:02:00.379] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:8:0)
[01:02:00.401] Build status: STARTED
[01:02:00.401] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:9:0) after 1m46s
[01:03:46.450] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:9:0)
[01:03:46.469] Build status: STARTED
[01:03:46.469] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:10:0) after 8m47s
[01:12:33.507] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:10:0)
[01:12:33.527] Build status: STARTED
[01:12:33.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:11:0) after 8m23s
[01:20:56.588] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:11:0)
[01:20:56.616] Build status: STARTED
[01:20:56.616] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:12:0) after 3m10s
[01:24:06.636] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:12:0)
[01:24:06.690] Build status: STARTED
[01:24:06.690] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:13:0) after 3m46s
[01:27:52.727] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:13:0)
[01:27:52.744] Build status: STARTED
[01:27:52.744] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:14:0) after 3m36s
[01:31:28.808] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:14:0)
[01:31:28.824] Build status: STARTED
[01:31:28.824] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:15:0) after 9m31s
[01:41:00.173] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:15:0)
[01:41:00.260] Build status: STARTED
[01:41:00.260] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:16:0) after 3m8s
[01:44:08.283] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:16:0)
[01:44:08.304] Build status: STARTED
[01:44:08.304] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:17:0) after 9m13s
[01:53:21.711] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:17:0)
[01:53:21.731] Build status: STARTED
[01:53:21.731] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:18:0) after 6m58s
[02:00:19.724] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:18:0)
[02:00:19.761] Build status: STARTED
[02:00:19.761] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:19:0) after 3m54s
[02:04:13.939] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:19:0)
[02:04:13.956] Build status: STARTED
[02:04:13.956] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:20:0) after 6m44s
[02:10:58.055] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:20:0)
[02:10:58.079] Build status: STARTED
[02:10:58.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:21:0) after 6m28s
[02:17:26.154] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:21:0)
[02:17:26.175] Build status: STARTED
[02:17:26.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:22:0) after 3m47s
[02:21:13.262] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:22:0)
[02:21:13.345] Build status: STARTED
[02:21:13.345] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:23:0) after 2m24s
[02:23:37.421] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:23:0)
[02:23:37.536] Build status: STARTED
[02:23:37.536] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:24:0) after 3m4s
[02:26:41.751] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:24:0)
[02:26:41.772] Build status: STARTED
[02:26:41.772] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:25:0) after 2m24s
[02:29:05.897] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:25:0)
[02:29:05.915] Build status: STARTED
[02:29:05.915] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:26:0) after 5m12s
[02:34:18.065] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:26:0)
[02:34:18.103] Build status: STARTED
[02:34:18.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:27:0) after 4m29s
[02:38:47.194] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:27:0)
[02:38:47.227] Build status: STARTED
[02:38:47.227] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:28:0) after 6m29s
[02:45:16.251] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:28:0)
[02:45:16.284] Build status: STARTED
[02:45:16.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:29:0) after 8m45s
[02:54:01.299] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:29:0)
[02:54:01.316] Build status: STARTED
[02:54:01.316] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:30:0) after 8m51s
[03:02:52.383] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:30:0)
[03:02:52.398] Build status: STARTED
[03:02:52.398] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:31:0) after 1m51s
[03:04:43.628] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:31:0)
[03:04:43.647] Build status: STARTED
[03:04:43.647] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:32:0) after 1m14s
[03:05:57.672] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:32:0)
[03:05:57.699] Build status: STARTED
[03:05:57.699] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:33:0) after 7m36s
[03:13:33.691] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:33:0)
[03:13:33.722] Build status: STARTED
[03:13:33.722] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:34:0) after 3m0s
[03:16:33.738] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:34:0)
[03:16:33.813] Build status: STARTED
[03:16:33.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:35:0) after 3m5s
[03:19:38.948] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:35:0)
[03:19:38.969] Build status: STARTED
[03:19:38.969] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:36:0) after 2m9s
[03:21:47.994] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:36:0)
[03:21:48.018] Build status: STARTED
[03:21:48.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:37:0) after 8m49s
[03:30:37.037] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:37:0)
[03:30:37.058] Build status: STARTED
[03:30:37.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:38:0) after 6m9s
[03:36:46.177] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:38:0)
[03:36:46.200] Build status: STARTED
[03:36:46.200] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:39:0) after 7m54s
[03:44:40.216] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:39:0)
[03:44:40.234] Build status: STARTED
[03:44:40.234] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:40:0) after 5m33s
[03:50:13.254] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:40:0)
[03:50:13.319] Build status: STARTED
[03:50:13.319] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:41:0) after 7m47s
[03:58:00.467] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:41:0)
[03:58:00.482] Build status: STARTED
[03:58:00.482] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:42:0) after 5m10s
[04:03:10.568] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:42:0)
[04:03:10.642] Build status: STARTED
[04:03:10.642] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:43:0) after 8m6s
[04:11:16.763] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:43:0)
[04:11:16.798] Build status: STARTED
[04:11:16.798] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:44:0) after 7m43s
[04:18:59.817] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:44:0)
[04:18:59.844] Build status: STARTED
[04:18:59.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:45:0) after 3m11s
[04:22:11.041] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:45:0)
[04:22:11.057] Build status: STARTED
[04:22:11.057] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:46:0) after 8m17s
[04:30:28.220] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:46:0)
[04:30:28.236] Build status: STARTED
[04:30:28.236] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:47:0) after 5m1s
[04:35:29.239] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:47:0)
[04:35:29.269] Build status: STARTED
[04:35:29.269] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:48:0) after 9m46s
[04:45:15.319] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:48:0)
[04:45:15.456] Build status: STARTED
[04:45:15.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:49:0) after 6m45s
[04:52:00.567] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:49:0)
[04:52:00.587] Build status: STARTED
[04:52:00.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8850563632132585120:50:0) after 4m12s
[04:54:53.526] Received PubSub notification, asking Buildbucket for the build status
[04:54:53.542] Build:
{
"id": "8685133567254774561",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 2107,
"createdBy": "project:chromium",
"createTime": "2026-04-08T00:30:04.559418373Z",
"startTime": "2026-04-08T00:34:14.346665876Z",
"endTime": "2026-04-08T04:54:52.821081187Z",
"updateTime": "2026-04-08T04:54:52.821081187Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1a2964f4a7e6ce62845ce76aa53d3fb1ff05ecda",
"ref": "refs/heads/main"
}
}
}
[04:54:53.542] Invocation finished in 4h24m53.019065551s with status FAILED