[17:32:40.721] New invocation is queued and will start shortly
[17:32:41.879] Starting the invocation (attempt 1)
[17:32:41.914] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/03affe5919ad23fb7a463fd319f8d7dfbafb1b41"
[17:32:41.914] Popped gitiles commit info from properties and tags
[17:32:41.914] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[17:32:41.914] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[17:32:41.914] Buildbucket request:
{
"requestId": "8858471824357804448",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "03affe5919ad23fb7a463fd319f8d7dfbafb1b41",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8858471824357804448"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[17:32:42.345] Scheduled build:
{
"id": "8693041762389203153",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1156,
"createdBy": "project:chromium",
"createTime": "2026-01-10T17:32:41.970590002Z",
"updateTime": "2026-01-10T17:32:41.970590002Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "03affe5919ad23fb7a463fd319f8d7dfbafb1b41",
"ref": "refs/heads/main"
}
}
}
[17:32:42.345] Task URL: https://cr-buildbucket.appspot.com/build/8693041762389203153
[17:32:42.345] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:2:0) after 9m3s
[17:32:43.304] Received PubSub notification, asking Buildbucket for the build status
[17:32:43.331] Build status: SCHEDULED
[17:32:46.409] Received PubSub notification, asking Buildbucket for the build status
[17:32:46.424] Build status: STARTED
[17:41:45.359] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:2:0)
[17:41:45.372] Build status: STARTED
[17:41:45.372] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:5:0) after 6m34s
[17:48:19.386] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:5:0)
[17:48:19.407] Build status: STARTED
[17:48:19.407] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:6:0) after 5m30s
[17:53:49.554] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:6:0)
[17:53:49.592] Build status: STARTED
[17:53:49.592] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:7:0) after 2m27s
[17:56:16.603] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:7:0)
[17:56:16.631] Build status: STARTED
[17:56:16.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:8:0) after 9m54s
[18:06:10.924] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:8:0)
[18:06:10.939] Build status: STARTED
[18:06:10.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:9:0) after 6m38s
[18:12:48.971] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:9:0)
[18:12:49.045] Build status: STARTED
[18:12:49.045] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:10:0) after 8m31s
[18:21:20.059] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:10:0)
[18:21:20.120] Build status: STARTED
[18:21:20.121] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:11:0) after 1m1s
[18:22:21.133] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:11:0)
[18:22:21.149] Build status: STARTED
[18:22:21.149] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:12:0) after 6m36s
[18:28:57.175] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:12:0)
[18:28:57.190] Build status: STARTED
[18:28:57.190] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:13:0) after 8m58s
[18:37:55.320] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:13:0)
[18:37:55.341] Build status: STARTED
[18:37:55.341] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:14:0) after 5m56s
[18:43:52.046] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:14:0)
[18:43:52.072] Build status: STARTED
[18:43:52.072] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:15:0) after 5m7s
[18:48:59.457] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:15:0)
[18:48:59.490] Build status: STARTED
[18:48:59.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:16:0) after 7m53s
[18:56:52.726] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:16:0)
[18:56:52.755] Build status: STARTED
[18:56:52.755] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:17:0) after 5m49s
[19:02:41.741] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:17:0)
[19:02:41.781] Build status: STARTED
[19:02:41.781] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:18:0) after 8m38s
[19:11:19.894] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:18:0)
[19:11:19.912] Build status: STARTED
[19:11:19.912] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:19:0) after 5m44s
[19:17:04.020] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:19:0)
[19:17:04.037] Build status: STARTED
[19:17:04.037] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:20:0) after 7m10s
[19:24:14.169] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:20:0)
[19:24:14.186] Build status: STARTED
[19:24:14.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:21:0) after 5m46s
[19:30:00.202] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:21:0)
[19:30:00.218] Build status: STARTED
[19:30:00.218] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:22:0) after 9m50s
[19:39:50.341] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:22:0)
[19:39:50.356] Build status: STARTED
[19:39:50.356] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:23:0) after 8m3s
[19:47:53.503] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:23:0)
[19:47:53.519] Build status: STARTED
[19:47:53.519] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:24:0) after 6m45s
[19:54:38.754] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:24:0)
[19:54:38.776] Build status: STARTED
[19:54:38.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:25:0) after 2m57s
[19:57:35.790] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:25:0)
[19:57:35.806] Build status: STARTED
[19:57:35.806] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:26:0) after 1m3s
[19:58:38.842] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:26:0)
[19:58:38.857] Build status: STARTED
[19:58:38.857] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:27:0) after 1m14s
[19:59:53.017] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:27:0)
[19:59:53.117] Build status: STARTED
[19:59:53.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:28:0) after 2m37s
[20:02:30.245] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:28:0)
[20:02:30.261] Build status: STARTED
[20:02:30.261] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:29:0) after 9m44s
[20:12:14.497] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:29:0)
[20:12:14.557] Build status: STARTED
[20:12:14.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:30:0) after 5m42s
[20:17:56.571] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:30:0)
[20:17:56.589] Build status: STARTED
[20:17:56.589] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:31:0) after 7m12s
[20:25:08.815] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:31:0)
[20:25:08.831] Build status: STARTED
[20:25:08.831] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:32:0) after 8m29s
[20:33:37.845] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:32:0)
[20:33:37.858] Build status: STARTED
[20:33:37.859] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:33:0) after 4m18s
[20:37:55.953] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:33:0)
[20:37:56.033] Build status: STARTED
[20:37:56.033] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:34:0) after 2m39s
[20:40:35.149] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:34:0)
[20:40:35.164] Build status: STARTED
[20:40:35.164] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:35:0) after 9m39s
[20:50:14.182] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:35:0)
[20:50:14.197] Build status: STARTED
[20:50:14.198] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:36:0) after 6m32s
[20:56:46.295] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:36:0)
[20:56:46.308] Build status: STARTED
[20:56:46.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:37:0) after 7m47s
[21:04:33.333] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:37:0)
[21:04:33.351] Build status: STARTED
[21:04:33.351] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:38:0) after 2m37s
[21:07:10.638] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:38:0)
[21:07:10.652] Build status: STARTED
[21:07:10.652] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:39:0) after 8m45s
[21:15:55.972] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:39:0)
[21:15:55.984] Build status: STARTED
[21:15:55.984] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:40:0) after 4m22s
[21:20:18.001] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:40:0)
[21:20:18.018] Build status: STARTED
[21:20:18.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:41:0) after 1m6s
[21:21:24.030] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:41:0)
[21:21:24.049] Build status: STARTED
[21:21:24.049] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:42:0) after 6m8s
[21:27:32.196] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:42:0)
[21:27:32.210] Build status: STARTED
[21:27:32.210] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:43:0) after 9m42s
[21:37:14.469] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:43:0)
[21:37:14.488] Build status: STARTED
[21:37:14.488] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858471824357804448:44:0) after 6m33s
[21:42:38.873] Received PubSub notification, asking Buildbucket for the build status
[21:42:38.897] Build:
{
"id": "8693041762389203153",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1156,
"createdBy": "project:chromium",
"createTime": "2026-01-10T17:32:41.970590002Z",
"startTime": "2026-01-10T17:32:46.208106400Z",
"endTime": "2026-01-10T21:42:38.269338671Z",
"updateTime": "2026-01-10T21:42:38.269338671Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "03affe5919ad23fb7a463fd319f8d7dfbafb1b41",
"ref": "refs/heads/main"
}
}
}
[21:42:38.897] Invocation finished in 4h9m58.18878893s with status FAILED