[02:49:09.228] New invocation is queued and will start shortly
[02:49:10.232] Starting the invocation (attempt 1)
[02:49:10.254] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0cc62ad8bfc15d5fb47602b0a42e05c5f734fab7"
[02:49:10.254] Popped gitiles commit info from properties and tags
[02:49:10.254] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[02:49:10.254] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[02:49:10.254] Buildbucket request:
{
"requestId": "8858436813966018624",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0cc62ad8bfc15d5fb47602b0a42e05c5f734fab7",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8858436813966018624"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[02:49:10.707] Scheduled build:
{
"id": "8693006752151980897",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1158,
"createdBy": "project:chromium",
"createTime": "2026-01-11T02:49:10.334359570Z",
"updateTime": "2026-01-11T02:49:10.334359570Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0cc62ad8bfc15d5fb47602b0a42e05c5f734fab7",
"ref": "refs/heads/main"
}
}
}
[02:49:10.707] Task URL: https://cr-buildbucket.appspot.com/build/8693006752151980897
[02:49:10.707] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:2:0) after 3m42s
[02:49:11.656] Received PubSub notification, asking Buildbucket for the build status
[02:49:11.669] Build status: SCHEDULED
[02:49:13.605] Received PubSub notification, asking Buildbucket for the build status
[02:49:13.629] Build status: STARTED
[02:52:53.008] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:2:0)
[02:52:53.026] Build status: STARTED
[02:52:53.026] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:5:0) after 3m16s
[02:56:09.039] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:5:0)
[02:56:09.102] Build status: STARTED
[02:56:09.102] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:6:0) after 4m22s
[03:00:31.378] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:6:0)
[03:00:31.396] Build status: STARTED
[03:00:31.396] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:7:0) after 3m33s
[03:04:04.409] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:7:0)
[03:04:04.424] Build status: STARTED
[03:04:04.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:8:0) after 3m3s
[03:07:07.433] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:8:0)
[03:07:07.479] Build status: STARTED
[03:07:07.479] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:9:0) after 6m0s
[03:13:07.683] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:9:0)
[03:13:07.699] Build status: STARTED
[03:13:07.699] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:10:0) after 4m19s
[03:17:26.770] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:10:0)
[03:17:26.791] Build status: STARTED
[03:17:26.791] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:11:0) after 1m15s
[03:18:41.805] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:11:0)
[03:18:41.822] Build status: STARTED
[03:18:41.822] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:12:0) after 1m3s
[03:19:44.951] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:12:0)
[03:19:44.964] Build status: STARTED
[03:19:44.964] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:13:0) after 8m1s
[03:27:45.980] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:13:0)
[03:27:45.994] Build status: STARTED
[03:27:45.994] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:14:0) after 7m5s
[03:34:51.223] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:14:0)
[03:34:51.238] Build status: STARTED
[03:34:51.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:15:0) after 2m14s
[03:37:05.259] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:15:0)
[03:37:05.272] Build status: STARTED
[03:37:05.272] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:16:0) after 8m55s
[03:46:00.290] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:16:0)
[03:46:00.308] Build status: STARTED
[03:46:00.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:17:0) after 1m1s
[03:47:01.408] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:17:0)
[03:47:01.427] Build status: STARTED
[03:47:01.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:18:0) after 1m4s
[03:48:05.561] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:18:0)
[03:48:05.577] Build status: STARTED
[03:48:05.577] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:19:0) after 8m8s
[03:56:13.786] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:19:0)
[03:56:13.803] Build status: STARTED
[03:56:13.803] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:20:0) after 7m41s
[04:03:55.108] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:20:0)
[04:03:55.120] Build status: STARTED
[04:03:55.120] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:21:0) after 6m20s
[04:10:15.136] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:21:0)
[04:10:15.159] Build status: STARTED
[04:10:15.159] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:22:0) after 6m1s
[04:16:16.208] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:22:0)
[04:16:16.229] Build status: STARTED
[04:16:16.229] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:23:0) after 6m36s
[04:22:52.247] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:23:0)
[04:22:52.260] Build status: STARTED
[04:22:52.260] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:24:0) after 3m27s
[04:26:19.304] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:24:0)
[04:26:19.330] Build status: STARTED
[04:26:19.330] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:25:0) after 1m37s
[04:27:56.526] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:25:0)
[04:27:56.548] Build status: STARTED
[04:27:56.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:26:0) after 3m37s
[04:31:33.693] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:26:0)
[04:31:33.720] Build status: STARTED
[04:31:33.720] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:27:0) after 1m22s
[04:32:56.035] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:27:0)
[04:32:56.050] Build status: STARTED
[04:32:56.050] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:28:0) after 4m25s
[04:37:21.229] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:28:0)
[04:37:21.248] Build status: STARTED
[04:37:21.248] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:29:0) after 9m26s
[04:46:47.337] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:29:0)
[04:46:47.352] Build status: STARTED
[04:46:47.352] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:30:0) after 1m36s
[04:48:23.663] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:30:0)
[04:48:23.678] Build status: STARTED
[04:48:23.678] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:31:0) after 9m55s
[04:58:18.796] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:31:0)
[04:58:18.873] Build status: STARTED
[04:58:18.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:32:0) after 1m25s
[04:59:43.887] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:32:0)
[04:59:43.901] Build status: STARTED
[04:59:43.901] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:33:0) after 6m40s
[05:06:24.124] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:33:0)
[05:06:24.147] Build status: STARTED
[05:06:24.147] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:34:0) after 7m39s
[05:14:03.424] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:34:0)
[05:14:03.446] Build status: STARTED
[05:14:03.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:35:0) after 1m39s
[05:15:42.680] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:35:0)
[05:15:42.700] Build status: STARTED
[05:15:42.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:36:0) after 5m46s
[05:21:29.096] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:36:0)
[05:21:29.118] Build status: STARTED
[05:21:29.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:37:0) after 8m35s
[05:30:04.390] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:37:0)
[05:30:04.407] Build status: STARTED
[05:30:04.407] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:38:0) after 8m23s
[05:38:27.434] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:38:0)
[05:38:27.502] Build status: STARTED
[05:38:27.502] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:39:0) after 3m5s
[05:41:32.711] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:39:0)
[05:41:32.739] Build status: STARTED
[05:41:32.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:40:0) after 6m18s
[05:47:50.787] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:40:0)
[05:47:50.809] Build status: STARTED
[05:47:50.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:41:0) after 4m1s
[05:51:51.836] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:41:0)
[05:51:51.849] Build status: STARTED
[05:51:51.849] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:42:0) after 4m11s
[05:56:02.966] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:42:0)
[05:56:02.982] Build status: STARTED
[05:56:02.982] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:43:0) after 3m6s
[05:59:09.002] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:43:0)
[05:59:09.095] Build status: STARTED
[05:59:09.095] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:44:0) after 2m58s
[06:02:07.109] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:44:0)
[06:02:07.130] Build status: STARTED
[06:02:07.130] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:45:0) after 7m23s
[06:09:30.400] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:45:0)
[06:09:30.423] Build status: STARTED
[06:09:30.423] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:46:0) after 9m33s
[06:19:03.478] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:46:0)
[06:19:03.492] Build status: STARTED
[06:19:03.492] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:47:0) after 3m26s
[06:22:29.530] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:47:0)
[06:22:29.546] Build status: STARTED
[06:22:29.546] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:48:0) after 5m41s
[06:28:10.568] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:48:0)
[06:28:10.614] Build status: STARTED
[06:28:10.614] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:49:0) after 5m7s
[06:33:17.697] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:49:0)
[06:33:17.716] Build status: STARTED
[06:33:17.716] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:50:0) after 8m55s
[06:42:12.789] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:50:0)
[06:42:12.802] Build status: STARTED
[06:42:12.802] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:51:0) after 8m3s
[06:50:15.986] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:51:0)
[06:50:16.001] Build status: STARTED
[06:50:16.001] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:52:0) after 7m33s
[06:57:49.131] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:52:0)
[06:57:49.153] Build status: STARTED
[06:57:49.153] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858436813966018624:53:0) after 1m43s
[06:59:09.557] Received PubSub notification, asking Buildbucket for the build status
[06:59:09.579] Build:
{
"id": "8693006752151980897",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1158,
"createdBy": "project:chromium",
"createTime": "2026-01-11T02:49:10.334359570Z",
"startTime": "2026-01-11T02:49:13.420901596Z",
"endTime": "2026-01-11T06:59:09.230575382Z",
"updateTime": "2026-01-11T06:59:09.230575382Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0cc62ad8bfc15d5fb47602b0a42e05c5f734fab7",
"ref": "refs/heads/main"
}
}
}
[06:59:09.579] Invocation finished in 4h10m0.358803105s with status FAILED