[22:16:56.662] New invocation is queued and will start shortly
[22:16:57.740] Starting the invocation (attempt 1)
[22:16:57.761] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/9e35aff8283b43cab954a0d922d9b9853029c6b5"
[22:16:57.761] Popped gitiles commit info from properties and tags
[22:16:57.761] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[22:16:57.761] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[22:16:57.761] Buildbucket request:
{
"requestId": "8858363342942964416",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9e35aff8283b43cab954a0d922d9b9853029c6b5",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8858363342942964416"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[22:16:58.393] Scheduled build:
{
"id": "8692933281004485073",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1162,
"createdBy": "project:chromium",
"createTime": "2026-01-11T22:16:57.880070070Z",
"updateTime": "2026-01-11T22:16:57.880070070Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9e35aff8283b43cab954a0d922d9b9853029c6b5",
"ref": "refs/heads/main"
}
}
}
[22:16:58.393] Task URL: https://cr-buildbucket.appspot.com/build/8692933281004485073
[22:16:58.393] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:2:0) after 4m10s
[22:16:59.417] Received PubSub notification, asking Buildbucket for the build status
[22:16:59.439] Build status: SCHEDULED
[22:17:01.224] Received PubSub notification, asking Buildbucket for the build status
[22:17:01.239] Build status: STARTED
[22:21:08.506] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:2:0)
[22:21:08.519] Build status: STARTED
[22:21:08.520] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:5:0) after 3m34s
[22:24:42.532] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:5:0)
[22:24:42.570] Build status: STARTED
[22:24:42.570] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:6:0) after 2m54s
[22:27:36.583] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:6:0)
[22:27:36.653] Build status: STARTED
[22:27:36.653] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:7:0) after 2m59s
[22:30:35.667] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:7:0)
[22:30:35.701] Build status: STARTED
[22:30:35.701] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:8:0) after 8m7s
[22:38:42.718] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:8:0)
[22:38:42.750] Build status: STARTED
[22:38:42.750] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:9:0) after 1m31s
[22:40:13.764] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:9:0)
[22:40:13.805] Build status: STARTED
[22:40:13.805] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:10:0) after 3m21s
[22:43:35.028] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:10:0)
[22:43:35.049] Build status: STARTED
[22:43:35.049] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:11:0) after 1m55s
[22:45:30.063] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:11:0)
[22:45:30.085] Build status: STARTED
[22:45:30.085] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:12:0) after 2m38s
[22:48:08.113] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:12:0)
[22:48:08.129] Build status: STARTED
[22:48:08.129] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:13:0) after 9m46s
[22:57:54.142] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:13:0)
[22:57:54.258] Build status: STARTED
[22:57:54.258] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:14:0) after 5m35s
[23:03:29.374] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:14:0)
[23:03:29.426] Build status: STARTED
[23:03:29.426] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:15:0) after 9m22s
[23:12:51.701] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:15:0)
[23:12:51.732] Build status: STARTED
[23:12:51.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:16:0) after 6m15s
[23:19:06.805] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:16:0)
[23:19:06.852] Build status: STARTED
[23:19:06.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:17:0) after 6m42s
[23:25:48.866] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:17:0)
[23:25:48.883] Build status: STARTED
[23:25:48.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:18:0) after 7m38s
[23:33:27.005] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:18:0)
[23:33:27.076] Build status: STARTED
[23:33:27.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:19:0) after 6m38s
[23:40:05.826] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:19:0)
[23:40:05.841] Build status: STARTED
[23:40:05.841] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:20:0) after 3m14s
[23:43:19.910] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:20:0)
[23:43:19.926] Build status: STARTED
[23:43:19.926] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:21:0) after 5m12s
[23:48:31.944] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:21:0)
[23:48:31.969] Build status: STARTED
[23:48:31.969] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:22:0) after 6m47s
[23:55:18.989] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:22:0)
[23:55:19.010] Build status: STARTED
[23:55:19.010] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:23:0) after 5m16s
[00:00:35.029] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:23:0)
[00:00:35.047] Build status: STARTED
[00:00:35.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:24:0) after 7m1s
[00:07:36.110] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:24:0)
[00:07:36.128] Build status: STARTED
[00:07:36.128] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:25:0) after 8m43s
[00:16:19.162] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:25:0)
[00:16:19.187] Build status: STARTED
[00:16:19.187] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:26:0) after 4m35s
[00:20:54.205] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:26:0)
[00:20:54.223] Build status: STARTED
[00:20:54.223] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:27:0) after 1m30s
[00:22:24.261] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:27:0)
[00:22:24.282] Build status: STARTED
[00:22:24.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:28:0) after 8m41s
[00:31:05.382] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:28:0)
[00:31:05.439] Build status: STARTED
[00:31:05.439] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:29:0) after 9m44s
[00:40:49.638] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:29:0)
[00:40:49.659] Build status: STARTED
[00:40:49.659] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:30:0) after 7m23s
[00:48:12.676] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:30:0)
[00:48:12.690] Build status: STARTED
[00:48:12.690] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:31:0) after 8m0s
[00:56:12.744] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:31:0)
[00:56:12.759] Build status: STARTED
[00:56:12.759] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:32:0) after 2m49s
[00:59:02.252] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:32:0)
[00:59:02.279] Build status: STARTED
[00:59:02.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:33:0) after 4m55s
[01:03:57.295] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:33:0)
[01:03:57.323] Build status: STARTED
[01:03:57.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:34:0) after 3m0s
[01:06:57.585] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:34:0)
[01:06:57.630] Build status: STARTED
[01:06:57.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:35:0) after 2m48s
[01:09:45.775] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:35:0)
[01:09:45.918] Build status: STARTED
[01:09:45.918] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:36:0) after 9m34s
[01:19:20.113] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:36:0)
[01:19:20.127] Build status: STARTED
[01:19:20.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:37:0) after 9m55s
[01:29:15.389] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:37:0)
[01:29:15.404] Build status: STARTED
[01:29:15.404] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:38:0) after 6m58s
[01:36:13.416] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:38:0)
[01:36:13.437] Build status: STARTED
[01:36:13.437] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:39:0) after 8m4s
[01:44:17.453] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:39:0)
[01:44:17.468] Build status: STARTED
[01:44:17.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:40:0) after 8m35s
[01:52:52.551] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:40:0)
[01:52:52.565] Build status: STARTED
[01:52:52.565] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:41:0) after 1m2s
[01:53:54.777] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:41:0)
[01:53:54.792] Build status: STARTED
[01:53:54.792] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:42:0) after 8m34s
[02:02:28.858] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:42:0)
[02:02:28.893] Build status: STARTED
[02:02:28.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:43:0) after 7m32s
[02:10:00.906] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:43:0)
[02:10:00.933] Build status: STARTED
[02:10:00.933] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:44:0) after 1m42s
[02:11:43.006] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:44:0)
[02:11:43.033] Build status: STARTED
[02:11:43.033] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:45:0) after 6m38s
[02:18:21.047] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:45:0)
[02:18:21.062] Build status: STARTED
[02:18:21.062] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:46:0) after 9m7s
[02:27:28.204] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:46:0)
[02:27:28.220] Build status: STARTED
[02:27:28.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858363342942964416:47:0) after 6m13s
[02:27:33.750] Received PubSub notification, asking Buildbucket for the build status
[02:27:33.918] Build:
{
"id": "8692933281004485073",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1162,
"createdBy": "project:chromium",
"createTime": "2026-01-11T22:16:57.880070070Z",
"startTime": "2026-01-11T22:17:01.052405713Z",
"endTime": "2026-01-12T02:27:33.486497735Z",
"updateTime": "2026-01-12T02:27:33.486497735Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9e35aff8283b43cab954a0d922d9b9853029c6b5",
"ref": "refs/heads/main"
}
}
}
[02:27:33.918] Invocation finished in 4h10m37.269131646s with status FAILED