[21:08:22.696] New invocation is queued and will start shortly
[21:08:22.696] Triggered by user:rerunner@pigweed-service-accounts.iam.gserviceaccount.com
[21:08:24.041] Starting the invocation (attempt 1)
[21:08:24.072] Ignoring gitiles_ref tag without the buildset tag
[21:08:24.072] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:08:24.072] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:08:24.072] Buildbucket request:
{
"requestId": "8843328560317437968",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-rust"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler.appspot.com",
"invocation": "8843328560317437968",
"job": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust",
"triggers": [
{
"buildbucket": {
"tags": [
"parent_buildername:rerunner",
"user_agent:rerunner"
]
},
"id": "28058846-54e1-46f4-b2f6-f75abfa97b43",
"title": "rerunner/0",
"url": "https://cr-buildbucket.appspot.com/build/8677898818629237473"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8843328560317437968"
},
{
"key": "scheduler_job_id",
"value": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust"
},
{
"key": "user_agent",
"value": "luci-scheduler"
},
{
"key": "parent_buildername",
"value": "rerunner"
},
{
"key": "user_agent",
"value": "rerunner"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:08:24.437] Scheduled build:
{
"id": "8677898498146389969",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-rust"
},
"createdBy": "project:pigweed",
"createTime": "2026-06-26T21:08:24.145568226Z",
"updateTime": "2026-06-26T21:08:24.145568226Z",
"status": "SCHEDULED",
"input": {
}
}
[21:08:24.437] Task URL: https://cr-buildbucket.appspot.com/build/8677898498146389969
[21:08:24.437] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:2:0) after 1m19s
[21:08:25.419] Received PubSub notification, asking Buildbucket for the build status
[21:08:25.441] Build status: SCHEDULED
[21:09:43.473] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:2:0)
[21:09:43.497] Build status: SCHEDULED
[21:09:43.497] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:4:0) after 8m27s
[21:18:10.511] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:4:0)
[21:18:10.531] Build status: SCHEDULED
[21:18:10.531] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:5:0) after 5m44s
[21:23:54.546] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:5:0)
[21:23:54.565] Build status: SCHEDULED
[21:23:54.565] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:6:0) after 3m43s
[21:27:37.579] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:6:0)
[21:27:37.681] Build status: SCHEDULED
[21:27:37.681] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:7:0) after 2m0s
[21:29:37.743] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:7:0)
[21:29:37.760] Build status: SCHEDULED
[21:29:37.761] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:8:0) after 3m11s
[21:32:48.743] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:8:0)
[21:32:48.762] Build status: SCHEDULED
[21:32:48.762] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:9:0) after 2m28s
[21:35:16.775] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:9:0)
[21:35:16.823] Build status: SCHEDULED
[21:35:16.823] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:10:0) after 4m41s
[21:39:57.837] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:10:0)
[21:39:57.854] Build status: SCHEDULED
[21:39:57.854] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:11:0) after 5m18s
[21:45:15.999] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:11:0)
[21:45:16.070] Build status: SCHEDULED
[21:45:16.071] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:12:0) after 6m20s
[21:51:36.085] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:12:0)
[21:51:36.150] Build status: SCHEDULED
[21:51:36.150] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:13:0) after 6m45s
[21:58:21.455] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:13:0)
[21:58:21.479] Build status: SCHEDULED
[21:58:21.479] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:14:0) after 3m35s
[22:01:56.491] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:14:0)
[22:01:56.519] Build status: SCHEDULED
[22:01:56.519] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:15:0) after 3m1s
[22:04:57.534] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:15:0)
[22:04:57.555] Build status: SCHEDULED
[22:04:57.555] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:16:0) after 9m21s
[22:14:18.619] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:16:0)
[22:14:18.646] Build status: SCHEDULED
[22:14:18.646] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:17:0) after 9m3s
[22:23:21.726] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:17:0)
[22:23:21.757] Build status: SCHEDULED
[22:23:21.757] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:18:0) after 3m3s
[22:26:24.839] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:18:0)
[22:26:24.922] Build status: SCHEDULED
[22:26:24.922] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:19:0) after 8m50s
[22:35:14.940] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:19:0)
[22:35:14.969] Build status: SCHEDULED
[22:35:14.969] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:20:0) after 6m44s
[22:41:58.984] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:20:0)
[22:41:59.016] Build status: SCHEDULED
[22:41:59.016] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:21:0) after 4m43s
[22:46:42.029] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:21:0)
[22:46:42.051] Build status: SCHEDULED
[22:46:42.052] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:22:0) after 9m7s
[22:55:49.018] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:22:0)
[22:55:49.046] Build status: SCHEDULED
[22:55:49.046] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:23:0) after 2m2s
[22:57:51.041] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:23:0)
[22:57:51.059] Build status: SCHEDULED
[22:57:51.059] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:24:0) after 1m42s
[22:59:33.074] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:24:0)
[22:59:33.096] Build status: SCHEDULED
[22:59:33.096] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:25:0) after 4m49s
[23:04:22.121] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:25:0)
[23:04:22.144] Build status: SCHEDULED
[23:04:22.144] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:26:0) after 7m23s
[23:11:45.209] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:26:0)
[23:11:45.244] Build status: SCHEDULED
[23:11:45.244] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:27:0) after 5m33s
[23:17:18.263] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:27:0)
[23:17:18.294] Build status: SCHEDULED
[23:17:18.294] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:28:0) after 3m13s
[23:20:31.309] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:28:0)
[23:20:31.328] Build status: SCHEDULED
[23:20:31.328] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:29:0) after 9m20s
[23:29:51.341] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:29:0)
[23:29:51.386] Build status: SCHEDULED
[23:29:51.386] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:30:0) after 4m28s
[23:34:19.451] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:30:0)
[23:34:19.478] Build status: SCHEDULED
[23:34:19.478] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:31:0) after 8m40s
[23:42:59.533] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:31:0)
[23:42:59.552] Build status: SCHEDULED
[23:42:59.552] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:32:0) after 9m54s
[23:51:23.823] Received PubSub notification, asking Buildbucket for the build status
[23:51:23.843] Build status: STARTED
[23:52:53.581] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:32:0)
[23:52:53.599] Build status: STARTED
[23:52:53.599] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-rust:8843328560317437968:34:0) after 2m43s
[23:55:11.462] Received PubSub notification, asking Buildbucket for the build status
[23:55:11.485] Build:
{
"id": "8677898498146389969",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-rust"
},
"createdBy": "project:pigweed",
"createTime": "2026-06-26T21:08:24.145568226Z",
"startTime": "2026-06-26T23:51:23.665313597Z",
"endTime": "2026-06-26T23:55:11.096684375Z",
"updateTime": "2026-06-26T23:55:11.096684375Z",
"status": "FAILURE",
"input": {
}
}
[23:55:11.485] Invocation finished in 2h46m48.793734412s with status FAILED