[07:37:00.213] New invocation is queued and will start shortly
[07:37:00.213] Triggered by user:rerunner@pigweed-service-accounts.iam.gserviceaccount.com
[07:37:03.329] Starting the invocation (attempt 1)
[07:37:03.358] Ignoring gitiles_ref tag without the buildset tag
[07:37:03.358] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[07:37:03.358] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[07:37:03.358] Buildbucket request:
{
"requestId": "8839030953215966944",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-host-cpp23"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler.appspot.com",
"invocation": "8839030953215966944",
"job": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23",
"triggers": [
{
"buildbucket": {
"tags": [
"parent_buildername:rerunner",
"user_agent:rerunner"
]
},
"id": "237345b3-f825-4c0c-833c-63d9717f5479",
"title": "rerunner/0",
"url": "https://cr-buildbucket.appspot.com/build/8673601228110617617"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8839030953215966944"
},
{
"key": "scheduler_job_id",
"value": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23"
},
{
"key": "user_agent",
"value": "luci-scheduler"
},
{
"key": "parent_buildername",
"value": "rerunner"
},
{
"key": "user_agent",
"value": "rerunner"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[07:37:03.764] Scheduled build:
{
"id": "8673600889181927009",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-host-cpp23"
},
"createdBy": "project:pigweed",
"createTime": "2026-08-13T07:37:03.436848882Z",
"updateTime": "2026-08-13T07:37:03.436848882Z",
"status": "SCHEDULED",
"input": {
}
}
[07:37:03.764] Task URL: https://cr-buildbucket.appspot.com/build/8673600889181927009
[07:37:03.764] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:2:0) after 3m15s
[07:37:04.692] Received PubSub notification, asking Buildbucket for the build status
[07:37:04.713] Build status: SCHEDULED
[07:40:18.777] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:2:0)
[07:40:18.823] Build status: SCHEDULED
[07:40:18.824] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:4:0) after 6m40s
[07:46:58.963] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:4:0)
[07:46:58.986] Build status: SCHEDULED
[07:46:58.986] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:5:0) after 5m58s
[07:52:57.007] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:5:0)
[07:52:57.078] Build status: SCHEDULED
[07:52:57.079] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:6:0) after 1m50s
[07:54:47.093] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:6:0)
[07:54:47.130] Build status: SCHEDULED
[07:54:47.130] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:7:0) after 5m41s
[08:00:28.145] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:7:0)
[08:00:28.170] Build status: SCHEDULED
[08:00:28.170] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:8:0) after 7m35s
[08:08:03.317] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:8:0)
[08:08:03.333] Build status: SCHEDULED
[08:08:03.333] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:9:0) after 7m4s
[08:15:07.345] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:9:0)
[08:15:07.368] Build status: SCHEDULED
[08:15:07.368] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:10:0) after 8m23s
[08:23:30.387] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:10:0)
[08:23:30.412] Build status: SCHEDULED
[08:23:30.412] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:11:0) after 7m46s
[08:31:16.416] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:11:0)
[08:31:16.434] Build status: SCHEDULED
[08:31:16.435] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:12:0) after 7m15s
[08:38:31.499] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:12:0)
[08:38:31.524] Build status: SCHEDULED
[08:38:31.524] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:13:0) after 1m17s
[08:39:48.562] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:13:0)
[08:39:48.607] Build status: SCHEDULED
[08:39:48.607] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:14:0) after 4m45s
[08:44:33.620] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:14:0)
[08:44:33.749] Build status: SCHEDULED
[08:44:33.749] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:15:0) after 1m49s
[08:46:22.802] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:15:0)
[08:46:22.840] Build status: SCHEDULED
[08:46:22.840] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:16:0) after 9m20s
[08:55:42.906] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:16:0)
[08:55:42.927] Build status: SCHEDULED
[08:55:42.927] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:17:0) after 5m6s
[09:00:49.014] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:17:0)
[09:00:49.053] Build status: SCHEDULED
[09:00:49.053] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:18:0) after 2m31s
[09:03:20.068] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:18:0)
[09:03:20.090] Build status: SCHEDULED
[09:03:20.090] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:19:0) after 1m1s
[09:04:21.106] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:19:0)
[09:04:21.128] Build status: SCHEDULED
[09:04:21.128] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:20:0) after 5m13s
[09:09:34.148] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:20:0)
[09:09:34.170] Build status: SCHEDULED
[09:09:34.170] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:21:0) after 8m36s
[09:18:10.286] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:21:0)
[09:18:10.310] Build status: SCHEDULED
[09:18:10.310] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:22:0) after 5m57s
[09:24:07.406] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:22:0)
[09:24:07.428] Build status: SCHEDULED
[09:24:07.428] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:23:0) after 4m28s
[09:28:35.470] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:23:0)
[09:28:35.492] Build status: SCHEDULED
[09:28:35.492] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:24:0) after 6m21s
[09:34:56.605] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:24:0)
[09:34:56.627] Build status: SCHEDULED
[09:34:56.627] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:25:0) after 3m11s
[09:38:07.693] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:25:0)
[09:38:07.734] Build status: SCHEDULED
[09:38:07.734] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:26:0) after 5m17s
[09:43:24.965] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:26:0)
[09:43:24.985] Build status: SCHEDULED
[09:43:24.985] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:27:0) after 1m10s
[09:44:35.158] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:27:0)
[09:44:35.177] Build status: SCHEDULED
[09:44:35.177] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:28:0) after 3m0s
[09:47:35.192] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:28:0)
[09:47:35.220] Build status: SCHEDULED
[09:47:35.220] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:29:0) after 5m22s
[09:52:57.283] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:29:0)
[09:52:57.397] Build status: SCHEDULED
[09:52:57.397] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:30:0) after 9m34s
[10:02:31.428] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:30:0)
[10:02:31.450] Build status: SCHEDULED
[10:02:31.450] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:31:0) after 9m27s
[10:11:58.544] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:31:0)
[10:11:58.668] Build status: SCHEDULED
[10:11:58.668] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:32:0) after 2m56s
[10:14:54.709] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:32:0)
[10:14:54.732] Build status: SCHEDULED
[10:14:54.732] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:33:0) after 6m39s
[10:21:33.800] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:33:0)
[10:21:33.832] Build status: SCHEDULED
[10:21:33.832] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:34:0) after 8m32s
[10:30:05.939] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:34:0)
[10:30:05.961] Build status: SCHEDULED
[10:30:05.961] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:35:0) after 6m34s
[10:36:39.978] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:35:0)
[10:36:40.021] Build status: SCHEDULED
[10:36:40.021] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:36:0) after 7m56s
[10:44:36.070] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:36:0)
[10:44:36.090] Build status: SCHEDULED
[10:44:36.090] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:37:0) after 2m18s
[10:46:54.107] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:37:0)
[10:46:54.162] Build status: SCHEDULED
[10:46:54.162] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-host-cpp23:8839030953215966944:38:0) after 5m29s
[10:49:36.298] Received PubSub notification, asking Buildbucket for the build status
[10:49:36.323] Build status: STARTED
[10:50:30.895] Received PubSub notification, asking Buildbucket for the build status
[10:50:30.916] Build:
{
"id": "8673600889181927009",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-host-cpp23"
},
"createdBy": "project:pigweed",
"createTime": "2026-08-13T07:37:03.436848882Z",
"startTime": "2026-08-13T10:49:35.483791121Z",
"endTime": "2026-08-13T10:50:30.644472343Z",
"updateTime": "2026-08-13T10:50:30.644472343Z",
"status": "FAILURE",
"input": {
}
}
[10:50:30.917] Invocation finished in 3h13m30.710122287s with status FAILED