[07:28:47.571] New invocation is queued and will start shortly
[07:28:47.571] Triggered by user:rerunner@pigweed-service-accounts.iam.gserviceaccount.com
[07:28:48.742] Starting the invocation (attempt 1)
[07:28:48.765] Ignoring gitiles_ref tag without the buildset tag
[07:28:48.765] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[07:28:48.765] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[07:28:48.766] Buildbucket request:
{
"requestId": "8839212663717738400",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler.appspot.com",
"invocation": "8839212663717738400",
"job": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz",
"triggers": [
{
"buildbucket": {
"tags": [
"parent_buildername:rerunner",
"user_agent:rerunner"
]
},
"id": "85040f09-a091-430f-9fc9-1450a4f26d6b",
"title": "rerunner/0",
"url": "https://cr-buildbucket.appspot.com/build/8673782914845165729"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8839212663717738400"
},
{
"key": "scheduler_job_id",
"value": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz"
},
{
"key": "user_agent",
"value": "luci-scheduler"
},
{
"key": "parent_buildername",
"value": "rerunner"
},
{
"key": "user_agent",
"value": "rerunner"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[07:28:49.263] Scheduled build:
{
"id": "8673782601748055649",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"createdBy": "project:pigweed",
"createTime": "2026-08-11T07:28:48.829555968Z",
"updateTime": "2026-08-11T07:28:48.829555968Z",
"status": "SCHEDULED",
"input": {
}
}
[07:28:49.263] Task URL: https://cr-buildbucket.appspot.com/build/8673782601748055649
[07:28:49.263] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:2:0) after 6m11s
[07:28:50.109] Received PubSub notification, asking Buildbucket for the build status
[07:28:50.133] Build status: SCHEDULED
[07:35:00.475] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:2:0)
[07:35:00.503] Build status: SCHEDULED
[07:35:00.503] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:4:0) after 7m18s
[07:42:18.678] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:4:0)
[07:42:18.759] Build status: SCHEDULED
[07:42:18.759] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:5:0) after 5m46s
[07:48:04.772] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:5:0)
[07:48:04.795] Build status: SCHEDULED
[07:48:04.795] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:6:0) after 1m18s
[07:49:22.810] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:6:0)
[07:49:22.832] Build status: SCHEDULED
[07:49:22.832] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:7:0) after 5m39s
[07:55:01.967] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:7:0)
[07:55:02.041] Build status: SCHEDULED
[07:55:02.041] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:8:0) after 9m32s
[08:04:34.057] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:8:0)
[08:04:34.081] Build status: SCHEDULED
[08:04:34.081] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:9:0) after 3m21s
[08:07:55.190] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:9:0)
[08:07:55.218] Build status: SCHEDULED
[08:07:55.218] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:10:0) after 9m4s
[08:16:59.301] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:10:0)
[08:16:59.343] Build status: SCHEDULED
[08:16:59.344] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:11:0) after 2m12s
[08:19:11.358] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:11:0)
[08:19:11.384] Build status: SCHEDULED
[08:19:11.384] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:12:0) after 6m34s
[08:25:45.467] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:12:0)
[08:25:45.492] Build status: SCHEDULED
[08:25:45.493] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:13:0) after 3m22s
[08:29:07.510] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:13:0)
[08:29:07.544] Build status: SCHEDULED
[08:29:07.544] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:14:0) after 4m35s
[08:33:42.560] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:14:0)
[08:33:42.586] Build status: SCHEDULED
[08:33:42.586] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:15:0) after 8m42s
[08:42:24.687] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:15:0)
[08:42:24.715] Build status: SCHEDULED
[08:42:24.715] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:16:0) after 1m55s
[08:44:19.716] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:16:0)
[08:44:19.739] Build status: SCHEDULED
[08:44:19.739] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:17:0) after 1m23s
[08:45:42.791] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:17:0)
[08:45:42.907] Build status: SCHEDULED
[08:45:42.907] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:18:0) after 9m4s
[08:54:47.003] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:18:0)
[08:54:47.025] Build status: SCHEDULED
[08:54:47.025] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:19:0) after 5m9s
[08:59:56.047] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:19:0)
[08:59:56.073] Build status: SCHEDULED
[08:59:56.073] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:20:0) after 1m12s
[09:01:08.069] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:20:0)
[09:01:08.095] Build status: SCHEDULED
[09:01:08.095] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:21:0) after 7m31s
[09:08:39.144] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:21:0)
[09:08:39.167] Build status: SCHEDULED
[09:08:39.167] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:22:0) after 8m38s
[09:17:17.184] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:22:0)
[09:17:17.201] Build status: SCHEDULED
[09:17:17.201] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:23:0) after 3m59s
[09:21:16.185] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:23:0)
[09:21:16.203] Build status: SCHEDULED
[09:21:16.203] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:24:0) after 2m7s
[09:23:23.309] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:24:0)
[09:23:23.332] Build status: SCHEDULED
[09:23:23.332] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:25:0) after 7m52s
[09:31:15.347] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:25:0)
[09:31:15.366] Build status: SCHEDULED
[09:31:15.366] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:26:0) after 6m36s
[09:37:51.431] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:26:0)
[09:37:51.473] Build status: SCHEDULED
[09:37:51.473] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:27:0) after 8m40s
[09:46:31.506] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:27:0)
[09:46:31.523] Build status: SCHEDULED
[09:46:31.523] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:28:0) after 4m41s
[09:51:12.499] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:28:0)
[09:51:12.516] Build status: SCHEDULED
[09:51:12.516] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:29:0) after 5m37s
[09:56:49.614] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:29:0)
[09:56:49.632] Build status: SCHEDULED
[09:56:49.632] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:30:0) after 1m43s
[09:58:32.663] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:30:0)
[09:58:32.686] Build status: SCHEDULED
[09:58:32.686] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:31:0) after 4m57s
[10:03:29.700] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:31:0)
[10:03:29.803] Build status: SCHEDULED
[10:03:29.803] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:32:0) after 6m56s
[10:09:00.666] Received PubSub notification, asking Buildbucket for the build status
[10:09:00.684] Build status: STARTED
[10:10:25.822] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:32:0)
[10:10:25.868] Build status: STARTED
[10:10:25.868] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839212663717738400:34:0) after 4m24s
[10:13:26.575] Received PubSub notification, asking Buildbucket for the build status
[10:13:26.593] Build:
{
"id": "8673782601748055649",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"createdBy": "project:pigweed",
"createTime": "2026-08-11T07:28:48.829555968Z",
"startTime": "2026-08-11T10:09:00.492534371Z",
"endTime": "2026-08-11T10:13:26.219767902Z",
"updateTime": "2026-08-11T10:13:26.219767902Z",
"status": "FAILURE",
"input": {
}
}
[10:13:26.593] Invocation finished in 2h44m39.026349847s with status FAILED