[21:32:30.038] New invocation is queued and will start shortly
[21:32:30.038] Triggered by user:rerunner@pigweed-service-accounts.iam.gserviceaccount.com
[21:32:31.102] Starting the invocation (attempt 1)
[21:32:31.129] Ignoring gitiles_ref tag without the buildset tag
[21:32:31.129] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:32:31.129] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:32:31.129] Buildbucket request:
{
"requestId": "8849034651552903376",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler.appspot.com",
"invocation": "8849034651552903376",
"job": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz",
"triggers": [
{
"buildbucket": {
"tags": [
"parent_buildername:rerunner",
"user_agent:rerunner"
]
},
"id": "e416b16f-b4f5-46cf-b964-4f01e0612503",
"title": "rerunner/0",
"url": "https://cr-buildbucket.appspot.com/build/8683604828595404577"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8849034651552903376"
},
{
"key": "scheduler_job_id",
"value": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz"
},
{
"key": "user_agent",
"value": "luci-scheduler"
},
{
"key": "parent_buildername",
"value": "rerunner"
},
{
"key": "user_agent",
"value": "rerunner"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:32:32.325] Scheduled build:
{
"id": "8683604589255903633",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"createdBy": "project:pigweed",
"createTime": "2026-04-24T21:32:31.607826163Z",
"updateTime": "2026-04-24T21:32:31.607826163Z",
"status": "SCHEDULED",
"input": {
}
}
[21:32:32.325] Task URL: https://cr-buildbucket.appspot.com/build/8683604589255903633
[21:32:32.325] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:2:0) after 5m46s
[21:32:32.470] Received PubSub notification, asking Buildbucket for the build status
[21:32:32.492] Build status: SCHEDULED
[21:38:18.388] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:2:0)
[21:38:18.411] Build status: SCHEDULED
[21:38:18.411] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:4:0) after 9m7s
[21:47:25.419] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:4:0)
[21:47:25.441] Build status: SCHEDULED
[21:47:25.441] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:5:0) after 2m54s
[21:50:19.515] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:5:0)
[21:50:19.562] Build status: SCHEDULED
[21:50:19.562] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:6:0) after 1m0s
[21:51:19.846] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:6:0)
[21:51:19.862] Build status: SCHEDULED
[21:51:19.862] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:7:0) after 4m42s
[21:56:01.931] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:7:0)
[21:56:01.953] Build status: SCHEDULED
[21:56:01.953] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:8:0) after 3m9s
[21:59:11.012] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:8:0)
[21:59:11.044] Build status: SCHEDULED
[21:59:11.044] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:9:0) after 1m12s
[22:00:23.063] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:9:0)
[22:00:23.084] Build status: SCHEDULED
[22:00:23.084] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:10:0) after 8m58s
[22:09:21.139] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:10:0)
[22:09:21.157] Build status: SCHEDULED
[22:09:21.157] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:11:0) after 2m30s
[22:11:51.184] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:11:0)
[22:11:51.212] Build status: SCHEDULED
[22:11:51.212] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:12:0) after 5m56s
[22:17:47.374] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:12:0)
[22:17:47.425] Build status: SCHEDULED
[22:17:47.425] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:13:0) after 4m33s
[22:22:20.483] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:13:0)
[22:22:20.509] Build status: SCHEDULED
[22:22:20.509] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:14:0) after 5m44s
[22:28:04.525] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:14:0)
[22:28:04.596] Build status: SCHEDULED
[22:28:04.596] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:15:0) after 4m43s
[22:32:47.744] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:15:0)
[22:32:47.763] Build status: SCHEDULED
[22:32:47.763] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:16:0) after 3m47s
[22:36:34.819] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:16:0)
[22:36:34.836] Build status: SCHEDULED
[22:36:34.836] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:17:0) after 3m54s
[22:40:28.874] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:17:0)
[22:40:28.925] Build status: SCHEDULED
[22:40:28.925] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:18:0) after 7m42s
[22:48:10.939] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:18:0)
[22:48:10.968] Build status: SCHEDULED
[22:48:10.968] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:19:0) after 6m21s
[22:54:32.036] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:19:0)
[22:54:32.055] Build status: SCHEDULED
[22:54:32.055] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:20:0) after 4m17s
[22:58:49.067] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:20:0)
[22:58:49.086] Build status: SCHEDULED
[22:58:49.086] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:21:0) after 3m7s
[23:01:56.083] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:21:0)
[23:01:56.103] Build status: SCHEDULED
[23:01:56.103] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:22:0) after 3m28s
[23:05:24.170] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:22:0)
[23:05:24.261] Build status: SCHEDULED
[23:05:24.261] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:23:0) after 8m29s
[23:13:53.349] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:23:0)
[23:13:53.380] Build status: SCHEDULED
[23:13:53.380] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:24:0) after 8m59s
[23:22:52.362] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:24:0)
[23:22:52.387] Build status: SCHEDULED
[23:22:52.388] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:25:0) after 9m58s
[23:32:50.465] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:25:0)
[23:32:50.504] Build status: SCHEDULED
[23:32:50.504] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:26:0) after 8m54s
[23:41:44.518] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:26:0)
[23:41:44.534] Build status: SCHEDULED
[23:41:44.535] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:27:0) after 5m32s
[23:47:16.551] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:27:0)
[23:47:16.572] Build status: SCHEDULED
[23:47:16.572] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8849034651552903376:28:0) after 7m37s
[23:50:20.775] Received PubSub notification, asking Buildbucket for the build status
[23:50:20.794] Build status: STARTED
[23:53:18.344] Received PubSub notification, asking Buildbucket for the build status
[23:53:18.365] Build:
{
"id": "8683604589255903633",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"createdBy": "project:pigweed",
"createTime": "2026-04-24T21:32:31.607826163Z",
"startTime": "2026-04-24T23:50:20.605566784Z",
"endTime": "2026-04-24T23:53:17.926625361Z",
"updateTime": "2026-04-24T23:53:17.926625361Z",
"status": "FAILURE",
"input": {
}
}
[23:53:18.365] Invocation finished in 2h20m48.332028117s with status FAILED