[18:50:55.495] New invocation is queued and will start shortly
[18:50:55.495] Triggered by user:rerunner@pigweed-service-accounts.iam.gserviceaccount.com
[18:50:56.564] Starting the invocation (attempt 1)
[18:50:56.591] Ignoring gitiles_ref tag without the buildset tag
[18:50:56.591] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[18:50:56.593] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[18:50:56.593] Buildbucket request:
{
"requestId": "8844061983924531264",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler.appspot.com",
"invocation": "8844061983924531264",
"job": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz",
"triggers": [
{
"buildbucket": {
"tags": [
"parent_buildername:rerunner",
"user_agent:rerunner"
]
},
"id": "8d0c7f87-0fc6-4670-a44e-154f8fec805c",
"title": "rerunner/0",
"url": "https://cr-buildbucket.appspot.com/build/8678632340508391505"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8844061983924531264"
},
{
"key": "scheduler_job_id",
"value": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz"
},
{
"key": "user_agent",
"value": "luci-scheduler"
},
{
"key": "parent_buildername",
"value": "rerunner"
},
{
"key": "user_agent",
"value": "rerunner"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[18:50:57.102] Scheduled build:
{
"id": "8678631921931619169",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"createdBy": "project:pigweed",
"createTime": "2026-06-18T18:50:56.718010641Z",
"updateTime": "2026-06-18T18:50:56.718010641Z",
"status": "SCHEDULED",
"input": {
}
}
[18:50:57.102] Task URL: https://cr-buildbucket.appspot.com/build/8678631921931619169
[18:50:57.102] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:2:0) after 4m56s
[18:50:58.023] Received PubSub notification, asking Buildbucket for the build status
[18:50:58.044] Build status: SCHEDULED
[18:55:53.307] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:2:0)
[18:55:53.328] Build status: SCHEDULED
[18:55:53.328] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:4:0) after 5m7s
[19:01:00.573] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:4:0)
[19:01:00.598] Build status: SCHEDULED
[19:01:00.598] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:5:0) after 1m50s
[19:02:50.614] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:5:0)
[19:02:50.686] Build status: SCHEDULED
[19:02:50.686] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:6:0) after 5m40s
[19:08:30.795] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:6:0)
[19:08:30.833] Build status: SCHEDULED
[19:08:30.833] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:7:0) after 7m10s
[19:15:40.889] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:7:0)
[19:15:40.909] Build status: SCHEDULED
[19:15:40.909] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:8:0) after 7m48s
[19:23:29.091] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:8:0)
[19:23:29.120] Build status: SCHEDULED
[19:23:29.121] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:9:0) after 8m18s
[19:31:47.121] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:9:0)
[19:31:47.139] Build status: SCHEDULED
[19:31:47.139] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:10:0) after 6m37s
[19:38:24.187] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:10:0)
[19:38:24.234] Build status: SCHEDULED
[19:38:24.235] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:11:0) after 9m45s
[19:48:09.312] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:11:0)
[19:48:09.331] Build status: SCHEDULED
[19:48:09.331] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:12:0) after 1m21s
[19:49:30.438] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:12:0)
[19:49:30.462] Build status: SCHEDULED
[19:49:30.462] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:13:0) after 1m7s
[19:50:37.530] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:13:0)
[19:50:37.549] Build status: SCHEDULED
[19:50:37.549] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:14:0) after 9m16s
[19:59:53.681] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:14:0)
[19:59:53.703] Build status: SCHEDULED
[19:59:53.703] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:15:0) after 1m22s
[20:01:15.703] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:15:0)
[20:01:15.722] Build status: SCHEDULED
[20:01:15.722] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:16:0) after 3m23s
[20:04:38.901] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:16:0)
[20:04:38.931] Build status: SCHEDULED
[20:04:38.931] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:17:0) after 4m34s
[20:09:12.901] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:17:0)
[20:09:12.930] Build status: SCHEDULED
[20:09:12.930] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:18:0) after 5m52s
[20:15:05.028] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:18:0)
[20:15:05.051] Build status: SCHEDULED
[20:15:05.051] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:19:0) after 3m10s
[20:18:15.048] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:19:0)
[20:18:15.083] Build status: SCHEDULED
[20:18:15.083] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:20:0) after 5m19s
[20:23:34.195] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:20:0)
[20:23:34.216] Build status: SCHEDULED
[20:23:34.216] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:21:0) after 1m44s
[20:25:18.230] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:21:0)
[20:25:18.247] Build status: SCHEDULED
[20:25:18.247] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:22:0) after 7m26s
[20:32:44.272] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:22:0)
[20:32:44.298] Build status: SCHEDULED
[20:32:44.298] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:23:0) after 5m48s
[20:38:32.316] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:23:0)
[20:38:32.344] Build status: SCHEDULED
[20:38:32.344] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:24:0) after 3m20s
[20:41:52.399] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:24:0)
[20:41:52.432] Build status: SCHEDULED
[20:41:52.432] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:25:0) after 6m17s
[20:48:09.454] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:25:0)
[20:48:09.496] Build status: SCHEDULED
[20:48:09.496] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:26:0) after 9m37s
[20:57:46.604] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:26:0)
[20:57:46.643] Build status: SCHEDULED
[20:57:46.643] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:27:0) after 7m2s
[21:04:48.720] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:27:0)
[21:04:48.739] Build status: SCHEDULED
[21:04:48.739] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:28:0) after 8m32s
[21:13:20.777] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:28:0)
[21:13:20.797] Build status: SCHEDULED
[21:13:20.797] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:29:0) after 4m46s
[21:18:06.962] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:29:0)
[21:18:06.981] Build status: SCHEDULED
[21:18:06.981] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:30:0) after 8m43s
[21:26:50.122] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:30:0)
[21:26:50.142] Build status: SCHEDULED
[21:26:50.142] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:31:0) after 8m10s
[21:28:15.085] Received PubSub notification, asking Buildbucket for the build status
[21:28:15.160] Build status: STARTED
[21:35:00.156] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:31:0)
[21:35:00.174] Build status: STARTED
[21:35:00.174] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8844061983924531264:33:0) after 9m6s
[21:36:10.233] Received PubSub notification, asking Buildbucket for the build status
[21:36:10.257] Build:
{
"id": "8678631921931619169",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"createdBy": "project:pigweed",
"createTime": "2026-06-18T18:50:56.718010641Z",
"startTime": "2026-06-18T21:28:14.832720730Z",
"endTime": "2026-06-18T21:35:36.492407887Z",
"updateTime": "2026-06-18T21:35:36.492407887Z",
"status": "FAILURE",
"input": {
}
}
[21:36:10.257] Invocation finished in 2h45m14.822300073s with status FAILED