[04:34:10.488] New invocation is queued and will start shortly
[04:34:10.488] Triggered by user:rerunner@pigweed-service-accounts.iam.gserviceaccount.com
[04:34:11.973] Starting the invocation (attempt 1)
[04:34:12.002] Ignoring gitiles_ref tag without the buildset tag
[04:34:12.002] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:34:12.003] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:34:12.003] Buildbucket request:
{
"requestId": "8839042455803583568",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler.appspot.com",
"invocation": "8839042455803583568",
"job": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz",
"triggers": [
{
"buildbucket": {
"tags": [
"parent_buildername:rerunner",
"user_agent:rerunner"
]
},
"id": "951a730a-d2ec-4d99-bd36-184f192bd8ff",
"title": "rerunner/0",
"url": "https://cr-buildbucket.appspot.com/build/8673612725229104209"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8839042455803583568"
},
{
"key": "scheduler_job_id",
"value": "pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz"
},
{
"key": "user_agent",
"value": "luci-scheduler"
},
{
"key": "parent_buildername",
"value": "rerunner"
},
{
"key": "user_agent",
"value": "rerunner"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:34:12.373] Scheduled build:
{
"id": "8673612393480296337",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"createdBy": "project:pigweed",
"createTime": "2026-08-13T04:34:12.082823879Z",
"updateTime": "2026-08-13T04:34:12.082823879Z",
"status": "SCHEDULED",
"input": {
}
}
[04:34:12.373] Task URL: https://cr-buildbucket.appspot.com/build/8673612393480296337
[04:34:12.373] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:2:0) after 5m47s
[04:34:26.725] Received PubSub notification, asking Buildbucket for the build status
[04:34:26.744] Build status: SCHEDULED
[04:39:59.673] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:2:0)
[04:39:59.730] Build status: SCHEDULED
[04:39:59.730] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:4:0) after 1m20s
[04:41:19.864] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:4:0)
[04:41:19.889] Build status: SCHEDULED
[04:41:19.889] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:5:0) after 4m3s
[04:45:22.880] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:5:0)
[04:45:22.901] Build status: SCHEDULED
[04:45:22.902] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:6:0) after 3m14s
[04:48:36.914] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:6:0)
[04:48:36.936] Build status: SCHEDULED
[04:48:36.936] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:7:0) after 9m5s
[04:57:42.000] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:7:0)
[04:57:42.082] Build status: SCHEDULED
[04:57:42.082] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:8:0) after 4m10s
[05:01:52.106] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:8:0)
[05:01:52.126] Build status: SCHEDULED
[05:01:52.126] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:9:0) after 6m46s
[05:08:38.200] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:9:0)
[05:08:38.303] Build status: SCHEDULED
[05:08:38.303] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:10:0) after 1m34s
[05:10:12.363] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:10:0)
[05:10:12.524] Build status: SCHEDULED
[05:10:12.524] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:11:0) after 6m33s
[05:16:45.589] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:11:0)
[05:16:45.646] Build status: SCHEDULED
[05:16:45.646] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:12:0) after 1m15s
[05:18:00.860] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:12:0)
[05:18:00.889] Build status: SCHEDULED
[05:18:00.889] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:13:0) after 9m40s
[05:27:40.904] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:13:0)
[05:27:40.924] Build status: SCHEDULED
[05:27:40.924] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:14:0) after 2m16s
[05:29:56.991] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:14:0)
[05:29:57.023] Build status: SCHEDULED
[05:29:57.023] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:15:0) after 7m21s
[05:37:18.037] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:15:0)
[05:37:18.060] Build status: SCHEDULED
[05:37:18.060] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:16:0) after 1m10s
[05:38:28.171] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:16:0)
[05:38:28.194] Build status: SCHEDULED
[05:38:28.194] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:17:0) after 3m7s
[05:41:35.224] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:17:0)
[05:41:35.304] Build status: SCHEDULED
[05:41:35.304] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:18:0) after 6m43s
[05:48:18.409] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:18:0)
[05:48:18.432] Build status: SCHEDULED
[05:48:18.432] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:19:0) after 8m29s
[05:56:47.543] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:19:0)
[05:56:47.576] Build status: SCHEDULED
[05:56:47.576] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:20:0) after 4m18s
[06:01:05.650] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:20:0)
[06:01:05.668] Build status: SCHEDULED
[06:01:05.669] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:21:0) after 5m16s
[06:06:21.786] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:21:0)
[06:06:21.811] Build status: SCHEDULED
[06:06:21.811] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:22:0) after 5m43s
[06:12:04.843] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:22:0)
[06:12:04.872] Build status: SCHEDULED
[06:12:04.872] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:23:0) after 9m30s
[06:21:34.885] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:23:0)
[06:21:34.918] Build status: SCHEDULED
[06:21:34.918] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:24:0) after 2m8s
[06:23:42.933] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:24:0)
[06:23:42.954] Build status: SCHEDULED
[06:23:42.954] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:25:0) after 5m49s
[06:29:31.968] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:25:0)
[06:29:31.988] Build status: SCHEDULED
[06:29:31.988] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:26:0) after 6m18s
[06:35:50.104] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:26:0)
[06:35:50.365] Build status: SCHEDULED
[06:35:50.365] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:27:0) after 3m37s
[06:39:27.431] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:27:0)
[06:39:27.465] Build status: SCHEDULED
[06:39:27.465] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:28:0) after 7m45s
[06:47:12.501] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:28:0)
[06:47:12.518] Build status: SCHEDULED
[06:47:12.518] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:29:0) after 2m0s
[06:49:12.614] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:29:0)
[06:49:12.641] Build status: SCHEDULED
[06:49:12.641] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:30:0) after 3m24s
[06:52:36.704] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:30:0)
[06:52:36.724] Build status: SCHEDULED
[06:52:36.724] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:31:0) after 3m45s
[06:56:21.716] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:31:0)
[06:56:21.734] Build status: SCHEDULED
[06:56:21.734] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:32:0) after 3m57s
[07:00:18.762] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:32:0)
[07:00:18.785] Build status: SCHEDULED
[07:00:18.785] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:33:0) after 4m40s
[07:04:58.918] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:33:0)
[07:04:58.946] Build status: SCHEDULED
[07:04:58.946] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:34:0) after 2m58s
[07:05:38.327] Received PubSub notification, asking Buildbucket for the build status
[07:05:38.346] Build status: STARTED
[07:07:56.964] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:34:0)
[07:07:56.986] Build status: STARTED
[07:07:56.987] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-mac-arm-bazel-asan-fuzz:8839042455803583568:36:0) after 9m9s
[07:09:58.023] Received PubSub notification, asking Buildbucket for the build status
[07:09:58.052] Build:
{
"id": "8673612393480296337",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-mac-arm-bazel-asan-fuzz"
},
"createdBy": "project:pigweed",
"createTime": "2026-08-13T04:34:12.082823879Z",
"startTime": "2026-08-13T07:05:38.118945543Z",
"endTime": "2026-08-13T07:09:57.486750569Z",
"updateTime": "2026-08-13T07:09:57.486750569Z",
"status": "FAILURE",
"input": {
}
}
[07:09:58.052] Invocation finished in 2h35m47.569025203s with status FAILED