[12:50:38.702] New invocation is queued and will start shortly
[12:50:38.702] Triggered by user:rerunner@pigweed-service-accounts.iam.gserviceaccount.com
[12:50:40.296] Starting the invocation (attempt 1)
[12:50:40.325] Ignoring gitiles_ref tag without the buildset tag
[12:50:40.325] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[12:50:40.325] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[12:50:40.326] Buildbucket request:
{
"requestId": "8841094950823179520",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-linux-bazel-rpc"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler.appspot.com",
"invocation": "8841094950823179520",
"job": "pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc",
"triggers": [
{
"buildbucket": {
"tags": [
"parent_buildername:rerunner",
"user_agent:rerunner"
]
},
"id": "3860b879-1ddf-4f3e-a6ba-c636a1f3aa06",
"title": "rerunner/0",
"url": "https://cr-buildbucket.appspot.com/build/8675665249056486961"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8841094950823179520"
},
{
"key": "scheduler_job_id",
"value": "pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc"
},
{
"key": "user_agent",
"value": "luci-scheduler"
},
{
"key": "parent_buildername",
"value": "rerunner"
},
{
"key": "user_agent",
"value": "rerunner"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[12:50:40.945] Scheduled build:
{
"id": "8675664888339679889",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-linux-bazel-rpc"
},
"createdBy": "project:pigweed",
"createTime": "2026-07-21T12:50:40.450659897Z",
"updateTime": "2026-07-21T12:50:40.450659897Z",
"status": "SCHEDULED",
"input": {
}
}
[12:50:40.945] Task URL: https://cr-buildbucket.appspot.com/build/8675664888339679889
[12:50:40.946] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:2:0) after 4m33s
[12:50:41.782] Received PubSub notification, asking Buildbucket for the build status
[12:50:41.820] Build status: SCHEDULED
[12:55:14.013] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:2:0)
[12:55:14.029] Build status: SCHEDULED
[12:55:14.029] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:4:0) after 4m53s
[13:00:07.045] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:4:0)
[13:00:07.094] Build status: SCHEDULED
[13:00:07.094] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:5:0) after 5m43s
[13:05:50.110] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:5:0)
[13:05:50.128] Build status: SCHEDULED
[13:05:50.129] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:6:0) after 7m6s
[13:12:56.148] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:6:0)
[13:12:56.175] Build status: SCHEDULED
[13:12:56.175] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:7:0) after 8m6s
[13:21:02.194] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:7:0)
[13:21:02.283] Build status: SCHEDULED
[13:21:02.284] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:8:0) after 5m6s
[13:26:08.289] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:8:0)
[13:26:08.312] Build status: SCHEDULED
[13:26:08.312] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:9:0) after 4m34s
[13:30:42.376] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:9:0)
[13:30:42.448] Build status: SCHEDULED
[13:30:42.448] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:10:0) after 7m16s
[13:37:58.511] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:10:0)
[13:37:58.528] Build status: SCHEDULED
[13:37:58.528] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:11:0) after 9m39s
[13:47:37.605] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:11:0)
[13:47:37.624] Build status: SCHEDULED
[13:47:37.624] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:12:0) after 4m17s
[13:51:54.641] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:12:0)
[13:51:54.662] Build status: SCHEDULED
[13:51:54.662] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:13:0) after 3m29s
[13:55:23.680] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:13:0)
[13:55:23.722] Build status: SCHEDULED
[13:55:23.722] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:14:0) after 7m3s
[14:02:26.769] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:14:0)
[14:02:26.791] Build status: SCHEDULED
[14:02:26.791] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:15:0) after 6m13s
[14:08:39.865] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:15:0)
[14:08:39.896] Build status: SCHEDULED
[14:08:39.896] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:16:0) after 2m51s
[14:11:30.917] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:16:0)
[14:11:30.947] Build status: SCHEDULED
[14:11:30.947] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:17:0) after 2m13s
[14:13:43.974] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:17:0)
[14:13:43.998] Build status: SCHEDULED
[14:13:43.998] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:18:0) after 7m18s
[14:21:02.021] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:18:0)
[14:21:02.052] Build status: SCHEDULED
[14:21:02.052] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:19:0) after 6m6s
[14:27:08.068] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:19:0)
[14:27:08.086] Build status: SCHEDULED
[14:27:08.086] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:20:0) after 1m27s
[14:28:35.149] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:20:0)
[14:28:35.177] Build status: SCHEDULED
[14:28:35.177] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:21:0) after 9m45s
[14:38:20.280] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:21:0)
[14:38:20.304] Build status: SCHEDULED
[14:38:20.304] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:22:0) after 3m37s
[14:41:57.320] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:22:0)
[14:41:57.349] Build status: SCHEDULED
[14:41:57.349] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:23:0) after 3m4s
[14:45:01.472] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:23:0)
[14:45:01.500] Build status: SCHEDULED
[14:45:01.500] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:24:0) after 4m33s
[14:49:34.536] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:24:0)
[14:49:34.621] Build status: SCHEDULED
[14:49:34.621] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:25:0) after 7m47s
[14:57:21.714] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:25:0)
[14:57:21.784] Build status: SCHEDULED
[14:57:21.784] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:26:0) after 9m18s
[15:06:39.873] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:26:0)
[15:06:39.914] Build status: SCHEDULED
[15:06:39.914] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:27:0) after 3m42s
[15:10:21.938] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:27:0)
[15:10:21.958] Build status: SCHEDULED
[15:10:21.958] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:28:0) after 7m23s
[15:17:44.973] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:28:0)
[15:17:45.025] Build status: SCHEDULED
[15:17:45.025] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:29:0) after 2m49s
[15:20:34.138] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:29:0)
[15:20:34.161] Build status: SCHEDULED
[15:20:34.161] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:30:0) after 1m39s
[15:22:13.354] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:30:0)
[15:22:13.373] Build status: SCHEDULED
[15:22:13.373] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:31:0) after 5m19s
[15:27:32.439] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:31:0)
[15:27:32.469] Build status: SCHEDULED
[15:27:32.469] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:32:0) after 5m32s
[15:33:04.560] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:32:0)
[15:33:04.583] Build status: SCHEDULED
[15:33:04.583] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:33:0) after 5m54s
[15:38:58.749] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:33:0)
[15:38:58.767] Build status: SCHEDULED
[15:38:58.767] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:34:0) after 6m19s
[15:45:17.831] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:34:0)
[15:45:17.859] Build status: SCHEDULED
[15:45:17.859] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:35:0) after 9m12s
[15:54:29.918] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:35:0)
[15:54:30.119] Build status: SCHEDULED
[15:54:30.119] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:36:0) after 9m1s
[16:03:31.146] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:36:0)
[16:03:31.164] Build status: SCHEDULED
[16:03:31.165] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:37:0) after 7m56s
[16:11:27.287] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:37:0)
[16:11:27.313] Build status: SCHEDULED
[16:11:27.313] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:38:0) after 7m46s
[16:19:13.336] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:38:0)
[16:19:13.354] Build status: SCHEDULED
[16:19:13.354] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:39:0) after 6m33s
[16:25:46.377] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:39:0)
[16:25:46.397] Build status: SCHEDULED
[16:25:46.397] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:40:0) after 6m34s
[16:32:20.416] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:40:0)
[16:32:20.454] Build status: SCHEDULED
[16:32:20.454] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:41:0) after 1m28s
[16:33:48.469] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:41:0)
[16:33:48.490] Build status: SCHEDULED
[16:33:48.490] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:42:0) after 4m45s
[16:38:33.555] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:42:0)
[16:38:33.584] Build status: SCHEDULED
[16:38:33.584] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:43:0) after 5m40s
[16:44:13.603] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:43:0)
[16:44:13.662] Build status: SCHEDULED
[16:44:13.662] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:44:0) after 6m23s
[16:50:36.710] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:44:0)
[16:50:36.730] Build status: SCHEDULED
[16:50:36.731] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:45:0) after 8m30s
[16:59:06.750] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:45:0)
[16:59:06.895] Build status: SCHEDULED
[16:59:06.895] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-rpc:8841094950823179520:46:0) after 9m34s
[17:00:40.990] Received PubSub notification, asking Buildbucket for the build status
[17:00:41.019] Build status: STARTED
[17:07:39.528] Received PubSub notification, asking Buildbucket for the build status
[17:07:39.559] Build:
{
"id": "8675664888339679889",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-linux-bazel-rpc"
},
"createdBy": "project:pigweed",
"createTime": "2026-07-21T12:50:40.450659897Z",
"startTime": "2026-07-21T17:00:38.315899773Z",
"endTime": "2026-07-21T17:07:38.978496056Z",
"updateTime": "2026-07-21T17:07:38.978496056Z",
"status": "FAILURE",
"input": {
}
}
[17:07:39.559] Invocation finished in 4h17m0.860942393s with status FAILED