[03:55:45.466] New invocation is queued and will start shortly
[03:55:45.466] Triggered by user:rerunner@pigweed-service-accounts.iam.gserviceaccount.com
[03:55:46.678] Starting the invocation (attempt 1)
[03:55:46.700] Ignoring gitiles_ref tag without the buildset tag
[03:55:46.700] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[03:55:46.700] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[03:55:46.702] Buildbucket request:
{
"requestId": "8860788144736165056",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-linux-bazel-sanitizers-clang-tidy"
},
"properties": {
"$recipe_engine/scheduler": {
"hostname": "luci-scheduler.appspot.com",
"invocation": "8860788144736165056",
"job": "pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy",
"triggers": [
{
"buildbucket": {
"tags": [
"parent_buildername:rerunner",
"user_agent:rerunner"
]
},
"id": "4c7679b3-b574-4580-83a2-d803c0d91799",
"title": "rerunner/0",
"url": "https://cr-buildbucket.appspot.com/build/8695358308258884305"
}
]
}
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860788144736165056"
},
{
"key": "scheduler_job_id",
"value": "pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy"
},
{
"key": "user_agent",
"value": "luci-scheduler"
},
{
"key": "parent_buildername",
"value": "rerunner"
},
{
"key": "user_agent",
"value": "rerunner"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[03:55:47.022] Scheduled build:
{
"id": "8695358082717924049",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-linux-bazel-sanitizers-clang-tidy"
},
"createdBy": "project:pigweed",
"createTime": "2025-12-16T03:55:46.764948215Z",
"updateTime": "2025-12-16T03:55:46.764948215Z",
"status": "SCHEDULED",
"input": {
}
}
[03:55:47.022] Task URL: https://cr-buildbucket.appspot.com/build/8695358082717924049
[03:55:47.022] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:2:0) after 4m51s
[03:55:47.942] Received PubSub notification, asking Buildbucket for the build status
[03:55:47.962] Build status: SCHEDULED
[04:00:38.035] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:2:0)
[04:00:38.062] Build status: SCHEDULED
[04:00:38.062] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:4:0) after 8m55s
[04:09:33.185] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:4:0)
[04:09:33.203] Build status: SCHEDULED
[04:09:33.203] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:5:0) after 1m30s
[04:11:03.279] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:5:0)
[04:11:03.296] Build status: SCHEDULED
[04:11:03.296] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:6:0) after 6m18s
[04:17:21.312] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:6:0)
[04:17:21.392] Build status: SCHEDULED
[04:17:21.392] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:7:0) after 7m38s
[04:24:59.542] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:7:0)
[04:24:59.555] Build status: SCHEDULED
[04:24:59.555] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:8:0) after 1m32s
[04:26:31.668] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:8:0)
[04:26:31.748] Build status: SCHEDULED
[04:26:31.748] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:9:0) after 3m36s
[04:30:07.769] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:9:0)
[04:30:07.788] Build status: SCHEDULED
[04:30:07.788] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:10:0) after 1m12s
[04:31:19.841] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:10:0)
[04:31:19.855] Build status: SCHEDULED
[04:31:19.855] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:11:0) after 3m0s
[04:34:19.869] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:11:0)
[04:34:19.887] Build status: SCHEDULED
[04:34:19.888] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:12:0) after 6m50s
[04:41:09.902] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:12:0)
[04:41:10.014] Build status: SCHEDULED
[04:41:10.014] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:13:0) after 6m6s
[04:47:16.119] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:13:0)
[04:47:16.144] Build status: SCHEDULED
[04:47:16.144] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:14:0) after 9m47s
[04:57:03.201] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:14:0)
[04:57:03.216] Build status: SCHEDULED
[04:57:03.216] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:15:0) after 2m23s
[04:59:26.226] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:15:0)
[04:59:26.263] Build status: SCHEDULED
[04:59:26.263] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:16:0) after 7m55s
[05:07:21.379] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:16:0)
[05:07:21.395] Build status: SCHEDULED
[05:07:21.395] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:17:0) after 2m37s
[05:09:58.410] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:17:0)
[05:09:58.463] Build status: SCHEDULED
[05:09:58.463] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:18:0) after 8m36s
[05:18:34.532] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:18:0)
[05:18:34.556] Build status: SCHEDULED
[05:18:34.556] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:19:0) after 2m55s
[05:21:29.672] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:19:0)
[05:21:29.692] Build status: SCHEDULED
[05:21:29.692] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:20:0) after 8m31s
[05:30:00.704] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:20:0)
[05:30:00.746] Build status: SCHEDULED
[05:30:00.746] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:21:0) after 6m45s
[05:36:45.775] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:21:0)
[05:36:45.791] Build status: SCHEDULED
[05:36:45.791] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:22:0) after 9m28s
[05:46:13.808] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:22:0)
[05:46:13.829] Build status: SCHEDULED
[05:46:13.830] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:23:0) after 6m55s
[05:53:08.902] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:23:0)
[05:53:08.991] Build status: SCHEDULED
[05:53:08.991] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:24:0) after 9m41s
[06:02:50.006] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:24:0)
[06:02:50.020] Build status: SCHEDULED
[06:02:50.020] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:25:0) after 5m39s
[06:08:29.076] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:25:0)
[06:08:29.096] Build status: SCHEDULED
[06:08:29.096] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:26:0) after 8m17s
[06:16:46.408] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:26:0)
[06:16:46.435] Build status: SCHEDULED
[06:16:46.435] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:27:0) after 2m27s
[06:19:13.401] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:27:0)
[06:19:13.501] Build status: SCHEDULED
[06:19:13.501] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:28:0) after 4m15s
[06:23:28.515] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:28:0)
[06:23:28.532] Build status: SCHEDULED
[06:23:28.532] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:29:0) after 1m45s
[06:25:13.543] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:29:0)
[06:25:13.616] Build status: SCHEDULED
[06:25:13.617] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:30:0) after 2m11s
[06:27:24.769] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:30:0)
[06:27:24.785] Build status: SCHEDULED
[06:27:24.785] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:31:0) after 1m54s
[06:29:18.797] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:31:0)
[06:29:18.813] Build status: SCHEDULED
[06:29:18.813] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:32:0) after 9m13s
[06:38:31.825] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:32:0)
[06:38:31.840] Build status: SCHEDULED
[06:38:31.840] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:33:0) after 8m13s
[06:46:44.864] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:33:0)
[06:46:44.882] Build status: SCHEDULED
[06:46:44.882] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:34:0) after 5m47s
[06:52:31.928] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:34:0)
[06:52:31.944] Build status: SCHEDULED
[06:52:31.944] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:35:0) after 2m27s
[06:54:58.982] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:35:0)
[06:54:58.995] Build status: SCHEDULED
[06:54:58.995] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:36:0) after 2m7s
[06:57:06.015] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:36:0)
[06:57:06.044] Build status: SCHEDULED
[06:57:06.044] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:37:0) after 2m28s
[06:59:34.080] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:37:0)
[06:59:34.151] Build status: SCHEDULED
[06:59:34.151] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:38:0) after 3m16s
[07:02:50.165] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:38:0)
[07:02:50.182] Build status: SCHEDULED
[07:02:50.182] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:39:0) after 1m56s
[07:04:46.197] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:39:0)
[07:04:46.238] Build status: SCHEDULED
[07:04:46.238] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:40:0) after 6m32s
[07:11:18.298] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:40:0)
[07:11:18.317] Build status: SCHEDULED
[07:11:18.317] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:41:0) after 7m59s
[07:19:17.379] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:41:0)
[07:19:17.391] Build status: SCHEDULED
[07:19:17.391] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:42:0) after 4m58s
[07:24:11.130] Received PubSub notification, asking Buildbucket for the build status
[07:24:11.144] Build status: STARTED
[07:24:15.430] Handling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:42:0)
[07:24:15.446] Build status: STARTED
[07:24:15.446] Scheduling timer "check-buildbucket-build-status" (pigweed/pigweed.dev.ci-pigweed-linux-bazel-sanitizers-clang-tidy:8860788144736165056:44:0) after 8m24s
[07:25:59.426] Received PubSub notification, asking Buildbucket for the build status
[07:25:59.444] Build:
{
"id": "8695358082717924049",
"builder": {
"project": "pigweed",
"bucket": "pigweed.dev.ci",
"builder": "pigweed-linux-bazel-sanitizers-clang-tidy"
},
"createdBy": "project:pigweed",
"createTime": "2025-12-16T03:55:46.764948215Z",
"startTime": "2025-12-16T07:24:10.971093304Z",
"endTime": "2025-12-16T07:25:59.150646806Z",
"updateTime": "2025-12-16T07:25:59.150646806Z",
"status": "FAILURE",
"input": {
}
}
[07:25:59.444] Invocation finished in 3h30m13.98785131s with status FAILED