[16:51:50.848] New invocation is queued and will start shortly
[16:51:51.926] Starting the invocation (attempt 1)
[16:51:51.959] Ignoring gitiles_ref tag without the buildset tag
[16:51:51.959] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[16:51:51.959] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[16:51:51.959] Buildbucket request:
{
"requestId": "8852766784345543328",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-ubsan-fyi-rel"
},
"properties": {
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8852766784345543328"
},
{
"key": "scheduler_job_id",
"value": "chromium/mac-ubsan-fyi-rel"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[16:51:52.446] Scheduled build:
{
"id": "8687336722469807777",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-ubsan-fyi-rel"
},
"number": 628,
"createdBy": "project:chromium",
"createTime": "2026-03-14T16:51:52.015884482Z",
"updateTime": "2026-03-14T16:51:52.015884482Z",
"status": "SCHEDULED",
"input": {
}
}
[16:51:52.446] Task URL: https://cr-buildbucket.appspot.com/build/8687336722469807777
[16:51:52.446] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:2:0) after 5m26s
[16:51:53.375] Received PubSub notification, asking Buildbucket for the build status
[16:51:53.393] Build status: SCHEDULED
[16:51:57.446] Received PubSub notification, asking Buildbucket for the build status
[16:51:57.461] Build status: STARTED
[16:57:18.441] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:2:0)
[16:57:18.470] Build status: STARTED
[16:57:18.470] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:5:0) after 2m15s
[16:59:33.751] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:5:0)
[16:59:33.768] Build status: STARTED
[16:59:33.768] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:6:0) after 4m34s
[17:04:07.782] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:6:0)
[17:04:07.848] Build status: STARTED
[17:04:07.848] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:7:0) after 4m29s
[17:08:37.153] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:7:0)
[17:08:37.170] Build status: STARTED
[17:08:37.170] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:8:0) after 6m21s
[17:14:58.190] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:8:0)
[17:14:58.205] Build status: STARTED
[17:14:58.205] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:9:0) after 1m47s
[17:16:45.202] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:9:0)
[17:16:45.231] Build status: STARTED
[17:16:45.231] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:10:0) after 1m57s
[17:18:42.245] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:10:0)
[17:18:42.262] Build status: STARTED
[17:18:42.262] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:11:0) after 5m55s
[17:24:37.381] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:11:0)
[17:24:37.399] Build status: STARTED
[17:24:37.400] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:12:0) after 6m5s
[17:30:42.416] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:12:0)
[17:30:42.501] Build status: STARTED
[17:30:42.501] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:13:0) after 6m34s
[17:37:16.817] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:13:0)
[17:37:16.833] Build status: STARTED
[17:37:16.833] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:14:0) after 3m18s
[17:40:34.923] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:14:0)
[17:40:34.940] Build status: STARTED
[17:40:34.940] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:15:0) after 4m57s
[17:45:32.001] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:15:0)
[17:45:32.046] Build status: STARTED
[17:45:32.046] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:16:0) after 1m44s
[17:47:16.059] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:16:0)
[17:47:16.143] Build status: STARTED
[17:47:16.143] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:17:0) after 3m29s
[17:50:45.193] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:17:0)
[17:50:45.210] Build status: STARTED
[17:50:45.210] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:18:0) after 4m46s
[17:55:31.417] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:18:0)
[17:55:31.435] Build status: STARTED
[17:55:31.435] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:19:0) after 4m52s
[18:00:23.446] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:19:0)
[18:00:23.468] Build status: STARTED
[18:00:23.468] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:20:0) after 3m1s
[18:03:24.754] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:20:0)
[18:03:24.770] Build status: STARTED
[18:03:24.770] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:21:0) after 5m39s
[18:09:03.790] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:21:0)
[18:09:03.839] Build status: STARTED
[18:09:03.839] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:22:0) after 2m40s
[18:11:43.852] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:22:0)
[18:11:43.871] Build status: STARTED
[18:11:43.871] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:23:0) after 1m39s
[18:13:22.940] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:23:0)
[18:13:22.965] Build status: STARTED
[18:13:22.965] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:24:0) after 2m6s
[18:15:29.011] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:24:0)
[18:15:29.032] Build status: STARTED
[18:15:29.032] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:25:0) after 1m15s
[18:16:44.145] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:25:0)
[18:16:44.162] Build status: STARTED
[18:16:44.162] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:26:0) after 1m5s
[18:17:49.168] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:26:0)
[18:17:49.182] Build status: STARTED
[18:17:49.182] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:27:0) after 1m57s
[18:19:46.158] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:27:0)
[18:19:46.176] Build status: STARTED
[18:19:46.176] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:28:0) after 2m38s
[18:22:24.337] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:28:0)
[18:22:24.363] Build status: STARTED
[18:22:24.364] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:29:0) after 2m34s
[18:24:58.478] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:29:0)
[18:24:58.530] Build status: STARTED
[18:24:58.530] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:30:0) after 3m26s
[18:28:24.579] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:30:0)
[18:28:24.601] Build status: STARTED
[18:28:24.601] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:31:0) after 4m9s
[18:32:33.618] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:31:0)
[18:32:33.697] Build status: STARTED
[18:32:33.697] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:32:0) after 8m56s
[18:41:29.759] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:32:0)
[18:41:29.798] Build status: STARTED
[18:41:29.798] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:33:0) after 4m0s
[18:45:29.763] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:33:0)
[18:45:29.780] Build status: STARTED
[18:45:29.780] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:34:0) after 6m16s
[18:51:45.795] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:34:0)
[18:51:45.810] Build status: STARTED
[18:51:45.810] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:35:0) after 2m17s
[18:54:02.840] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:35:0)
[18:54:02.860] Build status: STARTED
[18:54:02.860] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:36:0) after 5m40s
[18:59:43.127] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:36:0)
[18:59:43.148] Build status: STARTED
[18:59:43.148] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:37:0) after 8m7s
[19:07:50.764] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:37:0)
[19:07:50.784] Build status: STARTED
[19:07:50.784] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:38:0) after 8m22s
[19:16:12.800] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:38:0)
[19:16:12.858] Build status: STARTED
[19:16:12.858] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:39:0) after 4m49s
[19:21:01.873] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:39:0)
[19:21:01.887] Build status: STARTED
[19:21:01.887] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:40:0) after 9m28s
[19:30:30.113] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:40:0)
[19:30:30.188] Build status: STARTED
[19:30:30.188] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:41:0) after 6m23s
[19:36:53.209] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:41:0)
[19:36:53.228] Build status: STARTED
[19:36:53.228] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:42:0) after 2m48s
[19:39:41.371] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:42:0)
[19:39:41.387] Build status: STARTED
[19:39:41.387] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:43:0) after 5m50s
[19:45:31.403] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:43:0)
[19:45:31.422] Build status: STARTED
[19:45:31.422] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:44:0) after 8m50s
[19:54:21.470] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:44:0)
[19:54:21.489] Build status: STARTED
[19:54:21.490] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:45:0) after 6m30s
[20:00:51.505] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:45:0)
[20:00:51.525] Build status: STARTED
[20:00:51.525] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:46:0) after 3m40s
[20:04:31.540] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:46:0)
[20:04:31.555] Build status: STARTED
[20:04:31.555] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:47:0) after 8m3s
[20:12:34.860] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:47:0)
[20:12:34.875] Build status: STARTED
[20:12:34.875] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:48:0) after 2m8s
[20:14:43.543] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:48:0)
[20:14:43.560] Build status: STARTED
[20:14:43.560] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:49:0) after 9m4s
[20:23:47.576] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:49:0)
[20:23:47.591] Build status: STARTED
[20:23:47.591] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:50:0) after 8m38s
[20:32:25.650] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:50:0)
[20:32:25.666] Build status: STARTED
[20:32:25.666] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:51:0) after 4m46s
[20:37:11.840] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:51:0)
[20:37:11.857] Build status: STARTED
[20:37:11.857] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:52:0) after 8m33s
[20:45:44.873] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:52:0)
[20:45:44.889] Build status: STARTED
[20:45:44.889] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:53:0) after 1m57s
[20:47:41.903] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:53:0)
[20:47:41.921] Build status: STARTED
[20:47:41.921] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:54:0) after 9m41s
[20:57:22.938] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:54:0)
[20:57:22.952] Build status: STARTED
[20:57:22.952] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:55:0) after 6m13s
[21:03:35.970] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:55:0)
[21:03:35.988] Build status: STARTED
[21:03:35.988] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:56:0) after 2m45s
[21:06:20.961] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:56:0)
[21:06:20.981] Build status: STARTED
[21:06:20.981] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:57:0) after 3m9s
[21:09:29.995] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:57:0)
[21:09:30.022] Build status: STARTED
[21:09:30.022] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:58:0) after 2m35s
[21:12:05.039] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:58:0)
[21:12:05.054] Build status: STARTED
[21:12:05.054] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:59:0) after 1m19s
[21:13:24.167] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:59:0)
[21:13:24.192] Build status: STARTED
[21:13:24.192] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:60:0) after 7m45s
[21:21:09.208] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:60:0)
[21:21:09.227] Build status: STARTED
[21:21:09.227] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:61:0) after 2m56s
[21:24:05.240] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:61:0)
[21:24:05.270] Build status: STARTED
[21:24:05.270] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:62:0) after 2m5s
[21:26:10.350] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:62:0)
[21:26:10.365] Build status: STARTED
[21:26:10.365] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:63:0) after 1m34s
[21:27:44.380] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:63:0)
[21:27:44.395] Build status: STARTED
[21:27:44.395] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:64:0) after 3m19s
[21:31:03.712] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:64:0)
[21:31:03.725] Build status: STARTED
[21:31:03.725] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8852766784345543328:65:0) after 9m4s
[21:38:38.560] Received PubSub notification, asking Buildbucket for the build status
[21:38:38.598] Build:
{
"id": "8687336722469807777",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-ubsan-fyi-rel"
},
"number": 628,
"createdBy": "project:chromium",
"createTime": "2026-03-14T16:51:52.015884482Z",
"startTime": "2026-03-14T16:51:57.262012484Z",
"endTime": "2026-03-14T21:37:54.536062937Z",
"updateTime": "2026-03-14T21:37:54.536062937Z",
"status": "FAILURE",
"input": {
}
}
[21:38:38.598] Invocation finished in 4h46m47.755497831s with status FAILED