[00:40:45.034] New invocation is queued and will start shortly
[00:40:46.104] Starting the invocation (attempt 1)
[00:40:46.129] Ignoring gitiles_ref tag without the buildset tag
[00:40:46.129] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[00:40:46.129] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[00:40:46.129] Buildbucket request:
{
"requestId": "8851015941151701440",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-ubsan-fyi-rel"
},
"properties": {
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851015941151701440"
},
{
"key": "scheduler_job_id",
"value": "chromium/mac-ubsan-fyi-rel"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[00:40:47.066] Scheduled build:
{
"id": "8685585879264782449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-ubsan-fyi-rel"
},
"number": 644,
"createdBy": "project:chromium",
"createTime": "2026-04-03T00:40:46.212813587Z",
"updateTime": "2026-04-03T00:40:46.212813587Z",
"status": "SCHEDULED",
"input": {
}
}
[00:40:47.067] Task URL: https://cr-buildbucket.appspot.com/build/8685585879264782449
[00:40:47.067] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:2:0) after 8m27s
[00:40:47.200] Received PubSub notification, asking Buildbucket for the build status
[00:40:47.215] Build status: SCHEDULED
[00:40:53.139] Received PubSub notification, asking Buildbucket for the build status
[00:40:53.202] Build status: STARTED
[00:49:14.081] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:2:0)
[00:49:14.102] Build status: STARTED
[00:49:14.102] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:5:0) after 1m34s
[00:50:48.167] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:5:0)
[00:50:48.197] Build status: STARTED
[00:50:48.197] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:6:0) after 8m43s
[00:59:31.379] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:6:0)
[00:59:31.402] Build status: STARTED
[00:59:31.403] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:7:0) after 4m59s
[01:04:30.475] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:7:0)
[01:04:30.492] Build status: STARTED
[01:04:30.492] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:8:0) after 1m12s
[01:05:42.510] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:8:0)
[01:05:42.528] Build status: STARTED
[01:05:42.528] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:9:0) after 5m29s
[01:11:11.544] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:9:0)
[01:11:11.561] Build status: STARTED
[01:11:11.561] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:10:0) after 7m36s
[01:18:47.576] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:10:0)
[01:18:47.606] Build status: STARTED
[01:18:47.606] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:11:0) after 8m9s
[01:26:56.612] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:11:0)
[01:26:56.635] Build status: STARTED
[01:26:56.635] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:12:0) after 5m3s
[01:31:59.702] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:12:0)
[01:31:59.717] Build status: STARTED
[01:31:59.717] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:13:0) after 3m20s
[01:35:19.732] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:13:0)
[01:35:19.750] Build status: STARTED
[01:35:19.751] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:14:0) after 6m30s
[01:41:49.765] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:14:0)
[01:41:49.782] Build status: STARTED
[01:41:49.782] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:15:0) after 6m45s
[01:48:34.802] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:15:0)
[01:48:34.886] Build status: STARTED
[01:48:34.886] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:16:0) after 3m53s
[01:52:27.855] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:16:0)
[01:52:27.871] Build status: STARTED
[01:52:27.871] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:17:0) after 8m12s
[02:00:39.846] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:17:0)
[02:00:39.860] Build status: STARTED
[02:00:39.860] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:18:0) after 5m51s
[02:06:30.926] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:18:0)
[02:06:30.942] Build status: STARTED
[02:06:30.942] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:19:0) after 7m7s
[02:13:37.957] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:19:0)
[02:13:37.979] Build status: STARTED
[02:13:37.979] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:20:0) after 4m8s
[02:17:45.993] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:20:0)
[02:17:46.037] Build status: STARTED
[02:17:46.037] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:21:0) after 1m6s
[02:18:52.061] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:21:0)
[02:18:52.113] Build status: STARTED
[02:18:52.113] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:22:0) after 2m17s
[02:21:09.199] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:22:0)
[02:21:09.345] Build status: STARTED
[02:21:09.345] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:23:0) after 4m30s
[02:25:39.353] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:23:0)
[02:25:39.368] Build status: STARTED
[02:25:39.368] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:24:0) after 3m42s
[02:29:21.369] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:24:0)
[02:29:21.385] Build status: STARTED
[02:29:21.385] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:25:0) after 2m5s
[02:31:26.400] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:25:0)
[02:31:26.416] Build status: STARTED
[02:31:26.416] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:26:0) after 6m2s
[02:37:28.437] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:26:0)
[02:37:28.461] Build status: STARTED
[02:37:28.461] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:27:0) after 3m42s
[02:41:10.488] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:27:0)
[02:41:10.513] Build status: STARTED
[02:41:10.513] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:28:0) after 4m41s
[02:45:51.542] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:28:0)
[02:45:51.618] Build status: STARTED
[02:45:51.618] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:29:0) after 1m41s
[02:47:32.637] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:29:0)
[02:47:32.694] Build status: STARTED
[02:47:32.694] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:30:0) after 9m38s
[02:57:10.755] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:30:0)
[02:57:10.772] Build status: STARTED
[02:57:10.772] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:31:0) after 6m27s
[03:03:37.785] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:31:0)
[03:03:37.803] Build status: STARTED
[03:03:37.803] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:32:0) after 1m39s
[03:05:16.916] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:32:0)
[03:05:16.962] Build status: STARTED
[03:05:16.962] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:33:0) after 6m20s
[03:11:37.089] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:33:0)
[03:11:37.117] Build status: STARTED
[03:11:37.117] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:34:0) after 7m49s
[03:19:26.258] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:34:0)
[03:19:26.283] Build status: STARTED
[03:19:26.283] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:35:0) after 2m18s
[03:21:44.303] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:35:0)
[03:21:44.317] Build status: STARTED
[03:21:44.317] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:36:0) after 7m13s
[03:28:57.351] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:36:0)
[03:28:57.424] Build status: STARTED
[03:28:57.424] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:37:0) after 1m54s
[03:30:51.477] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:37:0)
[03:30:51.491] Build status: STARTED
[03:30:51.491] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:38:0) after 4m42s
[03:35:33.507] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:38:0)
[03:35:33.522] Build status: STARTED
[03:35:33.522] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:39:0) after 8m47s
[03:44:20.544] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:39:0)
[03:44:20.611] Build status: STARTED
[03:44:20.611] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:40:0) after 9m20s
[03:53:40.628] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:40:0)
[03:53:40.885] Build status: STARTED
[03:53:40.885] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:41:0) after 9m15s
[04:02:55.900] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:41:0)
[04:02:56.134] Build status: STARTED
[04:02:56.134] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:42:0) after 1m41s
[04:04:37.128] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:42:0)
[04:04:37.207] Build status: STARTED
[04:04:37.207] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:43:0) after 5m53s
[04:10:30.232] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:43:0)
[04:10:30.275] Build status: STARTED
[04:10:30.275] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:44:0) after 1m29s
[04:11:59.296] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:44:0)
[04:11:59.312] Build status: STARTED
[04:11:59.312] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:45:0) after 2m21s
[04:14:20.318] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:45:0)
[04:14:20.376] Build status: STARTED
[04:14:20.376] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:46:0) after 1m58s
[04:16:18.417] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:46:0)
[04:16:18.435] Build status: STARTED
[04:16:18.435] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:47:0) after 2m24s
[04:18:42.491] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:47:0)
[04:18:42.525] Build status: STARTED
[04:18:42.525] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:48:0) after 8m29s
[04:27:11.539] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:48:0)
[04:27:11.564] Build status: STARTED
[04:27:11.564] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:49:0) after 1m47s
[04:28:58.643] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:49:0)
[04:28:58.673] Build status: STARTED
[04:28:58.673] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:50:0) after 9m58s
[04:38:56.917] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:50:0)
[04:38:56.935] Build status: STARTED
[04:38:56.935] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:51:0) after 2m33s
[04:41:30.144] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:51:0)
[04:41:30.204] Build status: STARTED
[04:41:30.204] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:52:0) after 2m19s
[04:43:49.318] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:52:0)
[04:43:49.360] Build status: STARTED
[04:43:49.360] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:53:0) after 8m40s
[04:52:29.372] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:53:0)
[04:52:29.387] Build status: STARTED
[04:52:29.387] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:54:0) after 3m20s
[04:55:49.369] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:54:0)
[04:55:49.391] Build status: STARTED
[04:55:49.391] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:55:0) after 9m32s
[05:05:21.364] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:55:0)
[05:05:21.380] Build status: STARTED
[05:05:21.380] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:56:0) after 5m56s
[05:11:17.398] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:56:0)
[05:11:17.415] Build status: STARTED
[05:11:17.415] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:57:0) after 5m5s
[05:16:22.425] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:57:0)
[05:16:22.476] Build status: STARTED
[05:16:22.476] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:58:0) after 3m16s
[05:19:38.530] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:58:0)
[05:19:38.546] Build status: STARTED
[05:19:38.546] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:59:0) after 9m49s
[05:29:27.564] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:59:0)
[05:29:27.591] Build status: STARTED
[05:29:27.591] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:60:0) after 9m43s
[05:39:10.612] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:60:0)
[05:39:10.662] Build status: STARTED
[05:39:10.662] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:61:0) after 1m22s
[05:40:32.716] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:61:0)
[05:40:32.737] Build status: STARTED
[05:40:32.737] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:62:0) after 5m48s
[05:46:20.736] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:62:0)
[05:46:20.814] Build status: STARTED
[05:46:20.814] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:63:0) after 6m9s
[05:52:29.829] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:63:0)
[05:52:29.847] Build status: STARTED
[05:52:29.847] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:64:0) after 3m16s
[05:55:45.923] Handling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:64:0)
[05:55:45.942] Build status: STARTED
[05:55:45.942] Scheduling timer "check-buildbucket-build-status" (chromium/mac-ubsan-fyi-rel:8851015941151701440:65:0) after 4m30s
[05:56:23.433] Received PubSub notification, asking Buildbucket for the build status
[05:56:23.451] Build:
{
"id": "8685585879264782449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-ubsan-fyi-rel"
},
"number": 644,
"createdBy": "project:chromium",
"createTime": "2026-04-03T00:40:46.212813587Z",
"startTime": "2026-04-03T00:40:52.781341469Z",
"endTime": "2026-04-03T05:56:22.972349983Z",
"updateTime": "2026-04-03T05:56:22.972349983Z",
"status": "FAILURE",
"input": {
}
}
[05:56:23.451] Invocation finished in 5h15m38.421882862s with status FAILED