[23:53:44.427] New invocation is queued and will start shortly
[23:53:48.159] Starting the invocation (attempt 1)
[23:53:48.192] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/cac03a4c378c3b0499e0cde7fa6b4681418a4f12"
[23:53:48.192] Popped gitiles commit info from properties and tags
[23:53:48.192] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[23:53:48.192] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[23:53:48.193] Buildbucket request:
{
"requestId": "8832084132794667648",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOS"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cac03a4c378c3b0499e0cde7fa6b4681418a4f12",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8832084132794667648"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOS"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[23:53:48.975] Scheduled build:
{
"id": "8666654067986060881",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOS"
},
"number": 38392,
"createdBy": "project:chromium",
"createTime": "2026-10-28T23:53:48.391628380Z",
"updateTime": "2026-10-28T23:53:48.391628380Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cac03a4c378c3b0499e0cde7fa6b4681418a4f12",
"ref": "refs/heads/main"
}
}
}
[23:53:48.975] Task URL: https://cr-buildbucket.appspot.com/build/8666654067986060881
[23:53:48.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:2:0) after 6m16s
[23:53:49.923] Received PubSub notification, asking Buildbucket for the build status
[23:53:50.003] Build status: SCHEDULED
[23:53:52.981] Received PubSub notification, asking Buildbucket for the build status
[23:53:53.008] Build status: STARTED
[00:00:05.280] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:2:0)
[00:00:05.325] Build status: STARTED
[00:00:05.325] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:5:0) after 1m47s
[00:01:52.414] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:5:0)
[00:01:52.438] Build status: STARTED
[00:01:52.438] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:6:0) after 8m40s
[00:10:32.523] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:6:0)
[00:10:32.547] Build status: STARTED
[00:10:32.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:7:0) after 5m37s
[00:16:09.677] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:7:0)
[00:16:09.718] Build status: STARTED
[00:16:09.718] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:8:0) after 7m17s
[00:23:26.729] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:8:0)
[00:23:26.759] Build status: STARTED
[00:23:26.759] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:9:0) after 2m52s
[00:26:18.787] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:9:0)
[00:26:18.828] Build status: STARTED
[00:26:18.828] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:10:0) after 1m44s
[00:28:02.902] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:10:0)
[00:28:02.943] Build status: STARTED
[00:28:02.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:11:0) after 4m57s
[00:32:59.968] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:11:0)
[00:32:59.993] Build status: STARTED
[00:32:59.993] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:12:0) after 5m27s
[00:38:27.080] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:12:0)
[00:38:27.099] Build status: STARTED
[00:38:27.099] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:13:0) after 1m8s
[00:39:35.179] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:13:0)
[00:39:35.203] Build status: STARTED
[00:39:35.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:14:0) after 9m3s
[00:48:38.257] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:14:0)
[00:48:38.301] Build status: STARTED
[00:48:38.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:15:0) after 4m22s
[00:53:00.449] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:15:0)
[00:53:00.471] Build status: STARTED
[00:53:00.471] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:16:0) after 3m26s
[00:56:26.522] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:16:0)
[00:56:26.539] Build status: STARTED
[00:56:26.540] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:17:0) after 4m29s
[01:00:55.521] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:17:0)
[01:00:55.552] Build status: STARTED
[01:00:55.553] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:18:0) after 8m12s
[01:09:07.579] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:18:0)
[01:09:07.602] Build status: STARTED
[01:09:07.602] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:19:0) after 5m19s
[01:14:26.771] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:19:0)
[01:14:26.790] Build status: STARTED
[01:14:26.790] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:20:0) after 2m58s
[01:17:24.973] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:20:0)
[01:17:25.005] Build status: STARTED
[01:17:25.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:21:0) after 2m1s
[01:19:26.038] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:21:0)
[01:19:26.065] Build status: STARTED
[01:19:26.065] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:22:0) after 3m38s
[01:23:04.229] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:22:0)
[01:23:04.255] Build status: STARTED
[01:23:04.255] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:23:0) after 1m54s
[01:24:58.462] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:23:0)
[01:24:58.482] Build status: STARTED
[01:24:58.482] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:24:0) after 7m18s
[01:32:16.516] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:24:0)
[01:32:16.556] Build status: STARTED
[01:32:16.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:25:0) after 6m40s
[01:38:56.614] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:25:0)
[01:38:56.636] Build status: STARTED
[01:38:56.636] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:26:0) after 7m48s
[01:46:44.632] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:26:0)
[01:46:44.665] Build status: STARTED
[01:46:44.665] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:27:0) after 5m35s
[01:52:19.752] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:27:0)
[01:52:19.780] Build status: STARTED
[01:52:19.780] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:28:0) after 7m58s
[02:00:17.896] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:28:0)
[02:00:17.914] Build status: STARTED
[02:00:17.914] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:29:0) after 4m5s
[02:04:22.953] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:29:0)
[02:04:23.010] Build status: STARTED
[02:04:23.011] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:30:0) after 8m45s
[02:13:08.067] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:30:0)
[02:13:08.092] Build status: STARTED
[02:13:08.092] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:31:0) after 2m55s
[02:16:03.186] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:31:0)
[02:16:03.227] Build status: STARTED
[02:16:03.227] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:32:0) after 8m51s
[02:24:54.360] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:32:0)
[02:24:54.382] Build status: STARTED
[02:24:54.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:33:0) after 7m57s
[02:32:51.506] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:33:0)
[02:32:51.531] Build status: STARTED
[02:32:51.531] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:34:0) after 4m59s
[02:37:50.553] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:34:0)
[02:37:50.571] Build status: STARTED
[02:37:50.571] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:35:0) after 4m28s
[02:42:18.683] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:35:0)
[02:42:18.707] Build status: STARTED
[02:42:18.708] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:36:0) after 6m52s
[02:49:10.688] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:36:0)
[02:49:10.709] Build status: STARTED
[02:49:10.709] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:37:0) after 4m9s
[02:53:19.846] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:37:0)
[02:53:19.864] Build status: STARTED
[02:53:19.864] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:38:0) after 1m39s
[02:54:58.890] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:38:0)
[02:54:58.947] Build status: STARTED
[02:54:58.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:39:0) after 4m4s
[02:59:03.013] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:39:0)
[02:59:03.039] Build status: STARTED
[02:59:03.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:40:0) after 7m47s
[03:06:50.067] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:40:0)
[03:06:50.096] Build status: STARTED
[03:06:50.096] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:41:0) after 9m19s
[03:16:09.130] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:41:0)
[03:16:09.155] Build status: STARTED
[03:16:09.155] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:42:0) after 5m23s
[03:21:32.291] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:42:0)
[03:21:32.308] Build status: STARTED
[03:21:32.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:43:0) after 6m11s
[03:27:43.362] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:43:0)
[03:27:43.379] Build status: STARTED
[03:27:43.379] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:44:0) after 5m21s
[03:33:04.407] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:44:0)
[03:33:04.447] Build status: STARTED
[03:33:04.447] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:45:0) after 8m1s
[03:41:05.486] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:45:0)
[03:41:05.518] Build status: STARTED
[03:41:05.518] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:46:0) after 3m12s
[03:44:17.561] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:46:0)
[03:44:17.635] Build status: STARTED
[03:44:17.635] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:47:0) after 3m34s
[03:47:51.663] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:47:0)
[03:47:51.680] Build status: STARTED
[03:47:51.680] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:48:0) after 6m21s
[03:54:12.693] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:48:0)
[03:54:12.732] Build status: STARTED
[03:54:12.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:49:0) after 5m38s
[03:59:50.766] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:49:0)
[03:59:50.949] Build status: STARTED
[03:59:50.949] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:50:0) after 2m35s
[04:02:25.998] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:50:0)
[04:02:26.017] Build status: STARTED
[04:02:26.017] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:51:0) after 9m53s
[04:12:19.043] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:51:0)
[04:12:19.103] Build status: STARTED
[04:12:19.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:52:0) after 6m35s
[04:18:54.132] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:52:0)
[04:18:54.155] Build status: STARTED
[04:18:54.155] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:53:0) after 4m16s
[04:23:10.185] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:53:0)
[04:23:10.206] Build status: STARTED
[04:23:10.206] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:54:0) after 5m27s
[04:28:37.331] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:54:0)
[04:28:37.444] Build status: STARTED
[04:28:37.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:55:0) after 1m45s
[04:30:22.538] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:55:0)
[04:30:22.556] Build status: STARTED
[04:30:22.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:56:0) after 2m17s
[04:32:39.585] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:56:0)
[04:32:39.642] Build status: STARTED
[04:32:39.642] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:57:0) after 6m1s
[04:38:40.683] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:57:0)
[04:38:40.704] Build status: STARTED
[04:38:40.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:58:0) after 9m16s
[04:47:56.701] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:58:0)
[04:47:56.719] Build status: STARTED
[04:47:56.719] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:59:0) after 4m39s
[04:52:35.848] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:59:0)
[04:52:35.871] Build status: STARTED
[04:52:35.871] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8832084132794667648:60:0) after 7m59s
[04:54:03.142] Received PubSub notification, asking Buildbucket for the build status
[04:54:03.161] Build:
{
"id": "8666654067986060881",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOS"
},
"number": 38392,
"createdBy": "project:chromium",
"createTime": "2026-10-28T23:53:48.391628380Z",
"startTime": "2026-10-28T23:53:52.669144972Z",
"endTime": "2026-10-29T04:54:02.662793983Z",
"updateTime": "2026-10-29T04:54:02.662793983Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cac03a4c378c3b0499e0cde7fa6b4681418a4f12",
"ref": "refs/heads/main"
}
}
}
[04:54:03.161] Invocation finished in 5h0m18.739708805s with status FAILED