[01:18:02.105] New invocation is queued and will start shortly
[01:18:03.194] Starting the invocation (attempt 1)
[01:18:03.248] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/e3ae124bcdc9e66600f487a9d18ae963ee27a409"
[01:18:03.248] Popped gitiles commit info from properties and tags
[01:18:03.248] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[01:18:03.248] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[01:18:03.249] Buildbucket request:
{
"requestId": "8854365683176866880",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOS"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e3ae124bcdc9e66600f487a9d18ae963ee27a409",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8854365683176866880"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOS"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[01:18:03.681] Scheduled build:
{
"id": "8688935621268586401",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOS"
},
"number": 34955,
"createdBy": "project:chromium",
"createTime": "2026-02-25T01:18:03.296496507Z",
"updateTime": "2026-02-25T01:18:03.296496507Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e3ae124bcdc9e66600f487a9d18ae963ee27a409",
"ref": "refs/heads/main"
}
}
}
[01:18:03.681] Task URL: https://cr-buildbucket.appspot.com/build/8688935621268586401
[01:18:03.681] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:2:0) after 2m27s
[01:18:04.712] Received PubSub notification, asking Buildbucket for the build status
[01:18:04.736] Build status: SCHEDULED
[01:18:09.886] Received PubSub notification, asking Buildbucket for the build status
[01:18:09.904] Build status: STARTED
[01:20:30.768] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:2:0)
[01:20:30.794] Build status: STARTED
[01:20:30.794] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:5:0) after 4m36s
[01:25:06.830] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:5:0)
[01:25:06.860] Build status: STARTED
[01:25:06.861] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:6:0) after 3m24s
[01:28:30.875] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:6:0)
[01:28:30.904] Build status: STARTED
[01:28:30.904] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:7:0) after 2m23s
[01:30:54.069] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:7:0)
[01:30:54.106] Build status: STARTED
[01:30:54.106] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:8:0) after 4m53s
[01:35:47.145] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:8:0)
[01:35:47.192] Build status: STARTED
[01:35:47.192] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:9:0) after 7m43s
[01:43:30.290] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:9:0)
[01:43:30.317] Build status: STARTED
[01:43:30.317] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:10:0) after 2m59s
[01:46:29.342] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:10:0)
[01:46:29.358] Build status: STARTED
[01:46:29.358] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:11:0) after 1m25s
[01:47:54.424] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:11:0)
[01:47:54.461] Build status: STARTED
[01:47:54.461] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:12:0) after 8m34s
[01:56:28.480] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:12:0)
[01:56:28.507] Build status: STARTED
[01:56:28.507] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:13:0) after 6m56s
[02:03:24.555] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:13:0)
[02:03:24.642] Build status: STARTED
[02:03:24.642] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:14:0) after 3m8s
[02:06:32.749] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:14:0)
[02:06:32.765] Build status: STARTED
[02:06:32.765] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:15:0) after 5m10s
[02:11:42.860] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:15:0)
[02:11:42.880] Build status: STARTED
[02:11:42.880] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:16:0) after 9m39s
[02:21:21.903] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:16:0)
[02:21:21.961] Build status: STARTED
[02:21:21.961] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:17:0) after 9m48s
[02:31:10.092] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:17:0)
[02:31:10.105] Build status: STARTED
[02:31:10.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:18:0) after 3m11s
[02:34:21.121] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:18:0)
[02:34:21.149] Build status: STARTED
[02:34:21.149] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:19:0) after 3m36s
[02:37:57.252] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:19:0)
[02:37:57.272] Build status: STARTED
[02:37:57.272] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:20:0) after 7m59s
[02:45:56.346] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:20:0)
[02:45:56.389] Build status: STARTED
[02:45:56.389] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:21:0) after 2m14s
[02:48:10.424] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:21:0)
[02:48:10.437] Build status: STARTED
[02:48:10.438] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:22:0) after 6m18s
[02:54:28.570] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:22:0)
[02:54:28.601] Build status: STARTED
[02:54:28.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:23:0) after 7m45s
[03:02:13.767] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:23:0)
[03:02:13.795] Build status: STARTED
[03:02:13.795] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:24:0) after 7m24s
[03:09:37.863] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:24:0)
[03:09:37.881] Build status: STARTED
[03:09:37.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:25:0) after 4m0s
[03:13:37.949] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:25:0)
[03:13:38.016] Build status: STARTED
[03:13:38.016] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:26:0) after 6m52s
[03:20:30.037] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:26:0)
[03:20:30.052] Build status: STARTED
[03:20:30.052] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:27:0) after 5m9s
[03:25:39.158] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:27:0)
[03:25:39.174] Build status: STARTED
[03:25:39.174] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:28:0) after 4m32s
[03:30:11.197] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:28:0)
[03:30:11.249] Build status: STARTED
[03:30:11.249] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:29:0) after 7m23s
[03:37:34.284] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:29:0)
[03:37:34.349] Build status: STARTED
[03:37:34.349] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:30:0) after 2m49s
[03:40:23.428] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:30:0)
[03:40:23.443] Build status: STARTED
[03:40:23.443] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:31:0) after 7m44s
[03:48:07.461] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:31:0)
[03:48:07.488] Build status: STARTED
[03:48:07.489] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:32:0) after 3m46s
[03:51:53.656] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:32:0)
[03:51:53.671] Build status: STARTED
[03:51:53.671] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:33:0) after 2m50s
[03:54:43.690] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:33:0)
[03:54:43.712] Build status: STARTED
[03:54:43.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:34:0) after 2m37s
[03:57:20.735] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:34:0)
[03:57:20.793] Build status: STARTED
[03:57:20.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:35:0) after 4m48s
[04:02:08.817] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:35:0)
[04:02:08.877] Build status: STARTED
[04:02:08.877] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:36:0) after 1m58s
[04:04:06.896] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:36:0)
[04:04:06.968] Build status: STARTED
[04:04:06.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:37:0) after 5m36s
[04:09:43.006] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:37:0)
[04:09:43.024] Build status: STARTED
[04:09:43.024] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:38:0) after 5m37s
[04:15:20.049] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:38:0)
[04:15:20.078] Build status: STARTED
[04:15:20.078] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:39:0) after 7m57s
[04:23:17.174] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:39:0)
[04:23:17.203] Build status: STARTED
[04:23:17.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:40:0) after 4m48s
[04:28:05.219] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:40:0)
[04:28:05.243] Build status: STARTED
[04:28:05.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:41:0) after 1m26s
[04:29:31.261] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:41:0)
[04:29:31.275] Build status: STARTED
[04:29:31.275] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:42:0) after 6m37s
[04:36:08.307] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:42:0)
[04:36:08.324] Build status: STARTED
[04:36:08.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:43:0) after 7m3s
[04:43:11.345] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:43:0)
[04:43:11.378] Build status: STARTED
[04:43:11.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:44:0) after 1m55s
[04:45:06.470] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:44:0)
[04:45:06.492] Build status: STARTED
[04:45:06.492] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:45:0) after 6m46s
[04:51:52.577] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:45:0)
[04:51:52.591] Build status: STARTED
[04:51:52.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:46:0) after 9m23s
[05:01:15.600] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:46:0)
[05:01:15.621] Build status: STARTED
[05:01:15.621] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:47:0) after 5m21s
[05:06:36.659] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:47:0)
[05:06:36.767] Build status: STARTED
[05:06:36.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:48:0) after 8m12s
[05:14:48.934] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:48:0)
[05:14:48.953] Build status: STARTED
[05:14:48.953] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:49:0) after 3m59s
[05:18:47.993] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:49:0)
[05:18:48.004] Build status: STARTED
[05:18:48.004] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:50:0) after 9m8s
[05:27:56.083] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:50:0)
[05:27:56.101] Build status: STARTED
[05:27:56.101] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:51:0) after 1m29s
[05:29:25.158] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:51:0)
[05:29:25.185] Build status: STARTED
[05:29:25.185] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:52:0) after 3m51s
[05:33:16.159] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:52:0)
[05:33:16.178] Build status: STARTED
[05:33:16.178] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:53:0) after 4m10s
[05:37:26.188] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:53:0)
[05:37:26.203] Build status: STARTED
[05:37:26.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:54:0) after 3m46s
[05:41:12.254] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:54:0)
[05:41:12.269] Build status: STARTED
[05:41:12.269] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:55:0) after 5m42s
[05:46:54.451] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:55:0)
[05:46:54.471] Build status: STARTED
[05:46:54.471] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:56:0) after 6m13s
[05:53:07.587] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:56:0)
[05:53:07.601] Build status: STARTED
[05:53:07.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:57:0) after 1m9s
[05:54:16.670] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:57:0)
[05:54:16.688] Build status: STARTED
[05:54:16.689] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:58:0) after 5m5s
[05:59:21.709] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:58:0)
[05:59:21.791] Build status: STARTED
[05:59:21.791] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:59:0) after 3m28s
[06:02:49.852] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:59:0)
[06:02:49.868] Build status: STARTED
[06:02:49.868] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:60:0) after 2m33s
[06:05:22.894] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:60:0)
[06:05:22.912] Build status: STARTED
[06:05:22.912] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:61:0) after 1m24s
[06:06:46.932] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:61:0)
[06:06:46.947] Build status: STARTED
[06:06:46.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:62:0) after 6m50s
[06:13:36.987] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:62:0)
[06:13:37.001] Build status: STARTED
[06:13:37.001] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:63:0) after 1m8s
[06:14:45.028] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:63:0)
[06:14:45.044] Build status: STARTED
[06:14:45.044] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:64:0) after 8m27s
[06:23:12.219] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:64:0)
[06:23:12.239] Build status: STARTED
[06:23:12.239] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:65:0) after 6m16s
[06:29:28.295] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:65:0)
[06:29:28.308] Build status: STARTED
[06:29:28.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:66:0) after 4m37s
[06:34:05.336] Handling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:66:0)
[06:34:05.402] Build status: STARTED
[06:34:05.402] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOS:8854365683176866880:67:0) after 1m59s
[06:34:37.463] Received PubSub notification, asking Buildbucket for the build status
[06:34:37.474] Build:
{
"id": "8688935621268586401",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOS"
},
"number": 34955,
"createdBy": "project:chromium",
"createTime": "2026-02-25T01:18:03.296496507Z",
"startTime": "2026-02-25T01:18:09.460321829Z",
"endTime": "2026-02-25T06:34:37.259027624Z",
"updateTime": "2026-02-25T06:34:37.259027624Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e3ae124bcdc9e66600f487a9d18ae963ee27a409",
"ref": "refs/heads/main"
}
}
}
[06:34:37.474] Invocation finished in 5h16m35.381152572s with status FAILED