[19:32:03.980] New invocation is queued and will start shortly
[19:32:05.147] Starting the invocation (attempt 1)
[19:32:05.200] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/36d14a3fe8abb389182679b1edd9397768460bff"
[19:32:05.200] Popped gitiles commit info from properties and tags
[19:32:05.200] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[19:32:05.200] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[19:32:05.200] Buildbucket request:
{
"requestId": "8848679840763892576",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "36d14a3fe8abb389182679b1edd9397768460bff",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8848679840763892576"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[19:32:05.882] Scheduled build:
{
"id": "8683249778764626449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41423,
"createdBy": "project:chromium",
"createTime": "2026-04-28T19:32:05.259503413Z",
"updateTime": "2026-04-28T19:32:05.259503413Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "36d14a3fe8abb389182679b1edd9397768460bff",
"ref": "refs/heads/main"
}
},
"canary": true
}
[19:32:05.882] Task URL: https://cr-buildbucket.appspot.com/build/8683249778764626449
[19:32:05.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:2:0) after 2m32s
[19:32:06.831] Received PubSub notification, asking Buildbucket for the build status
[19:32:06.944] Build status: SCHEDULED
[19:32:39.111] Received PubSub notification, asking Buildbucket for the build status
[19:32:39.139] Build status: STARTED
[19:34:38.048] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:2:0)
[19:34:38.075] Build status: STARTED
[19:34:38.075] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:5:0) after 5m6s
[19:39:44.179] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:5:0)
[19:39:44.196] Build status: STARTED
[19:39:44.196] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:6:0) after 1m7s
[19:40:51.220] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:6:0)
[19:40:51.243] Build status: STARTED
[19:40:51.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:7:0) after 4m5s
[19:44:56.339] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:7:0)
[19:44:56.410] Build status: STARTED
[19:44:56.410] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:8:0) after 2m28s
[19:47:24.423] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:8:0)
[19:47:24.440] Build status: STARTED
[19:47:24.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:9:0) after 4m24s
[19:51:48.598] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:9:0)
[19:51:48.617] Build status: STARTED
[19:51:48.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:10:0) after 8m17s
[20:00:05.630] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:10:0)
[20:00:05.668] Build status: STARTED
[20:00:05.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:11:0) after 9m11s
[20:09:16.678] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:11:0)
[20:09:16.704] Build status: STARTED
[20:09:16.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:12:0) after 4m32s
[20:13:48.787] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:12:0)
[20:13:48.802] Build status: STARTED
[20:13:48.802] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:13:0) after 7m55s
[20:21:43.812] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:13:0)
[20:21:43.838] Build status: STARTED
[20:21:43.838] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:14:0) after 4m25s
[20:26:08.842] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:14:0)
[20:26:08.907] Build status: STARTED
[20:26:08.908] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:15:0) after 1m30s
[20:27:38.948] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:15:0)
[20:27:38.962] Build status: STARTED
[20:27:38.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:16:0) after 6m3s
[20:33:42.098] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:16:0)
[20:33:42.121] Build status: STARTED
[20:33:42.121] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:17:0) after 3m54s
[20:37:36.105] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:17:0)
[20:37:36.118] Build status: STARTED
[20:37:36.119] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:18:0) after 5m1s
[20:42:37.133] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:18:0)
[20:42:37.149] Build status: STARTED
[20:42:37.149] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:19:0) after 5m2s
[20:47:39.323] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:19:0)
[20:47:39.338] Build status: STARTED
[20:47:39.338] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:20:0) after 8m1s
[20:55:40.363] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:20:0)
[20:55:40.388] Build status: STARTED
[20:55:40.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:21:0) after 9m30s
[21:05:10.455] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:21:0)
[21:05:10.489] Build status: STARTED
[21:05:10.489] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:22:0) after 4m36s
[21:09:46.589] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:22:0)
[21:09:46.612] Build status: STARTED
[21:09:46.612] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:23:0) after 6m8s
[21:15:54.719] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:23:0)
[21:15:54.740] Build status: STARTED
[21:15:54.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:24:0) after 3m23s
[21:19:17.802] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:24:0)
[21:19:17.818] Build status: STARTED
[21:19:17.818] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:25:0) after 5m58s
[21:25:15.832] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:25:0)
[21:25:15.848] Build status: STARTED
[21:25:15.848] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:26:0) after 1m44s
[21:26:59.886] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:26:0)
[21:26:59.954] Build status: STARTED
[21:26:59.954] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:27:0) after 3m43s
[21:30:43.037] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:27:0)
[21:30:43.079] Build status: STARTED
[21:30:43.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:28:0) after 2m41s
[21:33:24.098] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:28:0)
[21:33:24.138] Build status: STARTED
[21:33:24.138] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:29:0) after 2m15s
[21:35:39.211] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:29:0)
[21:35:39.231] Build status: STARTED
[21:35:39.231] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:30:0) after 5m21s
[21:41:00.245] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:30:0)
[21:41:00.269] Build status: STARTED
[21:41:00.269] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:31:0) after 6m4s
[21:47:04.367] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:31:0)
[21:47:04.391] Build status: STARTED
[21:47:04.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:32:0) after 3m23s
[21:50:27.382] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:32:0)
[21:50:27.408] Build status: STARTED
[21:50:27.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:33:0) after 4m2s
[21:54:29.424] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:33:0)
[21:54:29.444] Build status: STARTED
[21:54:29.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:34:0) after 8m54s
[22:03:23.563] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:34:0)
[22:03:23.597] Build status: STARTED
[22:03:23.597] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:35:0) after 2m25s
[22:05:48.663] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:35:0)
[22:05:48.686] Build status: STARTED
[22:05:48.686] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:36:0) after 1m48s
[22:07:36.699] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:36:0)
[22:07:36.717] Build status: STARTED
[22:07:36.718] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:37:0) after 1m50s
[22:09:26.690] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:37:0)
[22:09:26.709] Build status: STARTED
[22:09:26.709] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:38:0) after 9m22s
[22:18:48.728] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:38:0)
[22:18:48.746] Build status: STARTED
[22:18:48.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:39:0) after 9m26s
[22:28:14.790] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:39:0)
[22:28:14.805] Build status: STARTED
[22:28:14.806] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:40:0) after 1m28s
[22:29:42.875] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:40:0)
[22:29:42.892] Build status: STARTED
[22:29:42.892] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:41:0) after 2m28s
[22:32:11.009] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:41:0)
[22:32:11.026] Build status: STARTED
[22:32:11.026] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:42:0) after 8m55s
[22:41:06.113] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:42:0)
[22:41:06.130] Build status: STARTED
[22:41:06.130] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:43:0) after 5m9s
[22:46:15.123] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:43:0)
[22:46:15.151] Build status: STARTED
[22:46:15.152] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:44:0) after 4m29s
[22:50:44.168] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:44:0)
[22:50:44.205] Build status: STARTED
[22:50:44.206] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:45:0) after 1m0s
[22:51:44.275] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:45:0)
[22:51:44.291] Build status: STARTED
[22:51:44.291] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:46:0) after 2m34s
[22:54:18.359] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:46:0)
[22:54:18.373] Build status: STARTED
[22:54:18.373] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:47:0) after 6m40s
[23:00:58.441] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:47:0)
[23:00:58.529] Build status: STARTED
[23:00:58.529] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:48:0) after 6m17s
[23:07:15.696] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:48:0)
[23:07:15.717] Build status: STARTED
[23:07:15.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:49:0) after 1m49s
[23:09:04.736] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:49:0)
[23:09:04.757] Build status: STARTED
[23:09:04.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:50:0) after 2m2s
[23:11:06.787] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:50:0)
[23:11:06.804] Build status: STARTED
[23:11:06.804] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:51:0) after 9m43s
[23:20:49.861] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:51:0)
[23:20:49.896] Build status: STARTED
[23:20:49.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:52:0) after 9m51s
[23:30:41.023] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:52:0)
[23:30:41.104] Build status: STARTED
[23:30:41.104] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:53:0) after 7m25s
[23:38:06.129] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:53:0)
[23:38:06.199] Build status: STARTED
[23:38:06.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:54:0) after 1m50s
[23:39:56.216] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:54:0)
[23:39:56.237] Build status: STARTED
[23:39:56.237] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:55:0) after 1m34s
[23:41:30.341] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:55:0)
[23:41:30.364] Build status: STARTED
[23:41:30.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:56:0) after 2m17s
[23:43:47.423] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:56:0)
[23:43:47.443] Build status: STARTED
[23:43:47.443] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:57:0) after 7m7s
[23:50:54.459] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:57:0)
[23:50:54.493] Build status: STARTED
[23:50:54.493] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:58:0) after 7m47s
[23:58:41.653] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:58:0)
[23:58:41.732] Build status: STARTED
[23:58:41.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:59:0) after 9m54s
[00:08:35.704] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:59:0)
[00:08:35.732] Build status: STARTED
[00:08:35.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:60:0) after 1m58s
[00:10:33.746] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:60:0)
[00:10:33.764] Build status: STARTED
[00:10:33.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848679840763892576:61:0) after 3m59s
[00:13:25.098] Received PubSub notification, asking Buildbucket for the build status
[00:13:25.163] Build:
{
"id": "8683249778764626449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41423,
"createdBy": "project:chromium",
"createTime": "2026-04-28T19:32:05.259503413Z",
"startTime": "2026-04-28T19:32:38.897844791Z",
"endTime": "2026-04-29T00:13:24.805649542Z",
"updateTime": "2026-04-29T00:13:24.805649542Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "36d14a3fe8abb389182679b1edd9397768460bff",
"ref": "refs/heads/main"
}
},
"canary": true
}
[00:13:25.163] Invocation finished in 4h41m21.19390415s with status FAILED