[05:47:06.685] New invocation is queued and will start shortly
[05:47:07.885] Starting the invocation (attempt 1)
[05:47:07.910] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/14f422b2bb11c8856a2a0cf9331786fe6ad487bf"
[05:47:07.910] Popped gitiles commit info from properties and tags
[05:47:07.910] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[05:47:07.910] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[05:47:07.910] Buildbucket request:
{
"requestId": "8837044733765578608",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "14f422b2bb11c8856a2a0cf9331786fe6ad487bf",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8837044733765578608"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[05:47:08.418] Scheduled build:
{
"id": "8671614671750152385",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 43113,
"createdBy": "project:chromium",
"createTime": "2026-09-04T05:47:07.988042896Z",
"updateTime": "2026-09-04T05:47:07.988042896Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "14f422b2bb11c8856a2a0cf9331786fe6ad487bf",
"ref": "refs/heads/main"
}
}
}
[05:47:08.419] Task URL: https://cr-buildbucket.appspot.com/build/8671614671750152385
[05:47:08.419] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:2:0) after 4m16s
[05:47:30.626] Received PubSub notification, asking Buildbucket for the build status
[05:47:30.647] Build status: STARTED
[05:47:45.595] Received PubSub notification, asking Buildbucket for the build status
[05:47:45.991] Build status: STARTED
[05:51:24.514] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:2:0)
[05:51:24.529] Build status: STARTED
[05:51:24.529] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:5:0) after 6m36s
[05:58:00.597] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:5:0)
[05:58:00.620] Build status: STARTED
[05:58:00.620] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:6:0) after 4m5s
[06:02:05.639] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:6:0)
[06:02:05.662] Build status: STARTED
[06:02:05.662] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:7:0) after 1m0s
[06:03:05.720] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:7:0)
[06:03:05.750] Build status: STARTED
[06:03:05.750] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:8:0) after 2m0s
[06:05:05.772] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:8:0)
[06:05:05.792] Build status: STARTED
[06:05:05.792] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:9:0) after 8m51s
[06:13:56.914] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:9:0)
[06:13:56.980] Build status: STARTED
[06:13:56.980] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:10:0) after 4m11s
[06:18:07.996] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:10:0)
[06:18:08.021] Build status: STARTED
[06:18:08.021] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:11:0) after 3m19s
[06:21:27.034] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:11:0)
[06:21:27.093] Build status: STARTED
[06:21:27.093] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:12:0) after 7m4s
[06:28:31.235] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:12:0)
[06:28:31.260] Build status: STARTED
[06:28:31.260] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:13:0) after 4m55s
[06:33:26.329] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:13:0)
[06:33:26.369] Build status: STARTED
[06:33:26.369] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:14:0) after 5m10s
[06:38:36.439] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:14:0)
[06:38:36.487] Build status: STARTED
[06:38:36.487] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:15:0) after 6m45s
[06:45:21.623] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:15:0)
[06:45:21.657] Build status: STARTED
[06:45:21.658] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:16:0) after 4m42s
[06:50:03.680] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:16:0)
[06:50:03.754] Build status: STARTED
[06:50:03.755] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:17:0) after 2m53s
[06:52:56.774] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:17:0)
[06:52:56.826] Build status: STARTED
[06:52:56.826] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:18:0) after 3m39s
[06:56:35.848] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:18:0)
[06:56:35.880] Build status: STARTED
[06:56:35.880] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:19:0) after 1m7s
[06:57:42.951] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:19:0)
[06:57:42.967] Build status: STARTED
[06:57:42.967] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:20:0) after 5m2s
[07:02:45.067] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:20:0)
[07:02:45.105] Build status: STARTED
[07:02:45.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:21:0) after 1m31s
[07:04:16.125] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:21:0)
[07:04:16.176] Build status: STARTED
[07:04:16.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:22:0) after 5m17s
[07:09:33.188] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:22:0)
[07:09:33.269] Build status: STARTED
[07:09:33.269] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:23:0) after 6m16s
[07:15:49.330] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:23:0)
[07:15:49.354] Build status: STARTED
[07:15:49.354] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:24:0) after 5m52s
[07:21:41.475] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:24:0)
[07:21:41.587] Build status: STARTED
[07:21:41.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:25:0) after 1m43s
[07:23:24.668] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:25:0)
[07:23:24.690] Build status: STARTED
[07:23:24.690] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:26:0) after 3m33s
[07:26:57.736] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:26:0)
[07:26:57.765] Build status: STARTED
[07:26:57.765] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:27:0) after 1m55s
[07:28:52.836] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:27:0)
[07:28:52.855] Build status: STARTED
[07:28:52.855] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:28:0) after 2m28s
[07:31:20.878] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:28:0)
[07:31:20.924] Build status: STARTED
[07:31:20.924] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:29:0) after 7m3s
[07:38:23.978] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:29:0)
[07:38:24.014] Build status: STARTED
[07:38:24.014] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:30:0) after 4m22s
[07:42:46.117] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:30:0)
[07:42:46.138] Build status: STARTED
[07:42:46.138] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:31:0) after 2m57s
[07:45:43.165] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:31:0)
[07:45:43.193] Build status: STARTED
[07:45:43.193] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:32:0) after 8m56s
[07:54:39.216] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:32:0)
[07:54:39.247] Build status: STARTED
[07:54:39.247] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:33:0) after 1m9s
[07:55:48.253] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:33:0)
[07:55:48.293] Build status: STARTED
[07:55:48.293] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:34:0) after 1m45s
[07:57:33.355] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:34:0)
[07:57:33.374] Build status: STARTED
[07:57:33.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:35:0) after 4m8s
[08:01:41.393] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:35:0)
[08:01:41.430] Build status: STARTED
[08:01:41.430] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:36:0) after 6m39s
[08:08:20.463] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:36:0)
[08:08:20.514] Build status: STARTED
[08:08:20.514] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:37:0) after 1m51s
[08:10:11.544] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:37:0)
[08:10:11.576] Build status: STARTED
[08:10:11.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:38:0) after 3m55s
[08:14:06.601] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:38:0)
[08:14:06.618] Build status: STARTED
[08:14:06.618] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:39:0) after 8m29s
[08:22:35.641] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:39:0)
[08:22:35.660] Build status: STARTED
[08:22:35.660] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:40:0) after 2m56s
[08:25:31.683] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:40:0)
[08:25:31.708] Build status: STARTED
[08:25:31.708] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:41:0) after 9m59s
[08:35:30.733] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:41:0)
[08:35:30.757] Build status: STARTED
[08:35:30.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:42:0) after 3m28s
[08:38:58.877] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:42:0)
[08:38:58.900] Build status: STARTED
[08:38:58.900] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:43:0) after 6m48s
[08:45:46.981] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:43:0)
[08:45:47.011] Build status: STARTED
[08:45:47.011] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:44:0) after 7m8s
[08:52:55.151] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:44:0)
[08:52:55.171] Build status: STARTED
[08:52:55.171] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:45:0) after 8m34s
[09:01:29.191] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:45:0)
[09:01:29.219] Build status: STARTED
[09:01:29.219] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:46:0) after 9m17s
[09:10:46.289] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:46:0)
[09:10:46.305] Build status: STARTED
[09:10:46.305] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:47:0) after 8m8s
[09:18:54.460] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:47:0)
[09:18:54.522] Build status: STARTED
[09:18:54.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:48:0) after 1m12s
[09:20:06.515] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:48:0)
[09:20:06.537] Build status: STARTED
[09:20:06.537] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:49:0) after 1m54s
[09:22:00.607] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:49:0)
[09:22:00.656] Build status: STARTED
[09:22:00.656] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:50:0) after 9m52s
[09:31:52.721] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:50:0)
[09:31:52.772] Build status: STARTED
[09:31:52.772] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:51:0) after 3m50s
[09:35:42.746] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:51:0)
[09:35:42.767] Build status: STARTED
[09:35:42.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:52:0) after 6m42s
[09:42:24.878] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:52:0)
[09:42:24.920] Build status: STARTED
[09:42:24.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:53:0) after 4m14s
[09:46:39.060] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:53:0)
[09:46:39.143] Build status: STARTED
[09:46:39.143] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:54:0) after 8m36s
[09:55:15.165] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:54:0)
[09:55:15.184] Build status: STARTED
[09:55:15.184] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:55:0) after 1m29s
[09:56:44.253] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:55:0)
[09:56:44.303] Build status: STARTED
[09:56:44.303] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:56:0) after 9m48s
[10:06:32.322] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:56:0)
[10:06:32.342] Build status: STARTED
[10:06:32.342] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:57:0) after 5m53s
[10:12:25.450] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:57:0)
[10:12:25.481] Build status: STARTED
[10:12:25.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:58:0) after 3m23s
[10:15:48.505] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:58:0)
[10:15:48.524] Build status: STARTED
[10:15:48.524] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:59:0) after 8m11s
[10:23:59.543] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:59:0)
[10:23:59.562] Build status: STARTED
[10:23:59.562] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:60:0) after 7m48s
[10:31:47.637] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:60:0)
[10:31:47.717] Build status: STARTED
[10:31:47.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:61:0) after 5m9s
[10:36:56.804] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:61:0)
[10:36:56.856] Build status: STARTED
[10:36:56.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:62:0) after 4m22s
[10:41:18.838] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:62:0)
[10:41:18.859] Build status: STARTED
[10:41:18.859] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:63:0) after 1m9s
[10:42:27.938] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:63:0)
[10:42:27.970] Build status: STARTED
[10:42:27.970] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:64:0) after 2m1s
[10:44:29.040] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:64:0)
[10:44:29.070] Build status: STARTED
[10:44:29.070] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:65:0) after 7m12s
[10:51:41.144] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:65:0)
[10:51:41.169] Build status: STARTED
[10:51:41.169] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:66:0) after 2m38s
[10:54:19.220] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:66:0)
[10:54:19.243] Build status: STARTED
[10:54:19.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:67:0) after 1m43s
[10:56:02.382] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:67:0)
[10:56:02.404] Build status: STARTED
[10:56:02.404] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8837044733765578608:68:0) after 8m3s
[10:56:44.323] Received PubSub notification, asking Buildbucket for the build status
[10:56:44.352] Build:
{
"id": "8671614671750152385",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 43113,
"createdBy": "project:chromium",
"createTime": "2026-09-04T05:47:07.988042896Z",
"startTime": "2026-09-04T05:47:24.734177185Z",
"endTime": "2026-09-04T10:56:43.744682797Z",
"updateTime": "2026-09-04T10:56:43.744682797Z",
"status": "INFRA_FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "14f422b2bb11c8856a2a0cf9331786fe6ad487bf",
"ref": "refs/heads/main"
}
}
}
[10:56:44.352] Invocation finished in 5h9m37.670092718s with status FAILED