[02:13:56.851] New invocation is queued and will start shortly
[02:13:58.264] Starting the invocation (attempt 1)
[02:13:58.294] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/c099bd180a2db0fa6a313d43653529ba02665c84"
[02:13:58.294] Popped gitiles commit info from properties and tags
[02:13:58.294] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[02:13:58.295] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[02:13:58.295] Buildbucket request:
{
"requestId": "8836605160048332704",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c099bd180a2db0fa6a313d43653529ba02665c84",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8836605160048332704"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[02:13:59.229] Scheduled build:
{
"id": "8671175097729497121",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 43136,
"createdBy": "project:chromium",
"createTime": "2026-09-09T02:13:58.441284316Z",
"updateTime": "2026-09-09T02:13:58.441284316Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c099bd180a2db0fa6a313d43653529ba02665c84",
"ref": "refs/heads/main"
}
}
}
[02:13:59.229] Task URL: https://cr-buildbucket.appspot.com/build/8671175097729497121
[02:13:59.229] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:2:0) after 2m18s
[02:14:00.261] Received PubSub notification, asking Buildbucket for the build status
[02:14:00.283] Build status: SCHEDULED
[02:14:05.114] Received PubSub notification, asking Buildbucket for the build status
[02:14:05.192] Build status: STARTED
[02:16:17.238] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:2:0)
[02:16:17.259] Build status: STARTED
[02:16:17.259] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:5:0) after 7m1s
[02:23:18.384] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:5:0)
[02:23:18.471] Build status: STARTED
[02:23:18.471] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:6:0) after 3m45s
[02:27:03.510] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:6:0)
[02:27:03.526] Build status: STARTED
[02:27:03.526] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:7:0) after 3m59s
[02:31:02.552] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:7:0)
[02:31:02.586] Build status: STARTED
[02:31:02.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:8:0) after 1m34s
[02:32:36.555] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:8:0)
[02:32:36.587] Build status: STARTED
[02:32:36.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:9:0) after 4m59s
[02:37:35.568] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:9:0)
[02:37:35.585] Build status: STARTED
[02:37:35.585] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:10:0) after 4m20s
[02:41:55.701] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:10:0)
[02:41:55.805] Build status: STARTED
[02:41:55.805] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:11:0) after 2m47s
[02:44:42.853] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:11:0)
[02:44:42.878] Build status: STARTED
[02:44:42.878] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:12:0) after 4m17s
[02:48:59.950] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:12:0)
[02:48:59.969] Build status: STARTED
[02:48:59.969] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:13:0) after 4m3s
[02:53:03.208] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:13:0)
[02:53:03.225] Build status: STARTED
[02:53:03.225] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:14:0) after 9m42s
[03:02:45.244] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:14:0)
[03:02:45.267] Build status: STARTED
[03:02:45.267] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:15:0) after 7m17s
[03:10:02.310] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:15:0)
[03:10:02.329] Build status: STARTED
[03:10:02.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:16:0) after 5m19s
[03:15:21.414] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:16:0)
[03:15:21.440] Build status: STARTED
[03:15:21.440] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:17:0) after 4m51s
[03:20:12.503] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:17:0)
[03:20:12.526] Build status: STARTED
[03:20:12.526] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:18:0) after 2m39s
[03:22:51.624] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:18:0)
[03:22:51.657] Build status: STARTED
[03:22:51.657] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:19:0) after 7m19s
[03:30:10.771] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:19:0)
[03:30:10.847] Build status: STARTED
[03:30:10.847] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:20:0) after 6m30s
[03:36:41.023] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:20:0)
[03:36:41.045] Build status: STARTED
[03:36:41.045] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:21:0) after 7m41s
[03:44:22.065] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:21:0)
[03:44:22.086] Build status: STARTED
[03:44:22.086] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:22:0) after 2m7s
[03:46:29.146] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:22:0)
[03:46:29.169] Build status: STARTED
[03:46:29.169] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:23:0) after 8m33s
[03:55:02.270] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:23:0)
[03:55:02.306] Build status: STARTED
[03:55:02.306] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:24:0) after 4m41s
[03:59:43.393] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:24:0)
[03:59:43.414] Build status: STARTED
[03:59:43.414] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:25:0) after 1m4s
[04:00:47.473] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:25:0)
[04:00:47.552] Build status: STARTED
[04:00:47.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:26:0) after 5m40s
[04:06:27.913] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:26:0)
[04:06:27.946] Build status: STARTED
[04:06:27.946] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:27:0) after 9m4s
[04:15:32.001] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:27:0)
[04:15:32.036] Build status: STARTED
[04:15:32.036] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:28:0) after 1m35s
[04:17:07.111] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:28:0)
[04:17:07.189] Build status: STARTED
[04:17:07.189] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:29:0) after 8m37s
[04:25:44.207] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:29:0)
[04:25:44.223] Build status: STARTED
[04:25:44.223] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:30:0) after 5m30s
[04:31:14.360] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:30:0)
[04:31:14.376] Build status: STARTED
[04:31:14.377] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:31:0) after 5m47s
[04:37:01.466] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:31:0)
[04:37:01.539] Build status: STARTED
[04:37:01.539] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:32:0) after 6m49s
[04:43:50.559] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:32:0)
[04:43:50.629] Build status: STARTED
[04:43:50.629] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:33:0) after 5m29s
[04:49:19.653] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:33:0)
[04:49:19.674] Build status: STARTED
[04:49:19.674] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:34:0) after 6m35s
[04:55:54.705] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:34:0)
[04:55:54.805] Build status: STARTED
[04:55:54.805] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:35:0) after 7m53s
[05:03:47.856] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:35:0)
[05:03:47.878] Build status: STARTED
[05:03:47.878] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:36:0) after 3m52s
[05:07:40.040] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:36:0)
[05:07:40.070] Build status: STARTED
[05:07:40.070] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:37:0) after 5m24s
[05:13:04.093] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:37:0)
[05:13:04.111] Build status: STARTED
[05:13:04.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:38:0) after 1m16s
[05:14:20.138] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:38:0)
[05:14:20.188] Build status: STARTED
[05:14:20.188] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:39:0) after 5m16s
[05:19:36.266] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:39:0)
[05:19:36.287] Build status: STARTED
[05:19:36.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:40:0) after 6m19s
[05:25:55.398] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:40:0)
[05:25:55.422] Build status: STARTED
[05:25:55.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:41:0) after 6m44s
[05:32:39.642] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:41:0)
[05:32:39.664] Build status: STARTED
[05:32:39.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:42:0) after 1m18s
[05:33:57.717] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:42:0)
[05:33:57.738] Build status: STARTED
[05:33:57.738] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:43:0) after 4m12s
[05:38:09.761] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:43:0)
[05:38:09.781] Build status: STARTED
[05:38:09.781] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:44:0) after 5m30s
[05:43:39.935] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:44:0)
[05:43:39.951] Build status: STARTED
[05:43:39.951] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:45:0) after 9m18s
[05:52:58.063] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:45:0)
[05:52:58.083] Build status: STARTED
[05:52:58.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:46:0) after 3m6s
[05:56:04.277] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:46:0)
[05:56:04.293] Build status: STARTED
[05:56:04.293] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:47:0) after 9m28s
[06:05:32.274] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:47:0)
[06:05:32.292] Build status: STARTED
[06:05:32.292] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:48:0) after 9m47s
[06:15:19.357] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:48:0)
[06:15:19.377] Build status: STARTED
[06:15:19.377] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:49:0) after 2m6s
[06:17:25.401] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:49:0)
[06:17:25.453] Build status: STARTED
[06:17:25.454] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:50:0) after 4m53s
[06:22:18.476] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:50:0)
[06:22:18.502] Build status: STARTED
[06:22:18.502] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:51:0) after 6m39s
[06:28:57.500] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:51:0)
[06:28:57.518] Build status: STARTED
[06:28:57.518] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:52:0) after 4m24s
[06:33:21.544] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:52:0)
[06:33:21.631] Build status: STARTED
[06:33:21.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:53:0) after 4m17s
[06:37:38.660] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:53:0)
[06:37:38.682] Build status: STARTED
[06:37:38.682] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:54:0) after 2m35s
[06:40:13.719] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:54:0)
[06:40:13.737] Build status: STARTED
[06:40:13.737] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:55:0) after 8m25s
[06:48:38.789] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:55:0)
[06:48:38.865] Build status: STARTED
[06:48:38.865] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:56:0) after 3m54s
[06:52:32.886] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:56:0)
[06:52:32.906] Build status: STARTED
[06:52:32.906] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:57:0) after 6m50s
[06:59:23.030] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:57:0)
[06:59:23.051] Build status: STARTED
[06:59:23.051] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:58:0) after 7m13s
[07:06:36.075] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:58:0)
[07:06:36.096] Build status: STARTED
[07:06:36.096] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:59:0) after 1m7s
[07:07:43.119] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:59:0)
[07:07:43.142] Build status: STARTED
[07:07:43.142] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8836605160048332704:60:0) after 7m11s
[07:08:13.381] Received PubSub notification, asking Buildbucket for the build status
[07:08:13.396] Build:
{
"id": "8671175097729497121",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 43136,
"createdBy": "project:chromium",
"createTime": "2026-09-09T02:13:58.441284316Z",
"startTime": "2026-09-09T02:14:04.786880536Z",
"endTime": "2026-09-09T07:08:13.033923767Z",
"updateTime": "2026-09-09T07:08:13.033923767Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c099bd180a2db0fa6a313d43653529ba02665c84",
"ref": "refs/heads/main"
}
}
}
[07:08:13.397] Invocation finished in 4h54m16.550775794s with status FAILED