[05:35:16.637] New invocation is queued and will start shortly
[05:35:18.656] Starting the invocation (attempt 1)
[05:35:18.694] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/1a40eab3d2faacd167cc3d78d20f9da98d55b78e"
[05:35:18.694] Popped gitiles commit info from properties and tags
[05:35:18.694] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[05:35:18.694] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[05:35:18.694] Buildbucket request:
{
"requestId": "8848098308206642944",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1a40eab3d2faacd167cc3d78d20f9da98d55b78e",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8848098308206642944"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[05:35:19.588] Scheduled build:
{
"id": "8682668245288624353",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41458,
"createdBy": "project:chromium",
"createTime": "2026-05-05T05:35:18.800273651Z",
"updateTime": "2026-05-05T05:35:18.800273651Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1a40eab3d2faacd167cc3d78d20f9da98d55b78e",
"ref": "refs/heads/main"
}
}
}
[05:35:19.588] Task URL: https://cr-buildbucket.appspot.com/build/8682668245288624353
[05:35:19.588] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:2:0) after 6m23s
[05:35:39.715] Received PubSub notification, asking Buildbucket for the build status
[05:35:39.780] Build status: SCHEDULED
[05:35:54.911] Received PubSub notification, asking Buildbucket for the build status
[05:35:54.932] Build status: STARTED
[05:41:42.715] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:2:0)
[05:41:42.757] Build status: STARTED
[05:41:42.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:5:0) after 6m22s
[05:48:04.863] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:5:0)
[05:48:04.953] Build status: STARTED
[05:48:04.953] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:6:0) after 8m48s
[05:56:53.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:6:0)
[05:56:53.090] Build status: STARTED
[05:56:53.090] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:7:0) after 7m7s
[06:04:00.243] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:7:0)
[06:04:00.260] Build status: STARTED
[06:04:00.260] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:8:0) after 8m7s
[06:12:07.298] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:8:0)
[06:12:07.340] Build status: STARTED
[06:12:07.340] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:9:0) after 3m5s
[06:15:12.357] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:9:0)
[06:15:12.372] Build status: STARTED
[06:15:12.372] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:10:0) after 5m56s
[06:21:08.377] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:10:0)
[06:21:08.395] Build status: STARTED
[06:21:08.395] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:11:0) after 8m44s
[06:29:52.420] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:11:0)
[06:29:52.437] Build status: STARTED
[06:29:52.437] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:12:0) after 4m45s
[06:34:37.522] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:12:0)
[06:34:37.538] Build status: STARTED
[06:34:37.538] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:13:0) after 2m36s
[06:37:13.560] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:13:0)
[06:37:13.583] Build status: STARTED
[06:37:13.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:14:0) after 3m57s
[06:41:10.639] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:14:0)
[06:41:10.653] Build status: STARTED
[06:41:10.653] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:15:0) after 1m32s
[06:42:42.723] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:15:0)
[06:42:42.745] Build status: STARTED
[06:42:42.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:16:0) after 4m10s
[06:46:52.763] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:16:0)
[06:46:52.783] Build status: STARTED
[06:46:52.783] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:17:0) after 6m55s
[06:53:47.828] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:17:0)
[06:53:47.847] Build status: STARTED
[06:53:47.847] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:18:0) after 6m15s
[07:00:02.996] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:18:0)
[07:00:03.017] Build status: STARTED
[07:00:03.017] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:19:0) after 3m8s
[07:03:11.315] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:19:0)
[07:03:11.372] Build status: STARTED
[07:03:11.372] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:20:0) after 4m43s
[07:07:54.419] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:20:0)
[07:07:54.437] Build status: STARTED
[07:07:54.437] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:21:0) after 3m23s
[07:11:17.524] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:21:0)
[07:11:17.568] Build status: STARTED
[07:11:17.568] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:22:0) after 8m19s
[07:19:36.690] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:22:0)
[07:19:36.709] Build status: STARTED
[07:19:36.709] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:23:0) after 6m23s
[07:25:59.774] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:23:0)
[07:25:59.793] Build status: STARTED
[07:25:59.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:24:0) after 3m51s
[07:29:50.824] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:24:0)
[07:29:50.840] Build status: STARTED
[07:29:50.840] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:25:0) after 1m22s
[07:31:12.967] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:25:0)
[07:31:12.994] Build status: STARTED
[07:31:12.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:26:0) after 9m59s
[07:41:12.013] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:26:0)
[07:41:12.032] Build status: STARTED
[07:41:12.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:27:0) after 1m33s
[07:42:45.034] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:27:0)
[07:42:45.117] Build status: STARTED
[07:42:45.117] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:28:0) after 2m5s
[07:44:50.306] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:28:0)
[07:44:50.649] Build status: STARTED
[07:44:50.649] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:29:0) after 5m17s
[07:50:07.675] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:29:0)
[07:50:07.688] Build status: STARTED
[07:50:07.688] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:30:0) after 4m25s
[07:54:32.759] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:30:0)
[07:54:32.783] Build status: STARTED
[07:54:32.783] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:31:0) after 1m50s
[07:56:22.829] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:31:0)
[07:56:22.848] Build status: STARTED
[07:56:22.848] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:32:0) after 1m9s
[07:57:31.920] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:32:0)
[07:57:31.936] Build status: STARTED
[07:57:31.936] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:33:0) after 3m13s
[08:00:45.007] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:33:0)
[08:00:45.022] Build status: STARTED
[08:00:45.022] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:34:0) after 7m23s
[08:08:08.036] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:34:0)
[08:08:08.057] Build status: STARTED
[08:08:08.057] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:35:0) after 2m46s
[08:10:54.199] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:35:0)
[08:10:54.291] Build status: STARTED
[08:10:54.291] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:36:0) after 3m6s
[08:14:00.502] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:36:0)
[08:14:00.527] Build status: STARTED
[08:14:00.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:37:0) after 8m52s
[08:22:52.546] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:37:0)
[08:22:52.558] Build status: STARTED
[08:22:52.558] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:38:0) after 1m4s
[08:23:56.536] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:38:0)
[08:23:56.553] Build status: STARTED
[08:23:56.553] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:39:0) after 1m3s
[08:24:59.680] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:39:0)
[08:24:59.767] Build status: STARTED
[08:24:59.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:40:0) after 7m38s
[08:32:37.799] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:40:0)
[08:32:37.821] Build status: STARTED
[08:32:37.821] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:41:0) after 3m33s
[08:36:11.031] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:41:0)
[08:36:11.049] Build status: STARTED
[08:36:11.049] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:42:0) after 9m6s
[08:45:17.056] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:42:0)
[08:45:17.078] Build status: STARTED
[08:45:17.078] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:43:0) after 1m45s
[08:47:02.095] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:43:0)
[08:47:02.119] Build status: STARTED
[08:47:02.119] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:44:0) after 3m51s
[08:50:53.150] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:44:0)
[08:50:53.216] Build status: STARTED
[08:50:53.216] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:45:0) after 1m45s
[08:52:38.220] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:45:0)
[08:52:38.240] Build status: STARTED
[08:52:38.240] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:46:0) after 2m44s
[08:55:22.300] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:46:0)
[08:55:22.335] Build status: STARTED
[08:55:22.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:47:0) after 3m43s
[08:59:05.343] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:47:0)
[08:59:05.355] Build status: STARTED
[08:59:05.355] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:48:0) after 5m44s
[09:04:49.374] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:48:0)
[09:04:49.411] Build status: STARTED
[09:04:49.411] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:49:0) after 6m6s
[09:10:55.461] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:49:0)
[09:10:55.480] Build status: STARTED
[09:10:55.480] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:50:0) after 8m20s
[09:19:15.498] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:50:0)
[09:19:15.552] Build status: STARTED
[09:19:15.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:51:0) after 7m34s
[09:26:49.579] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:51:0)
[09:26:49.598] Build status: STARTED
[09:26:49.598] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:52:0) after 3m35s
[09:30:24.589] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:52:0)
[09:30:24.614] Build status: STARTED
[09:30:24.614] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:53:0) after 6m38s
[09:37:02.689] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:53:0)
[09:37:02.707] Build status: STARTED
[09:37:02.707] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:54:0) after 3m55s
[09:40:57.724] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:54:0)
[09:40:57.745] Build status: STARTED
[09:40:57.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:55:0) after 7m23s
[09:48:20.941] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:55:0)
[09:48:20.963] Build status: STARTED
[09:48:20.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:56:0) after 7m41s
[09:56:01.980] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:56:0)
[09:56:01.998] Build status: STARTED
[09:56:01.998] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:57:0) after 6m47s
[10:02:49.151] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:57:0)
[10:02:49.171] Build status: STARTED
[10:02:49.172] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848098308206642944:58:0) after 9m0s
[10:07:47.933] Received PubSub notification, asking Buildbucket for the build status
[10:07:47.951] Build:
{
"id": "8682668245288624353",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41458,
"createdBy": "project:chromium",
"createTime": "2026-05-05T05:35:18.800273651Z",
"startTime": "2026-05-05T05:35:41.275646290Z",
"endTime": "2026-05-05T10:07:47.624034981Z",
"updateTime": "2026-05-05T10:07:47.624034981Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1a40eab3d2faacd167cc3d78d20f9da98d55b78e",
"ref": "refs/heads/main"
}
}
}
[10:07:47.951] Invocation finished in 4h32m31.318595694s with status FAILED