[11:01:16.361] New invocation is queued and will start shortly
[11:01:17.551] Starting the invocation (attempt 1)
[11:01:17.580] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0c5915c78ebe158c46763ab04d35a0cc6e55d1fe"
[11:01:17.580] Popped gitiles commit info from properties and tags
[11:01:17.580] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[11:01:17.580] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[11:01:17.580] Buildbucket request:
{
"requestId": "8848802574079458336",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0c5915c78ebe158c46763ab04d35a0cc6e55d1fe",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8848802574079458336"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[11:01:18.160] Scheduled build:
{
"id": "8683372512095405969",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41410,
"createdBy": "project:chromium",
"createTime": "2026-04-27T11:01:17.634372519Z",
"updateTime": "2026-04-27T11:01:17.634372519Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0c5915c78ebe158c46763ab04d35a0cc6e55d1fe",
"ref": "refs/heads/main"
}
}
}
[11:01:18.160] Task URL: https://cr-buildbucket.appspot.com/build/8683372512095405969
[11:01:18.160] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:2:0) after 7m1s
[11:01:19.079] Received PubSub notification, asking Buildbucket for the build status
[11:01:19.107] Build status: SCHEDULED
[11:01:56.880] Received PubSub notification, asking Buildbucket for the build status
[11:01:56.905] Build status: STARTED
[11:08:19.308] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:2:0)
[11:08:19.328] Build status: STARTED
[11:08:19.328] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:5:0) after 4m26s
[11:12:45.395] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:5:0)
[11:12:45.428] Build status: STARTED
[11:12:45.428] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:6:0) after 1m22s
[11:14:07.447] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:6:0)
[11:14:07.468] Build status: STARTED
[11:14:07.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:7:0) after 6m55s
[11:21:02.462] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:7:0)
[11:21:02.487] Build status: STARTED
[11:21:02.487] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:8:0) after 2m54s
[11:23:56.552] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:8:0)
[11:23:56.570] Build status: STARTED
[11:23:56.570] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:9:0) after 5m13s
[11:29:09.605] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:9:0)
[11:29:09.623] Build status: STARTED
[11:29:09.623] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:10:0) after 4m31s
[11:33:40.671] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:10:0)
[11:33:40.700] Build status: STARTED
[11:33:40.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:11:0) after 1m43s
[11:35:23.799] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:11:0)
[11:35:23.817] Build status: STARTED
[11:35:23.817] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:12:0) after 6m53s
[11:42:16.827] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:12:0)
[11:42:16.873] Build status: STARTED
[11:42:16.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:13:0) after 8m23s
[11:50:39.899] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:13:0)
[11:50:39.923] Build status: STARTED
[11:50:39.923] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:14:0) after 1m1s
[11:51:40.941] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:14:0)
[11:51:41.017] Build status: STARTED
[11:51:41.017] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:15:0) after 7m39s
[11:59:20.115] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:15:0)
[11:59:20.170] Build status: STARTED
[11:59:20.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:16:0) after 5m42s
[12:05:02.185] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:16:0)
[12:05:02.258] Build status: STARTED
[12:05:02.258] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:17:0) after 7m45s
[12:12:47.325] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:17:0)
[12:12:47.348] Build status: STARTED
[12:12:47.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:18:0) after 5m33s
[12:18:20.320] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:18:0)
[12:18:20.337] Build status: STARTED
[12:18:20.337] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:19:0) after 9m14s
[12:27:34.572] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:19:0)
[12:27:34.592] Build status: STARTED
[12:27:34.592] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:20:0) after 5m47s
[12:33:21.719] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:20:0)
[12:33:21.800] Build status: STARTED
[12:33:21.800] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:21:0) after 6m48s
[12:40:09.966] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:21:0)
[12:40:10.098] Build status: STARTED
[12:40:10.098] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:22:0) after 2m29s
[12:42:40.141] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:22:0)
[12:42:40.159] Build status: STARTED
[12:42:40.159] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:23:0) after 6m36s
[12:49:16.175] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:23:0)
[12:49:16.192] Build status: STARTED
[12:49:16.192] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:24:0) after 2m5s
[12:51:21.258] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:24:0)
[12:51:21.282] Build status: STARTED
[12:51:21.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:25:0) after 2m52s
[12:54:13.298] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:25:0)
[12:54:13.316] Build status: STARTED
[12:54:13.316] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:26:0) after 6m49s
[13:01:02.321] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:26:0)
[13:01:02.342] Build status: STARTED
[13:01:02.342] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:27:0) after 3m4s
[13:04:06.387] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:27:0)
[13:04:06.474] Build status: STARTED
[13:04:06.474] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:28:0) after 2m20s
[13:06:26.542] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:28:0)
[13:06:26.560] Build status: STARTED
[13:06:26.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:29:0) after 7m12s
[13:13:38.643] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:29:0)
[13:13:38.657] Build status: STARTED
[13:13:38.657] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:30:0) after 3m51s
[13:17:29.725] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:30:0)
[13:17:29.793] Build status: STARTED
[13:17:29.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:31:0) after 5m14s
[13:22:43.807] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:31:0)
[13:22:43.826] Build status: STARTED
[13:22:43.826] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:32:0) after 2m24s
[13:25:07.846] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:32:0)
[13:25:07.884] Build status: STARTED
[13:25:07.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:33:0) after 1m10s
[13:26:17.903] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:33:0)
[13:26:17.922] Build status: STARTED
[13:26:17.922] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:34:0) after 5m35s
[13:31:52.943] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:34:0)
[13:31:52.972] Build status: STARTED
[13:31:52.972] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:35:0) after 9m58s
[13:41:50.993] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:35:0)
[13:41:51.017] Build status: STARTED
[13:41:51.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:36:0) after 1m57s
[13:43:48.032] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:36:0)
[13:43:48.081] Build status: STARTED
[13:43:48.082] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:37:0) after 1m27s
[13:45:15.243] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:37:0)
[13:45:15.264] Build status: STARTED
[13:45:15.264] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:38:0) after 4m16s
[13:49:31.382] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:38:0)
[13:49:31.398] Build status: STARTED
[13:49:31.398] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:39:0) after 5m9s
[13:54:40.414] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:39:0)
[13:54:40.438] Build status: STARTED
[13:54:40.438] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:40:0) after 6m9s
[14:00:49.507] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:40:0)
[14:00:49.671] Build status: STARTED
[14:00:49.671] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:41:0) after 9m58s
[14:10:47.688] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:41:0)
[14:10:47.719] Build status: STARTED
[14:10:47.719] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:42:0) after 7m17s
[14:18:04.726] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:42:0)
[14:18:04.752] Build status: STARTED
[14:18:04.752] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:43:0) after 8m14s
[14:26:18.778] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:43:0)
[14:26:18.800] Build status: STARTED
[14:26:18.801] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:44:0) after 2m17s
[14:28:35.912] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:44:0)
[14:28:35.931] Build status: STARTED
[14:28:35.931] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:45:0) after 7m20s
[14:35:55.935] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:45:0)
[14:35:55.950] Build status: STARTED
[14:35:55.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:46:0) after 4m2s
[14:39:57.985] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:46:0)
[14:39:57.999] Build status: STARTED
[14:39:57.999] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:47:0) after 7m25s
[14:47:23.020] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:47:0)
[14:47:23.036] Build status: STARTED
[14:47:23.036] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:48:0) after 1m44s
[14:49:07.053] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:48:0)
[14:49:07.074] Build status: STARTED
[14:49:07.074] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:49:0) after 8m35s
[14:57:42.077] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:49:0)
[14:57:42.096] Build status: STARTED
[14:57:42.096] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:50:0) after 5m31s
[15:03:13.116] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:50:0)
[15:03:13.137] Build status: STARTED
[15:03:13.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:51:0) after 5m29s
[15:08:42.235] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:51:0)
[15:08:42.255] Build status: STARTED
[15:08:42.256] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:52:0) after 2m9s
[15:10:51.273] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:52:0)
[15:10:51.292] Build status: STARTED
[15:10:51.292] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:53:0) after 1m40s
[15:12:31.302] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:53:0)
[15:12:31.323] Build status: STARTED
[15:12:31.323] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:54:0) after 6m19s
[15:18:50.349] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:54:0)
[15:18:50.376] Build status: STARTED
[15:18:50.376] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:55:0) after 3m52s
[15:22:42.410] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:55:0)
[15:22:42.432] Build status: STARTED
[15:22:42.432] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:56:0) after 3m23s
[15:26:05.506] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:56:0)
[15:26:05.609] Build status: STARTED
[15:26:05.609] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:57:0) after 9m4s
[15:35:09.626] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:57:0)
[15:35:09.643] Build status: STARTED
[15:35:09.643] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848802574079458336:58:0) after 8m11s
[15:38:19.230] Received PubSub notification, asking Buildbucket for the build status
[15:38:19.252] Build:
{
"id": "8683372512095405969",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41410,
"createdBy": "project:chromium",
"createTime": "2026-04-27T11:01:17.634372519Z",
"startTime": "2026-04-27T11:01:55.119314088Z",
"endTime": "2026-04-27T15:38:18.888540854Z",
"updateTime": "2026-04-27T15:38:18.888540854Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0c5915c78ebe158c46763ab04d35a0cc6e55d1fe",
"ref": "refs/heads/main"
}
}
}
[15:38:19.252] Invocation finished in 4h37m2.894345824s with status FAILED