[16:06:54.421] New invocation is queued and will start shortly
[16:06:55.549] Starting the invocation (attempt 1)
[16:06:55.574] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/5c301f5fc6b169415d03cfaf63df7b6d3bb43e1e"
[16:06:55.574] Popped gitiles commit info from properties and tags
[16:06:55.574] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[16:06:55.574] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[16:06:55.575] Buildbucket request:
{
"requestId": "8860198563008103232",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5c301f5fc6b169415d03cfaf63df7b6d3bb43e1e",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8860198563008103232"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[16:06:56.068] Scheduled build:
{
"id": "8694768501085053521",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 40305,
"createdBy": "project:chromium",
"createTime": "2025-12-22T16:06:55.625575158Z",
"updateTime": "2025-12-22T16:06:55.625575158Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5c301f5fc6b169415d03cfaf63df7b6d3bb43e1e",
"ref": "refs/heads/main"
}
}
}
[16:06:56.068] Task URL: https://cr-buildbucket.appspot.com/build/8694768501085053521
[16:06:56.068] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:2:0) after 6m48s
[16:06:57.003] Received PubSub notification, asking Buildbucket for the build status
[16:06:57.025] Build status: SCHEDULED
[16:08:10.928] Received PubSub notification, asking Buildbucket for the build status
[16:08:10.962] Build status: STARTED
[16:13:44.083] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:2:0)
[16:13:44.101] Build status: STARTED
[16:13:44.101] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:5:0) after 8m12s
[16:21:56.189] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:5:0)
[16:21:56.205] Build status: STARTED
[16:21:56.205] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:6:0) after 7m43s
[16:29:39.400] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:6:0)
[16:29:39.417] Build status: STARTED
[16:29:39.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:7:0) after 2m11s
[16:31:50.694] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:7:0)
[16:31:50.713] Build status: STARTED
[16:31:50.713] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:8:0) after 3m48s
[16:35:38.769] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:8:0)
[16:35:38.792] Build status: STARTED
[16:35:38.792] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:9:0) after 2m16s
[16:37:54.840] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:9:0)
[16:37:54.864] Build status: STARTED
[16:37:54.864] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:10:0) after 8m35s
[16:46:29.901] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:10:0)
[16:46:29.919] Build status: STARTED
[16:46:29.919] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:11:0) after 2m6s
[16:48:36.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:11:0)
[16:48:36.039] Build status: STARTED
[16:48:36.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:12:0) after 3m0s
[16:51:36.057] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:12:0)
[16:51:36.072] Build status: STARTED
[16:51:36.072] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:13:0) after 5m44s
[16:57:20.102] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:13:0)
[16:57:20.167] Build status: STARTED
[16:57:20.168] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:14:0) after 1m18s
[16:58:38.322] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:14:0)
[16:58:38.337] Build status: STARTED
[16:58:38.337] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:15:0) after 3m12s
[17:01:50.416] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:15:0)
[17:01:50.480] Build status: STARTED
[17:01:50.480] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:16:0) after 3m43s
[17:05:33.520] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:16:0)
[17:05:33.533] Build status: STARTED
[17:05:33.533] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:17:0) after 1m32s
[17:07:05.558] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:17:0)
[17:07:05.571] Build status: STARTED
[17:07:05.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:18:0) after 6m42s
[17:13:47.589] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:18:0)
[17:13:47.618] Build status: STARTED
[17:13:47.618] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:19:0) after 6m25s
[17:20:12.636] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:19:0)
[17:20:12.652] Build status: STARTED
[17:20:12.652] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:20:0) after 1m12s
[17:21:24.677] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:20:0)
[17:21:24.763] Build status: STARTED
[17:21:24.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:21:0) after 8m21s
[17:29:45.839] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:21:0)
[17:29:45.856] Build status: STARTED
[17:29:45.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:22:0) after 5m23s
[17:35:08.870] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:22:0)
[17:35:08.887] Build status: STARTED
[17:35:08.887] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:23:0) after 3m18s
[17:38:26.951] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:23:0)
[17:38:26.969] Build status: STARTED
[17:38:26.969] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:24:0) after 2m12s
[17:40:38.988] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:24:0)
[17:40:39.006] Build status: STARTED
[17:40:39.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:25:0) after 3m29s
[17:44:08.061] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:25:0)
[17:44:08.077] Build status: STARTED
[17:44:08.077] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:26:0) after 4m52s
[17:49:00.144] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:26:0)
[17:49:00.163] Build status: STARTED
[17:49:00.163] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:27:0) after 5m29s
[17:54:29.288] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:27:0)
[17:54:29.301] Build status: STARTED
[17:54:29.301] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:28:0) after 9m22s
[18:03:51.319] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:28:0)
[18:03:51.340] Build status: STARTED
[18:03:51.340] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:29:0) after 6m34s
[18:10:25.507] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:29:0)
[18:10:25.523] Build status: STARTED
[18:10:25.524] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:30:0) after 8m56s
[18:19:21.759] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:30:0)
[18:19:21.787] Build status: STARTED
[18:19:21.787] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:31:0) after 2m45s
[18:22:07.165] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:31:0)
[18:22:07.181] Build status: STARTED
[18:22:07.181] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:32:0) after 6m3s
[18:28:10.289] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:32:0)
[18:28:10.304] Build status: STARTED
[18:28:10.304] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:33:0) after 3m24s
[18:31:34.619] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:33:0)
[18:31:34.635] Build status: STARTED
[18:31:34.636] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:34:0) after 7m0s
[18:38:34.993] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:34:0)
[18:38:35.009] Build status: STARTED
[18:38:35.009] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:35:0) after 5m7s
[18:43:42.027] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:35:0)
[18:43:42.041] Build status: STARTED
[18:43:42.041] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:36:0) after 3m57s
[18:47:39.103] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:36:0)
[18:47:39.131] Build status: STARTED
[18:47:39.131] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:37:0) after 5m2s
[18:52:41.221] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:37:0)
[18:52:41.252] Build status: STARTED
[18:52:41.252] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:38:0) after 9m20s
[19:02:01.267] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:38:0)
[19:02:01.283] Build status: STARTED
[19:02:01.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:39:0) after 4m35s
[19:06:36.348] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:39:0)
[19:06:36.365] Build status: STARTED
[19:06:36.365] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:40:0) after 3m59s
[19:10:35.487] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:40:0)
[19:10:35.830] Build status: STARTED
[19:10:35.830] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:41:0) after 8m43s
[19:19:18.978] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:41:0)
[19:19:18.993] Build status: STARTED
[19:19:18.993] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:42:0) after 8m9s
[19:27:28.049] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:42:0)
[19:27:28.232] Build status: STARTED
[19:27:28.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:43:0) after 1m22s
[19:28:50.264] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:43:0)
[19:28:50.281] Build status: STARTED
[19:28:50.281] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:44:0) after 2m46s
[19:31:36.304] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:44:0)
[19:31:36.343] Build status: STARTED
[19:31:36.343] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:45:0) after 1m10s
[19:32:46.377] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:45:0)
[19:32:46.390] Build status: STARTED
[19:32:46.390] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:46:0) after 2m46s
[19:35:32.407] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:46:0)
[19:35:32.421] Build status: STARTED
[19:35:32.421] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:47:0) after 9m52s
[19:45:24.633] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:47:0)
[19:45:24.652] Build status: STARTED
[19:45:24.652] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:48:0) after 5m19s
[19:50:43.670] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:48:0)
[19:50:43.683] Build status: STARTED
[19:50:43.683] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:49:0) after 1m32s
[19:52:15.733] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:49:0)
[19:52:15.744] Build status: STARTED
[19:52:15.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:50:0) after 3m27s
[19:55:42.761] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:50:0)
[19:55:42.775] Build status: STARTED
[19:55:42.775] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:51:0) after 8m11s
[20:03:53.791] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:51:0)
[20:03:53.806] Build status: STARTED
[20:03:53.806] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:52:0) after 3m10s
[20:07:03.793] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:52:0)
[20:07:03.810] Build status: STARTED
[20:07:03.810] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:53:0) after 2m44s
[20:09:47.828] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:53:0)
[20:09:47.927] Build status: STARTED
[20:09:47.927] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:54:0) after 9m30s
[20:19:17.959] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:54:0)
[20:19:17.976] Build status: STARTED
[20:19:17.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:55:0) after 6m38s
[20:25:55.998] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:55:0)
[20:25:56.013] Build status: STARTED
[20:25:56.013] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:56:0) after 2m11s
[20:28:07.033] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:56:0)
[20:28:07.053] Build status: STARTED
[20:28:07.053] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:57:0) after 2m18s
[20:30:25.071] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:57:0)
[20:30:25.086] Build status: STARTED
[20:30:25.086] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:58:0) after 1m2s
[20:31:27.101] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:58:0)
[20:31:27.115] Build status: STARTED
[20:31:27.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8860198563008103232:59:0) after 9m30s
[20:36:30.769] Received PubSub notification, asking Buildbucket for the build status
[20:36:30.786] Build:
{
"id": "8694768501085053521",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 40305,
"createdBy": "project:chromium",
"createTime": "2025-12-22T16:06:55.625575158Z",
"startTime": "2025-12-22T16:08:10.743111139Z",
"endTime": "2025-12-22T20:36:07.061715200Z",
"updateTime": "2025-12-22T20:36:07.061715200Z",
"status": "INFRA_FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5c301f5fc6b169415d03cfaf63df7b6d3bb43e1e",
"ref": "refs/heads/main"
}
}
}
[20:36:30.786] Invocation finished in 4h29m36.379156874s with status FAILED