[15:40:23.792] New invocation is queued and will start shortly
[15:40:25.066] Starting the invocation (attempt 1)
[15:40:25.096] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/9d68b08ffafd1cd80da00ace8f6e8e68ccb31207"
[15:40:25.096] Popped gitiles commit info from properties and tags
[15:40:25.096] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:40:25.096] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:40:25.096] Buildbucket request:
{
"requestId": "8952337345798431984",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9d68b08ffafd1cd80da00ace8f6e8e68ccb31207",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8952337345798431984"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:40:25.583] Scheduled build:
{
"id": "8786907283636396641",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29060,
"createdBy": "project:chromium",
"createTime": "2023-03-11T15:40:25.160169638Z",
"updateTime": "2023-03-11T15:40:25.160169638Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9d68b08ffafd1cd80da00ace8f6e8e68ccb31207",
"ref": "refs/heads/main"
}
}
}
[15:40:25.583] Task URL: https://cr-buildbucket.appspot.com/build/8786907283636396641
[15:40:25.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:2:0) after 2m33s
[15:42:58.614] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:2:0)
[15:42:58.637] Build status: SCHEDULED
[15:42:58.637] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:3:0) after 6m4s
[15:42:59.863] Received PubSub notification, asking Buildbucket for the build status
[15:42:59.894] Build status: STARTED
[15:49:02.930] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:3:0)
[15:49:02.960] Build status: STARTED
[15:49:02.960] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:5:0) after 4m0s
[15:53:02.961] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:5:0)
[15:53:02.981] Build status: STARTED
[15:53:02.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:6:0) after 8m58s
[16:02:01.022] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:6:0)
[16:02:01.045] Build status: STARTED
[16:02:01.045] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:7:0) after 5m24s
[16:07:25.066] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:7:0)
[16:07:25.088] Build status: STARTED
[16:07:25.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:8:0) after 5m52s
[16:13:17.368] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:8:0)
[16:13:17.572] Build status: STARTED
[16:13:17.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:9:0) after 1m1s
[16:14:18.591] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:9:0)
[16:14:18.621] Build status: STARTED
[16:14:18.621] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:10:0) after 3m12s
[16:17:30.636] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:10:0)
[16:17:30.660] Build status: STARTED
[16:17:30.661] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:11:0) after 2m19s
[16:19:49.683] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:11:0)
[16:19:49.717] Build status: STARTED
[16:19:49.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:12:0) after 5m8s
[16:24:57.734] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:12:0)
[16:24:57.807] Build status: STARTED
[16:24:57.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:13:0) after 7m29s
[16:32:26.824] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:13:0)
[16:32:26.851] Build status: STARTED
[16:32:26.851] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:14:0) after 7m30s
[16:39:56.870] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:14:0)
[16:39:56.893] Build status: STARTED
[16:39:56.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:15:0) after 8m43s
[16:48:40.128] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:15:0)
[16:48:40.151] Build status: STARTED
[16:48:40.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:16:0) after 2m21s
[16:51:01.545] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:16:0)
[16:51:01.825] Build status: STARTED
[16:51:01.825] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:17:0) after 1m14s
[16:52:15.862] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:17:0)
[16:52:15.888] Build status: STARTED
[16:52:15.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:18:0) after 8m51s
[17:01:07.131] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:18:0)
[17:01:07.161] Build status: STARTED
[17:01:07.161] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:19:0) after 3m42s
[17:04:49.225] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:19:0)
[17:04:49.247] Build status: STARTED
[17:04:49.247] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:20:0) after 3m55s
[17:08:44.400] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:20:0)
[17:08:44.422] Build status: STARTED
[17:08:44.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:21:0) after 2m49s
[17:11:33.627] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:21:0)
[17:11:33.651] Build status: STARTED
[17:11:33.651] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:22:0) after 8m45s
[17:20:18.752] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:22:0)
[17:20:18.779] Build status: STARTED
[17:20:18.779] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:23:0) after 7m54s
[17:28:12.840] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:23:0)
[17:28:12.858] Build status: STARTED
[17:28:12.859] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:24:0) after 7m33s
[17:35:45.926] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:24:0)
[17:35:45.944] Build status: STARTED
[17:35:45.944] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:25:0) after 5m53s
[17:41:39.022] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:25:0)
[17:41:39.044] Build status: STARTED
[17:41:39.044] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:26:0) after 8m54s
[17:50:33.066] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:26:0)
[17:50:33.088] Build status: STARTED
[17:50:33.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:27:0) after 1m39s
[17:52:12.212] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:27:0)
[17:52:12.232] Build status: STARTED
[17:52:12.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:28:0) after 5m27s
[17:57:39.254] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:28:0)
[17:57:39.277] Build status: STARTED
[17:57:39.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:29:0) after 2m44s
[18:00:23.300] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:29:0)
[18:00:23.329] Build status: STARTED
[18:00:23.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:30:0) after 1m18s
[18:01:41.346] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:30:0)
[18:01:41.387] Build status: STARTED
[18:01:41.387] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:31:0) after 7m31s
[18:09:12.491] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:31:0)
[18:09:12.544] Build status: STARTED
[18:09:12.544] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:32:0) after 2m21s
[18:11:33.566] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:32:0)
[18:11:33.650] Build status: STARTED
[18:11:33.650] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:33:0) after 4m21s
[18:15:54.675] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:33:0)
[18:15:54.708] Build status: STARTED
[18:15:54.708] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:34:0) after 1m48s
[18:17:42.731] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:34:0)
[18:17:42.750] Build status: STARTED
[18:17:42.750] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:35:0) after 3m4s
[18:20:48.603] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:35:0)
[18:20:48.619] Build status: STARTED
[18:20:48.619] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:36:0) after 7m11s
[18:27:59.808] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:36:0)
[18:27:59.834] Build status: STARTED
[18:27:59.834] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:37:0) after 9m52s
[18:37:51.853] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:37:0)
[18:37:51.888] Build status: STARTED
[18:37:51.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:38:0) after 5m57s
[18:43:48.912] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:38:0)
[18:43:48.936] Build status: STARTED
[18:43:48.936] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:39:0) after 4m46s
[18:48:34.953] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:39:0)
[18:48:34.978] Build status: STARTED
[18:48:34.978] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:40:0) after 6m40s
[18:55:15.000] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:40:0)
[18:55:15.038] Build status: STARTED
[18:55:15.038] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:41:0) after 6m43s
[19:01:58.087] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:41:0)
[19:01:58.104] Build status: STARTED
[19:01:58.104] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:42:0) after 3m51s
[19:05:49.126] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:42:0)
[19:05:49.150] Build status: STARTED
[19:05:49.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:43:0) after 2m28s
[19:08:17.194] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:43:0)
[19:08:17.213] Build status: STARTED
[19:08:17.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:44:0) after 6m7s
[19:14:24.381] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:44:0)
[19:14:24.406] Build status: STARTED
[19:14:24.406] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:45:0) after 3m41s
[19:18:05.512] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:45:0)
[19:18:05.556] Build status: STARTED
[19:18:05.556] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:46:0) after 1m5s
[19:19:10.585] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:46:0)
[19:19:10.609] Build status: STARTED
[19:19:10.609] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:47:0) after 7m11s
[19:26:21.635] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:47:0)
[19:26:21.657] Build status: STARTED
[19:26:21.657] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:48:0) after 7m8s
[19:33:29.779] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:48:0)
[19:33:29.799] Build status: STARTED
[19:33:29.799] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:49:0) after 9m31s
[19:43:00.822] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:49:0)
[19:43:00.843] Build status: STARTED
[19:43:00.843] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:50:0) after 3m56s
[19:46:56.884] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:50:0)
[19:46:56.946] Build status: STARTED
[19:46:56.946] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:51:0) after 9m7s
[19:56:04.040] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:51:0)
[19:56:04.066] Build status: STARTED
[19:56:04.066] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:52:0) after 3m33s
[19:59:37.221] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:52:0)
[19:59:37.251] Build status: STARTED
[19:59:37.251] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:53:0) after 8m35s
[20:08:12.309] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:53:0)
[20:08:12.331] Build status: STARTED
[20:08:12.331] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:54:0) after 2m10s
[20:10:22.352] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:54:0)
[20:10:22.382] Build status: STARTED
[20:10:22.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:55:0) after 7m35s
[20:17:57.446] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:55:0)
[20:17:57.467] Build status: STARTED
[20:17:57.467] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:56:0) after 5m23s
[20:23:20.494] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:56:0)
[20:23:20.513] Build status: STARTED
[20:23:20.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:57:0) after 1m44s
[20:25:04.651] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:57:0)
[20:25:04.673] Build status: STARTED
[20:25:04.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:58:0) after 1m41s
[20:26:45.705] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:58:0)
[20:26:45.785] Build status: STARTED
[20:26:45.785] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:59:0) after 6m14s
[20:32:59.769] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:59:0)
[20:32:59.836] Build status: STARTED
[20:32:59.836] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:60:0) after 6m55s
[20:39:54.857] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:60:0)
[20:39:54.883] Build status: STARTED
[20:39:54.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:61:0) after 2m10s
[20:42:05.035] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:61:0)
[20:42:05.057] Build status: STARTED
[20:42:05.057] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:62:0) after 8m19s
[20:50:24.125] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:62:0)
[20:50:24.150] Build status: STARTED
[20:50:24.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:63:0) after 4m40s
[20:55:04.264] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:63:0)
[20:55:04.287] Build status: STARTED
[20:55:04.287] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8952337345798431984:64:0) after 4m51s
[20:57:07.408] Received PubSub notification, asking Buildbucket for the build status
[20:57:07.431] Build:
{
"id": "8786907283636396641",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29060,
"createdBy": "project:chromium",
"createTime": "2023-03-11T15:40:25.160169638Z",
"startTime": "2023-03-11T15:42:59.192983Z",
"endTime": "2023-03-11T20:57:06.761623389Z",
"updateTime": "2023-03-11T20:57:06.761623389Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9d68b08ffafd1cd80da00ace8f6e8e68ccb31207",
"ref": "refs/heads/main"
}
}
}
[20:57:07.431] Invocation finished in 5h16m43.71610472s with status FAILED