[16:42:57.994] New invocation is queued and will start shortly
[16:42:59.261] Starting the invocation (attempt 1)
[16:42:59.314] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/52070be21bff4c7b760cc2fcbd6c1b53db6f72ba"
[16:42:59.314] Popped gitiles commit info from properties and tags
[16:42:59.314] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[16:42:59.314] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[16:42:59.314] Buildbucket request:
{
"requestId": "8945266845782356208",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "52070be21bff4c7b760cc2fcbd6c1b53db6f72ba",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8945266845782356208"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[16:42:59.969] Scheduled build:
{
"id": "8779836783663136529",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29490,
"createdBy": "project:chromium",
"createTime": "2023-05-28T16:42:59.389437416Z",
"updateTime": "2023-05-28T16:42:59.389437416Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "52070be21bff4c7b760cc2fcbd6c1b53db6f72ba",
"ref": "refs/heads/main"
}
}
}
[16:42:59.969] Task URL: https://cr-buildbucket.appspot.com/build/8779836783663136529
[16:42:59.969] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:2:0) after 9m40s
[16:45:52.641] Received PubSub notification, asking Buildbucket for the build status
[16:45:52.676] Build status: STARTED
[16:52:39.987] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:2:0)
[16:52:40.015] Build status: STARTED
[16:52:40.015] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:4:0) after 6m19s
[16:58:59.095] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:4:0)
[16:58:59.136] Build status: STARTED
[16:58:59.136] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:5:0) after 3m40s
[17:02:39.152] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:5:0)
[17:02:39.181] Build status: STARTED
[17:02:39.181] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:6:0) after 5m11s
[17:07:50.218] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:6:0)
[17:07:50.244] Build status: STARTED
[17:07:50.245] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:7:0) after 8m10s
[17:16:00.252] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:7:0)
[17:16:00.274] Build status: STARTED
[17:16:00.274] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:8:0) after 9m59s
[17:25:59.682] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:8:0)
[17:25:59.724] Build status: STARTED
[17:25:59.724] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:9:0) after 3m20s
[17:29:19.758] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:9:0)
[17:29:19.781] Build status: STARTED
[17:29:19.781] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:10:0) after 2m33s
[17:31:52.794] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:10:0)
[17:31:52.812] Build status: STARTED
[17:31:52.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:11:0) after 6m10s
[17:38:02.827] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:11:0)
[17:38:02.857] Build status: STARTED
[17:38:02.857] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:12:0) after 1m59s
[17:40:01.874] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:12:0)
[17:40:01.890] Build status: STARTED
[17:40:01.890] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:13:0) after 6m10s
[17:46:12.060] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:13:0)
[17:46:12.208] Build status: STARTED
[17:46:12.208] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:14:0) after 1m9s
[17:47:21.227] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:14:0)
[17:47:21.263] Build status: STARTED
[17:47:21.263] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:15:0) after 1m7s
[17:48:28.408] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:15:0)
[17:48:28.456] Build status: STARTED
[17:48:28.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:16:0) after 2m40s
[17:51:08.503] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:16:0)
[17:51:08.523] Build status: STARTED
[17:51:08.523] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:17:0) after 8m2s
[17:59:10.530] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:17:0)
[17:59:10.548] Build status: STARTED
[17:59:10.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:18:0) after 5m31s
[18:04:41.637] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:18:0)
[18:04:41.661] Build status: STARTED
[18:04:41.661] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:19:0) after 8m21s
[18:13:02.684] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:19:0)
[18:13:02.789] Build status: STARTED
[18:13:02.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:20:0) after 1m25s
[18:14:27.811] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:20:0)
[18:14:27.831] Build status: STARTED
[18:14:27.831] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:21:0) after 5m16s
[18:19:44.005] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:21:0)
[18:19:44.029] Build status: STARTED
[18:19:44.029] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:22:0) after 5m41s
[18:25:25.154] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:22:0)
[18:25:25.183] Build status: STARTED
[18:25:25.183] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:23:0) after 1m18s
[18:26:43.203] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:23:0)
[18:26:43.224] Build status: STARTED
[18:26:43.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:24:0) after 7m31s
[18:34:14.240] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:24:0)
[18:34:14.287] Build status: STARTED
[18:34:14.287] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:25:0) after 5m17s
[18:39:31.311] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:25:0)
[18:39:31.331] Build status: STARTED
[18:39:31.331] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:26:0) after 4m38s
[18:44:09.377] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:26:0)
[18:44:09.395] Build status: STARTED
[18:44:09.396] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:27:0) after 8m49s
[18:52:58.512] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:27:0)
[18:52:58.532] Build status: STARTED
[18:52:58.533] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:28:0) after 9m23s
[19:02:21.688] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:28:0)
[19:02:21.709] Build status: STARTED
[19:02:21.709] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:29:0) after 8m28s
[19:10:49.730] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:29:0)
[19:10:49.749] Build status: STARTED
[19:10:49.749] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:30:0) after 1m22s
[19:12:11.766] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:30:0)
[19:12:11.831] Build status: STARTED
[19:12:11.831] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:31:0) after 5m17s
[19:17:28.876] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:31:0)
[19:17:28.898] Build status: STARTED
[19:17:28.898] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:32:0) after 5m54s
[19:23:23.138] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:32:0)
[19:23:23.158] Build status: STARTED
[19:23:23.158] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:33:0) after 8m53s
[19:32:16.180] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:33:0)
[19:32:16.198] Build status: STARTED
[19:32:16.198] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:34:0) after 4m9s
[19:36:25.342] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:34:0)
[19:36:25.360] Build status: STARTED
[19:36:25.360] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:35:0) after 8m31s
[19:44:56.381] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:35:0)
[19:44:56.405] Build status: STARTED
[19:44:56.405] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:36:0) after 4m18s
[19:49:14.443] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:36:0)
[19:49:14.588] Build status: STARTED
[19:49:14.588] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:37:0) after 9m13s
[19:58:27.603] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:37:0)
[19:58:27.623] Build status: STARTED
[19:58:27.623] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:38:0) after 5m57s
[20:04:24.646] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:38:0)
[20:04:24.685] Build status: STARTED
[20:04:24.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:39:0) after 1m30s
[20:05:55.048] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:39:0)
[20:05:55.094] Build status: STARTED
[20:05:55.094] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:40:0) after 4m49s
[20:10:44.115] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:40:0)
[20:10:44.137] Build status: STARTED
[20:10:44.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:41:0) after 5m20s
[20:16:04.165] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:41:0)
[20:16:04.186] Build status: STARTED
[20:16:04.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:42:0) after 7m51s
[20:23:55.218] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:42:0)
[20:23:55.247] Build status: STARTED
[20:23:55.247] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:43:0) after 6m19s
[20:30:14.308] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:43:0)
[20:30:14.368] Build status: STARTED
[20:30:14.368] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:44:0) after 9m54s
[20:40:08.389] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:44:0)
[20:40:08.408] Build status: STARTED
[20:40:08.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:45:0) after 2m0s
[20:42:08.545] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:45:0)
[20:42:08.599] Build status: STARTED
[20:42:08.600] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:46:0) after 1m55s
[20:44:03.608] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:46:0)
[20:44:03.629] Build status: STARTED
[20:44:03.629] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:47:0) after 4m31s
[20:48:34.896] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:47:0)
[20:48:34.917] Build status: STARTED
[20:48:34.917] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:48:0) after 8m41s
[20:57:16.008] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:48:0)
[20:57:16.028] Build status: STARTED
[20:57:16.028] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:49:0) after 7m24s
[21:04:40.136] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:49:0)
[21:04:40.201] Build status: STARTED
[21:04:40.201] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:50:0) after 5m5s
[21:09:45.229] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:50:0)
[21:09:45.252] Build status: STARTED
[21:09:45.252] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:51:0) after 2m10s
[21:11:55.434] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:51:0)
[21:11:55.453] Build status: STARTED
[21:11:55.453] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:52:0) after 7m54s
[21:19:49.649] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:52:0)
[21:19:49.671] Build status: STARTED
[21:19:49.671] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:53:0) after 2m33s
[21:22:22.697] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:53:0)
[21:22:22.726] Build status: STARTED
[21:22:22.726] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:54:0) after 2m11s
[21:24:33.750] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:54:0)
[21:24:33.771] Build status: STARTED
[21:24:33.771] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:55:0) after 3m5s
[21:27:38.793] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:55:0)
[21:27:38.814] Build status: STARTED
[21:27:38.814] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:56:0) after 4m55s
[21:32:33.898] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:56:0)
[21:32:33.924] Build status: STARTED
[21:32:33.925] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:57:0) after 5m1s
[21:37:35.055] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:57:0)
[21:37:35.076] Build status: STARTED
[21:37:35.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:58:0) after 9m16s
[21:46:51.098] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:58:0)
[21:46:51.122] Build status: STARTED
[21:46:51.122] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:59:0) after 3m55s
[21:50:46.204] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:59:0)
[21:50:46.224] Build status: STARTED
[21:50:46.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:60:0) after 5m16s
[21:56:02.292] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:60:0)
[21:56:02.312] Build status: STARTED
[21:56:02.312] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8945266845782356208:61:0) after 7m19s
[21:56:23.604] Received PubSub notification, asking Buildbucket for the build status
[21:56:23.622] Build:
{
"id": "8779836783663136529",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29490,
"createdBy": "project:chromium",
"createTime": "2023-05-28T16:42:59.389437416Z",
"startTime": "2023-05-28T16:45:52.123750Z",
"endTime": "2023-05-28T21:56:23.352837188Z",
"updateTime": "2023-05-28T21:56:23.352837188Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "52070be21bff4c7b760cc2fcbd6c1b53db6f72ba",
"ref": "refs/heads/main"
}
}
}
[21:56:23.622] Invocation finished in 5h13m25.636985414s with status FAILED