[12:16:10.276] New invocation is queued and will start shortly
[12:16:12.102] Starting the invocation (attempt 1)
[12:16:12.139] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/5ad07afb026792f4031fb90631c31a15dd6b0356"
[12:16:12.139] Popped gitiles commit info from properties and tags
[12:16:12.139] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[12:16:12.139] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[12:16:12.140] Buildbucket request:
{
"requestId": "9001997332078416240",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5ad07afb026792f4031fb90631c31a15dd6b0356",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "9001997332078416240"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[12:16:12.655] Scheduled build:
{
"id": "8836567269359504161",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23068,
"createdBy": "project:chromium",
"createTime": "2021-09-09T12:16:12.223655160Z",
"updateTime": "2021-09-09T12:16:12.476161735Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5ad07afb026792f4031fb90631c31a15dd6b0356",
"ref": "refs/heads/main"
}
},
"canary": true
}
[12:16:12.655] Task URL: https://cr-buildbucket.appspot.com/build/8836567269359504161
[12:16:12.655] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:2:0) after 7m45s
[12:19:02.265] Received PubSub notification, asking Buildbucket for the build status
[12:19:02.314] Build status: STARTED
[12:23:58.083] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:2:0)
[12:23:58.112] Build status: STARTED
[12:23:58.112] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:4:0) after 2m26s
[12:26:24.365] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:4:0)
[12:26:24.396] Build status: STARTED
[12:26:24.396] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:5:0) after 5m25s
[12:31:49.431] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:5:0)
[12:31:49.480] Build status: STARTED
[12:31:49.480] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:6:0) after 2m25s
[12:34:14.681] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:6:0)
[12:34:14.731] Build status: STARTED
[12:34:14.731] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:7:0) after 3m57s
[12:38:11.817] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:7:0)
[12:38:11.863] Build status: STARTED
[12:38:11.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:8:0) after 6m45s
[12:44:57.168] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:8:0)
[12:44:57.227] Build status: STARTED
[12:44:57.227] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:9:0) after 8m18s
[12:53:15.518] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:9:0)
[12:53:15.557] Build status: STARTED
[12:53:15.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:10:0) after 4m16s
[12:57:31.689] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:10:0)
[12:57:31.724] Build status: STARTED
[12:57:31.724] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:11:0) after 6m16s
[13:03:47.745] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:11:0)
[13:03:47.781] Build status: STARTED
[13:03:47.781] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:12:0) after 5m48s
[13:09:35.922] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:12:0)
[13:09:35.957] Build status: STARTED
[13:09:35.957] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:13:0) after 7m55s
[13:17:31.115] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:13:0)
[13:17:31.148] Build status: STARTED
[13:17:31.148] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:14:0) after 8m42s
[13:26:13.414] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:14:0)
[13:26:13.447] Build status: STARTED
[13:26:13.447] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:15:0) after 7m13s
[13:33:26.491] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:15:0)
[13:33:26.528] Build status: STARTED
[13:33:26.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:16:0) after 1m32s
[13:34:58.552] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:16:0)
[13:34:58.589] Build status: STARTED
[13:34:58.589] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:17:0) after 6m10s
[13:41:08.670] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:17:0)
[13:41:08.722] Build status: STARTED
[13:41:08.722] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:18:0) after 4m28s
[13:45:37.061] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:18:0)
[13:45:37.116] Build status: STARTED
[13:45:37.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:19:0) after 6m41s
[13:52:18.259] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:19:0)
[13:52:18.316] Build status: STARTED
[13:52:18.316] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:20:0) after 1m52s
[13:54:10.503] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:20:0)
[13:54:10.531] Build status: STARTED
[13:54:10.531] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:21:0) after 1m10s
[13:55:20.551] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:21:0)
[13:55:20.581] Build status: STARTED
[13:55:20.581] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:22:0) after 2m2s
[13:57:22.704] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:22:0)
[13:57:22.735] Build status: STARTED
[13:57:22.735] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:23:0) after 1m6s
[13:58:28.905] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:23:0)
[13:58:28.948] Build status: STARTED
[13:58:28.948] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:24:0) after 1m7s
[13:59:36.076] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:24:0)
[13:59:36.115] Build status: STARTED
[13:59:36.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:25:0) after 3m56s
[14:03:32.237] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:25:0)
[14:03:32.316] Build status: STARTED
[14:03:32.316] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:26:0) after 6m45s
[14:10:17.415] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:26:0)
[14:10:17.569] Build status: STARTED
[14:10:17.569] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:27:0) after 5m2s
[14:15:19.643] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:27:0)
[14:15:19.679] Build status: STARTED
[14:15:19.679] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:28:0) after 2m34s
[14:17:54.097] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:28:0)
[14:17:54.142] Build status: STARTED
[14:17:54.142] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:29:0) after 7m16s
[14:25:10.174] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:29:0)
[14:25:10.225] Build status: STARTED
[14:25:10.225] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:30:0) after 1m50s
[14:27:00.285] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:30:0)
[14:27:00.314] Build status: STARTED
[14:27:00.314] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:31:0) after 2m6s
[14:29:06.512] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:31:0)
[14:29:06.543] Build status: STARTED
[14:29:06.543] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:32:0) after 7m38s
[14:36:44.655] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:32:0)
[14:36:44.698] Build status: STARTED
[14:36:44.698] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:33:0) after 9m12s
[14:45:57.066] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:33:0)
[14:45:57.099] Build status: STARTED
[14:45:57.099] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:34:0) after 1m9s
[14:47:06.123] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:34:0)
[14:47:06.169] Build status: STARTED
[14:47:06.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:35:0) after 6m55s
[14:54:01.554] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:35:0)
[14:54:01.586] Build status: STARTED
[14:54:01.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:36:0) after 2m40s
[14:56:41.707] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:36:0)
[14:56:41.739] Build status: STARTED
[14:56:41.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:37:0) after 9m50s
[15:06:31.766] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:37:0)
[15:06:31.804] Build status: STARTED
[15:06:31.804] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:38:0) after 5m14s
[15:11:45.962] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:38:0)
[15:11:45.993] Build status: STARTED
[15:11:45.994] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:39:0) after 5m47s
[15:17:33.022] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:39:0)
[15:17:33.061] Build status: STARTED
[15:17:33.061] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:40:0) after 5m6s
[15:22:39.041] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:40:0)
[15:22:39.076] Build status: STARTED
[15:22:39.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:41:0) after 3m31s
[15:26:10.248] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:41:0)
[15:26:10.281] Build status: STARTED
[15:26:10.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:42:0) after 9m57s
[15:36:07.402] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:42:0)
[15:36:07.446] Build status: STARTED
[15:36:07.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:43:0) after 5m8s
[15:41:15.449] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:43:0)
[15:41:15.489] Build status: STARTED
[15:41:15.489] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:44:0) after 3m56s
[15:45:11.527] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:44:0)
[15:45:11.591] Build status: STARTED
[15:45:11.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:45:0) after 8m14s
[15:53:25.640] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:45:0)
[15:53:25.668] Build status: STARTED
[15:53:25.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:46:0) after 7m56s
[16:01:21.738] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:46:0)
[16:01:21.778] Build status: STARTED
[16:01:21.778] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:47:0) after 8m22s
[16:09:43.899] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:47:0)
[16:09:43.953] Build status: STARTED
[16:09:43.953] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:48:0) after 3m39s
[16:13:22.976] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:48:0)
[16:13:23.004] Build status: STARTED
[16:13:23.004] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:49:0) after 8m54s
[16:22:17.267] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:49:0)
[16:22:17.310] Build status: STARTED
[16:22:17.310] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:50:0) after 8m17s
[16:30:34.412] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:50:0)
[16:30:34.453] Build status: STARTED
[16:30:34.453] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:51:0) after 5m32s
[16:36:06.481] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:51:0)
[16:36:06.526] Build status: STARTED
[16:36:06.526] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:52:0) after 1m30s
[16:37:36.571] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:52:0)
[16:37:36.607] Build status: STARTED
[16:37:36.607] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:53:0) after 5m45s
[16:43:21.666] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:53:0)
[16:43:21.709] Build status: STARTED
[16:43:21.709] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:54:0) after 1m23s
[16:44:44.977] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:54:0)
[16:44:45.008] Build status: STARTED
[16:44:45.008] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:55:0) after 8m9s
[16:52:54.027] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:55:0)
[16:52:54.102] Build status: STARTED
[16:52:54.102] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:56:0) after 7m50s
[17:00:44.224] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:56:0)
[17:00:44.262] Build status: STARTED
[17:00:44.262] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:57:0) after 1m44s
[17:02:28.555] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:57:0)
[17:02:28.587] Build status: STARTED
[17:02:28.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:58:0) after 7m1s
[17:09:29.624] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:58:0)
[17:09:29.657] Build status: STARTED
[17:09:29.657] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:59:0) after 3m48s
[17:13:17.686] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:59:0)
[17:13:17.718] Build status: STARTED
[17:13:17.718] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:60:0) after 8m2s
[17:21:19.755] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:60:0)
[17:21:19.782] Build status: STARTED
[17:21:19.782] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:61:0) after 8m36s
[17:29:55.847] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:61:0)
[17:29:55.877] Build status: STARTED
[17:29:55.877] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:62:0) after 4m19s
[17:34:15.235] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:62:0)
[17:34:15.291] Build status: STARTED
[17:34:15.291] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:63:0) after 1m27s
[17:35:42.324] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:63:0)
[17:35:42.369] Build status: STARTED
[17:35:42.369] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001997332078416240:64:0) after 9m30s
[17:42:15.323] Received PubSub notification, asking Buildbucket for the build status
[17:42:15.380] Build:
{
"id": "8836567269359504161",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23068,
"createdBy": "project:chromium",
"createTime": "2021-09-09T12:16:12.223655160Z",
"startTime": "2021-09-09T12:19:01.525918Z",
"endTime": "2021-09-09T17:42:14.693753774Z",
"updateTime": "2021-09-09T17:42:14.694122235Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5ad07afb026792f4031fb90631c31a15dd6b0356",
"ref": "refs/heads/main"
}
},
"canary": true
}
[17:42:15.380] Invocation finished in 5h26m5.132585501s with status FAILED