[19:06:24.220] New invocation is queued and will start shortly
[19:06:26.830] Starting the invocation (attempt 1)
[19:06:26.867] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/9d3231c8dbf2a33c00ad08851b03568881221026"
[19:06:26.867] Popped gitiles commit info from properties and tags
[19:06:26.867] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[19:06:26.867] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[19:06:26.868] Buildbucket request:
{
"requestId": "8999525404379852016",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9d3231c8dbf2a33c00ad08851b03568881221026",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8999525404379852016"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[19:06:27.381] Scheduled build:
{
"id": "8834095340861741889",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23311,
"createdBy": "project:chromium",
"createTime": "2021-10-06T19:06:26.944640313Z",
"updateTime": "2021-10-06T19:06:27.165593151Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9d3231c8dbf2a33c00ad08851b03568881221026",
"ref": "refs/heads/main"
}
}
}
[19:06:27.381] Task URL: https://cr-buildbucket.appspot.com/build/8834095340861741889
[19:06:27.381] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:2:0) after 4m43s
[19:08:40.681] Received PubSub notification, asking Buildbucket for the build status
[19:08:40.709] Build status: STARTED
[19:11:10.401] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:2:0)
[19:11:10.501] Build status: STARTED
[19:11:10.501] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:4:0) after 1m13s
[19:12:23.625] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:4:0)
[19:12:23.657] Build status: STARTED
[19:12:23.657] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:5:0) after 2m15s
[19:14:38.718] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:5:0)
[19:14:38.751] Build status: STARTED
[19:14:38.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:6:0) after 8m55s
[19:23:33.774] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:6:0)
[19:23:33.803] Build status: STARTED
[19:23:33.804] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:7:0) after 8m43s
[19:32:16.826] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:7:0)
[19:32:16.864] Build status: STARTED
[19:32:16.864] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:8:0) after 2m36s
[19:34:52.888] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:8:0)
[19:34:52.921] Build status: STARTED
[19:34:52.921] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:9:0) after 1m13s
[19:36:05.962] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:9:0)
[19:36:06.004] Build status: STARTED
[19:36:06.004] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:10:0) after 8m4s
[19:44:10.359] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:10:0)
[19:44:10.404] Build status: STARTED
[19:44:10.404] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:11:0) after 1m11s
[19:45:21.436] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:11:0)
[19:45:21.471] Build status: STARTED
[19:45:21.471] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:12:0) after 6m56s
[19:52:17.528] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:12:0)
[19:52:17.562] Build status: STARTED
[19:52:17.562] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:13:0) after 4m39s
[19:56:56.600] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:13:0)
[19:56:56.634] Build status: STARTED
[19:56:56.634] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:14:0) after 1m33s
[19:58:29.766] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:14:0)
[19:58:29.802] Build status: STARTED
[19:58:29.802] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:15:0) after 1m39s
[20:00:08.962] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:15:0)
[20:00:08.998] Build status: STARTED
[20:00:08.998] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:16:0) after 3m1s
[20:03:10.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:16:0)
[20:03:10.070] Build status: STARTED
[20:03:10.070] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:17:0) after 8m29s
[20:11:39.098] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:17:0)
[20:11:39.151] Build status: STARTED
[20:11:39.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:18:0) after 8m42s
[20:20:21.501] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:18:0)
[20:20:21.537] Build status: STARTED
[20:20:21.537] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:19:0) after 4m26s
[20:24:47.520] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:19:0)
[20:24:47.548] Build status: STARTED
[20:24:47.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:20:0) after 1m41s
[20:26:28.854] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:20:0)
[20:26:28.884] Build status: STARTED
[20:26:28.884] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:21:0) after 3m56s
[20:30:25.060] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:21:0)
[20:30:25.162] Build status: STARTED
[20:30:25.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:22:0) after 1m18s
[20:31:43.282] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:22:0)
[20:31:43.311] Build status: STARTED
[20:31:43.311] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:23:0) after 1m42s
[20:33:25.758] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:23:0)
[20:33:25.790] Build status: STARTED
[20:33:25.790] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:24:0) after 8m26s
[20:41:51.872] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:24:0)
[20:41:51.932] Build status: STARTED
[20:41:51.932] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:25:0) after 6m28s
[20:48:20.112] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:25:0)
[20:48:20.140] Build status: STARTED
[20:48:20.140] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:26:0) after 3m57s
[20:52:17.224] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:26:0)
[20:52:17.252] Build status: STARTED
[20:52:17.252] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:27:0) after 8m31s
[21:00:48.352] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:27:0)
[21:00:48.386] Build status: STARTED
[21:00:48.386] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:28:0) after 7m51s
[21:08:39.930] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:28:0)
[21:08:39.980] Build status: STARTED
[21:08:39.980] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:29:0) after 3m49s
[21:12:29.011] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:29:0)
[21:12:29.059] Build status: STARTED
[21:12:29.059] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:30:0) after 2m57s
[21:15:26.091] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:30:0)
[21:15:26.121] Build status: STARTED
[21:15:26.121] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:31:0) after 4m18s
[21:19:44.150] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:31:0)
[21:19:44.299] Build status: STARTED
[21:19:44.299] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:32:0) after 4m43s
[21:24:27.391] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:32:0)
[21:24:27.429] Build status: STARTED
[21:24:27.429] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:33:0) after 6m4s
[21:30:31.434] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:33:0)
[21:30:31.470] Build status: STARTED
[21:30:31.470] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:34:0) after 8m43s
[21:39:14.466] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:34:0)
[21:39:14.575] Build status: STARTED
[21:39:14.575] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:35:0) after 5m42s
[21:44:56.991] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:35:0)
[21:44:57.035] Build status: STARTED
[21:44:57.035] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:36:0) after 6m51s
[21:51:48.064] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:36:0)
[21:51:48.103] Build status: STARTED
[21:51:48.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:37:0) after 4m46s
[21:56:34.126] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:37:0)
[21:56:34.156] Build status: STARTED
[21:56:34.156] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:38:0) after 5m59s
[22:02:33.196] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:38:0)
[22:02:33.226] Build status: STARTED
[22:02:33.226] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:39:0) after 5m10s
[22:07:43.362] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:39:0)
[22:07:43.394] Build status: STARTED
[22:07:43.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:40:0) after 1m21s
[22:09:04.548] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:40:0)
[22:09:04.602] Build status: STARTED
[22:09:04.602] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:41:0) after 6m14s
[22:15:18.635] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:41:0)
[22:15:18.675] Build status: STARTED
[22:15:18.675] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:42:0) after 3m48s
[22:19:06.699] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:42:0)
[22:19:06.732] Build status: STARTED
[22:19:06.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:43:0) after 8m59s
[22:28:05.768] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:43:0)
[22:28:05.805] Build status: STARTED
[22:28:05.805] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:44:0) after 3m56s
[22:32:01.828] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:44:0)
[22:32:01.980] Build status: STARTED
[22:32:01.980] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:45:0) after 9m57s
[22:41:59.103] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:45:0)
[22:41:59.132] Build status: STARTED
[22:41:59.132] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:46:0) after 1m54s
[22:43:53.188] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:46:0)
[22:43:53.296] Build status: STARTED
[22:43:53.296] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:47:0) after 5m2s
[22:48:55.322] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:47:0)
[22:48:55.352] Build status: STARTED
[22:48:55.352] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:48:0) after 9m56s
[22:58:51.433] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:48:0)
[22:58:51.465] Build status: STARTED
[22:58:51.465] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:49:0) after 4m50s
[23:03:41.695] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:49:0)
[23:03:41.723] Build status: STARTED
[23:03:41.723] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:50:0) after 4m20s
[23:08:01.847] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:50:0)
[23:08:01.884] Build status: STARTED
[23:08:01.884] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:51:0) after 2m46s
[23:10:47.961] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:51:0)
[23:10:48.018] Build status: STARTED
[23:10:48.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:52:0) after 2m20s
[23:13:08.108] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:52:0)
[23:13:08.151] Build status: STARTED
[23:13:08.152] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:53:0) after 5m4s
[23:18:12.178] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:53:0)
[23:18:12.214] Build status: STARTED
[23:18:12.214] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:54:0) after 2m41s
[23:20:53.369] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:54:0)
[23:20:53.429] Build status: STARTED
[23:20:53.429] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:55:0) after 2m30s
[23:23:23.459] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:55:0)
[23:23:23.491] Build status: STARTED
[23:23:23.491] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:56:0) after 8m38s
[23:32:01.998] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:56:0)
[23:32:02.026] Build status: STARTED
[23:32:02.026] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:57:0) after 4m45s
[23:36:47.122] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:57:0)
[23:36:47.162] Build status: STARTED
[23:36:47.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:58:0) after 6m47s
[23:43:34.299] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:58:0)
[23:43:34.347] Build status: STARTED
[23:43:34.347] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:59:0) after 7m6s
[23:50:40.595] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:59:0)
[23:50:40.624] Build status: STARTED
[23:50:40.624] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:60:0) after 4m36s
[23:55:16.747] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:60:0)
[23:55:16.779] Build status: STARTED
[23:55:16.779] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:61:0) after 9m50s
[00:05:06.797] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:61:0)
[00:05:06.823] Build status: STARTED
[00:05:06.823] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:62:0) after 4m11s
[00:09:17.856] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:62:0)
[00:09:17.952] Build status: STARTED
[00:09:17.953] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:63:0) after 9m34s
[00:18:51.983] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:63:0)
[00:18:52.018] Build status: STARTED
[00:18:52.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:64:0) after 6m24s
[00:25:16.232] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:64:0)
[00:25:16.281] Build status: STARTED
[00:25:16.281] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999525404379852016:65:0) after 4m52s
[00:27:37.445] Received PubSub notification, asking Buildbucket for the build status
[00:27:37.488] Build:
{
"id": "8834095340861741889",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23311,
"createdBy": "project:chromium",
"createTime": "2021-10-06T19:06:26.944640313Z",
"startTime": "2021-10-06T19:08:39.997960Z",
"endTime": "2021-10-07T00:27:36.677608298Z",
"updateTime": "2021-10-07T00:27:36.677992987Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9d3231c8dbf2a33c00ad08851b03568881221026",
"ref": "refs/heads/main"
}
}
}
[00:27:37.488] Invocation finished in 5h21m13.283652133s with status FAILED