[21:09:00.375] New invocation is queued and will start shortly
[21:09:01.457] Starting the invocation (attempt 1)
[21:09:01.500] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/cdd29645e08dbd3e2b9a4ba3d14bc1b7187b1f12"
[21:09:01.500] Popped gitiles commit info from properties and tags
[21:09:01.500] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:09:01.500] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:09:01.500] Buildbucket request:
{
"requestId": "8995259633473429488",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cdd29645e08dbd3e2b9a4ba3d14bc1b7187b1f12",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8995259633473429488"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:09:02.559] Scheduled build:
{
"id": "8829829571528151553",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23593,
"createdBy": "project:chromium",
"createTime": "2021-11-22T21:09:01.597892053Z",
"updateTime": "2021-11-22T21:09:01.597892053Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cdd29645e08dbd3e2b9a4ba3d14bc1b7187b1f12",
"ref": "refs/heads/main"
}
}
}
[21:09:02.559] Task URL: https://cr-buildbucket.appspot.com/build/8829829571528151553
[21:09:02.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:2:0) after 6m58s
[21:11:37.516] Received PubSub notification, asking Buildbucket for the build status
[21:11:37.615] Build status: STARTED
[21:16:01.078] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:2:0)
[21:16:01.175] Build status: STARTED
[21:16:01.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:4:0) after 7m59s
[21:24:00.299] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:4:0)
[21:24:00.334] Build status: STARTED
[21:24:00.334] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:5:0) after 7m48s
[21:31:48.780] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:5:0)
[21:31:48.812] Build status: STARTED
[21:31:48.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:6:0) after 7m3s
[21:38:52.710] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:6:0)
[21:38:52.737] Build status: STARTED
[21:38:52.737] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:7:0) after 9m36s
[21:48:28.778] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:7:0)
[21:48:28.871] Build status: STARTED
[21:48:28.871] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:8:0) after 2m25s
[21:50:53.917] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:8:0)
[21:50:53.949] Build status: STARTED
[21:50:53.949] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:9:0) after 2m47s
[21:53:41.259] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:9:0)
[21:53:41.294] Build status: STARTED
[21:53:41.294] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:10:0) after 1m52s
[21:55:33.426] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:10:0)
[21:55:33.609] Build status: STARTED
[21:55:33.610] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:11:0) after 8m11s
[22:03:44.666] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:11:0)
[22:03:44.704] Build status: STARTED
[22:03:44.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:12:0) after 3m22s
[22:07:06.803] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:12:0)
[22:07:06.843] Build status: STARTED
[22:07:06.843] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:13:0) after 1m4s
[22:08:11.001] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:13:0)
[22:08:11.043] Build status: STARTED
[22:08:11.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:14:0) after 1m54s
[22:10:05.277] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:14:0)
[22:10:05.327] Build status: STARTED
[22:10:05.327] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:15:0) after 7m35s
[22:17:40.348] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:15:0)
[22:17:40.551] Build status: STARTED
[22:17:40.551] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:16:0) after 4m46s
[22:22:26.660] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:16:0)
[22:22:26.696] Build status: STARTED
[22:22:26.696] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:17:0) after 4m38s
[22:27:04.727] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:17:0)
[22:27:04.766] Build status: STARTED
[22:27:04.766] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:18:0) after 3m34s
[22:30:39.082] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:18:0)
[22:30:39.123] Build status: STARTED
[22:30:39.123] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:19:0) after 8m12s
[22:38:51.364] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:19:0)
[22:38:51.400] Build status: STARTED
[22:38:51.400] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:20:0) after 1m40s
[22:40:31.424] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:20:0)
[22:40:31.459] Build status: STARTED
[22:40:31.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:21:0) after 9m19s
[22:49:50.513] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:21:0)
[22:49:50.544] Build status: STARTED
[22:49:50.544] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:22:0) after 2m22s
[22:52:12.567] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:22:0)
[22:52:12.612] Build status: STARTED
[22:52:12.612] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:23:0) after 8m52s
[23:01:04.823] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:23:0)
[23:01:04.869] Build status: STARTED
[23:01:04.869] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:24:0) after 4m54s
[23:05:59.062] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:24:0)
[23:05:59.105] Build status: STARTED
[23:05:59.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:25:0) after 2m38s
[23:08:37.270] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:25:0)
[23:08:37.315] Build status: STARTED
[23:08:37.315] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:26:0) after 5m7s
[23:13:44.439] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:26:0)
[23:13:44.476] Build status: STARTED
[23:13:44.476] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:27:0) after 3m13s
[23:16:57.507] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:27:0)
[23:16:57.543] Build status: STARTED
[23:16:57.543] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:28:0) after 6m16s
[23:23:13.636] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:28:0)
[23:23:13.677] Build status: STARTED
[23:23:13.677] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:29:0) after 6m4s
[23:29:17.713] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:29:0)
[23:29:17.755] Build status: STARTED
[23:29:17.755] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:30:0) after 6m11s
[23:35:28.804] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:30:0)
[23:35:28.968] Build status: STARTED
[23:35:28.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:31:0) after 4m42s
[23:40:11.013] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:31:0)
[23:40:11.123] Build status: STARTED
[23:40:11.123] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:32:0) after 1m18s
[23:41:29.262] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:32:0)
[23:41:29.299] Build status: STARTED
[23:41:29.299] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:33:0) after 5m8s
[23:46:37.406] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:33:0)
[23:46:37.502] Build status: STARTED
[23:46:37.502] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:34:0) after 9m9s
[23:55:46.549] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:34:0)
[23:55:46.590] Build status: STARTED
[23:55:46.590] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:35:0) after 1m47s
[23:57:33.616] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:35:0)
[23:57:33.777] Build status: STARTED
[23:57:33.777] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:36:0) after 6m55s
[00:04:29.201] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:36:0)
[00:04:29.240] Build status: STARTED
[00:04:29.240] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:37:0) after 5m47s
[00:10:16.805] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:37:0)
[00:10:16.851] Build status: STARTED
[00:10:16.851] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:38:0) after 3m31s
[00:13:48.098] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:38:0)
[00:13:48.137] Build status: STARTED
[00:13:48.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:39:0) after 6m50s
[00:20:38.283] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:39:0)
[00:20:38.314] Build status: STARTED
[00:20:38.314] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:40:0) after 3m53s
[00:24:31.374] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:40:0)
[00:24:31.405] Build status: STARTED
[00:24:31.405] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:41:0) after 4m7s
[00:28:38.431] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:41:0)
[00:28:38.461] Build status: STARTED
[00:28:38.461] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:42:0) after 1m22s
[00:30:00.593] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:42:0)
[00:30:00.625] Build status: STARTED
[00:30:00.625] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:43:0) after 4m50s
[00:34:50.649] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:43:0)
[00:34:50.681] Build status: STARTED
[00:34:50.681] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:44:0) after 9m18s
[00:44:08.686] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:44:0)
[00:44:08.717] Build status: STARTED
[00:44:08.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:45:0) after 3m27s
[00:47:35.905] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:45:0)
[00:47:35.982] Build status: STARTED
[00:47:35.982] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:46:0) after 3m39s
[00:51:15.165] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:46:0)
[00:51:15.205] Build status: STARTED
[00:51:15.205] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:47:0) after 3m23s
[00:54:38.476] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:47:0)
[00:54:38.504] Build status: STARTED
[00:54:38.504] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:48:0) after 1m52s
[00:56:31.110] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:48:0)
[00:56:31.237] Build status: STARTED
[00:56:31.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:49:0) after 8m43s
[01:05:14.340] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:49:0)
[01:05:14.375] Build status: STARTED
[01:05:14.375] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:50:0) after 8m42s
[01:13:56.417] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:50:0)
[01:13:56.481] Build status: STARTED
[01:13:56.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:51:0) after 3m44s
[01:17:40.551] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:51:0)
[01:17:40.590] Build status: STARTED
[01:17:40.590] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:52:0) after 8m16s
[01:25:56.617] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:52:0)
[01:25:56.787] Build status: STARTED
[01:25:56.787] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:53:0) after 3m17s
[01:29:13.884] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:53:0)
[01:29:13.913] Build status: STARTED
[01:29:13.913] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:54:0) after 1m45s
[01:30:59.021] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:54:0)
[01:30:59.052] Build status: STARTED
[01:30:59.052] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:55:0) after 9m59s
[01:40:58.114] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:55:0)
[01:40:58.146] Build status: STARTED
[01:40:58.146] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:56:0) after 7m23s
[01:48:21.301] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:56:0)
[01:48:21.339] Build status: STARTED
[01:48:21.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:57:0) after 1m21s
[01:49:42.378] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:57:0)
[01:49:42.411] Build status: STARTED
[01:49:42.411] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:58:0) after 3m19s
[01:53:01.615] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:58:0)
[01:53:01.684] Build status: STARTED
[01:53:01.684] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:59:0) after 9m34s
[02:02:35.711] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:59:0)
[02:02:35.744] Build status: STARTED
[02:02:35.744] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:60:0) after 8m18s
[02:10:53.777] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:60:0)
[02:10:53.813] Build status: STARTED
[02:10:53.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:61:0) after 6m34s
[02:17:27.901] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:61:0)
[02:17:27.937] Build status: STARTED
[02:17:27.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:62:0) after 7m5s
[02:24:33.076] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:62:0)
[02:24:33.111] Build status: STARTED
[02:24:33.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:63:0) after 4m40s
[02:29:13.266] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:63:0)
[02:29:13.314] Build status: STARTED
[02:29:13.314] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995259633473429488:64:0) after 8m44s
[02:33:05.638] Received PubSub notification, asking Buildbucket for the build status
[02:33:05.670] Build:
{
"id": "8829829571528151553",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23593,
"createdBy": "project:chromium",
"createTime": "2021-11-22T21:09:01.597892053Z",
"startTime": "2021-11-22T21:11:34.510629Z",
"endTime": "2021-11-23T02:33:04.980484266Z",
"updateTime": "2021-11-23T02:33:04.980484266Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cdd29645e08dbd3e2b9a4ba3d14bc1b7187b1f12",
"ref": "refs/heads/main"
}
}
}
[02:33:05.670] Invocation finished in 5h24m5.31166041s with status FAILED