[21:04:49.342] New invocation is queued and will start shortly
[21:04:50.882] Starting the invocation (attempt 1)
[21:04:50.925] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/5bd1f43d81447ee04f133062a1cf633fc8a3f647"
[21:04:50.925] Popped gitiles commit info from properties and tags
[21:04:50.925] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:04:50.925] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:04:50.926] Buildbucket request:
{
"requestId": "9012292126384088208",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5bd1f43d81447ee04f133062a1cf633fc8a3f647",
"ref": "refs/heads/master"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "9012292126384088208"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:04:51.626] Scheduled build:
{
"id": "8846862063971714048",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 22144,
"createdBy": "project:chromium",
"createTime": "2021-05-18T21:04:51.009058Z",
"updateTime": "2021-05-18T21:04:51.525634Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5bd1f43d81447ee04f133062a1cf633fc8a3f647",
"ref": "refs/heads/master"
}
},
"canary": true
}
[21:04:51.626] Task URL: https://cr-buildbucket.appspot.com/build/8846862063971714048
[21:04:51.626] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:2:0) after 8m49s
[21:07:51.611] Received PubSub notification, asking Buildbucket for the build status
[21:07:51.640] Build status: STARTED
[21:13:40.809] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:2:0)
[21:13:40.847] Build status: STARTED
[21:13:40.847] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:4:0) after 8m41s
[21:22:21.867] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:4:0)
[21:22:21.893] Build status: STARTED
[21:22:21.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:5:0) after 2m13s
[21:24:34.994] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:5:0)
[21:24:35.021] Build status: STARTED
[21:24:35.021] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:6:0) after 4m8s
[21:28:43.067] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:6:0)
[21:28:43.097] Build status: STARTED
[21:28:43.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:7:0) after 7m15s
[21:35:58.167] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:7:0)
[21:35:58.199] Build status: STARTED
[21:35:58.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:8:0) after 2m43s
[21:38:41.247] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:8:0)
[21:38:41.295] Build status: STARTED
[21:38:41.295] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:9:0) after 1m59s
[21:40:40.633] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:9:0)
[21:40:40.684] Build status: STARTED
[21:40:40.684] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:10:0) after 1m49s
[21:42:29.949] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:10:0)
[21:42:29.990] Build status: STARTED
[21:42:29.990] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:11:0) after 1m36s
[21:44:06.157] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:11:0)
[21:44:06.186] Build status: STARTED
[21:44:06.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:12:0) after 9m22s
[21:53:28.292] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:12:0)
[21:53:28.333] Build status: STARTED
[21:53:28.333] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:13:0) after 5m35s
[21:59:03.719] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:13:0)
[21:59:03.746] Build status: STARTED
[21:59:03.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:14:0) after 8m4s
[22:07:07.916] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:14:0)
[22:07:07.964] Build status: STARTED
[22:07:07.964] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:15:0) after 8m49s
[22:15:57.985] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:15:0)
[22:15:58.014] Build status: STARTED
[22:15:58.014] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:16:0) after 6m50s
[22:22:48.041] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:16:0)
[22:22:48.084] Build status: STARTED
[22:22:48.084] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:17:0) after 5m19s
[22:28:07.108] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:17:0)
[22:28:07.141] Build status: STARTED
[22:28:07.141] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:18:0) after 9m51s
[22:37:58.316] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:18:0)
[22:37:58.369] Build status: STARTED
[22:37:58.369] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:19:0) after 9m24s
[22:47:22.607] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:19:0)
[22:47:22.644] Build status: STARTED
[22:47:22.644] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:20:0) after 4m14s
[22:51:36.752] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:20:0)
[22:51:36.789] Build status: STARTED
[22:51:36.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:21:0) after 7m14s
[22:58:50.987] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:21:0)
[22:58:51.048] Build status: STARTED
[22:58:51.048] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:22:0) after 3m38s
[23:02:29.358] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:22:0)
[23:02:29.415] Build status: STARTED
[23:02:29.415] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:23:0) after 3m19s
[23:05:48.563] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:23:0)
[23:05:48.604] Build status: STARTED
[23:05:48.604] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:24:0) after 5m27s
[23:11:15.678] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:24:0)
[23:11:15.713] Build status: STARTED
[23:11:15.713] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:25:0) after 7m28s
[23:18:44.169] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:25:0)
[23:18:44.206] Build status: STARTED
[23:18:44.206] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:26:0) after 2m42s
[23:21:26.238] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:26:0)
[23:21:26.267] Build status: STARTED
[23:21:26.267] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:27:0) after 3m24s
[23:24:50.273] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:27:0)
[23:24:50.884] Build status: STARTED
[23:24:50.884] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:28:0) after 1m24s
[23:26:14.912] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:28:0)
[23:26:14.950] Build status: STARTED
[23:26:14.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:29:0) after 9m10s
[23:35:24.977] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:29:0)
[23:35:25.006] Build status: STARTED
[23:35:25.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:30:0) after 8m56s
[23:44:21.013] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:30:0)
[23:44:21.095] Build status: STARTED
[23:44:21.095] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:31:0) after 3m39s
[23:48:00.257] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:31:0)
[23:48:00.286] Build status: STARTED
[23:48:00.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:32:0) after 5m1s
[23:53:01.372] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:32:0)
[23:53:01.402] Build status: STARTED
[23:53:01.402] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:33:0) after 9m34s
[00:02:35.714] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:33:0)
[00:02:35.754] Build status: STARTED
[00:02:35.754] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:34:0) after 2m8s
[00:04:43.780] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:34:0)
[00:04:43.807] Build status: STARTED
[00:04:43.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:35:0) after 1m43s
[00:06:26.835] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:35:0)
[00:06:26.864] Build status: STARTED
[00:06:26.864] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:36:0) after 7m31s
[00:13:57.991] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:36:0)
[00:13:58.066] Build status: STARTED
[00:13:58.066] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:37:0) after 1m49s
[00:15:47.126] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:37:0)
[00:15:47.158] Build status: STARTED
[00:15:47.158] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:38:0) after 2m2s
[00:17:49.186] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:38:0)
[00:17:49.230] Build status: STARTED
[00:17:49.230] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:39:0) after 3m33s
[00:21:22.257] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:39:0)
[00:21:22.285] Build status: STARTED
[00:21:22.285] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:40:0) after 9m2s
[00:30:24.381] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:40:0)
[00:30:24.452] Build status: STARTED
[00:30:24.452] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:41:0) after 9m27s
[00:39:51.576] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:41:0)
[00:39:51.606] Build status: STARTED
[00:39:51.607] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:42:0) after 4m8s
[00:43:59.749] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:42:0)
[00:43:59.789] Build status: STARTED
[00:43:59.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:43:0) after 6m52s
[00:50:52.126] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:43:0)
[00:50:52.184] Build status: STARTED
[00:50:52.184] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:44:0) after 1m43s
[00:52:35.212] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:44:0)
[00:52:35.249] Build status: STARTED
[00:52:35.249] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:45:0) after 3m13s
[00:55:48.491] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:45:0)
[00:55:48.522] Build status: STARTED
[00:55:48.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:46:0) after 1m15s
[00:57:03.547] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:46:0)
[00:57:03.582] Build status: STARTED
[00:57:03.582] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:47:0) after 1m59s
[00:59:02.680] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:47:0)
[00:59:02.709] Build status: STARTED
[00:59:02.709] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:48:0) after 6m36s
[01:05:38.992] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:48:0)
[01:05:39.022] Build status: STARTED
[01:05:39.022] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:49:0) after 7m44s
[01:13:23.130] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:49:0)
[01:13:23.193] Build status: STARTED
[01:13:23.193] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:50:0) after 8m2s
[01:21:25.389] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:50:0)
[01:21:25.435] Build status: STARTED
[01:21:25.435] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:51:0) after 2m25s
[01:23:50.568] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:51:0)
[01:23:50.597] Build status: STARTED
[01:23:50.597] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:52:0) after 9m8s
[01:32:58.658] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:52:0)
[01:32:58.691] Build status: STARTED
[01:32:58.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:53:0) after 6m33s
[01:39:31.765] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:53:0)
[01:39:31.794] Build status: STARTED
[01:39:31.794] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:54:0) after 9m0s
[01:48:31.891] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:54:0)
[01:48:31.922] Build status: STARTED
[01:48:31.922] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:55:0) after 3m38s
[01:52:10.209] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:55:0)
[01:52:10.243] Build status: STARTED
[01:52:10.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:56:0) after 2m56s
[01:55:06.268] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:56:0)
[01:55:06.299] Build status: STARTED
[01:55:06.299] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9012292126384088208:57:0) after 7m32s
[01:56:04.897] Received PubSub notification, asking Buildbucket for the build status
[01:56:04.931] Build:
{
"id": "8846862063971714048",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 22144,
"createdBy": "project:chromium",
"createTime": "2021-05-18T21:04:51.009058Z",
"startTime": "2021-05-18T21:07:50.505131Z",
"endTime": "2021-05-19T01:56:04.555021047Z",
"updateTime": "2021-05-19T01:56:04.555388671Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5bd1f43d81447ee04f133062a1cf633fc8a3f647",
"ref": "refs/heads/master"
}
},
"canary": true
}
[01:56:04.931] Invocation finished in 4h51m15.606166447s with status FAILED