[08:47:02.338] New invocation is queued and will start shortly
[08:47:03.643] Starting the invocation (attempt 1)
[08:47:03.700] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/81b7a7b94b9419b3c65cd9a5df7b63d5fca934dc"
[08:47:03.700] Popped gitiles commit info from properties and tags
[08:47:03.700] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[08:47:03.700] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[08:47:03.700] Buildbucket request:
{
"requestId": "9008442874155222144",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "81b7a7b94b9419b3c65cd9a5df7b63d5fca934dc",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "9008442874155222144"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[08:47:04.274] Scheduled build:
{
"id": "8843012811985973249",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 22440,
"createdBy": "project:chromium",
"createTime": "2021-06-30T08:47:03.765463694Z",
"updateTime": "2021-06-30T08:47:04.116493735Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "81b7a7b94b9419b3c65cd9a5df7b63d5fca934dc",
"ref": "refs/heads/main"
}
},
"canary": true
}
[08:47:04.274] Task URL: https://cr-buildbucket.appspot.com/build/8843012811985973249
[08:47:04.274] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:2:0) after 9m7s
[08:49:59.819] Received PubSub notification, asking Buildbucket for the build status
[08:49:59.849] Build status: STARTED
[08:56:11.299] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:2:0)
[08:56:11.335] Build status: STARTED
[08:56:11.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:4:0) after 6m22s
[09:02:33.402] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:4:0)
[09:02:33.478] Build status: STARTED
[09:02:33.478] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:5:0) after 8m9s
[09:10:42.464] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:5:0)
[09:10:42.501] Build status: STARTED
[09:10:42.501] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:6:0) after 4m48s
[09:15:30.639] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:6:0)
[09:15:30.694] Build status: STARTED
[09:15:30.694] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:7:0) after 5m33s
[09:21:03.814] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:7:0)
[09:21:03.847] Build status: STARTED
[09:21:03.847] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:8:0) after 6m40s
[09:27:43.867] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:8:0)
[09:27:43.896] Build status: STARTED
[09:27:43.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:9:0) after 1m48s
[09:29:31.931] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:9:0)
[09:29:31.965] Build status: STARTED
[09:29:31.965] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:10:0) after 8m19s
[09:37:50.985] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:10:0)
[09:37:51.016] Build status: STARTED
[09:37:51.016] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:11:0) after 1m19s
[09:39:10.041] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:11:0)
[09:39:10.079] Build status: STARTED
[09:39:10.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:12:0) after 3m43s
[09:42:53.115] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:12:0)
[09:42:53.157] Build status: STARTED
[09:42:53.157] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:13:0) after 3m36s
[09:46:29.225] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:13:0)
[09:46:29.257] Build status: STARTED
[09:46:29.257] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:14:0) after 6m33s
[09:53:02.277] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:14:0)
[09:53:02.306] Build status: STARTED
[09:53:02.306] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:15:0) after 4m29s
[09:57:31.298] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:15:0)
[09:57:31.342] Build status: STARTED
[09:57:31.342] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:16:0) after 3m56s
[10:01:27.323] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:16:0)
[10:01:27.361] Build status: STARTED
[10:01:27.361] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:17:0) after 7m0s
[10:08:27.467] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:17:0)
[10:08:27.497] Build status: STARTED
[10:08:27.497] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:18:0) after 1m42s
[10:10:09.704] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:18:0)
[10:10:09.760] Build status: STARTED
[10:10:09.760] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:19:0) after 9m34s
[10:19:43.827] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:19:0)
[10:19:43.865] Build status: STARTED
[10:19:43.865] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:20:0) after 1m39s
[10:21:22.918] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:20:0)
[10:21:22.947] Build status: STARTED
[10:21:22.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:21:0) after 2m45s
[10:24:07.978] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:21:0)
[10:24:08.011] Build status: STARTED
[10:24:08.011] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:22:0) after 7m55s
[10:32:03.151] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:22:0)
[10:32:03.263] Build status: STARTED
[10:32:03.263] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:23:0) after 6m46s
[10:38:49.372] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:23:0)
[10:38:49.403] Build status: STARTED
[10:38:49.403] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:24:0) after 9m45s
[10:48:34.512] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:24:0)
[10:48:34.540] Build status: STARTED
[10:48:34.540] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:25:0) after 5m29s
[10:54:03.609] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:25:0)
[10:54:03.639] Build status: STARTED
[10:54:03.639] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:26:0) after 3m48s
[10:57:51.691] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:26:0)
[10:57:51.727] Build status: STARTED
[10:57:51.728] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:27:0) after 2m0s
[10:59:51.771] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:27:0)
[10:59:51.799] Build status: STARTED
[10:59:51.799] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:28:0) after 5m7s
[11:04:58.823] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:28:0)
[11:04:58.855] Build status: STARTED
[11:04:58.855] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:29:0) after 6m15s
[11:11:13.880] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:29:0)
[11:11:13.911] Build status: STARTED
[11:11:13.911] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:30:0) after 7m7s
[11:18:20.945] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:30:0)
[11:18:20.979] Build status: STARTED
[11:18:20.979] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:31:0) after 8m3s
[11:26:24.006] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:31:0)
[11:26:24.135] Build status: STARTED
[11:26:24.135] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:32:0) after 3m33s
[11:29:57.225] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:32:0)
[11:29:57.250] Build status: STARTED
[11:29:57.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:33:0) after 9m33s
[11:39:30.276] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:33:0)
[11:39:30.306] Build status: STARTED
[11:39:30.306] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:34:0) after 5m17s
[11:44:47.459] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:34:0)
[11:44:47.485] Build status: STARTED
[11:44:47.485] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:35:0) after 2m43s
[11:47:30.600] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:35:0)
[11:47:30.645] Build status: STARTED
[11:47:30.645] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:36:0) after 2m48s
[11:50:18.764] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:36:0)
[11:50:18.792] Build status: STARTED
[11:50:18.792] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:37:0) after 9m57s
[12:00:15.950] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:37:0)
[12:00:15.985] Build status: STARTED
[12:00:15.985] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:38:0) after 3m19s
[12:03:35.112] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:38:0)
[12:03:35.157] Build status: STARTED
[12:03:35.157] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:39:0) after 8m44s
[12:12:19.191] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:39:0)
[12:12:19.220] Build status: STARTED
[12:12:19.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:40:0) after 4m14s
[12:16:33.244] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:40:0)
[12:16:33.283] Build status: STARTED
[12:16:33.283] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:41:0) after 1m26s
[12:17:59.316] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:41:0)
[12:17:59.366] Build status: STARTED
[12:17:59.366] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:42:0) after 3m58s
[12:21:57.396] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:42:0)
[12:21:57.474] Build status: STARTED
[12:21:57.474] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:43:0) after 1m24s
[12:23:21.506] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:43:0)
[12:23:21.549] Build status: STARTED
[12:23:21.549] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:44:0) after 9m4s
[12:32:25.658] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:44:0)
[12:32:25.726] Build status: STARTED
[12:32:25.726] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:45:0) after 1m48s
[12:34:13.864] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:45:0)
[12:34:13.957] Build status: STARTED
[12:34:13.957] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:46:0) after 1m0s
[12:35:13.983] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:46:0)
[12:35:14.020] Build status: STARTED
[12:35:14.020] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:47:0) after 8m12s
[12:43:26.054] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:47:0)
[12:43:26.087] Build status: STARTED
[12:43:26.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:48:0) after 5m21s
[12:48:47.327] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:48:0)
[12:48:47.358] Build status: STARTED
[12:48:47.358] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:49:0) after 9m32s
[12:58:19.387] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:49:0)
[12:58:19.434] Build status: STARTED
[12:58:19.434] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:50:0) after 1m6s
[12:59:25.465] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:50:0)
[12:59:25.502] Build status: STARTED
[12:59:25.502] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:51:0) after 9m0s
[13:08:25.671] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:51:0)
[13:08:25.707] Build status: STARTED
[13:08:25.707] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:52:0) after 5m34s
[13:13:59.769] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:52:0)
[13:13:59.804] Build status: STARTED
[13:13:59.804] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:53:0) after 7m2s
[13:21:01.851] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:53:0)
[13:21:01.882] Build status: STARTED
[13:21:01.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:54:0) after 2m57s
[13:23:59.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:54:0)
[13:23:59.052] Build status: STARTED
[13:23:59.052] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:55:0) after 5m45s
[13:29:44.082] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:55:0)
[13:29:44.129] Build status: STARTED
[13:29:44.129] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:56:0) after 2m57s
[13:32:41.214] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:56:0)
[13:32:41.249] Build status: STARTED
[13:32:41.249] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:57:0) after 4m30s
[13:37:11.275] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:57:0)
[13:37:11.302] Build status: STARTED
[13:37:11.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:58:0) after 4m19s
[13:41:30.562] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:58:0)
[13:41:30.592] Build status: STARTED
[13:41:30.592] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:59:0) after 2m23s
[13:43:53.631] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:59:0)
[13:43:53.667] Build status: STARTED
[13:43:53.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:60:0) after 8m31s
[13:52:24.812] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:60:0)
[13:52:24.841] Build status: STARTED
[13:52:24.841] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:61:0) after 7m48s
[14:00:12.957] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:61:0)
[14:00:13.021] Build status: STARTED
[14:00:13.021] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:62:0) after 1m40s
[14:01:53.390] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:62:0)
[14:01:53.419] Build status: STARTED
[14:01:53.419] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9008442874155222144:63:0) after 4m0s
[14:03:15.016] Received PubSub notification, asking Buildbucket for the build status
[14:03:15.067] Build:
{
"id": "8843012811985973249",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 22440,
"createdBy": "project:chromium",
"createTime": "2021-06-30T08:47:03.765463694Z",
"startTime": "2021-06-30T08:49:59.419676Z",
"endTime": "2021-06-30T14:03:14.603749717Z",
"updateTime": "2021-06-30T14:03:14.604276254Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "81b7a7b94b9419b3c65cd9a5df7b63d5fca934dc",
"ref": "refs/heads/main"
}
},
"canary": true
}
[14:03:15.067] Invocation finished in 5h16m12.753492338s with status FAILED