[23:07:30.302] New invocation is queued and will start shortly
[23:07:31.371] Starting the invocation (attempt 1)
[23:07:31.399] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/df7874fb2460ecd589d68ae25ddcafbd86aab4ed"
[23:07:31.399] Popped gitiles commit info from properties and tags
[23:07:31.399] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[23:07:31.399] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[23:07:31.399] Buildbucket request:
{
"requestId": "9001775159757439808",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "df7874fb2460ecd589d68ae25ddcafbd86aab4ed",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "9001775159757439808"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[23:07:31.933] Scheduled build:
{
"id": "8836345097871583073",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23079,
"createdBy": "project:chromium",
"createTime": "2021-09-11T23:07:31.465375762Z",
"updateTime": "2021-09-11T23:07:31.756993254Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "df7874fb2460ecd589d68ae25ddcafbd86aab4ed",
"ref": "refs/heads/main"
}
},
"canary": true
}
[23:07:31.933] Task URL: https://cr-buildbucket.appspot.com/build/8836345097871583073
[23:07:31.933] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:2:0) after 5m6s
[23:11:14.927] Received PubSub notification, asking Buildbucket for the build status
[23:11:14.956] Build status: STARTED
[23:12:38.132] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:2:0)
[23:12:38.161] Build status: STARTED
[23:12:38.161] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:4:0) after 6m10s
[23:18:48.197] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:4:0)
[23:18:48.233] Build status: STARTED
[23:18:48.233] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:5:0) after 9m52s
[23:28:40.631] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:5:0)
[23:28:40.658] Build status: STARTED
[23:28:40.658] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:6:0) after 7m27s
[23:36:08.144] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:6:0)
[23:36:08.183] Build status: STARTED
[23:36:08.183] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:7:0) after 7m9s
[23:43:17.374] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:7:0)
[23:43:17.564] Build status: STARTED
[23:43:17.564] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:8:0) after 6m5s
[23:49:23.161] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:8:0)
[23:49:23.192] Build status: STARTED
[23:49:23.192] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:9:0) after 6m17s
[23:55:40.532] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:9:0)
[23:55:40.574] Build status: STARTED
[23:55:40.574] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:10:0) after 7m1s
[00:02:41.608] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:10:0)
[00:02:41.640] Build status: STARTED
[00:02:41.640] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:11:0) after 3m22s
[00:06:03.949] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:11:0)
[00:06:04.004] Build status: STARTED
[00:06:04.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:12:0) after 7m35s
[00:13:39.022] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:12:0)
[00:13:39.051] Build status: STARTED
[00:13:39.051] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:13:0) after 3m4s
[00:16:43.070] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:13:0)
[00:16:43.109] Build status: STARTED
[00:16:43.109] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:14:0) after 8m41s
[00:25:24.703] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:14:0)
[00:25:24.738] Build status: STARTED
[00:25:24.738] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:15:0) after 8m46s
[00:34:10.804] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:15:0)
[00:34:10.839] Build status: STARTED
[00:34:10.839] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:16:0) after 5m41s
[00:39:51.935] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:16:0)
[00:39:51.965] Build status: STARTED
[00:39:51.965] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:17:0) after 4m44s
[00:44:35.991] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:17:0)
[00:44:36.023] Build status: STARTED
[00:44:36.023] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:18:0) after 5m39s
[00:50:15.046] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:18:0)
[00:50:15.166] Build status: STARTED
[00:50:15.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:19:0) after 7m29s
[00:57:44.404] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:19:0)
[00:57:44.436] Build status: STARTED
[00:57:44.436] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:20:0) after 9m9s
[01:06:53.558] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:20:0)
[01:06:53.585] Build status: STARTED
[01:06:53.585] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:21:0) after 1m30s
[01:08:23.812] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:21:0)
[01:08:23.848] Build status: STARTED
[01:08:23.848] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:22:0) after 3m21s
[01:11:45.033] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:22:0)
[01:11:45.066] Build status: STARTED
[01:11:45.066] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:23:0) after 1m44s
[01:13:29.088] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:23:0)
[01:13:29.116] Build status: STARTED
[01:13:29.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:24:0) after 4m9s
[01:17:38.158] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:24:0)
[01:17:38.191] Build status: STARTED
[01:17:38.191] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:25:0) after 4m44s
[01:22:22.558] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:25:0)
[01:22:22.587] Build status: STARTED
[01:22:22.588] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:26:0) after 2m31s
[01:24:53.605] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:26:0)
[01:24:53.635] Build status: STARTED
[01:24:53.635] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:27:0) after 3m13s
[01:28:06.711] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:27:0)
[01:28:06.787] Build status: STARTED
[01:28:06.787] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:28:0) after 2m17s
[01:30:23.804] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:28:0)
[01:30:23.829] Build status: STARTED
[01:30:23.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:29:0) after 4m6s
[01:34:29.864] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:29:0)
[01:34:29.893] Build status: STARTED
[01:34:29.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:30:0) after 9m47s
[01:44:16.924] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:30:0)
[01:44:16.954] Build status: STARTED
[01:44:16.954] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:31:0) after 3m33s
[01:47:49.975] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:31:0)
[01:47:50.103] Build status: STARTED
[01:47:50.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:32:0) after 1m3s
[01:48:53.124] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:32:0)
[01:48:53.177] Build status: STARTED
[01:48:53.177] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:33:0) after 1m26s
[01:50:19.198] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:33:0)
[01:50:19.234] Build status: STARTED
[01:50:19.234] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:34:0) after 9m25s
[01:59:44.441] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:34:0)
[01:59:44.473] Build status: STARTED
[01:59:44.473] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:35:0) after 9m56s
[02:09:40.501] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:35:0)
[02:09:40.557] Build status: STARTED
[02:09:40.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:36:0) after 8m11s
[02:17:51.582] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:36:0)
[02:17:51.871] Build status: STARTED
[02:17:51.871] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:37:0) after 1m8s
[02:18:59.894] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:37:0)
[02:18:59.933] Build status: STARTED
[02:18:59.933] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:38:0) after 3m23s
[02:22:22.951] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:38:0)
[02:22:23.006] Build status: STARTED
[02:22:23.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:39:0) after 4m58s
[02:27:21.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:39:0)
[02:27:21.053] Build status: STARTED
[02:27:21.053] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:40:0) after 5m58s
[02:33:19.211] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:40:0)
[02:33:19.250] Build status: STARTED
[02:33:19.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:41:0) after 3m14s
[02:36:33.288] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:41:0)
[02:36:33.321] Build status: STARTED
[02:36:33.321] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:42:0) after 4m48s
[02:41:21.339] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:42:0)
[02:41:21.376] Build status: STARTED
[02:41:21.376] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:43:0) after 3m54s
[02:45:15.400] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:43:0)
[02:45:15.427] Build status: STARTED
[02:45:15.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:44:0) after 3m41s
[02:48:56.540] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:44:0)
[02:48:56.581] Build status: STARTED
[02:48:56.582] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:45:0) after 7m21s
[02:56:17.601] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:45:0)
[02:56:17.631] Build status: STARTED
[02:56:17.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:46:0) after 5m53s
[03:02:10.651] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:46:0)
[03:02:10.707] Build status: STARTED
[03:02:10.707] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:47:0) after 3m53s
[03:06:03.725] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:47:0)
[03:06:03.754] Build status: STARTED
[03:06:03.754] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:48:0) after 9m37s
[03:15:40.798] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:48:0)
[03:15:40.834] Build status: STARTED
[03:15:40.834] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:49:0) after 1m16s
[03:16:56.917] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:49:0)
[03:16:56.958] Build status: STARTED
[03:16:56.958] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:50:0) after 3m12s
[03:20:08.949] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:50:0)
[03:20:08.986] Build status: STARTED
[03:20:08.987] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:51:0) after 4m55s
[03:25:04.169] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:51:0)
[03:25:04.241] Build status: STARTED
[03:25:04.241] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:52:0) after 6m21s
[03:31:25.362] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:52:0)
[03:31:25.406] Build status: STARTED
[03:31:25.406] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:53:0) after 9m7s
[03:40:32.426] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:53:0)
[03:40:32.472] Build status: STARTED
[03:40:32.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:54:0) after 7m13s
[03:47:45.491] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:54:0)
[03:47:45.528] Build status: STARTED
[03:47:45.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:55:0) after 3m18s
[03:51:03.576] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:55:0)
[03:51:03.604] Build status: STARTED
[03:51:03.604] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:56:0) after 5m12s
[03:56:15.602] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:56:0)
[03:56:15.755] Build status: STARTED
[03:56:15.755] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:57:0) after 4m46s
[04:01:01.897] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:57:0)
[04:01:01.923] Build status: STARTED
[04:01:01.923] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:58:0) after 6m17s
[04:07:18.951] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:58:0)
[04:07:18.986] Build status: STARTED
[04:07:18.986] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:59:0) after 5m5s
[04:12:24.003] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:59:0)
[04:12:24.039] Build status: STARTED
[04:12:24.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:60:0) after 4m27s
[04:16:51.112] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:60:0)
[04:16:51.147] Build status: STARTED
[04:16:51.148] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:61:0) after 9m44s
[04:26:35.385] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:61:0)
[04:26:35.422] Build status: STARTED
[04:26:35.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9001775159757439808:62:0) after 4m32s
[04:29:48.299] Received PubSub notification, asking Buildbucket for the build status
[04:29:48.326] Build:
{
"id": "8836345097871583073",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23079,
"createdBy": "project:chromium",
"createTime": "2021-09-11T23:07:31.465375762Z",
"startTime": "2021-09-11T23:11:14.197561Z",
"endTime": "2021-09-12T04:29:47.876699434Z",
"updateTime": "2021-09-12T04:29:47.877454262Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "df7874fb2460ecd589d68ae25ddcafbd86aab4ed",
"ref": "refs/heads/main"
}
},
"canary": true
}
[04:29:48.326] Invocation finished in 5h22m18.042824976s with status FAILED