[19:42:05.094] New invocation is queued and will start shortly
[19:42:06.214] Starting the invocation (attempt 1)
[19:42:06.245] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0711a2656d3e9b88f59e7902e12abe05b8e503cb"
[19:42:06.245] Popped gitiles commit info from properties and tags
[19:42:06.245] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[19:42:06.245] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[19:42:06.245] Buildbucket request:
{
"requestId": "8995989877822169248",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0711a2656d3e9b88f59e7902e12abe05b8e503cb",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8995989877822169248"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[19:42:06.746] Scheduled build:
{
"id": "8830559815882868961",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23557,
"createdBy": "project:chromium",
"createTime": "2021-11-14T19:42:06.311013818Z",
"updateTime": "2021-11-14T19:42:06.311013818Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0711a2656d3e9b88f59e7902e12abe05b8e503cb",
"ref": "refs/heads/main"
}
},
"canary": true
}
[19:42:06.746] Task URL: https://cr-buildbucket.appspot.com/build/8830559815882868961
[19:42:06.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:2:0) after 2m3s
[19:44:09.966] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:2:0)
[19:44:09.994] Build status: SCHEDULED
[19:44:09.994] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:3:0) after 4m52s
[19:44:21.259] Received PubSub notification, asking Buildbucket for the build status
[19:44:21.308] Build status: STARTED
[19:49:01.989] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:3:0)
[19:49:02.021] Build status: STARTED
[19:49:02.021] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:5:0) after 4m52s
[19:53:54.046] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:5:0)
[19:53:54.081] Build status: STARTED
[19:53:54.081] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:6:0) after 5m22s
[19:59:16.777] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:6:0)
[19:59:16.812] Build status: STARTED
[19:59:16.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:7:0) after 6m25s
[20:05:41.840] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:7:0)
[20:05:41.880] Build status: STARTED
[20:05:41.880] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:8:0) after 8m48s
[20:14:30.514] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:8:0)
[20:14:30.548] Build status: STARTED
[20:14:30.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:9:0) after 2m14s
[20:16:44.961] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:9:0)
[20:16:44.994] Build status: STARTED
[20:16:44.994] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:10:0) after 7m20s
[20:24:05.017] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:10:0)
[20:24:05.084] Build status: STARTED
[20:24:05.084] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:11:0) after 1m27s
[20:25:32.379] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:11:0)
[20:25:32.410] Build status: STARTED
[20:25:32.410] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:12:0) after 3m23s
[20:28:55.980] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:12:0)
[20:28:56.009] Build status: STARTED
[20:28:56.009] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:13:0) after 2m56s
[20:31:52.667] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:13:0)
[20:31:53.004] Build status: STARTED
[20:31:53.004] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:14:0) after 4m50s
[20:36:43.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:14:0)
[20:36:43.058] Build status: STARTED
[20:36:43.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:15:0) after 1m59s
[20:38:42.085] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:15:0)
[20:38:42.118] Build status: STARTED
[20:38:42.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:16:0) after 5m1s
[20:43:43.239] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:16:0)
[20:43:43.272] Build status: STARTED
[20:43:43.272] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:17:0) after 7m54s
[20:51:37.393] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:17:0)
[20:51:37.422] Build status: STARTED
[20:51:37.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:18:0) after 8m18s
[20:59:55.442] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:18:0)
[20:59:55.468] Build status: STARTED
[20:59:55.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:19:0) after 9m54s
[21:09:49.546] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:19:0)
[21:09:49.609] Build status: STARTED
[21:09:49.609] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:20:0) after 4m44s
[21:14:33.637] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:20:0)
[21:14:33.732] Build status: STARTED
[21:14:33.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:21:0) after 3m24s
[21:17:57.851] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:21:0)
[21:17:57.886] Build status: STARTED
[21:17:57.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:22:0) after 1m30s
[21:19:27.958] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:22:0)
[21:19:27.990] Build status: STARTED
[21:19:27.990] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:23:0) after 7m31s
[21:26:59.011] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:23:0)
[21:26:59.143] Build status: STARTED
[21:26:59.143] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:24:0) after 4m15s
[21:31:14.288] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:24:0)
[21:31:14.345] Build status: STARTED
[21:31:14.345] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:25:0) after 6m2s
[21:37:16.850] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:25:0)
[21:37:16.881] Build status: STARTED
[21:37:16.881] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:26:0) after 5m35s
[21:42:51.902] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:26:0)
[21:42:51.937] Build status: STARTED
[21:42:51.938] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:27:0) after 1m52s
[21:44:43.956] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:27:0)
[21:44:43.991] Build status: STARTED
[21:44:43.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:28:0) after 7m49s
[21:52:33.012] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:28:0)
[21:52:33.083] Build status: STARTED
[21:52:33.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:29:0) after 3m40s
[21:56:13.102] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:29:0)
[21:56:13.134] Build status: STARTED
[21:56:13.134] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:30:0) after 9m50s
[22:06:03.154] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:30:0)
[22:06:03.181] Build status: STARTED
[22:06:03.181] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:31:0) after 8m28s
[22:14:31.399] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:31:0)
[22:14:31.443] Build status: STARTED
[22:14:31.443] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:32:0) after 3m49s
[22:18:20.463] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:32:0)
[22:18:20.488] Build status: STARTED
[22:18:20.488] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:33:0) after 7m56s
[22:26:16.510] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:33:0)
[22:26:16.547] Build status: STARTED
[22:26:16.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:34:0) after 1m51s
[22:28:07.794] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:34:0)
[22:28:07.831] Build status: STARTED
[22:28:07.831] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:35:0) after 6m54s
[22:35:01.856] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:35:0)
[22:35:01.889] Build status: STARTED
[22:35:01.889] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:36:0) after 2m6s
[22:37:08.371] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:36:0)
[22:37:08.446] Build status: STARTED
[22:37:08.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:37:0) after 5m41s
[22:42:49.785] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:37:0)
[22:42:49.816] Build status: STARTED
[22:42:49.816] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:38:0) after 2m18s
[22:45:07.939] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:38:0)
[22:45:07.976] Build status: STARTED
[22:45:07.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:39:0) after 2m32s
[22:47:40.004] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:39:0)
[22:47:40.032] Build status: STARTED
[22:47:40.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:40:0) after 3m35s
[22:51:15.089] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:40:0)
[22:51:15.169] Build status: STARTED
[22:51:15.169] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:41:0) after 1m5s
[22:52:20.187] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:41:0)
[22:52:20.213] Build status: STARTED
[22:52:20.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:42:0) after 9m9s
[23:01:29.186] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:42:0)
[23:01:29.212] Build status: STARTED
[23:01:29.212] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:43:0) after 1m2s
[23:02:31.231] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:43:0)
[23:02:31.261] Build status: STARTED
[23:02:31.261] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:44:0) after 4m23s
[23:06:54.281] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:44:0)
[23:06:54.310] Build status: STARTED
[23:06:54.310] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:45:0) after 1m38s
[23:08:32.484] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:45:0)
[23:08:32.584] Build status: STARTED
[23:08:32.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:46:0) after 6m59s
[23:15:32.267] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:46:0)
[23:15:32.297] Build status: STARTED
[23:15:32.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:47:0) after 5m37s
[23:21:09.550] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:47:0)
[23:21:09.580] Build status: STARTED
[23:21:09.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:48:0) after 1m35s
[23:22:44.691] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:48:0)
[23:22:44.747] Build status: STARTED
[23:22:44.748] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:49:0) after 2m57s
[23:25:41.848] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:49:0)
[23:25:41.877] Build status: STARTED
[23:25:41.877] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:50:0) after 5m53s
[23:31:35.104] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:50:0)
[23:31:35.141] Build status: STARTED
[23:31:35.141] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:51:0) after 9m2s
[23:40:37.693] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:51:0)
[23:40:37.736] Build status: STARTED
[23:40:37.736] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:52:0) after 5m26s
[23:46:03.958] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:52:0)
[23:46:03.992] Build status: STARTED
[23:46:03.992] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:53:0) after 5m56s
[23:52:00.011] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:53:0)
[23:52:00.040] Build status: STARTED
[23:52:00.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:54:0) after 3m21s
[23:55:21.070] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:54:0)
[23:55:21.099] Build status: STARTED
[23:55:21.099] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:55:0) after 8m56s
[00:04:17.263] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:55:0)
[00:04:17.307] Build status: STARTED
[00:04:17.307] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:56:0) after 4m27s
[00:08:44.669] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:56:0)
[00:08:44.702] Build status: STARTED
[00:08:44.703] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:57:0) after 7m36s
[00:16:20.731] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:57:0)
[00:16:20.794] Build status: STARTED
[00:16:20.794] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:58:0) after 1m0s
[00:17:20.842] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:58:0)
[00:17:20.873] Build status: STARTED
[00:17:20.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:59:0) after 9m1s
[00:26:21.903] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:59:0)
[00:26:21.936] Build status: STARTED
[00:26:21.936] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:60:0) after 6m6s
[00:32:27.957] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:60:0)
[00:32:27.987] Build status: STARTED
[00:32:27.987] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:61:0) after 7m14s
[00:39:42.007] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:61:0)
[00:39:42.040] Build status: STARTED
[00:39:42.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:62:0) after 5m6s
[00:44:48.260] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:62:0)
[00:44:48.293] Build status: STARTED
[00:44:48.293] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:63:0) after 4m4s
[00:48:52.337] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:63:0)
[00:48:52.434] Build status: STARTED
[00:48:52.435] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:64:0) after 9m24s
[00:58:16.554] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:64:0)
[00:58:16.584] Build status: STARTED
[00:58:16.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995989877822169248:65:0) after 7m4s
[01:00:25.207] Received PubSub notification, asking Buildbucket for the build status
[01:00:25.238] Build:
{
"id": "8830559815882868961",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23557,
"createdBy": "project:chromium",
"createTime": "2021-11-14T19:42:06.311013818Z",
"startTime": "2021-11-14T19:44:20.329797Z",
"endTime": "2021-11-15T01:00:24.863315198Z",
"updateTime": "2021-11-15T01:00:24.863315198Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0711a2656d3e9b88f59e7902e12abe05b8e503cb",
"ref": "refs/heads/main"
}
},
"canary": true
}
[01:00:25.238] Invocation finished in 5h18m20.159574768s with status FAILED