[16:42:52.086] New invocation is queued and will start shortly
[16:42:53.161] Starting the invocation (attempt 1)
[16:42:53.192] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d8333668b11d44f13f7406065f5f97a4bb19dfe9"
[16:42:53.192] Popped gitiles commit info from properties and tags
[16:42:53.192] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[16:42:53.192] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[16:42:53.192] Buildbucket request:
{
"requestId": "9003067716546187728",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d8333668b11d44f13f7406065f5f97a4bb19dfe9",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "9003067716546187728"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[16:42:53.701] Scheduled build:
{
"id": "8837637654666837985",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23014,
"createdBy": "project:chromium",
"createTime": "2021-08-28T16:42:53.247869738Z",
"updateTime": "2021-08-28T16:42:53.556130069Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d8333668b11d44f13f7406065f5f97a4bb19dfe9",
"ref": "refs/heads/main"
}
}
}
[16:42:53.701] Task URL: https://cr-buildbucket.appspot.com/build/8837637654666837985
[16:42:53.701] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:2:0) after 3m39s
[16:45:23.457] Received PubSub notification, asking Buildbucket for the build status
[16:45:23.491] Build status: STARTED
[16:46:32.716] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:2:0)
[16:46:32.747] Build status: STARTED
[16:46:32.747] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:4:0) after 4m56s
[16:51:28.879] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:4:0)
[16:51:28.911] Build status: STARTED
[16:51:28.911] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:5:0) after 5m19s
[16:56:48.613] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:5:0)
[16:56:48.639] Build status: STARTED
[16:56:48.639] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:6:0) after 4m17s
[17:01:05.666] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:6:0)
[17:01:05.700] Build status: STARTED
[17:01:05.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:7:0) after 4m22s
[17:05:28.208] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:7:0)
[17:05:28.244] Build status: STARTED
[17:05:28.244] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:8:0) after 8m56s
[17:14:24.362] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:8:0)
[17:14:24.396] Build status: STARTED
[17:14:24.396] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:9:0) after 1m15s
[17:15:39.410] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:9:0)
[17:15:39.436] Build status: STARTED
[17:15:39.436] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:10:0) after 9m8s
[17:24:47.403] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:10:0)
[17:24:47.459] Build status: STARTED
[17:24:47.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:11:0) after 1m4s
[17:25:51.475] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:11:0)
[17:25:51.557] Build status: STARTED
[17:25:51.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:12:0) after 1m50s
[17:27:41.596] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:12:0)
[17:27:41.624] Build status: STARTED
[17:27:41.624] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:13:0) after 5m19s
[17:33:00.885] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:13:0)
[17:33:00.910] Build status: STARTED
[17:33:00.910] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:14:0) after 2m38s
[17:35:39.117] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:14:0)
[17:35:39.146] Build status: STARTED
[17:35:39.146] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:15:0) after 4m8s
[17:39:47.269] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:15:0)
[17:39:47.297] Build status: STARTED
[17:39:47.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:16:0) after 7m5s
[17:46:52.756] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:16:0)
[17:46:52.879] Build status: STARTED
[17:46:52.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:17:0) after 3m27s
[17:50:20.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:17:0)
[17:50:20.095] Build status: STARTED
[17:50:20.095] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:18:0) after 3m33s
[17:53:53.455] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:18:0)
[17:53:53.496] Build status: STARTED
[17:53:53.496] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:19:0) after 6m29s
[18:00:22.603] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:19:0)
[18:00:22.638] Build status: STARTED
[18:00:22.638] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:20:0) after 3m30s
[18:03:52.658] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:20:0)
[18:03:52.685] Build status: STARTED
[18:03:52.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:21:0) after 8m23s
[18:12:15.812] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:21:0)
[18:12:16.051] Build status: STARTED
[18:12:16.051] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:22:0) after 1m47s
[18:14:03.068] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:22:0)
[18:14:03.099] Build status: STARTED
[18:14:03.099] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:23:0) after 7m53s
[18:21:56.117] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:23:0)
[18:21:56.144] Build status: STARTED
[18:21:56.145] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:24:0) after 7m30s
[18:29:26.828] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:24:0)
[18:29:26.858] Build status: STARTED
[18:29:26.858] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:25:0) after 1m55s
[18:31:22.308] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:25:0)
[18:31:22.381] Build status: STARTED
[18:31:22.381] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:26:0) after 8m8s
[18:39:31.279] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:26:0)
[18:39:31.308] Build status: STARTED
[18:39:31.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:27:0) after 4m1s
[18:43:32.519] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:27:0)
[18:43:32.543] Build status: STARTED
[18:43:32.543] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:28:0) after 3m0s
[18:46:32.543] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:28:0)
[18:46:32.569] Build status: STARTED
[18:46:32.569] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:29:0) after 9m32s
[18:56:04.894] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:29:0)
[18:56:04.933] Build status: STARTED
[18:56:04.933] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:30:0) after 2m28s
[18:58:34.261] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:30:0)
[18:58:34.289] Build status: STARTED
[18:58:34.289] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:31:0) after 9m43s
[19:08:17.317] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:31:0)
[19:08:17.365] Build status: STARTED
[19:08:17.365] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:32:0) after 5m42s
[19:13:59.447] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:32:0)
[19:13:59.478] Build status: STARTED
[19:13:59.478] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:33:0) after 3m24s
[19:17:23.509] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:33:0)
[19:17:23.562] Build status: STARTED
[19:17:23.562] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:34:0) after 2m51s
[19:20:14.730] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:34:0)
[19:20:14.756] Build status: STARTED
[19:20:14.756] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:35:0) after 2m13s
[19:22:27.777] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:35:0)
[19:22:27.806] Build status: STARTED
[19:22:27.806] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:36:0) after 7m24s
[19:29:51.824] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:36:0)
[19:29:51.850] Build status: STARTED
[19:29:51.850] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:37:0) after 6m2s
[19:35:54.322] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:37:0)
[19:35:54.350] Build status: STARTED
[19:35:54.350] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:38:0) after 5m19s
[19:41:13.578] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:38:0)
[19:41:13.665] Build status: STARTED
[19:41:13.665] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:39:0) after 3m54s
[19:45:08.308] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:39:0)
[19:45:08.339] Build status: STARTED
[19:45:08.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:40:0) after 9m39s
[19:54:47.361] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:40:0)
[19:54:47.393] Build status: STARTED
[19:54:47.393] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:41:0) after 7m48s
[20:02:35.390] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:41:0)
[20:02:35.418] Build status: STARTED
[20:02:35.418] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:42:0) after 9m9s
[20:11:44.453] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:42:0)
[20:11:44.540] Build status: STARTED
[20:11:44.540] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:43:0) after 9m43s
[20:21:27.614] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:43:0)
[20:21:27.642] Build status: STARTED
[20:21:27.642] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:44:0) after 1m29s
[20:22:56.713] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:44:0)
[20:22:56.748] Build status: STARTED
[20:22:56.748] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:45:0) after 6m49s
[20:29:45.869] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:45:0)
[20:29:45.894] Build status: STARTED
[20:29:45.894] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:46:0) after 4m43s
[20:34:28.886] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:46:0)
[20:34:28.931] Build status: STARTED
[20:34:28.931] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:47:0) after 2m36s
[20:37:04.949] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:47:0)
[20:37:04.984] Build status: STARTED
[20:37:04.984] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:48:0) after 2m22s
[20:39:27.014] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:48:0)
[20:39:27.050] Build status: STARTED
[20:39:27.050] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:49:0) after 8m45s
[20:48:12.213] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:49:0)
[20:48:12.270] Build status: STARTED
[20:48:12.270] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:50:0) after 2m32s
[20:50:44.302] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:50:0)
[20:50:44.456] Build status: STARTED
[20:50:44.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:51:0) after 2m16s
[20:53:00.613] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:51:0)
[20:53:00.671] Build status: STARTED
[20:53:00.671] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:52:0) after 5m28s
[20:58:28.688] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:52:0)
[20:58:28.723] Build status: STARTED
[20:58:28.723] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:53:0) after 8m48s
[21:07:16.753] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:53:0)
[21:07:16.786] Build status: STARTED
[21:07:16.786] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:54:0) after 9m33s
[21:16:49.806] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:54:0)
[21:16:49.835] Build status: STARTED
[21:16:49.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:55:0) after 3m24s
[21:20:13.956] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:55:0)
[21:20:14.000] Build status: STARTED
[21:20:14.000] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:56:0) after 7m41s
[21:27:55.025] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:56:0)
[21:27:55.080] Build status: STARTED
[21:27:55.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:57:0) after 7m58s
[21:35:53.101] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:57:0)
[21:35:53.133] Build status: STARTED
[21:35:53.133] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:58:0) after 7m8s
[21:43:01.168] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:58:0)
[21:43:01.278] Build status: STARTED
[21:43:01.278] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003067716546187728:59:0) after 7m29s
[21:47:19.418] Received PubSub notification, asking Buildbucket for the build status
[21:47:19.449] Build:
{
"id": "8837637654666837985",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23014,
"createdBy": "project:chromium",
"createTime": "2021-08-28T16:42:53.247869738Z",
"startTime": "2021-08-28T16:45:22.793696Z",
"endTime": "2021-08-28T21:47:18.858547580Z",
"updateTime": "2021-08-28T21:47:18.859833107Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d8333668b11d44f13f7406065f5f97a4bb19dfe9",
"ref": "refs/heads/main"
}
}
}
[21:47:19.449] Invocation finished in 5h4m27.377339994s with status FAILED