[21:09:10.313] New invocation is queued and will start shortly
[21:09:11.509] Starting the invocation (attempt 1)
[21:09:11.541] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/003f58dd6b2a711cb43613f96141265f079a23d4"
[21:09:11.541] Popped gitiles commit info from properties and tags
[21:09:11.541] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:09:11.541] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:09:11.541] Buildbucket request:
{
"requestId": "8999608277528161776",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "003f58dd6b2a711cb43613f96141265f079a23d4",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8999608277528161776"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:09:12.151] Scheduled build:
{
"id": "8834178215417618689",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23307,
"createdBy": "project:chromium",
"createTime": "2021-10-05T21:09:11.608580930Z",
"updateTime": "2021-10-05T21:09:11.891457272Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "003f58dd6b2a711cb43613f96141265f079a23d4",
"ref": "refs/heads/main"
}
}
}
[21:09:12.151] Task URL: https://cr-buildbucket.appspot.com/build/8834178215417618689
[21:09:12.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:2:0) after 5m6s
[21:11:26.579] Received PubSub notification, asking Buildbucket for the build status
[21:11:26.609] Build status: STARTED
[21:14:18.184] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:2:0)
[21:14:18.214] Build status: STARTED
[21:14:18.214] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:4:0) after 5m26s
[21:19:44.302] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:4:0)
[21:19:44.335] Build status: STARTED
[21:19:44.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:5:0) after 6m33s
[21:26:17.545] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:5:0)
[21:26:17.582] Build status: STARTED
[21:26:17.582] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:6:0) after 5m28s
[21:31:45.604] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:6:0)
[21:31:45.791] Build status: STARTED
[21:31:45.791] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:7:0) after 2m1s
[21:33:46.923] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:7:0)
[21:33:46.953] Build status: STARTED
[21:33:46.953] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:8:0) after 2m51s
[21:36:38.085] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:8:0)
[21:36:38.132] Build status: STARTED
[21:36:38.132] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:9:0) after 2m37s
[21:39:15.261] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:9:0)
[21:39:15.289] Build status: STARTED
[21:39:15.289] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:10:0) after 3m27s
[21:42:42.444] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:10:0)
[21:42:42.483] Build status: STARTED
[21:42:42.483] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:11:0) after 1m57s
[21:44:39.510] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:11:0)
[21:44:39.539] Build status: STARTED
[21:44:39.539] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:12:0) after 8m20s
[21:52:59.588] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:12:0)
[21:52:59.633] Build status: STARTED
[21:52:59.633] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:13:0) after 1m28s
[21:54:28.001] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:13:0)
[21:54:28.031] Build status: STARTED
[21:54:28.031] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:14:0) after 2m50s
[21:57:18.056] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:14:0)
[21:57:18.092] Build status: STARTED
[21:57:18.092] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:15:0) after 9m47s
[22:07:05.469] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:15:0)
[22:07:05.517] Build status: STARTED
[22:07:05.517] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:16:0) after 6m50s
[22:13:55.908] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:16:0)
[22:13:56.068] Build status: STARTED
[22:13:56.068] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:17:0) after 5m0s
[22:18:56.160] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:17:0)
[22:18:56.194] Build status: STARTED
[22:18:56.194] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:18:0) after 1m23s
[22:20:19.216] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:18:0)
[22:20:19.244] Build status: STARTED
[22:20:19.244] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:19:0) after 4m18s
[22:24:37.389] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:19:0)
[22:24:37.430] Build status: STARTED
[22:24:37.430] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:20:0) after 4m10s
[22:28:47.692] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:20:0)
[22:28:47.771] Build status: STARTED
[22:28:47.771] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:21:0) after 9m7s
[22:37:55.214] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:21:0)
[22:37:55.246] Build status: STARTED
[22:37:55.246] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:22:0) after 1m24s
[22:39:19.853] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:22:0)
[22:39:19.881] Build status: STARTED
[22:39:19.881] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:23:0) after 4m7s
[22:43:27.165] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:23:0)
[22:43:27.196] Build status: STARTED
[22:43:27.196] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:24:0) after 8m12s
[22:51:39.481] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:24:0)
[22:51:39.513] Build status: STARTED
[22:51:39.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:25:0) after 6m18s
[22:57:57.674] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:25:0)
[22:57:57.711] Build status: STARTED
[22:57:57.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:26:0) after 8m32s
[23:06:29.800] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:26:0)
[23:06:29.835] Build status: STARTED
[23:06:29.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:27:0) after 8m45s
[23:15:14.860] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:27:0)
[23:15:14.942] Build status: STARTED
[23:15:14.942] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:28:0) after 9m57s
[23:25:12.229] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:28:0)
[23:25:12.266] Build status: STARTED
[23:25:12.266] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:29:0) after 7m49s
[23:33:01.297] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:29:0)
[23:33:01.327] Build status: STARTED
[23:33:01.327] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:30:0) after 8m15s
[23:41:16.460] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:30:0)
[23:41:16.490] Build status: STARTED
[23:41:16.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:31:0) after 4m28s
[23:45:44.700] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:31:0)
[23:45:44.731] Build status: STARTED
[23:45:44.731] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:32:0) after 1m45s
[23:47:29.801] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:32:0)
[23:47:29.850] Build status: STARTED
[23:47:29.850] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:33:0) after 9m42s
[23:57:11.876] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:33:0)
[23:57:11.903] Build status: STARTED
[23:57:11.903] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:34:0) after 5m58s
[00:03:10.124] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:34:0)
[00:03:10.156] Build status: STARTED
[00:03:10.156] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:35:0) after 2m51s
[00:06:01.504] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:35:0)
[00:06:01.545] Build status: STARTED
[00:06:01.545] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:36:0) after 3m38s
[00:09:39.610] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:36:0)
[00:09:39.662] Build status: STARTED
[00:09:39.662] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:37:0) after 3m42s
[00:13:22.274] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:37:0)
[00:13:22.307] Build status: STARTED
[00:13:22.307] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:38:0) after 1m59s
[00:15:21.471] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:38:0)
[00:15:21.508] Build status: STARTED
[00:15:21.508] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:39:0) after 7m47s
[00:23:08.540] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:39:0)
[00:23:08.575] Build status: STARTED
[00:23:08.575] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:40:0) after 8m22s
[00:31:30.622] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:40:0)
[00:31:30.693] Build status: STARTED
[00:31:30.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:41:0) after 3m3s
[00:34:33.772] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:41:0)
[00:34:33.808] Build status: STARTED
[00:34:33.808] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:42:0) after 1m36s
[00:36:09.952] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:42:0)
[00:36:10.048] Build status: STARTED
[00:36:10.048] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:43:0) after 1m48s
[00:37:58.252] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:43:0)
[00:37:58.318] Build status: STARTED
[00:37:58.318] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:44:0) after 3m45s
[00:41:43.727] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:44:0)
[00:41:43.758] Build status: STARTED
[00:41:43.758] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:45:0) after 6m53s
[00:48:36.854] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:45:0)
[00:48:36.883] Build status: STARTED
[00:48:36.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:46:0) after 2m17s
[00:50:53.917] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:46:0)
[00:50:53.968] Build status: STARTED
[00:50:53.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:47:0) after 1m59s
[00:52:53.010] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:47:0)
[00:52:53.051] Build status: STARTED
[00:52:53.051] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:48:0) after 4m29s
[00:57:22.169] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:48:0)
[00:57:22.208] Build status: STARTED
[00:57:22.208] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:49:0) after 4m2s
[01:01:24.322] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:49:0)
[01:01:24.353] Build status: STARTED
[01:01:24.353] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:50:0) after 6m25s
[01:07:49.382] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:50:0)
[01:07:49.413] Build status: STARTED
[01:07:49.413] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:51:0) after 1m42s
[01:09:31.579] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:51:0)
[01:09:31.610] Build status: STARTED
[01:09:31.610] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:52:0) after 2m15s
[01:11:47.269] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:52:0)
[01:11:47.308] Build status: STARTED
[01:11:47.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:53:0) after 4m1s
[01:15:48.662] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:53:0)
[01:15:48.791] Build status: STARTED
[01:15:48.791] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:54:0) after 5m37s
[01:21:25.889] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:54:0)
[01:21:25.920] Build status: STARTED
[01:21:25.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:55:0) after 4m4s
[01:25:29.978] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:55:0)
[01:25:30.005] Build status: STARTED
[01:25:30.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:56:0) after 6m12s
[01:31:42.462] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:56:0)
[01:31:42.491] Build status: STARTED
[01:31:42.491] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:57:0) after 7m27s
[01:39:09.569] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:57:0)
[01:39:09.669] Build status: STARTED
[01:39:09.669] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:58:0) after 2m51s
[01:42:00.697] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:58:0)
[01:42:00.726] Build status: STARTED
[01:42:00.726] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:59:0) after 8m2s
[01:50:02.718] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:59:0)
[01:50:02.806] Build status: STARTED
[01:50:02.806] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:60:0) after 8m28s
[01:58:30.862] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:60:0)
[01:58:30.905] Build status: STARTED
[01:58:30.905] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:61:0) after 2m44s
[02:01:14.930] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:61:0)
[02:01:14.977] Build status: STARTED
[02:01:14.977] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:62:0) after 9m4s
[02:10:19.101] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:62:0)
[02:10:19.185] Build status: STARTED
[02:10:19.185] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:63:0) after 1m30s
[02:11:49.207] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:63:0)
[02:11:49.241] Build status: STARTED
[02:11:49.241] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:64:0) after 9m32s
[02:21:21.531] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:64:0)
[02:21:21.559] Build status: STARTED
[02:21:21.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999608277528161776:65:0) after 9m9s
[02:29:33.877] Received PubSub notification, asking Buildbucket for the build status
[02:29:33.905] Build:
{
"id": "8834178215417618689",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23307,
"createdBy": "project:chromium",
"createTime": "2021-10-05T21:09:11.608580930Z",
"startTime": "2021-10-05T21:11:25.827310Z",
"endTime": "2021-10-06T02:29:33.558272650Z",
"updateTime": "2021-10-06T02:29:33.558604506Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "003f58dd6b2a711cb43613f96141265f079a23d4",
"ref": "refs/heads/main"
}
}
}
[02:29:33.905] Invocation finished in 5h20m23.692708323s with status FAILED