[02:36:46.552] New invocation is queued and will start shortly
[02:36:47.660] Starting the invocation (attempt 1)
[02:36:47.690] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/4b8a565314a7669a515e3662029a8086fff67ab3"
[02:36:47.690] Popped gitiles commit info from properties and tags
[02:36:47.690] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[02:36:47.690] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[02:36:47.690] Buildbucket request:
{
"requestId": "9006744826721850928",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4b8a565314a7669a515e3662029a8086fff67ab3",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "9006744826721850928"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[02:36:48.550] Scheduled build:
{
"id": "8841314764750590896",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 22564,
"createdBy": "project:chromium",
"createTime": "2021-07-19T02:36:47.801183Z",
"updateTime": "2021-07-19T02:36:48.469264Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4b8a565314a7669a515e3662029a8086fff67ab3",
"ref": "refs/heads/main"
}
}
}
[02:36:48.550] Task URL: https://cr-buildbucket.appspot.com/build/8841314764750590896
[02:36:48.550] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:2:0) after 4m54s
[02:40:18.804] Received PubSub notification, asking Buildbucket for the build status
[02:40:18.830] Build status: STARTED
[02:41:42.877] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:2:0)
[02:41:42.912] Build status: STARTED
[02:41:42.912] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:4:0) after 3m39s
[02:45:22.033] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:4:0)
[02:45:22.075] Build status: STARTED
[02:45:22.075] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:5:0) after 2m54s
[02:48:16.235] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:5:0)
[02:48:16.265] Build status: STARTED
[02:48:16.265] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:6:0) after 3m23s
[02:51:39.280] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:6:0)
[02:51:39.323] Build status: STARTED
[02:51:39.323] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:7:0) after 9m26s
[03:01:05.342] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:7:0)
[03:01:05.373] Build status: STARTED
[03:01:05.373] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:8:0) after 8m4s
[03:09:09.391] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:8:0)
[03:09:09.423] Build status: STARTED
[03:09:09.423] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:9:0) after 6m13s
[03:15:22.623] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:9:0)
[03:15:22.658] Build status: STARTED
[03:15:22.658] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:10:0) after 9m34s
[03:24:56.678] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:10:0)
[03:24:56.710] Build status: STARTED
[03:24:56.710] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:11:0) after 6m23s
[03:31:20.494] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:11:0)
[03:31:20.532] Build status: STARTED
[03:31:20.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:12:0) after 2m36s
[03:33:56.548] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:12:0)
[03:33:56.586] Build status: STARTED
[03:33:56.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:13:0) after 7m29s
[03:41:25.707] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:13:0)
[03:41:26.374] Build status: STARTED
[03:41:26.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:14:0) after 6m56s
[03:48:22.392] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:14:0)
[03:48:22.425] Build status: STARTED
[03:48:22.425] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:15:0) after 4m18s
[03:52:40.447] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:15:0)
[03:52:40.472] Build status: STARTED
[03:52:40.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:16:0) after 6m27s
[03:59:07.493] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:16:0)
[03:59:07.520] Build status: STARTED
[03:59:07.520] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:17:0) after 4m20s
[04:03:27.547] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:17:0)
[04:03:27.585] Build status: STARTED
[04:03:27.585] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:18:0) after 6m17s
[04:09:44.636] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:18:0)
[04:09:44.673] Build status: STARTED
[04:09:44.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:19:0) after 9m49s
[04:19:33.642] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:19:0)
[04:19:33.772] Build status: STARTED
[04:19:33.772] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:20:0) after 7m41s
[04:27:14.820] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:20:0)
[04:27:14.904] Build status: STARTED
[04:27:14.904] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:21:0) after 4m15s
[04:31:29.965] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:21:0)
[04:31:29.993] Build status: STARTED
[04:31:29.993] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:22:0) after 5m0s
[04:36:30.013] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:22:0)
[04:36:30.044] Build status: STARTED
[04:36:30.044] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:23:0) after 4m7s
[04:40:37.052] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:23:0)
[04:40:37.091] Build status: STARTED
[04:40:37.091] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:24:0) after 4m16s
[04:44:53.346] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:24:0)
[04:44:53.374] Build status: STARTED
[04:44:53.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:25:0) after 5m15s
[04:50:08.407] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:25:0)
[04:50:08.436] Build status: STARTED
[04:50:08.436] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:26:0) after 7m0s
[04:57:08.456] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:26:0)
[04:57:08.490] Build status: STARTED
[04:57:08.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:27:0) after 8m27s
[05:05:35.518] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:27:0)
[05:05:35.549] Build status: STARTED
[05:05:35.549] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:28:0) after 8m36s
[05:14:11.570] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:28:0)
[05:14:11.599] Build status: STARTED
[05:14:11.599] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:29:0) after 5m43s
[05:19:54.637] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:29:0)
[05:19:54.692] Build status: STARTED
[05:19:54.692] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:30:0) after 2m20s
[05:22:14.981] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:30:0)
[05:22:15.018] Build status: STARTED
[05:22:15.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:31:0) after 2m17s
[05:24:32.043] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:31:0)
[05:24:32.074] Build status: STARTED
[05:24:32.074] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:32:0) after 1m32s
[05:26:04.096] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:32:0)
[05:26:04.132] Build status: STARTED
[05:26:04.132] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:33:0) after 7m48s
[05:33:52.150] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:33:0)
[05:33:52.180] Build status: STARTED
[05:33:52.180] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:34:0) after 3m43s
[05:37:35.216] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:34:0)
[05:37:35.250] Build status: STARTED
[05:37:35.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:35:0) after 2m50s
[05:40:25.489] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:35:0)
[05:40:25.527] Build status: STARTED
[05:40:25.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:36:0) after 3m5s
[05:43:30.741] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:36:0)
[05:43:30.767] Build status: STARTED
[05:43:30.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:37:0) after 9m19s
[05:52:49.794] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:37:0)
[05:52:49.835] Build status: STARTED
[05:52:49.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:38:0) after 9m2s
[06:01:51.855] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:38:0)
[06:01:51.888] Build status: STARTED
[06:01:51.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:39:0) after 5m25s
[06:07:16.913] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:39:0)
[06:07:16.946] Build status: STARTED
[06:07:16.946] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:40:0) after 8m43s
[06:16:00.095] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:40:0)
[06:16:00.126] Build status: STARTED
[06:16:00.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:41:0) after 4m22s
[06:20:22.126] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:41:0)
[06:20:22.157] Build status: STARTED
[06:20:22.157] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:42:0) after 7m44s
[06:28:06.282] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:42:0)
[06:28:06.458] Build status: STARTED
[06:28:06.458] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:43:0) after 7m4s
[06:35:10.486] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:43:0)
[06:35:10.520] Build status: STARTED
[06:35:10.520] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:44:0) after 5m13s
[06:40:23.526] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:44:0)
[06:40:23.584] Build status: STARTED
[06:40:23.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:45:0) after 3m28s
[06:43:51.602] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:45:0)
[06:43:51.663] Build status: STARTED
[06:43:51.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:46:0) after 7m19s
[06:51:10.683] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:46:0)
[06:51:10.710] Build status: STARTED
[06:51:10.710] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:47:0) after 7m10s
[06:58:20.735] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:47:0)
[06:58:20.763] Build status: STARTED
[06:58:20.763] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:48:0) after 8m53s
[07:07:13.835] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:48:0)
[07:07:13.907] Build status: STARTED
[07:07:13.907] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:49:0) after 2m25s
[07:09:38.924] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:49:0)
[07:09:39.025] Build status: STARTED
[07:09:39.026] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:50:0) after 3m37s
[07:13:16.049] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:50:0)
[07:13:16.083] Build status: STARTED
[07:13:16.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:51:0) after 4m34s
[07:17:50.223] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:51:0)
[07:17:50.254] Build status: STARTED
[07:17:50.254] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:52:0) after 3m1s
[07:20:51.264] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:52:0)
[07:20:51.297] Build status: STARTED
[07:20:51.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:53:0) after 4m56s
[07:25:47.351] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:53:0)
[07:25:47.377] Build status: STARTED
[07:25:47.377] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:54:0) after 3m12s
[07:28:59.401] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:54:0)
[07:28:59.442] Build status: STARTED
[07:28:59.442] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:55:0) after 6m33s
[07:35:32.487] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:55:0)
[07:35:32.511] Build status: STARTED
[07:35:32.511] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:56:0) after 2m39s
[07:38:11.533] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:56:0)
[07:38:11.565] Build status: STARTED
[07:38:11.565] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:57:0) after 9m52s
[07:48:03.589] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:57:0)
[07:48:03.625] Build status: STARTED
[07:48:03.625] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:58:0) after 9m45s
[07:57:48.773] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:58:0)
[07:57:48.824] Build status: STARTED
[07:57:48.824] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:59:0) after 2m17s
[08:00:05.847] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:59:0)
[08:00:05.878] Build status: STARTED
[08:00:05.878] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:60:0) after 3m58s
[08:04:03.898] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:60:0)
[08:04:03.928] Build status: STARTED
[08:04:03.928] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:61:0) after 7m19s
[08:11:22.946] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:61:0)
[08:11:22.991] Build status: STARTED
[08:11:22.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:62:0) after 2m41s
[08:14:04.013] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:62:0)
[08:14:04.047] Build status: STARTED
[08:14:04.048] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006744826721850928:63:0) after 7m27s
[08:15:16.357] Received PubSub notification, asking Buildbucket for the build status
[08:15:16.394] Build:
{
"id": "8841314764750590896",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 22564,
"createdBy": "project:chromium",
"createTime": "2021-07-19T02:36:47.801183Z",
"startTime": "2021-07-19T02:40:18.162180Z",
"endTime": "2021-07-19T08:15:15.917040536Z",
"updateTime": "2021-07-19T08:15:15.917564549Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4b8a565314a7669a515e3662029a8086fff67ab3",
"ref": "refs/heads/main"
}
}
}
[08:15:16.394] Invocation finished in 5h38m29.856168012s with status FAILED