[01:00:58.906] New invocation is queued and will start shortly
[01:01:00.219] Starting the invocation (attempt 1)
[01:01:00.262] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/e62b54cfb3111a1911d866c73f01877e14a3c655"
[01:01:00.262] Popped gitiles commit info from properties and tags
[01:01:00.262] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[01:01:00.262] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[01:01:00.262] Buildbucket request:
{
"requestId": "8999321902321931456",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e62b54cfb3111a1911d866c73f01877e14a3c655",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8999321902321931456"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[01:01:00.906] Scheduled build:
{
"id": "8833891840067321953",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23321,
"createdBy": "project:chromium",
"createTime": "2021-10-09T01:01:00.425141590Z",
"updateTime": "2021-10-09T01:01:00.671181491Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e62b54cfb3111a1911d866c73f01877e14a3c655",
"ref": "refs/heads/main"
}
}
}
[01:01:00.906] Task URL: https://cr-buildbucket.appspot.com/build/8833891840067321953
[01:01:00.906] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:2:0) after 7m26s
[01:03:20.300] Received PubSub notification, asking Buildbucket for the build status
[01:03:20.325] Build status: STARTED
[01:08:27.089] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:2:0)
[01:08:27.123] Build status: STARTED
[01:08:27.123] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:4:0) after 7m38s
[01:16:05.190] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:4:0)
[01:16:05.223] Build status: STARTED
[01:16:05.223] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:5:0) after 5m53s
[01:21:58.280] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:5:0)
[01:21:58.306] Build status: STARTED
[01:21:58.306] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:6:0) after 4m49s
[01:26:47.475] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:6:0)
[01:26:47.501] Build status: STARTED
[01:26:47.501] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:7:0) after 5m58s
[01:32:45.529] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:7:0)
[01:32:45.560] Build status: STARTED
[01:32:45.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:8:0) after 7m8s
[01:39:54.054] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:8:0)
[01:39:54.084] Build status: STARTED
[01:39:54.085] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:9:0) after 6m4s
[01:45:58.168] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:9:0)
[01:45:58.200] Build status: STARTED
[01:45:58.200] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:10:0) after 5m7s
[01:51:05.174] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:10:0)
[01:51:05.238] Build status: STARTED
[01:51:05.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:11:0) after 6m6s
[01:57:11.548] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:11:0)
[01:57:11.582] Build status: STARTED
[01:57:11.582] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:12:0) after 3m51s
[02:01:02.624] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:12:0)
[02:01:02.678] Build status: STARTED
[02:01:02.678] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:13:0) after 5m18s
[02:06:20.785] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:13:0)
[02:06:20.811] Build status: STARTED
[02:06:20.811] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:14:0) after 2m47s
[02:09:07.894] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:14:0)
[02:09:07.964] Build status: STARTED
[02:09:07.964] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:15:0) after 7m58s
[02:17:06.056] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:15:0)
[02:17:06.101] Build status: STARTED
[02:17:06.101] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:16:0) after 6m26s
[02:23:32.121] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:16:0)
[02:23:32.155] Build status: STARTED
[02:23:32.155] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:17:0) after 5m27s
[02:28:59.176] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:17:0)
[02:28:59.209] Build status: STARTED
[02:28:59.209] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:18:0) after 7m0s
[02:35:59.518] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:18:0)
[02:35:59.550] Build status: STARTED
[02:35:59.550] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:19:0) after 3m4s
[02:39:03.571] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:19:0)
[02:39:03.605] Build status: STARTED
[02:39:03.605] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:20:0) after 2m17s
[02:41:20.718] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:20:0)
[02:41:20.801] Build status: STARTED
[02:41:20.801] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:21:0) after 1m58s
[02:43:18.823] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:21:0)
[02:43:18.858] Build status: STARTED
[02:43:18.858] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:22:0) after 1m38s
[02:44:57.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:22:0)
[02:44:57.062] Build status: STARTED
[02:44:57.062] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:23:0) after 2m58s
[02:47:55.096] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:23:0)
[02:47:55.126] Build status: STARTED
[02:47:55.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:24:0) after 2m7s
[02:50:03.051] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:24:0)
[02:50:03.087] Build status: STARTED
[02:50:03.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:25:0) after 5m37s
[02:55:40.115] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:25:0)
[02:55:40.184] Build status: STARTED
[02:55:40.184] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:26:0) after 1m36s
[02:57:16.203] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:26:0)
[02:57:16.377] Build status: STARTED
[02:57:16.377] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:27:0) after 8m57s
[03:06:13.401] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:27:0)
[03:06:13.432] Build status: STARTED
[03:06:13.432] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:28:0) after 6m5s
[03:12:18.454] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:28:0)
[03:12:18.491] Build status: STARTED
[03:12:18.491] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:29:0) after 6m33s
[03:18:51.523] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:29:0)
[03:18:51.551] Build status: STARTED
[03:18:51.551] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:30:0) after 3m15s
[03:22:06.626] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:30:0)
[03:22:06.668] Build status: STARTED
[03:22:06.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:31:0) after 3m37s
[03:25:43.686] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:31:0)
[03:25:43.715] Build status: STARTED
[03:25:43.715] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:32:0) after 2m6s
[03:27:49.770] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:32:0)
[03:27:49.802] Build status: STARTED
[03:27:49.802] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:33:0) after 4m20s
[03:32:09.824] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:33:0)
[03:32:09.858] Build status: STARTED
[03:32:09.858] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:34:0) after 8m45s
[03:40:54.921] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:34:0)
[03:40:54.951] Build status: STARTED
[03:40:54.951] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:35:0) after 6m57s
[03:47:51.982] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:35:0)
[03:47:52.014] Build status: STARTED
[03:47:52.014] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:36:0) after 6m45s
[03:54:37.214] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:36:0)
[03:54:37.247] Build status: STARTED
[03:54:37.247] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:37:0) after 3m4s
[03:57:41.282] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:37:0)
[03:57:41.372] Build status: STARTED
[03:57:41.372] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:38:0) after 8m33s
[04:06:14.528] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:38:0)
[04:06:14.563] Build status: STARTED
[04:06:14.563] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:39:0) after 1m28s
[04:07:42.587] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:39:0)
[04:07:42.622] Build status: STARTED
[04:07:42.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:40:0) after 9m12s
[04:16:54.661] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:40:0)
[04:16:54.701] Build status: STARTED
[04:16:54.701] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:41:0) after 7m56s
[04:24:50.724] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:41:0)
[04:24:50.757] Build status: STARTED
[04:24:50.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:42:0) after 8m12s
[04:33:02.797] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:42:0)
[04:33:02.834] Build status: STARTED
[04:33:02.834] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:43:0) after 7m27s
[04:40:29.859] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:43:0)
[04:40:29.923] Build status: STARTED
[04:40:29.923] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:44:0) after 6m48s
[04:47:18.063] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:44:0)
[04:47:18.097] Build status: STARTED
[04:47:18.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:45:0) after 3m37s
[04:50:55.158] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:45:0)
[04:50:55.196] Build status: STARTED
[04:50:55.196] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:46:0) after 8m45s
[04:59:40.234] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:46:0)
[04:59:40.369] Build status: STARTED
[04:59:40.369] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:47:0) after 8m21s
[05:08:01.521] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:47:0)
[05:08:01.559] Build status: STARTED
[05:08:01.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:48:0) after 5m47s
[05:13:48.735] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:48:0)
[05:13:48.774] Build status: STARTED
[05:13:48.774] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:49:0) after 2m15s
[05:16:03.885] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:49:0)
[05:16:03.914] Build status: STARTED
[05:16:03.914] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:50:0) after 6m7s
[05:22:11.464] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:50:0)
[05:22:11.497] Build status: STARTED
[05:22:11.497] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:51:0) after 1m28s
[05:23:39.598] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:51:0)
[05:23:39.627] Build status: STARTED
[05:23:39.627] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:52:0) after 7m49s
[05:31:28.648] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:52:0)
[05:31:28.678] Build status: STARTED
[05:31:28.678] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:53:0) after 4m41s
[05:36:09.703] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:53:0)
[05:36:09.732] Build status: STARTED
[05:36:09.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:54:0) after 2m18s
[05:38:27.756] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:54:0)
[05:38:27.805] Build status: STARTED
[05:38:27.805] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:55:0) after 8m41s
[05:47:08.893] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:55:0)
[05:47:08.945] Build status: STARTED
[05:47:08.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:56:0) after 9m25s
[05:56:34.420] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:56:0)
[05:56:34.451] Build status: STARTED
[05:56:34.451] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:57:0) after 7m3s
[06:03:37.477] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:57:0)
[06:03:37.513] Build status: STARTED
[06:03:37.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:58:0) after 7m2s
[06:10:39.659] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:58:0)
[06:10:39.776] Build status: STARTED
[06:10:39.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:59:0) after 4m27s
[06:15:06.903] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:59:0)
[06:15:06.927] Build status: STARTED
[06:15:06.927] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8999321902321931456:60:0) after 7m21s
[06:22:06.043] Received PubSub notification, asking Buildbucket for the build status
[06:22:06.079] Build:
{
"id": "8833891840067321953",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23321,
"createdBy": "project:chromium",
"createTime": "2021-10-09T01:01:00.425141590Z",
"startTime": "2021-10-09T01:03:17.024931Z",
"endTime": "2021-10-09T06:22:05.674839555Z",
"updateTime": "2021-10-09T06:22:05.675568725Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e62b54cfb3111a1911d866c73f01877e14a3c655",
"ref": "refs/heads/main"
}
}
}
[06:22:06.079] Invocation finished in 5h21m7.188223313s with status FAILED