[22:21:33.018] New invocation is queued and will start shortly
[22:21:34.483] Starting the invocation (attempt 1)
[22:21:34.523] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d0c1b5d507f334cb7c15ea61ee29f14e08cf2ee7"
[22:21:34.523] Popped gitiles commit info from properties and tags
[22:21:34.523] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[22:21:34.523] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[22:21:34.524] Buildbucket request:
{
"requestId": "8998607157152443200",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d0c1b5d507f334cb7c15ea61ee29f14e08cf2ee7",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8998607157152443200"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[22:21:35.061] Scheduled build:
{
"id": "8833177094848761249",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23362,
"createdBy": "project:chromium",
"createTime": "2021-10-16T22:21:34.582806720Z",
"updateTime": "2021-10-16T22:21:34.813516221Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d0c1b5d507f334cb7c15ea61ee29f14e08cf2ee7",
"ref": "refs/heads/main"
}
}
}
[22:21:35.061] Task URL: https://cr-buildbucket.appspot.com/build/8833177094848761249
[22:21:35.061] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:2:0) after 2m30s
[22:22:47.724] Received PubSub notification, asking Buildbucket for the build status
[22:22:47.780] Build status: STARTED
[22:24:05.084] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:2:0)
[22:24:05.115] Build status: STARTED
[22:24:05.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:4:0) after 9m18s
[22:33:23.137] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:4:0)
[22:33:23.173] Build status: STARTED
[22:33:23.173] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:5:0) after 3m18s
[22:36:41.372] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:5:0)
[22:36:41.463] Build status: STARTED
[22:36:41.463] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:6:0) after 6m21s
[22:43:02.475] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:6:0)
[22:43:02.503] Build status: STARTED
[22:43:02.503] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:7:0) after 2m44s
[22:45:46.835] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:7:0)
[22:45:46.875] Build status: STARTED
[22:45:46.875] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:8:0) after 5m46s
[22:51:32.963] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:8:0)
[22:51:32.995] Build status: STARTED
[22:51:32.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:9:0) after 1m9s
[22:52:42.014] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:9:0)
[22:52:42.046] Build status: STARTED
[22:52:42.046] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:10:0) after 1m30s
[22:54:12.474] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:10:0)
[22:54:12.500] Build status: STARTED
[22:54:12.500] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:11:0) after 8m20s
[23:02:32.525] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:11:0)
[23:02:32.559] Build status: STARTED
[23:02:32.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:12:0) after 1m9s
[23:03:41.577] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:12:0)
[23:03:41.617] Build status: STARTED
[23:03:41.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:13:0) after 2m37s
[23:06:18.723] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:13:0)
[23:06:18.753] Build status: STARTED
[23:06:18.753] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:14:0) after 2m44s
[23:09:02.775] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:14:0)
[23:09:02.821] Build status: STARTED
[23:09:02.821] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:15:0) after 7m53s
[23:16:55.838] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:15:0)
[23:16:55.947] Build status: STARTED
[23:16:55.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:16:0) after 1m23s
[23:18:19.013] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:16:0)
[23:18:19.047] Build status: STARTED
[23:18:19.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:17:0) after 9m49s
[23:28:08.055] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:17:0)
[23:28:08.096] Build status: STARTED
[23:28:08.096] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:18:0) after 3m37s
[23:31:45.112] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:18:0)
[23:31:45.154] Build status: STARTED
[23:31:45.154] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:19:0) after 6m6s
[23:37:51.169] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:19:0)
[23:37:51.277] Build status: STARTED
[23:37:51.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:20:0) after 4m13s
[23:42:04.291] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:20:0)
[23:42:04.320] Build status: STARTED
[23:42:04.320] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:21:0) after 1m44s
[23:43:48.338] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:21:0)
[23:43:48.369] Build status: STARTED
[23:43:48.369] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:22:0) after 7m31s
[23:51:19.389] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:22:0)
[23:51:19.422] Build status: STARTED
[23:51:19.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:23:0) after 6m34s
[23:57:53.640] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:23:0)
[23:57:53.667] Build status: STARTED
[23:57:53.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:24:0) after 3m10s
[00:01:03.682] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:24:0)
[00:01:03.716] Build status: STARTED
[00:01:03.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:25:0) after 8m14s
[00:09:17.823] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:25:0)
[00:09:17.856] Build status: STARTED
[00:09:17.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:26:0) after 6m6s
[00:15:23.974] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:26:0)
[00:15:24.011] Build status: STARTED
[00:15:24.011] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:27:0) after 6m36s
[00:22:00.249] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:27:0)
[00:22:00.278] Build status: STARTED
[00:22:00.278] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:28:0) after 8m6s
[00:30:06.359] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:28:0)
[00:30:06.394] Build status: STARTED
[00:30:06.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:29:0) after 2m30s
[00:32:36.423] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:29:0)
[00:32:36.460] Build status: STARTED
[00:32:36.460] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:30:0) after 9m20s
[00:41:56.583] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:30:0)
[00:41:56.608] Build status: STARTED
[00:41:56.609] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:31:0) after 1m25s
[00:43:21.624] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:31:0)
[00:43:22.108] Build status: STARTED
[00:43:22.108] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:32:0) after 8m51s
[00:52:13.120] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:32:0)
[00:52:13.148] Build status: STARTED
[00:52:13.148] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:33:0) after 2m45s
[00:54:58.172] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:33:0)
[00:54:58.201] Build status: STARTED
[00:54:58.201] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:34:0) after 2m36s
[00:57:34.170] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:34:0)
[00:57:34.201] Build status: STARTED
[00:57:34.201] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:35:0) after 8m36s
[01:06:10.221] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:35:0)
[01:06:10.269] Build status: STARTED
[01:06:10.269] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:36:0) after 7m34s
[01:13:44.289] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:36:0)
[01:13:44.358] Build status: STARTED
[01:13:44.358] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:37:0) after 4m13s
[01:17:57.631] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:37:0)
[01:17:57.657] Build status: STARTED
[01:17:57.658] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:38:0) after 3m15s
[01:21:14.047] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:38:0)
[01:21:14.088] Build status: STARTED
[01:21:14.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:39:0) after 2m9s
[01:23:23.474] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:39:0)
[01:23:23.505] Build status: STARTED
[01:23:23.505] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:40:0) after 3m56s
[01:27:19.526] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:40:0)
[01:27:19.552] Build status: STARTED
[01:27:19.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:41:0) after 6m2s
[01:33:21.572] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:41:0)
[01:33:21.629] Build status: STARTED
[01:33:21.629] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:42:0) after 5m13s
[01:38:34.669] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:42:0)
[01:38:34.697] Build status: STARTED
[01:38:34.697] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:43:0) after 9m36s
[01:48:10.714] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:43:0)
[01:48:10.744] Build status: STARTED
[01:48:10.744] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:44:0) after 3m31s
[01:51:41.746] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:44:0)
[01:51:41.776] Build status: STARTED
[01:51:41.777] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:45:0) after 4m4s
[01:55:45.795] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:45:0)
[01:55:45.822] Build status: STARTED
[01:55:45.822] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:46:0) after 1m30s
[01:57:15.856] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:46:0)
[01:57:15.886] Build status: STARTED
[01:57:15.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:47:0) after 6m22s
[02:03:38.039] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:47:0)
[02:03:38.174] Build status: STARTED
[02:03:38.174] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:48:0) after 5m30s
[02:09:08.191] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:48:0)
[02:09:08.222] Build status: STARTED
[02:09:08.222] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:49:0) after 1m19s
[02:10:27.347] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:49:0)
[02:10:27.378] Build status: STARTED
[02:10:27.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:50:0) after 7m23s
[02:17:50.397] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:50:0)
[02:17:50.427] Build status: STARTED
[02:17:50.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:51:0) after 6m0s
[02:23:50.517] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:51:0)
[02:23:50.554] Build status: STARTED
[02:23:50.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:52:0) after 5m29s
[02:29:19.673] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:52:0)
[02:29:19.701] Build status: STARTED
[02:29:19.701] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:53:0) after 3m46s
[02:33:05.721] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:53:0)
[02:33:05.790] Build status: STARTED
[02:33:05.790] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:54:0) after 6m48s
[02:39:53.817] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:54:0)
[02:39:53.848] Build status: STARTED
[02:39:53.848] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:55:0) after 4m52s
[02:44:45.975] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:55:0)
[02:44:46.006] Build status: STARTED
[02:44:46.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:56:0) after 6m25s
[02:51:11.014] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:56:0)
[02:51:11.040] Build status: STARTED
[02:51:11.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:57:0) after 7m8s
[02:58:19.995] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:57:0)
[02:58:20.024] Build status: STARTED
[02:58:20.024] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:58:0) after 1m25s
[02:59:45.142] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:58:0)
[02:59:45.169] Build status: STARTED
[02:59:45.169] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:59:0) after 6m24s
[03:06:09.192] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:59:0)
[03:06:09.220] Build status: STARTED
[03:06:09.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:60:0) after 2m48s
[03:08:57.355] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:60:0)
[03:08:57.381] Build status: STARTED
[03:08:57.381] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:61:0) after 8m46s
[03:17:43.491] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:61:0)
[03:17:43.524] Build status: STARTED
[03:17:43.524] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:62:0) after 1m4s
[03:18:47.641] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:62:0)
[03:18:47.671] Build status: STARTED
[03:18:47.671] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:63:0) after 8m56s
[03:27:43.915] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:63:0)
[03:27:43.951] Build status: STARTED
[03:27:43.951] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:64:0) after 6m46s
[03:34:30.087] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:64:0)
[03:34:30.130] Build status: STARTED
[03:34:30.130] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:65:0) after 1m23s
[03:35:53.143] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:65:0)
[03:35:53.170] Build status: STARTED
[03:35:53.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:66:0) after 2m39s
[03:38:32.190] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:66:0)
[03:38:32.217] Build status: STARTED
[03:38:32.217] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8998607157152443200:67:0) after 7m56s
[03:41:00.000] Received PubSub notification, asking Buildbucket for the build status
[03:41:00.029] Build:
{
"id": "8833177094848761249",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23362,
"createdBy": "project:chromium",
"createTime": "2021-10-16T22:21:34.582806720Z",
"startTime": "2021-10-16T22:22:46.738272Z",
"endTime": "2021-10-17T03:40:59.718713084Z",
"updateTime": "2021-10-17T03:40:59.719132157Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d0c1b5d507f334cb7c15ea61ee29f14e08cf2ee7",
"ref": "refs/heads/main"
}
}
}
[03:41:00.029] Invocation finished in 5h19m27.026890475s with status FAILED