[15:00:35.448] New invocation is queued and will start shortly
[15:00:36.848] Starting the invocation (attempt 1)
[15:00:36.875] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/c51b8a1c0b2cd0992a6c7a7705ef6f6ee9942994"
[15:00:36.875] Popped gitiles commit info from properties and tags
[15:00:36.875] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:00:36.875] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:00:36.875] Buildbucket request:
{
"requestId": "8942102392882332656",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c51b8a1c0b2cd0992a6c7a7705ef6f6ee9942994",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8942102392882332656"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:00:37.293] Scheduled build:
{
"id": "8776672330668257809",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29853,
"createdBy": "project:chromium",
"createTime": "2023-07-02T15:00:36.936768229Z",
"updateTime": "2023-07-02T15:00:36.936768229Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c51b8a1c0b2cd0992a6c7a7705ef6f6ee9942994",
"ref": "refs/heads/main"
}
},
"canary": true
}
[15:00:37.293] Task URL: https://cr-buildbucket.appspot.com/build/8776672330668257809
[15:00:37.293] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:2:0) after 5m28s
[15:04:03.846] Received PubSub notification, asking Buildbucket for the build status
[15:04:03.860] Build status: STARTED
[15:06:05.311] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:2:0)
[15:06:05.365] Build status: STARTED
[15:06:05.365] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:4:0) after 7m17s
[15:13:22.414] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:4:0)
[15:13:22.445] Build status: STARTED
[15:13:22.445] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:5:0) after 8m27s
[15:21:49.497] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:5:0)
[15:21:49.513] Build status: STARTED
[15:21:49.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:6:0) after 2m32s
[15:24:21.595] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:6:0)
[15:24:21.614] Build status: STARTED
[15:24:21.614] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:7:0) after 4m4s
[15:28:25.686] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:7:0)
[15:28:25.740] Build status: STARTED
[15:28:25.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:8:0) after 1m24s
[15:29:49.727] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:8:0)
[15:29:49.749] Build status: STARTED
[15:29:49.749] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:9:0) after 1m47s
[15:31:36.956] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:9:0)
[15:31:36.976] Build status: STARTED
[15:31:36.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:10:0) after 1m29s
[15:33:06.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:10:0)
[15:33:06.046] Build status: STARTED
[15:33:06.046] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:11:0) after 2m31s
[15:35:37.067] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:11:0)
[15:35:37.091] Build status: STARTED
[15:35:37.091] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:12:0) after 7m46s
[15:43:23.119] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:12:0)
[15:43:23.136] Build status: STARTED
[15:43:23.136] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:13:0) after 6m59s
[15:50:22.250] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:13:0)
[15:50:22.274] Build status: STARTED
[15:50:22.274] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:14:0) after 7m32s
[15:57:54.438] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:14:0)
[15:57:54.471] Build status: STARTED
[15:57:54.471] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:15:0) after 9m3s
[16:06:57.495] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:15:0)
[16:06:57.519] Build status: STARTED
[16:06:57.519] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:16:0) after 8m28s
[16:15:25.720] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:16:0)
[16:15:25.744] Build status: STARTED
[16:15:25.744] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:17:0) after 5m54s
[16:21:19.760] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:17:0)
[16:21:19.914] Build status: STARTED
[16:21:19.914] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:18:0) after 6m13s
[16:27:32.954] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:18:0)
[16:27:32.972] Build status: STARTED
[16:27:32.972] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:19:0) after 5m15s
[16:32:47.987] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:19:0)
[16:32:48.016] Build status: STARTED
[16:32:48.016] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:20:0) after 8m23s
[16:41:11.325] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:20:0)
[16:41:11.346] Build status: STARTED
[16:41:11.346] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:21:0) after 7m5s
[16:48:16.369] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:21:0)
[16:48:16.388] Build status: STARTED
[16:48:16.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:22:0) after 2m4s
[16:50:20.740] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:22:0)
[16:50:20.763] Build status: STARTED
[16:50:20.763] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:23:0) after 4m10s
[16:54:31.377] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:23:0)
[16:54:31.394] Build status: STARTED
[16:54:31.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:24:0) after 4m31s
[16:59:02.513] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:24:0)
[16:59:02.776] Build status: STARTED
[16:59:02.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:25:0) after 8m31s
[17:07:33.790] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:25:0)
[17:07:33.811] Build status: STARTED
[17:07:33.811] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:26:0) after 9m49s
[17:17:23.001] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:26:0)
[17:17:23.021] Build status: STARTED
[17:17:23.021] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:27:0) after 6m26s
[17:23:49.455] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:27:0)
[17:23:49.475] Build status: STARTED
[17:23:49.475] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:28:0) after 8m32s
[17:32:21.555] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:28:0)
[17:32:21.577] Build status: STARTED
[17:32:21.577] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:29:0) after 2m34s
[17:34:55.896] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:29:0)
[17:34:55.922] Build status: STARTED
[17:34:55.922] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:30:0) after 8m16s
[17:43:11.929] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:30:0)
[17:43:11.945] Build status: STARTED
[17:43:11.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:31:0) after 7m20s
[17:50:32.047] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:31:0)
[17:50:32.064] Build status: STARTED
[17:50:32.064] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:32:0) after 3m22s
[17:53:54.093] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:32:0)
[17:53:54.165] Build status: STARTED
[17:53:54.165] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:33:0) after 2m59s
[17:56:53.285] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:33:0)
[17:56:53.302] Build status: STARTED
[17:56:53.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:34:0) after 9m9s
[18:06:02.408] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:34:0)
[18:06:02.436] Build status: STARTED
[18:06:02.437] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:35:0) after 4m39s
[18:10:41.669] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:35:0)
[18:10:41.687] Build status: STARTED
[18:10:41.687] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:36:0) after 5m27s
[18:16:08.747] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:36:0)
[18:16:08.764] Build status: STARTED
[18:16:08.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:37:0) after 1m31s
[18:17:39.905] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:37:0)
[18:17:40.047] Build status: STARTED
[18:17:40.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:38:0) after 9m22s
[18:27:02.065] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:38:0)
[18:27:02.083] Build status: STARTED
[18:27:02.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:39:0) after 3m39s
[18:30:41.408] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:39:0)
[18:30:41.428] Build status: STARTED
[18:30:41.428] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:40:0) after 7m8s
[18:37:49.516] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:40:0)
[18:37:49.555] Build status: STARTED
[18:37:49.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:41:0) after 2m6s
[18:39:55.577] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:41:0)
[18:39:55.596] Build status: STARTED
[18:39:55.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:42:0) after 8m1s
[18:47:56.612] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:42:0)
[18:47:56.630] Build status: STARTED
[18:47:56.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:43:0) after 7m43s
[18:55:40.346] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:43:0)
[18:55:40.378] Build status: STARTED
[18:55:40.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:44:0) after 9m10s
[19:04:50.451] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:44:0)
[19:04:50.470] Build status: STARTED
[19:04:50.470] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:45:0) after 9m23s
[19:14:13.598] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:45:0)
[19:14:13.622] Build status: STARTED
[19:14:13.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:46:0) after 5m7s
[19:19:20.758] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:46:0)
[19:19:20.778] Build status: STARTED
[19:19:20.778] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:47:0) after 8m11s
[19:27:32.028] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:47:0)
[19:27:32.046] Build status: STARTED
[19:27:32.046] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:48:0) after 3m25s
[19:30:57.063] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:48:0)
[19:30:57.082] Build status: STARTED
[19:30:57.082] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:49:0) after 4m6s
[19:35:03.097] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:49:0)
[19:35:03.129] Build status: STARTED
[19:35:03.129] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:50:0) after 5m17s
[19:40:20.145] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:50:0)
[19:40:20.164] Build status: STARTED
[19:40:20.164] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:51:0) after 6m54s
[19:47:14.191] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:51:0)
[19:47:14.209] Build status: STARTED
[19:47:14.210] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:52:0) after 6m15s
[19:53:29.228] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:52:0)
[19:53:29.286] Build status: STARTED
[19:53:29.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:53:0) after 2m4s
[19:55:33.363] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:53:0)
[19:55:33.382] Build status: STARTED
[19:55:33.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:54:0) after 3m18s
[19:58:51.403] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:54:0)
[19:58:51.420] Build status: STARTED
[19:58:51.420] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:55:0) after 8m47s
[20:07:38.446] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:55:0)
[20:07:38.470] Build status: STARTED
[20:07:38.470] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:56:0) after 2m0s
[20:09:38.624] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:56:0)
[20:09:38.639] Build status: STARTED
[20:09:38.639] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:57:0) after 9m33s
[20:19:11.657] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:57:0)
[20:19:11.674] Build status: STARTED
[20:19:11.674] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:58:0) after 1m37s
[20:20:48.701] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:58:0)
[20:20:48.718] Build status: STARTED
[20:20:48.718] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:59:0) after 3m29s
[20:24:17.820] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:59:0)
[20:24:17.843] Build status: STARTED
[20:24:17.843] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:60:0) after 1m38s
[20:25:55.940] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:60:0)
[20:25:55.959] Build status: STARTED
[20:25:55.959] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:61:0) after 7m56s
[20:33:51.982] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:61:0)
[20:33:51.998] Build status: STARTED
[20:33:51.998] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942102392882332656:62:0) after 6m6s
[20:37:54.517] Received PubSub notification, asking Buildbucket for the build status
[20:37:54.587] Build:
{
"id": "8776672330668257809",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29853,
"createdBy": "project:chromium",
"createTime": "2023-07-02T15:00:36.936768229Z",
"startTime": "2023-07-02T15:04:02.760980Z",
"endTime": "2023-07-02T20:37:53.894206348Z",
"updateTime": "2023-07-02T20:37:53.894206348Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c51b8a1c0b2cd0992a6c7a7705ef6f6ee9942994",
"ref": "refs/heads/main"
}
},
"canary": true
}
[20:37:54.587] Invocation finished in 5h37m19.14588253s with status FAILED