[03:51:44.612] New invocation is queued and will start shortly
[03:51:45.746] Starting the invocation (attempt 1)
[03:51:45.782] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/09c4710ccb72e317821b1137b44de8691a423c45"
[03:51:45.782] Popped gitiles commit info from properties and tags
[03:51:45.783] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[03:51:45.783] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[03:51:45.783] Buildbucket request:
{
"requestId": "8944771785144810768",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "09c4710ccb72e317821b1137b44de8691a423c45",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8944771785144810768"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[03:51:46.197] Scheduled build:
{
"id": "8779341723188151905",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29552,
"createdBy": "project:chromium",
"createTime": "2023-06-03T03:51:45.849664513Z",
"updateTime": "2023-06-03T03:51:45.849664513Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "09c4710ccb72e317821b1137b44de8691a423c45",
"ref": "refs/heads/main"
}
},
"canary": true
}
[03:51:46.197] Task URL: https://cr-buildbucket.appspot.com/build/8779341723188151905
[03:51:46.197] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:2:0) after 3m44s
[03:55:03.300] Received PubSub notification, asking Buildbucket for the build status
[03:55:03.317] Build status: STARTED
[03:55:30.231] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:2:0)
[03:55:30.250] Build status: STARTED
[03:55:30.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:4:0) after 8m24s
[04:03:54.266] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:4:0)
[04:03:54.304] Build status: STARTED
[04:03:54.304] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:5:0) after 9m28s
[04:13:22.329] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:5:0)
[04:13:22.348] Build status: STARTED
[04:13:22.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:6:0) after 9m8s
[04:22:30.363] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:6:0)
[04:22:30.383] Build status: STARTED
[04:22:30.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:7:0) after 6m55s
[04:29:25.404] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:7:0)
[04:29:25.488] Build status: STARTED
[04:29:25.488] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:8:0) after 2m20s
[04:31:45.506] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:8:0)
[04:31:45.532] Build status: STARTED
[04:31:45.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:9:0) after 2m44s
[04:34:29.552] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:9:0)
[04:34:29.576] Build status: STARTED
[04:34:29.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:10:0) after 9m16s
[04:43:45.635] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:10:0)
[04:43:45.674] Build status: STARTED
[04:43:45.674] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:11:0) after 2m49s
[04:46:34.810] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:11:0)
[04:46:34.833] Build status: STARTED
[04:46:34.833] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:12:0) after 4m37s
[04:51:11.902] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:12:0)
[04:51:11.925] Build status: STARTED
[04:51:11.925] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:13:0) after 1m38s
[04:52:49.943] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:13:0)
[04:52:49.963] Build status: STARTED
[04:52:49.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:14:0) after 9m33s
[05:02:23.953] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:14:0)
[05:02:24.232] Build status: STARTED
[05:02:24.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:15:0) after 1m39s
[05:04:03.389] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:15:0)
[05:04:03.407] Build status: STARTED
[05:04:03.407] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:16:0) after 2m23s
[05:06:26.555] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:16:0)
[05:06:26.573] Build status: STARTED
[05:06:26.573] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:17:0) after 4m5s
[05:10:31.591] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:17:0)
[05:10:31.608] Build status: STARTED
[05:10:31.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:18:0) after 5m16s
[05:15:47.627] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:18:0)
[05:15:47.649] Build status: STARTED
[05:15:47.649] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:19:0) after 6m35s
[05:22:22.669] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:19:0)
[05:22:22.691] Build status: STARTED
[05:22:22.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:20:0) after 6m11s
[05:28:33.711] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:20:0)
[05:28:33.751] Build status: STARTED
[05:28:33.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:21:0) after 6m8s
[05:34:41.874] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:21:0)
[05:34:41.893] Build status: STARTED
[05:34:41.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:22:0) after 4m32s
[05:39:14.000] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:22:0)
[05:39:14.024] Build status: STARTED
[05:39:14.024] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:23:0) after 8m18s
[05:47:32.151] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:23:0)
[05:47:32.219] Build status: STARTED
[05:47:32.219] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:24:0) after 2m48s
[05:50:20.291] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:24:0)
[05:50:20.312] Build status: STARTED
[05:50:20.312] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:25:0) after 5m24s
[05:55:44.962] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:25:0)
[05:55:44.983] Build status: STARTED
[05:55:44.983] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:26:0) after 4m53s
[06:00:38.092] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:26:0)
[06:00:38.122] Build status: STARTED
[06:00:38.122] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:27:0) after 3m46s
[06:04:24.244] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:27:0)
[06:04:24.260] Build status: STARTED
[06:04:24.260] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:28:0) after 3m35s
[06:07:59.539] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:28:0)
[06:07:59.556] Build status: STARTED
[06:07:59.556] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:29:0) after 1m8s
[06:09:07.572] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:29:0)
[06:09:07.592] Build status: STARTED
[06:09:07.592] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:30:0) after 2m44s
[06:11:51.617] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:30:0)
[06:11:51.638] Build status: STARTED
[06:11:51.638] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:31:0) after 8m19s
[06:20:10.665] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:31:0)
[06:20:10.689] Build status: STARTED
[06:20:10.689] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:32:0) after 9m20s
[06:29:30.805] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:32:0)
[06:29:30.819] Build status: STARTED
[06:29:30.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:33:0) after 5m0s
[06:34:30.841] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:33:0)
[06:34:30.866] Build status: STARTED
[06:34:30.866] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:34:0) after 1m46s
[06:36:16.883] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:34:0)
[06:36:16.909] Build status: STARTED
[06:36:16.909] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:35:0) after 6m1s
[06:42:17.933] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:35:0)
[06:42:17.952] Build status: STARTED
[06:42:17.952] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:36:0) after 2m11s
[06:44:29.077] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:36:0)
[06:44:29.121] Build status: STARTED
[06:44:29.121] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:37:0) after 2m59s
[06:47:28.216] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:37:0)
[06:47:28.364] Build status: STARTED
[06:47:28.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:38:0) after 2m58s
[06:50:26.388] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:38:0)
[06:50:26.411] Build status: STARTED
[06:50:26.411] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:39:0) after 2m45s
[06:53:11.589] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:39:0)
[06:53:11.609] Build status: STARTED
[06:53:11.609] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:40:0) after 2m48s
[06:55:59.787] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:40:0)
[06:55:59.842] Build status: STARTED
[06:55:59.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:41:0) after 5m4s
[07:01:03.970] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:41:0)
[07:01:04.004] Build status: STARTED
[07:01:04.004] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:42:0) after 5m14s
[07:06:18.030] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:42:0)
[07:06:18.050] Build status: STARTED
[07:06:18.050] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:43:0) after 9m4s
[07:15:22.065] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:43:0)
[07:15:22.109] Build status: STARTED
[07:15:22.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:44:0) after 4m11s
[07:19:33.136] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:44:0)
[07:19:33.160] Build status: STARTED
[07:19:33.160] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:45:0) after 3m17s
[07:22:50.289] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:45:0)
[07:22:50.343] Build status: STARTED
[07:22:50.343] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:46:0) after 4m31s
[07:27:21.361] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:46:0)
[07:27:21.387] Build status: STARTED
[07:27:21.387] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:47:0) after 5m45s
[07:33:06.467] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:47:0)
[07:33:06.483] Build status: STARTED
[07:33:06.483] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:48:0) after 6m36s
[07:39:42.682] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:48:0)
[07:39:42.703] Build status: STARTED
[07:39:42.703] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:49:0) after 8m17s
[07:47:59.819] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:49:0)
[07:47:59.838] Build status: STARTED
[07:47:59.838] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:50:0) after 6m45s
[07:54:44.864] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:50:0)
[07:54:44.882] Build status: STARTED
[07:54:44.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:51:0) after 2m37s
[07:57:21.899] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:51:0)
[07:57:21.919] Build status: STARTED
[07:57:21.919] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:52:0) after 7m54s
[08:05:16.094] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:52:0)
[08:05:16.124] Build status: STARTED
[08:05:16.124] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:53:0) after 5m37s
[08:10:53.464] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:53:0)
[08:10:53.507] Build status: STARTED
[08:10:53.507] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:54:0) after 8m33s
[08:19:26.522] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:54:0)
[08:19:26.539] Build status: STARTED
[08:19:26.539] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:55:0) after 6m31s
[08:25:57.604] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:55:0)
[08:25:57.622] Build status: STARTED
[08:25:57.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:56:0) after 5m52s
[08:31:49.640] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:56:0)
[08:31:49.661] Build status: STARTED
[08:31:49.661] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:57:0) after 4m1s
[08:35:50.713] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:57:0)
[08:35:50.740] Build status: STARTED
[08:35:50.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:58:0) after 2m10s
[08:38:00.760] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:58:0)
[08:38:00.782] Build status: STARTED
[08:38:00.782] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:59:0) after 5m18s
[08:43:19.020] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:59:0)
[08:43:19.049] Build status: STARTED
[08:43:19.049] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:60:0) after 9m2s
[08:52:21.107] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:60:0)
[08:52:21.127] Build status: STARTED
[08:52:21.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:61:0) after 1m25s
[08:53:46.154] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:61:0)
[08:53:46.174] Build status: STARTED
[08:53:46.174] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:62:0) after 2m38s
[08:56:24.224] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:62:0)
[08:56:24.243] Build status: STARTED
[08:56:24.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:63:0) after 8m33s
[09:04:57.555] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:63:0)
[09:04:57.575] Build status: STARTED
[09:04:57.575] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8944771785144810768:64:0) after 3m47s
[09:05:36.918] Received PubSub notification, asking Buildbucket for the build status
[09:05:36.942] Build:
{
"id": "8779341723188151905",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29552,
"createdBy": "project:chromium",
"createTime": "2023-06-03T03:51:45.849664513Z",
"startTime": "2023-06-03T03:55:02.382562Z",
"endTime": "2023-06-03T09:05:36.188927818Z",
"updateTime": "2023-06-03T09:05:36.188927818Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "09c4710ccb72e317821b1137b44de8691a423c45",
"ref": "refs/heads/main"
}
},
"canary": true
}
[09:05:36.942] Invocation finished in 5h13m52.341497475s with status FAILED