[20:26:07.424] New invocation is queued and will start shortly
[20:26:10.989] Starting the invocation (attempt 1)
[20:26:11.020] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/2987a72c6998f5d37ce8491f447f6f43b8f96d5a"
[20:26:11.020] Popped gitiles commit info from properties and tags
[20:26:11.020] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[20:26:11.020] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[20:26:11.020] Buildbucket request:
{
"requestId": "8935015348740219808",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2987a72c6998f5d37ce8491f447f6f43b8f96d5a",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8935015348740219808"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[20:26:11.663] Scheduled build:
{
"id": "8769585284247378833",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 30334,
"createdBy": "project:chromium",
"createTime": "2023-09-18T20:26:11.087091574Z",
"updateTime": "2023-09-18T20:26:11.087091574Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2987a72c6998f5d37ce8491f447f6f43b8f96d5a",
"ref": "refs/heads/main"
}
}
}
[20:26:11.663] Task URL: https://cr-buildbucket.appspot.com/build/8769585284247378833
[20:26:11.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:2:0) after 3m45s
[20:26:12.620] Received PubSub notification, asking Buildbucket for the build status
[20:26:12.642] Build status: SCHEDULED
[20:29:49.060] Received PubSub notification, asking Buildbucket for the build status
[20:29:49.089] Build status: STARTED
[20:29:56.907] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:2:0)
[20:29:56.929] Build status: STARTED
[20:29:56.929] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:5:0) after 1m59s
[20:31:56.087] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:5:0)
[20:31:56.173] Build status: STARTED
[20:31:56.173] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:6:0) after 4m3s
[20:35:59.701] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:6:0)
[20:35:59.727] Build status: STARTED
[20:35:59.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:7:0) after 5m31s
[20:41:30.751] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:7:0)
[20:41:30.772] Build status: STARTED
[20:41:30.772] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:8:0) after 6m29s
[20:47:59.859] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:8:0)
[20:47:59.886] Build status: STARTED
[20:47:59.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:9:0) after 5m56s
[20:53:56.267] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:9:0)
[20:53:56.347] Build status: STARTED
[20:53:56.347] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:10:0) after 1m36s
[20:55:32.698] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:10:0)
[20:55:32.725] Build status: STARTED
[20:55:32.725] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:11:0) after 7m24s
[21:02:56.900] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:11:0)
[21:02:56.930] Build status: STARTED
[21:02:56.930] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:12:0) after 2m6s
[21:05:02.954] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:12:0)
[21:05:03.006] Build status: STARTED
[21:05:03.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:13:0) after 4m39s
[21:09:42.027] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:13:0)
[21:09:42.063] Build status: STARTED
[21:09:42.063] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:14:0) after 5m54s
[21:15:36.276] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:14:0)
[21:15:36.298] Build status: STARTED
[21:15:36.298] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:15:0) after 3m0s
[21:18:36.472] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:15:0)
[21:18:36.496] Build status: STARTED
[21:18:36.496] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:16:0) after 9m0s
[21:27:36.919] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:16:0)
[21:27:36.978] Build status: STARTED
[21:27:36.978] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:17:0) after 9m11s
[21:36:48.608] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:17:0)
[21:36:48.625] Build status: STARTED
[21:36:48.625] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:18:0) after 9m49s
[21:46:37.664] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:18:0)
[21:46:37.696] Build status: STARTED
[21:46:37.696] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:19:0) after 7m31s
[21:54:08.763] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:19:0)
[21:54:08.784] Build status: STARTED
[21:54:08.784] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:20:0) after 6m26s
[22:00:34.808] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:20:0)
[22:00:34.856] Build status: STARTED
[22:00:34.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:21:0) after 8m7s
[22:08:41.966] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:21:0)
[22:08:42.025] Build status: STARTED
[22:08:42.025] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:22:0) after 6m57s
[22:15:39.550] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:22:0)
[22:15:39.580] Build status: STARTED
[22:15:39.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:23:0) after 5m20s
[22:21:00.802] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:23:0)
[22:21:00.829] Build status: STARTED
[22:21:00.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:24:0) after 4m48s
[22:25:48.899] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:24:0)
[22:25:48.924] Build status: STARTED
[22:25:48.924] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:25:0) after 7m41s
[22:33:29.968] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:25:0)
[22:33:30.088] Build status: STARTED
[22:33:30.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:26:0) after 6m3s
[22:41:49.662] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:26:0)
[22:41:49.693] Build status: STARTED
[22:41:49.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:27:0) after 7m49s
[22:49:38.917] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:27:0)
[22:49:39.030] Build status: STARTED
[22:49:39.030] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:28:0) after 9m47s
[22:59:26.063] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:28:0)
[22:59:26.092] Build status: STARTED
[22:59:26.092] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:29:0) after 9m13s
[23:08:39.100] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:29:0)
[23:08:39.140] Build status: STARTED
[23:08:39.140] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:30:0) after 4m14s
[23:12:53.373] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:30:0)
[23:12:53.393] Build status: STARTED
[23:12:53.393] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:31:0) after 4m25s
[23:17:18.519] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:31:0)
[23:17:18.565] Build status: STARTED
[23:17:18.565] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:32:0) after 3m33s
[23:20:51.605] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:32:0)
[23:20:51.628] Build status: STARTED
[23:20:51.628] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:33:0) after 6m36s
[23:27:27.668] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:33:0)
[23:27:27.686] Build status: STARTED
[23:27:27.686] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:34:0) after 6m49s
[23:34:17.060] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:34:0)
[23:34:17.081] Build status: STARTED
[23:34:17.081] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:35:0) after 2m25s
[23:36:42.230] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:35:0)
[23:36:42.249] Build status: STARTED
[23:36:42.249] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:36:0) after 4m34s
[23:41:16.269] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:36:0)
[23:41:16.290] Build status: STARTED
[23:41:16.290] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:37:0) after 1m56s
[23:43:12.323] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:37:0)
[23:43:12.344] Build status: STARTED
[23:43:12.344] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:38:0) after 8m24s
[23:51:36.516] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:38:0)
[23:51:36.538] Build status: STARTED
[23:51:36.538] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:39:0) after 6m54s
[23:58:31.153] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:39:0)
[23:58:31.175] Build status: STARTED
[23:58:31.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:40:0) after 4m24s
[00:02:55.538] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:40:0)
[00:02:55.579] Build status: STARTED
[00:02:55.579] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:41:0) after 5m11s
[00:08:06.916] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:41:0)
[00:08:06.931] Build status: STARTED
[00:08:06.931] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:42:0) after 3m37s
[00:11:43.948] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:42:0)
[00:11:43.970] Build status: STARTED
[00:11:43.970] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:43:0) after 8m41s
[00:20:24.992] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:43:0)
[00:20:25.014] Build status: STARTED
[00:20:25.014] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:44:0) after 4m11s
[00:24:36.049] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:44:0)
[00:24:36.076] Build status: STARTED
[00:24:36.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:45:0) after 3m48s
[00:28:24.112] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:45:0)
[00:28:24.172] Build status: STARTED
[00:28:24.172] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:46:0) after 6m50s
[00:35:14.209] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:46:0)
[00:35:14.297] Build status: STARTED
[00:35:14.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:47:0) after 5m7s
[00:40:21.512] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:47:0)
[00:40:21.539] Build status: STARTED
[00:40:21.539] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:48:0) after 7m31s
[00:47:52.596] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:48:0)
[00:47:52.705] Build status: STARTED
[00:47:52.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:49:0) after 1m28s
[00:49:20.746] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:49:0)
[00:49:20.789] Build status: STARTED
[00:49:20.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:50:0) after 9m1s
[00:58:21.857] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:50:0)
[00:58:21.886] Build status: STARTED
[00:58:21.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:51:0) after 6m55s
[01:05:17.076] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:51:0)
[01:05:17.097] Build status: STARTED
[01:05:17.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:52:0) after 4m3s
[01:09:20.503] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:52:0)
[01:09:20.522] Build status: STARTED
[01:09:20.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:53:0) after 3m55s
[01:13:15.719] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:53:0)
[01:13:15.736] Build status: STARTED
[01:13:15.736] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:54:0) after 4m32s
[01:17:48.044] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:54:0)
[01:17:48.075] Build status: STARTED
[01:17:48.075] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:55:0) after 2m23s
[01:20:11.176] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:55:0)
[01:20:11.201] Build status: STARTED
[01:20:11.201] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:56:0) after 6m19s
[01:26:30.338] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:56:0)
[01:26:30.361] Build status: STARTED
[01:26:30.361] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:57:0) after 4m20s
[01:30:50.380] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:57:0)
[01:30:50.406] Build status: STARTED
[01:30:50.406] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:58:0) after 4m23s
[01:35:13.529] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:58:0)
[01:35:13.549] Build status: STARTED
[01:35:13.549] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:59:0) after 9m20s
[01:44:33.863] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:59:0)
[01:44:33.893] Build status: STARTED
[01:44:33.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:60:0) after 7m51s
[01:52:24.914] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:60:0)
[01:52:25.033] Build status: STARTED
[01:52:25.033] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:61:0) after 3m11s
[01:55:36.056] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:61:0)
[01:55:36.101] Build status: STARTED
[01:55:36.101] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:62:0) after 3m42s
[01:59:18.082] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:62:0)
[01:59:18.104] Build status: STARTED
[01:59:18.104] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:63:0) after 2m56s
[02:02:14.143] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:63:0)
[02:02:14.175] Build status: STARTED
[02:02:14.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:64:0) after 3m0s
[02:05:14.203] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:64:0)
[02:05:14.242] Build status: STARTED
[02:05:14.242] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:65:0) after 6m54s
[02:12:08.425] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:65:0)
[02:12:08.449] Build status: STARTED
[02:12:08.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:66:0) after 6m2s
[02:18:10.562] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:66:0)
[02:18:10.582] Build status: STARTED
[02:18:10.582] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:67:0) after 4m15s
[02:22:25.648] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:67:0)
[02:22:25.682] Build status: STARTED
[02:22:25.682] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8935015348740219808:68:0) after 8m36s
[02:24:34.757] Received PubSub notification, asking Buildbucket for the build status
[02:24:34.776] Build:
{
"id": "8769585284247378833",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 30334,
"createdBy": "project:chromium",
"createTime": "2023-09-18T20:26:11.087091574Z",
"startTime": "2023-09-18T20:29:48.422002Z",
"endTime": "2023-09-19T02:24:34.493163665Z",
"updateTime": "2023-09-19T02:24:34.493163665Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2987a72c6998f5d37ce8491f447f6f43b8f96d5a",
"ref": "refs/heads/main"
}
}
}
[02:24:34.776] Invocation finished in 5h58m27.358153019s with status FAILED