[04:39:42.597] New invocation is queued and will start shortly
[04:39:43.798] Starting the invocation (attempt 1)
[04:39:43.828] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/5fa8a402a4f1b66fc2b0d84a90e7c5c7f25f21a3"
[04:39:43.828] Popped gitiles commit info from properties and tags
[04:39:43.828] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:39:43.828] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:39:43.828] Buildbucket request:
{
"requestId": "8930545043628033680",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5fa8a402a4f1b66fc2b0d84a90e7c5c7f25f21a3",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8930545043628033680"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:39:44.971] Scheduled build:
{
"id": "8765114981609889809",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 30880,
"createdBy": "project:chromium",
"createTime": "2023-11-07T04:39:43.899724020Z",
"updateTime": "2023-11-07T04:39:43.899724020Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5fa8a402a4f1b66fc2b0d84a90e7c5c7f25f21a3",
"ref": "refs/heads/main"
}
}
}
[04:39:44.971] Task URL: https://cr-buildbucket.appspot.com/build/8765114981609889809
[04:39:44.971] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:2:0) after 8m4s
[04:39:45.845] Received PubSub notification, asking Buildbucket for the build status
[04:39:45.879] Build status: SCHEDULED
[04:40:45.503] Received PubSub notification, asking Buildbucket for the build status
[04:40:45.537] Build status: STARTED
[04:47:49.101] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:2:0)
[04:47:49.139] Build status: STARTED
[04:47:49.139] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:5:0) after 1m44s
[04:49:33.180] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:5:0)
[04:49:33.201] Build status: STARTED
[04:49:33.201] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:6:0) after 9m20s
[04:58:53.259] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:6:0)
[04:58:53.284] Build status: STARTED
[04:58:53.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:7:0) after 7m41s
[05:06:34.328] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:7:0)
[05:06:34.366] Build status: STARTED
[05:06:34.366] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:8:0) after 8m38s
[05:15:12.373] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:8:0)
[05:15:12.412] Build status: STARTED
[05:15:12.412] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:9:0) after 9m59s
[05:25:11.552] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:9:0)
[05:25:11.576] Build status: STARTED
[05:25:11.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:10:0) after 2m18s
[05:27:29.595] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:10:0)
[05:27:29.617] Build status: STARTED
[05:27:29.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:11:0) after 8m51s
[05:36:20.799] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:11:0)
[05:36:20.836] Build status: STARTED
[05:36:20.836] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:12:0) after 4m40s
[05:41:01.021] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:12:0)
[05:41:01.056] Build status: STARTED
[05:41:01.056] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:13:0) after 8m8s
[05:49:09.215] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:13:0)
[05:49:09.241] Build status: STARTED
[05:49:09.241] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:14:0) after 7m42s
[05:56:51.333] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:14:0)
[05:56:51.357] Build status: STARTED
[05:56:51.357] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:15:0) after 1m11s
[05:58:02.724] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:15:0)
[05:58:02.745] Build status: STARTED
[05:58:02.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:16:0) after 5m30s
[06:03:34.372] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:16:0)
[06:03:34.391] Build status: STARTED
[06:03:34.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:17:0) after 4m4s
[06:07:38.414] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:17:0)
[06:07:38.441] Build status: STARTED
[06:07:38.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:18:0) after 2m30s
[06:10:08.666] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:18:0)
[06:10:08.727] Build status: STARTED
[06:10:08.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:19:0) after 4m12s
[06:14:20.804] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:19:0)
[06:14:20.825] Build status: STARTED
[06:14:20.825] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:20:0) after 8m14s
[06:22:34.964] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:20:0)
[06:22:35.001] Build status: STARTED
[06:22:35.001] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:21:0) after 7m44s
[06:30:19.153] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:21:0)
[06:30:19.202] Build status: STARTED
[06:30:19.202] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:22:0) after 1m29s
[06:31:48.260] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:22:0)
[06:31:48.279] Build status: STARTED
[06:31:48.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:23:0) after 7m34s
[06:39:22.407] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:23:0)
[06:39:22.424] Build status: STARTED
[06:39:22.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:24:0) after 4m37s
[06:43:59.502] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:24:0)
[06:43:59.524] Build status: STARTED
[06:43:59.524] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:25:0) after 7m59s
[06:51:58.525] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:25:0)
[06:51:58.551] Build status: STARTED
[06:51:58.551] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:26:0) after 6m11s
[06:58:09.595] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:26:0)
[06:58:09.616] Build status: STARTED
[06:58:09.616] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:27:0) after 7m44s
[07:05:53.736] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:27:0)
[07:05:53.756] Build status: STARTED
[07:05:53.756] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:28:0) after 7m1s
[07:12:54.821] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:28:0)
[07:12:54.844] Build status: STARTED
[07:12:54.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:29:0) after 8m31s
[07:21:25.866] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:29:0)
[07:21:25.886] Build status: STARTED
[07:21:25.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:30:0) after 3m28s
[07:24:53.923] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:30:0)
[07:24:53.940] Build status: STARTED
[07:24:53.940] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:31:0) after 4m39s
[07:29:33.340] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:31:0)
[07:29:33.360] Build status: STARTED
[07:29:33.361] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:32:0) after 5m2s
[07:34:35.400] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:32:0)
[07:34:35.424] Build status: STARTED
[07:34:35.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:33:0) after 8m41s
[07:43:16.756] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:33:0)
[07:43:16.777] Build status: STARTED
[07:43:16.777] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:34:0) after 3m30s
[07:46:46.917] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:34:0)
[07:46:46.935] Build status: STARTED
[07:46:46.935] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:35:0) after 4m37s
[07:51:24.083] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:35:0)
[07:51:24.104] Build status: STARTED
[07:51:24.104] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:36:0) after 3m50s
[07:55:14.399] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:36:0)
[07:55:14.420] Build status: STARTED
[07:55:14.420] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:37:0) after 7m56s
[08:03:10.475] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:37:0)
[08:03:10.508] Build status: STARTED
[08:03:10.508] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:38:0) after 8m51s
[08:12:01.542] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:38:0)
[08:12:01.561] Build status: STARTED
[08:12:01.561] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:39:0) after 7m6s
[08:19:07.581] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:39:0)
[08:19:07.597] Build status: STARTED
[08:19:07.597] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:40:0) after 1m42s
[08:20:49.621] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:40:0)
[08:20:49.642] Build status: STARTED
[08:20:49.642] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:41:0) after 5m26s
[08:26:15.760] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:41:0)
[08:26:15.780] Build status: STARTED
[08:26:15.780] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:42:0) after 5m46s
[08:32:02.225] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:42:0)
[08:32:02.248] Build status: STARTED
[08:32:02.248] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:43:0) after 9m29s
[08:41:31.272] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:43:0)
[08:41:31.312] Build status: STARTED
[08:41:31.312] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:44:0) after 8m43s
[08:50:14.333] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:44:0)
[08:50:14.354] Build status: STARTED
[08:50:14.354] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:45:0) after 3m0s
[08:53:14.534] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:45:0)
[08:53:14.559] Build status: STARTED
[08:53:14.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:46:0) after 7m45s
[09:00:59.725] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:46:0)
[09:00:59.749] Build status: STARTED
[09:00:59.749] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:47:0) after 5m33s
[09:06:32.772] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:47:0)
[09:06:32.813] Build status: STARTED
[09:06:32.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:48:0) after 6m6s
[09:12:38.810] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:48:0)
[09:12:38.846] Build status: STARTED
[09:12:38.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:49:0) after 3m22s
[09:16:00.956] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:49:0)
[09:16:00.991] Build status: STARTED
[09:16:00.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:50:0) after 1m8s
[09:17:09.099] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:50:0)
[09:17:09.119] Build status: STARTED
[09:17:09.119] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:51:0) after 4m56s
[09:22:05.145] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:51:0)
[09:22:05.166] Build status: STARTED
[09:22:05.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:52:0) after 1m28s
[09:23:33.311] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:52:0)
[09:23:33.339] Build status: STARTED
[09:23:33.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:53:0) after 4m15s
[09:27:48.544] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:53:0)
[09:27:48.641] Build status: STARTED
[09:27:48.641] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:54:0) after 3m59s
[09:31:47.735] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:54:0)
[09:31:47.766] Build status: STARTED
[09:31:47.766] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:55:0) after 5m30s
[09:37:18.069] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:55:0)
[09:37:18.119] Build status: STARTED
[09:37:18.119] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:56:0) after 1m55s
[09:39:13.142] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:56:0)
[09:39:13.160] Build status: STARTED
[09:39:13.160] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:57:0) after 7m11s
[09:46:24.366] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:57:0)
[09:46:24.388] Build status: STARTED
[09:46:24.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:58:0) after 6m59s
[09:53:23.519] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:58:0)
[09:53:23.540] Build status: STARTED
[09:53:23.540] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:59:0) after 5m39s
[09:59:02.813] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:59:0)
[09:59:02.835] Build status: STARTED
[09:59:02.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:60:0) after 9m54s
[10:08:57.626] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:60:0)
[10:08:57.647] Build status: STARTED
[10:08:57.647] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:61:0) after 1m53s
[10:10:50.692] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:61:0)
[10:10:50.757] Build status: STARTED
[10:10:50.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:62:0) after 8m20s
[10:19:10.817] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:62:0)
[10:19:10.940] Build status: STARTED
[10:19:10.940] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:63:0) after 3m35s
[10:22:45.977] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:63:0)
[10:22:46.000] Build status: STARTED
[10:22:46.000] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:64:0) after 4m3s
[10:26:49.250] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:64:0)
[10:26:49.303] Build status: STARTED
[10:26:49.303] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:65:0) after 2m47s
[10:29:36.615] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:65:0)
[10:29:36.635] Build status: STARTED
[10:29:36.635] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:66:0) after 6m33s
[10:36:09.843] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:66:0)
[10:36:09.863] Build status: STARTED
[10:36:09.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:67:0) after 9m10s
[10:45:20.473] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:67:0)
[10:45:20.539] Build status: STARTED
[10:45:20.539] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8930545043628033680:68:0) after 9m6s
[10:52:02.042] Received PubSub notification, asking Buildbucket for the build status
[10:52:02.063] Build:
{
"id": "8765114981609889809",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 30880,
"createdBy": "project:chromium",
"createTime": "2023-11-07T04:39:43.899724020Z",
"startTime": "2023-11-07T04:40:44.890015Z",
"endTime": "2023-11-07T10:52:01.143986Z",
"updateTime": "2023-11-07T10:52:01.143986Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5fa8a402a4f1b66fc2b0d84a90e7c5c7f25f21a3",
"ref": "refs/heads/main"
}
}
}
[10:52:02.063] Invocation finished in 6h12m19.47257056s with status FAILED