[21:33:12.372] New invocation is queued and will start shortly
[21:33:13.592] Starting the invocation (attempt 1)
[21:33:13.623] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/1c573e1605e224f0b4fbda16f05d079d9f289b1c"
[21:33:13.623] Popped gitiles commit info from properties and tags
[21:33:13.623] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:33:13.623] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:33:13.623] Buildbucket request:
{
"requestId": "8941715303792695472",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1c573e1605e224f0b4fbda16f05d079d9f289b1c",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8941715303792695472"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:33:14.038] Scheduled build:
{
"id": "8776285241755529153",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29871,
"createdBy": "project:chromium",
"createTime": "2023-07-06T21:33:13.690592087Z",
"updateTime": "2023-07-06T21:33:13.690592087Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1c573e1605e224f0b4fbda16f05d079d9f289b1c",
"ref": "refs/heads/main"
}
},
"canary": true
}
[21:33:14.038] Task URL: https://cr-buildbucket.appspot.com/build/8776285241755529153
[21:33:14.038] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:2:0) after 7m49s
[21:36:40.544] Received PubSub notification, asking Buildbucket for the build status
[21:36:40.566] Build status: STARTED
[21:41:03.093] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:2:0)
[21:41:03.112] Build status: STARTED
[21:41:03.112] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:4:0) after 2m5s
[21:43:08.170] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:4:0)
[21:43:08.232] Build status: STARTED
[21:43:08.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:5:0) after 5m2s
[21:48:10.266] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:5:0)
[21:48:10.301] Build status: STARTED
[21:48:10.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:6:0) after 3m41s
[21:51:51.323] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:6:0)
[21:51:51.341] Build status: STARTED
[21:51:51.341] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:7:0) after 5m7s
[21:56:58.407] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:7:0)
[21:56:58.424] Build status: STARTED
[21:56:58.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:8:0) after 1m39s
[21:58:37.719] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:8:0)
[21:58:37.736] Build status: STARTED
[21:58:37.736] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:9:0) after 9m9s
[22:07:47.059] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:9:0)
[22:07:47.089] Build status: STARTED
[22:07:47.089] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:10:0) after 4m43s
[22:12:30.580] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:10:0)
[22:12:30.598] Build status: STARTED
[22:12:30.598] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:11:0) after 1m34s
[22:14:04.835] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:11:0)
[22:14:04.858] Build status: STARTED
[22:14:04.858] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:12:0) after 6m8s
[22:20:12.910] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:12:0)
[22:20:12.926] Build status: STARTED
[22:20:12.926] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:13:0) after 9m47s
[22:29:59.977] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:13:0)
[22:30:00.021] Build status: STARTED
[22:30:00.021] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:14:0) after 5m27s
[22:35:27.219] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:14:0)
[22:35:27.237] Build status: STARTED
[22:35:27.237] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:15:0) after 8m26s
[22:43:53.331] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:15:0)
[22:43:53.352] Build status: STARTED
[22:43:53.352] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:16:0) after 5m58s
[22:49:51.381] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:16:0)
[22:49:51.439] Build status: STARTED
[22:49:51.439] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:17:0) after 6m23s
[22:56:14.567] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:17:0)
[22:56:14.584] Build status: STARTED
[22:56:14.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:18:0) after 8m42s
[23:04:56.709] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:18:0)
[23:04:56.727] Build status: STARTED
[23:04:56.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:19:0) after 5m29s
[23:10:25.812] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:19:0)
[23:10:25.826] Build status: STARTED
[23:10:25.826] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:20:0) after 2m6s
[23:12:31.921] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:20:0)
[23:12:31.939] Build status: STARTED
[23:12:31.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:21:0) after 8m8s
[23:20:40.448] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:21:0)
[23:20:40.472] Build status: STARTED
[23:20:40.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:22:0) after 7m26s
[23:28:06.680] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:22:0)
[23:28:06.706] Build status: STARTED
[23:28:06.707] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:23:0) after 6m8s
[23:34:14.731] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:23:0)
[23:34:14.749] Build status: STARTED
[23:34:14.749] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:24:0) after 1m9s
[23:35:23.851] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:24:0)
[23:35:23.872] Build status: STARTED
[23:35:23.872] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:25:0) after 7m5s
[23:42:28.915] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:25:0)
[23:42:28.944] Build status: STARTED
[23:42:28.944] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:26:0) after 2m51s
[23:45:20.176] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:26:0)
[23:45:20.193] Build status: STARTED
[23:45:20.193] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:27:0) after 1m12s
[23:46:32.570] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:27:0)
[23:46:32.593] Build status: STARTED
[23:46:32.593] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:28:0) after 7m53s
[23:54:25.619] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:28:0)
[23:54:25.682] Build status: STARTED
[23:54:25.682] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:29:0) after 7m6s
[00:01:31.669] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:29:0)
[00:01:31.692] Build status: STARTED
[00:01:31.692] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:30:0) after 1m31s
[00:03:02.717] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:30:0)
[00:03:02.739] Build status: STARTED
[00:03:02.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:31:0) after 8m8s
[00:11:10.828] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:31:0)
[00:11:10.844] Build status: STARTED
[00:11:10.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:32:0) after 8m31s
[00:19:42.070] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:32:0)
[00:19:42.153] Build status: STARTED
[00:19:42.153] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:33:0) after 6m41s
[00:26:23.445] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:33:0)
[00:26:23.462] Build status: STARTED
[00:26:23.462] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:34:0) after 6m0s
[00:32:23.490] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:34:0)
[00:32:23.507] Build status: STARTED
[00:32:23.507] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:35:0) after 1m29s
[00:33:52.616] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:35:0)
[00:33:52.638] Build status: STARTED
[00:33:52.638] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:36:0) after 6m3s
[00:39:56.098] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:36:0)
[00:39:56.115] Build status: STARTED
[00:39:56.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:37:0) after 9m38s
[00:49:34.529] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:37:0)
[00:49:34.546] Build status: STARTED
[00:49:34.546] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:38:0) after 1m41s
[00:51:15.843] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:38:0)
[00:51:15.879] Build status: STARTED
[00:51:15.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:39:0) after 5m25s
[00:56:41.197] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:39:0)
[00:56:41.217] Build status: STARTED
[00:56:41.217] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:40:0) after 5m38s
[01:02:19.262] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:40:0)
[01:02:19.283] Build status: STARTED
[01:02:19.283] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:41:0) after 3m27s
[01:05:46.317] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:41:0)
[01:05:46.364] Build status: STARTED
[01:05:46.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:42:0) after 5m35s
[01:11:21.581] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:42:0)
[01:11:21.600] Build status: STARTED
[01:11:21.600] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:43:0) after 1m49s
[01:13:10.860] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:43:0)
[01:13:10.876] Build status: STARTED
[01:13:10.876] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:44:0) after 1m8s
[01:14:18.970] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:44:0)
[01:14:18.992] Build status: STARTED
[01:14:18.992] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:45:0) after 7m46s
[01:22:05.029] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:45:0)
[01:22:05.051] Build status: STARTED
[01:22:05.051] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:46:0) after 2m40s
[01:24:45.396] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:46:0)
[01:24:45.468] Build status: STARTED
[01:24:45.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:47:0) after 8m1s
[01:32:46.643] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:47:0)
[01:32:46.667] Build status: STARTED
[01:32:46.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:48:0) after 5m2s
[01:37:48.754] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:48:0)
[01:37:48.800] Build status: STARTED
[01:37:48.800] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:49:0) after 3m1s
[01:40:50.115] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:49:0)
[01:40:50.134] Build status: STARTED
[01:40:50.134] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:50:0) after 2m48s
[01:43:38.269] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:50:0)
[01:43:38.291] Build status: STARTED
[01:43:38.291] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:51:0) after 5m21s
[01:48:59.315] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:51:0)
[01:48:59.345] Build status: STARTED
[01:48:59.345] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:52:0) after 1m53s
[01:50:52.477] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:52:0)
[01:50:52.494] Build status: STARTED
[01:50:52.495] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:53:0) after 6m2s
[01:56:54.623] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:53:0)
[01:56:54.663] Build status: STARTED
[01:56:54.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:54:0) after 5m21s
[02:02:15.687] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:54:0)
[02:02:15.710] Build status: STARTED
[02:02:15.710] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:55:0) after 3m10s
[02:05:26.406] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:55:0)
[02:05:26.425] Build status: STARTED
[02:05:26.425] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:56:0) after 8m31s
[02:13:57.466] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:56:0)
[02:13:57.561] Build status: STARTED
[02:13:57.561] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:57:0) after 3m5s
[02:17:02.805] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:57:0)
[02:17:02.869] Build status: STARTED
[02:17:02.869] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:58:0) after 6m11s
[02:23:13.887] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:58:0)
[02:23:13.900] Build status: STARTED
[02:23:13.900] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:59:0) after 9m50s
[02:33:04.009] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:59:0)
[02:33:04.036] Build status: STARTED
[02:33:04.036] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:60:0) after 9m57s
[02:43:01.167] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:60:0)
[02:43:01.184] Build status: STARTED
[02:43:01.184] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:61:0) after 3m43s
[02:46:44.437] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:61:0)
[02:46:44.455] Build status: STARTED
[02:46:44.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:62:0) after 3m57s
[02:50:41.614] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:62:0)
[02:50:41.633] Build status: STARTED
[02:50:41.633] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:63:0) after 9m31s
[03:00:12.661] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:63:0)
[03:00:12.684] Build status: STARTED
[03:00:12.684] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:64:0) after 7m9s
[03:07:21.844] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:64:0)
[03:07:21.863] Build status: STARTED
[03:07:21.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:65:0) after 3m13s
[03:10:34.913] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:65:0)
[03:10:34.938] Build status: STARTED
[03:10:34.938] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8941715303792695472:66:0) after 7m49s
[03:14:17.650] Received PubSub notification, asking Buildbucket for the build status
[03:14:17.670] Build:
{
"id": "8776285241755529153",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29871,
"createdBy": "project:chromium",
"createTime": "2023-07-06T21:33:13.690592087Z",
"startTime": "2023-07-06T21:36:39.930874Z",
"endTime": "2023-07-07T03:14:17.436342835Z",
"updateTime": "2023-07-07T03:14:17.436342835Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1c573e1605e224f0b4fbda16f05d079d9f289b1c",
"ref": "refs/heads/main"
}
},
"canary": true
}
[03:14:17.670] Invocation finished in 5h41m5.3055151s with status FAILED