[15:02:30.574] New invocation is queued and will start shortly
[15:02:32.665] Starting the invocation (attempt 1)
[15:02:32.806] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/bfc238f2159c974c44075706593d802c1ff7c38e"
[15:02:32.806] Popped gitiles commit info from properties and tags
[15:02:32.806] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:02:32.806] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:02:32.806] Buildbucket request:
{
"requestId": "8936485260248967456",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bfc238f2159c974c44075706593d802c1ff7c38e",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8936485260248967456"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:02:34.186] Scheduled build:
{
"id": "8771055197179884113",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 30251,
"createdBy": "project:chromium",
"createTime": "2023-09-02T15:02:32.875718659Z",
"updateTime": "2023-09-02T15:02:32.875718659Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bfc238f2159c974c44075706593d802c1ff7c38e",
"ref": "refs/heads/main"
}
}
}
[15:02:34.186] Task URL: https://cr-buildbucket.appspot.com/build/8771055197179884113
[15:02:34.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:2:0) after 4m32s
[15:02:36.829] Received PubSub notification, asking Buildbucket for the build status
[15:02:36.861] Build status: SCHEDULED
[15:03:55.999] Received PubSub notification, asking Buildbucket for the build status
[15:03:56.018] Build status: STARTED
[15:07:06.199] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:2:0)
[15:07:06.275] Build status: STARTED
[15:07:06.275] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:5:0) after 7m16s
[15:14:22.307] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:5:0)
[15:14:22.449] Build status: STARTED
[15:14:22.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:6:0) after 7m16s
[15:21:38.461] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:6:0)
[15:21:38.541] Build status: STARTED
[15:21:38.541] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:7:0) after 8m49s
[15:30:27.693] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:7:0)
[15:30:27.712] Build status: STARTED
[15:30:27.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:8:0) after 8m40s
[15:39:07.732] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:8:0)
[15:39:07.751] Build status: STARTED
[15:39:07.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:9:0) after 4m23s
[15:43:30.845] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:9:0)
[15:43:30.862] Build status: STARTED
[15:43:30.862] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:10:0) after 6m53s
[15:50:24.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:10:0)
[15:50:24.042] Build status: STARTED
[15:50:24.042] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:11:0) after 5m48s
[15:56:12.161] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:11:0)
[15:56:12.236] Build status: STARTED
[15:56:12.237] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:12:0) after 5m40s
[16:01:52.854] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:12:0)
[16:01:52.873] Build status: STARTED
[16:01:52.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:13:0) after 3m51s
[16:05:43.845] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:13:0)
[16:05:43.867] Build status: STARTED
[16:05:43.867] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:14:0) after 6m11s
[16:11:54.884] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:14:0)
[16:11:54.905] Build status: STARTED
[16:11:54.906] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:15:0) after 7m48s
[16:19:42.938] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:15:0)
[16:19:42.959] Build status: STARTED
[16:19:42.959] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:16:0) after 9m11s
[16:28:54.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:16:0)
[16:28:54.045] Build status: STARTED
[16:28:54.045] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:17:0) after 9m4s
[16:37:58.087] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:17:0)
[16:37:58.129] Build status: STARTED
[16:37:58.129] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:18:0) after 4m34s
[16:42:32.169] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:18:0)
[16:42:32.187] Build status: STARTED
[16:42:32.187] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:19:0) after 3m14s
[16:45:46.318] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:19:0)
[16:45:46.337] Build status: STARTED
[16:45:46.337] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:20:0) after 5m18s
[16:51:04.363] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:20:0)
[16:51:04.382] Build status: STARTED
[16:51:04.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:21:0) after 9m1s
[17:00:05.553] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:21:0)
[17:00:05.568] Build status: STARTED
[17:00:05.568] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:22:0) after 6m49s
[17:06:54.580] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:22:0)
[17:06:54.599] Build status: STARTED
[17:06:54.599] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:23:0) after 6m34s
[17:13:29.002] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:23:0)
[17:13:29.049] Build status: STARTED
[17:13:29.049] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:24:0) after 5m41s
[17:19:10.274] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:24:0)
[17:19:10.294] Build status: STARTED
[17:19:10.294] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:25:0) after 5m1s
[17:24:11.309] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:25:0)
[17:24:11.340] Build status: STARTED
[17:24:11.340] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:26:0) after 3m0s
[17:27:11.354] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:26:0)
[17:27:11.388] Build status: STARTED
[17:27:11.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:27:0) after 8m53s
[17:36:04.513] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:27:0)
[17:36:04.545] Build status: STARTED
[17:36:04.545] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:28:0) after 5m49s
[17:41:53.570] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:28:0)
[17:41:53.591] Build status: STARTED
[17:41:53.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:29:0) after 2m40s
[17:44:33.706] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:29:0)
[17:44:33.730] Build status: STARTED
[17:44:33.730] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:30:0) after 8m17s
[17:52:50.851] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:30:0)
[17:52:50.890] Build status: STARTED
[17:52:50.890] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:31:0) after 9m5s
[18:01:56.015] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:31:0)
[18:01:56.061] Build status: STARTED
[18:01:56.061] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:32:0) after 2m9s
[18:04:05.176] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:32:0)
[18:04:05.202] Build status: STARTED
[18:04:05.202] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:33:0) after 2m27s
[18:06:32.544] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:33:0)
[18:06:32.559] Build status: STARTED
[18:06:32.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:34:0) after 8m3s
[18:14:35.852] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:34:0)
[18:14:35.868] Build status: STARTED
[18:14:35.868] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:35:0) after 9m58s
[18:24:33.990] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:35:0)
[18:24:34.009] Build status: STARTED
[18:24:34.009] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:36:0) after 8m15s
[18:32:49.074] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:36:0)
[18:32:49.088] Build status: STARTED
[18:32:49.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:37:0) after 4m6s
[18:36:55.106] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:37:0)
[18:36:55.126] Build status: STARTED
[18:36:55.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:38:0) after 4m40s
[18:41:35.139] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:38:0)
[18:41:35.158] Build status: STARTED
[18:41:35.158] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:39:0) after 9m51s
[18:51:26.290] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:39:0)
[18:51:26.308] Build status: STARTED
[18:51:26.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:40:0) after 9m47s
[19:01:13.411] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:40:0)
[19:01:13.435] Build status: STARTED
[19:01:13.435] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:41:0) after 2m44s
[19:03:57.528] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:41:0)
[19:03:57.545] Build status: STARTED
[19:03:57.545] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:42:0) after 9m39s
[19:13:36.712] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:42:0)
[19:13:36.729] Build status: STARTED
[19:13:36.729] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:43:0) after 6m9s
[19:19:46.037] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:43:0)
[19:19:46.057] Build status: STARTED
[19:19:46.057] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:44:0) after 1m10s
[19:20:56.075] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:44:0)
[19:20:56.095] Build status: STARTED
[19:20:56.095] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:45:0) after 1m0s
[19:21:56.136] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:45:0)
[19:21:56.164] Build status: STARTED
[19:21:56.164] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:46:0) after 6m44s
[19:28:40.271] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:46:0)
[19:28:40.292] Build status: STARTED
[19:28:40.292] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:47:0) after 2m19s
[19:30:59.316] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:47:0)
[19:30:59.396] Build status: STARTED
[19:30:59.396] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:48:0) after 6m58s
[19:37:57.557] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:48:0)
[19:37:57.590] Build status: STARTED
[19:37:57.590] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:49:0) after 9m58s
[19:47:55.609] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:49:0)
[19:47:55.631] Build status: STARTED
[19:47:55.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:50:0) after 1m28s
[19:49:23.645] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:50:0)
[19:49:23.665] Build status: STARTED
[19:49:23.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:51:0) after 3m7s
[19:52:30.709] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:51:0)
[19:52:30.734] Build status: STARTED
[19:52:30.734] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:52:0) after 4m5s
[19:56:35.760] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:52:0)
[19:56:35.778] Build status: STARTED
[19:56:35.778] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:53:0) after 4m49s
[20:01:24.878] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:53:0)
[20:01:24.900] Build status: STARTED
[20:01:24.900] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:54:0) after 6m42s
[20:08:06.916] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:54:0)
[20:08:06.989] Build status: STARTED
[20:08:06.989] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:55:0) after 2m54s
[20:11:01.051] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:55:0)
[20:11:01.072] Build status: STARTED
[20:11:01.072] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:56:0) after 6m9s
[20:17:10.101] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:56:0)
[20:17:10.122] Build status: STARTED
[20:17:10.122] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:57:0) after 3m33s
[20:20:43.136] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:57:0)
[20:20:43.244] Build status: STARTED
[20:20:43.244] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:58:0) after 1m49s
[20:22:32.266] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:58:0)
[20:22:32.285] Build status: STARTED
[20:22:32.285] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:59:0) after 8m40s
[20:31:12.414] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:59:0)
[20:31:12.440] Build status: STARTED
[20:31:12.440] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:60:0) after 1m29s
[20:32:41.470] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:60:0)
[20:32:41.490] Build status: STARTED
[20:32:41.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:61:0) after 5m27s
[20:38:08.515] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:61:0)
[20:38:08.535] Build status: STARTED
[20:38:08.535] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:62:0) after 1m2s
[20:39:10.551] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:62:0)
[20:39:10.568] Build status: STARTED
[20:39:10.568] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:63:0) after 4m43s
[20:43:53.730] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:63:0)
[20:43:53.745] Build status: STARTED
[20:43:53.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:64:0) after 2m11s
[20:46:04.763] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:64:0)
[20:46:04.793] Build status: STARTED
[20:46:04.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:65:0) after 4m22s
[20:50:26.771] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:65:0)
[20:50:26.789] Build status: STARTED
[20:50:26.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8936485260248967456:66:0) after 6m59s
[20:54:22.026] Received PubSub notification, asking Buildbucket for the build status
[20:54:22.049] Build:
{
"id": "8771055197179884113",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 30251,
"createdBy": "project:chromium",
"createTime": "2023-09-02T15:02:32.875718659Z",
"startTime": "2023-09-02T15:03:53.046908Z",
"endTime": "2023-09-02T20:54:20.958021120Z",
"updateTime": "2023-09-02T20:54:20.958021120Z",
"status": "INFRA_FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bfc238f2159c974c44075706593d802c1ff7c38e",
"ref": "refs/heads/main"
}
}
}
[20:54:22.049] Invocation finished in 5h51m51.484100059s with status FAILED