[18:03:05.258] New invocation is queued and will start shortly
[18:03:06.409] Starting the invocation (attempt 1)
[18:03:06.447] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/322c44d943bc0cbbad8e95c2b34852fb18feaabf"
[18:03:06.448] Popped gitiles commit info from properties and tags
[18:03:06.448] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[18:03:06.448] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[18:03:06.448] Buildbucket request:
{
"requestId": "8939463599153363696",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "322c44d943bc0cbbad8e95c2b34852fb18feaabf",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8939463599153363696"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[18:03:07.141] Scheduled build:
{
"id": "8774033537055578481",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 30088,
"createdBy": "project:chromium",
"createTime": "2023-07-31T18:03:06.631812654Z",
"updateTime": "2023-07-31T18:03:06.631812654Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "322c44d943bc0cbbad8e95c2b34852fb18feaabf",
"ref": "refs/heads/main"
}
},
"canary": true
}
[18:03:07.142] Task URL: https://cr-buildbucket.appspot.com/build/8774033537055578481
[18:03:07.142] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:2:0) after 8m10s
[18:03:08.155] Received PubSub notification, asking Buildbucket for the build status
[18:03:08.192] Build status: SCHEDULED
[18:06:50.588] Received PubSub notification, asking Buildbucket for the build status
[18:06:50.620] Build status: STARTED
[18:11:17.252] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:2:0)
[18:11:17.282] Build status: STARTED
[18:11:17.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:5:0) after 7m6s
[18:18:23.396] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:5:0)
[18:18:23.427] Build status: STARTED
[18:18:23.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:6:0) after 7m58s
[18:26:21.564] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:6:0)
[18:26:21.591] Build status: STARTED
[18:26:21.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:7:0) after 6m11s
[18:32:32.640] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:7:0)
[18:32:32.663] Build status: STARTED
[18:32:32.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:8:0) after 3m6s
[18:35:39.135] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:8:0)
[18:35:39.160] Build status: STARTED
[18:35:39.160] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:9:0) after 9m40s
[18:45:19.179] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:9:0)
[18:45:19.197] Build status: STARTED
[18:45:19.197] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:10:0) after 6m47s
[18:52:06.291] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:10:0)
[18:52:06.321] Build status: STARTED
[18:52:06.321] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:11:0) after 6m5s
[18:58:11.314] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:11:0)
[18:58:11.335] Build status: STARTED
[18:58:11.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:12:0) after 7m46s
[19:05:57.363] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:12:0)
[19:05:57.414] Build status: STARTED
[19:05:57.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:13:0) after 1m8s
[19:07:05.441] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:13:0)
[19:07:05.468] Build status: STARTED
[19:07:05.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:14:0) after 6m47s
[19:13:52.590] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:14:0)
[19:13:52.612] Build status: STARTED
[19:13:52.612] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:15:0) after 8m3s
[19:21:55.633] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:15:0)
[19:21:55.656] Build status: STARTED
[19:21:55.656] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:16:0) after 3m7s
[19:25:03.079] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:16:0)
[19:25:03.106] Build status: STARTED
[19:25:03.106] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:17:0) after 2m57s
[19:28:00.225] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:17:0)
[19:28:00.282] Build status: STARTED
[19:28:00.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:18:0) after 8m4s
[19:36:04.435] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:18:0)
[19:36:04.454] Build status: STARTED
[19:36:04.454] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:19:0) after 6m8s
[19:42:13.488] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:19:0)
[19:42:13.523] Build status: STARTED
[19:42:13.523] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:20:0) after 8m2s
[19:50:15.573] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:20:0)
[19:50:15.593] Build status: STARTED
[19:50:15.593] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:21:0) after 2m24s
[19:52:39.939] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:21:0)
[19:52:39.960] Build status: STARTED
[19:52:39.960] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:22:0) after 3m48s
[19:56:28.375] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:22:0)
[19:56:28.394] Build status: STARTED
[19:56:28.395] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:23:0) after 7m46s
[20:04:14.526] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:23:0)
[20:04:14.546] Build status: STARTED
[20:04:14.546] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:24:0) after 6m48s
[20:11:02.644] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:24:0)
[20:11:02.663] Build status: STARTED
[20:11:02.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:25:0) after 4m27s
[20:15:30.068] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:25:0)
[20:15:30.089] Build status: STARTED
[20:15:30.089] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:26:0) after 5m30s
[20:21:00.165] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:26:0)
[20:21:00.189] Build status: STARTED
[20:21:00.189] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:27:0) after 8m43s
[20:29:43.239] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:27:0)
[20:29:43.298] Build status: STARTED
[20:29:43.298] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:28:0) after 1m10s
[20:30:53.344] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:28:0)
[20:30:53.408] Build status: STARTED
[20:30:53.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:29:0) after 7m36s
[20:38:29.529] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:29:0)
[20:38:29.549] Build status: STARTED
[20:38:29.549] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:30:0) after 9m14s
[20:47:43.687] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:30:0)
[20:47:43.706] Build status: STARTED
[20:47:43.706] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:31:0) after 7m51s
[20:55:34.776] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:31:0)
[20:55:34.812] Build status: STARTED
[20:55:34.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:32:0) after 8m27s
[21:04:01.934] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:32:0)
[21:04:01.954] Build status: STARTED
[21:04:01.954] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:33:0) after 3m46s
[21:07:48.125] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:33:0)
[21:07:48.146] Build status: STARTED
[21:07:48.146] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:34:0) after 2m45s
[21:10:33.315] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:34:0)
[21:10:33.342] Build status: STARTED
[21:10:33.342] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:35:0) after 4m43s
[21:15:16.777] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:35:0)
[21:15:16.876] Build status: STARTED
[21:15:16.876] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:36:0) after 5m42s
[21:20:58.944] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:36:0)
[21:20:59.044] Build status: STARTED
[21:20:59.044] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:37:0) after 3m6s
[21:24:05.106] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:37:0)
[21:24:05.126] Build status: STARTED
[21:24:05.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:38:0) after 7m52s
[21:31:57.364] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:38:0)
[21:31:57.383] Build status: STARTED
[21:31:57.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:39:0) after 4m32s
[21:36:29.584] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:39:0)
[21:36:29.604] Build status: STARTED
[21:36:29.604] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:40:0) after 4m1s
[21:40:30.850] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:40:0)
[21:40:30.871] Build status: STARTED
[21:40:30.871] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:41:0) after 9m59s
[21:50:30.031] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:41:0)
[21:50:30.074] Build status: STARTED
[21:50:30.074] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:42:0) after 2m11s
[21:52:41.347] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:42:0)
[21:52:41.368] Build status: STARTED
[21:52:41.369] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:43:0) after 6m26s
[21:59:07.523] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:43:0)
[21:59:07.544] Build status: STARTED
[21:59:07.544] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:44:0) after 5m43s
[22:04:50.626] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:44:0)
[22:04:50.645] Build status: STARTED
[22:04:50.645] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:45:0) after 7m59s
[22:12:49.766] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:45:0)
[22:12:49.786] Build status: STARTED
[22:12:49.786] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:46:0) after 4m25s
[22:17:14.809] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:46:0)
[22:17:14.879] Build status: STARTED
[22:17:14.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:47:0) after 7m16s
[22:24:31.062] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:47:0)
[22:24:31.090] Build status: STARTED
[22:24:31.090] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:48:0) after 9m34s
[22:34:05.542] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:48:0)
[22:34:05.563] Build status: STARTED
[22:34:05.563] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:49:0) after 4m21s
[22:38:26.686] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:49:0)
[22:38:26.706] Build status: STARTED
[22:38:26.706] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:50:0) after 2m42s
[22:41:08.756] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:50:0)
[22:41:08.776] Build status: STARTED
[22:41:08.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:51:0) after 8m5s
[22:49:13.860] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:51:0)
[22:49:13.879] Build status: STARTED
[22:49:13.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:52:0) after 5m39s
[22:54:52.907] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:52:0)
[22:54:52.927] Build status: STARTED
[22:54:52.927] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:53:0) after 1m33s
[22:56:26.125] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:53:0)
[22:56:26.188] Build status: STARTED
[22:56:26.188] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:54:0) after 5m27s
[23:01:53.216] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:54:0)
[23:01:53.235] Build status: STARTED
[23:01:53.235] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:55:0) after 5m9s
[23:07:02.340] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:55:0)
[23:07:02.363] Build status: STARTED
[23:07:02.363] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:56:0) after 7m1s
[23:14:03.851] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:56:0)
[23:14:03.872] Build status: STARTED
[23:14:03.872] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:57:0) after 7m48s
[23:21:51.899] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:57:0)
[23:21:51.917] Build status: STARTED
[23:21:51.917] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:58:0) after 7m45s
[23:29:36.941] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:58:0)
[23:29:36.958] Build status: STARTED
[23:29:36.958] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:59:0) after 6m20s
[23:35:57.106] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:59:0)
[23:35:57.126] Build status: STARTED
[23:35:57.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:60:0) after 3m40s
[23:39:37.315] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:60:0)
[23:39:37.363] Build status: STARTED
[23:39:37.363] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:61:0) after 2m36s
[23:42:13.386] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:61:0)
[23:42:13.409] Build status: STARTED
[23:42:13.409] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:62:0) after 2m12s
[23:44:25.570] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:62:0)
[23:44:25.589] Build status: STARTED
[23:44:25.589] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8939463599153363696:63:0) after 5m28s
[23:45:40.665] Received PubSub notification, asking Buildbucket for the build status
[23:45:40.686] Build:
{
"id": "8774033537055578481",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 30088,
"createdBy": "project:chromium",
"createTime": "2023-07-31T18:03:06.631812654Z",
"startTime": "2023-07-31T18:06:50.105986Z",
"endTime": "2023-07-31T23:45:40.319697704Z",
"updateTime": "2023-07-31T23:45:40.319697704Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "322c44d943bc0cbbad8e95c2b34852fb18feaabf",
"ref": "refs/heads/main"
}
},
"canary": true
}
[23:45:40.686] Invocation finished in 5h42m35.439218556s with status FAILED