[03:17:04.148] New invocation is queued and will start shortly
[03:17:05.241] Starting the invocation (attempt 1)
[03:17:05.282] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/931b1943ea667fd94d210e3a4f3e825e6d56a7cb"
[03:17:05.282] Popped gitiles commit info from properties and tags
[03:17:05.282] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[03:17:05.282] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[03:17:05.282] Buildbucket request:
{
"requestId": "8996414237519863584",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "931b1943ea667fd94d210e3a4f3e825e6d56a7cb",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8996414237519863584"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[03:17:05.928] Scheduled build:
{
"id": "8830984175562734977",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23534,
"createdBy": "project:chromium",
"createTime": "2021-11-10T03:17:05.383679536Z",
"updateTime": "2021-11-10T03:17:05.383679536Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "931b1943ea667fd94d210e3a4f3e825e6d56a7cb",
"ref": "refs/heads/main"
}
}
}
[03:17:05.928] Task URL: https://cr-buildbucket.appspot.com/build/8830984175562734977
[03:17:05.928] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:2:0) after 6m15s
[03:19:35.648] Received PubSub notification, asking Buildbucket for the build status
[03:19:35.698] Build status: STARTED
[03:23:21.110] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:2:0)
[03:23:21.275] Build status: STARTED
[03:23:21.275] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:4:0) after 4m14s
[03:27:35.400] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:4:0)
[03:27:35.432] Build status: STARTED
[03:27:35.432] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:5:0) after 1m59s
[03:29:34.518] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:5:0)
[03:29:34.557] Build status: STARTED
[03:29:34.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:6:0) after 2m13s
[03:31:47.584] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:6:0)
[03:31:47.663] Build status: STARTED
[03:31:47.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:7:0) after 7m15s
[03:39:02.695] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:7:0)
[03:39:02.754] Build status: STARTED
[03:39:02.754] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:8:0) after 7m43s
[03:46:45.778] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:8:0)
[03:46:45.820] Build status: STARTED
[03:46:45.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:9:0) after 4m56s
[03:51:41.947] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:9:0)
[03:51:41.977] Build status: STARTED
[03:51:41.977] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:10:0) after 9m48s
[04:01:30.004] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:10:0)
[04:01:30.113] Build status: STARTED
[04:01:30.113] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:11:0) after 9m35s
[04:11:05.274] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:11:0)
[04:11:05.318] Build status: STARTED
[04:11:05.318] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:12:0) after 4m25s
[04:15:30.491] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:12:0)
[04:15:30.522] Build status: STARTED
[04:15:30.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:13:0) after 4m14s
[04:19:44.685] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:13:0)
[04:19:44.711] Build status: STARTED
[04:19:44.711] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:14:0) after 9m26s
[04:29:10.849] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:14:0)
[04:29:10.882] Build status: STARTED
[04:29:10.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:15:0) after 3m42s
[04:32:52.932] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:15:0)
[04:32:52.988] Build status: STARTED
[04:32:52.988] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:16:0) after 5m2s
[04:37:55.014] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:16:0)
[04:37:55.047] Build status: STARTED
[04:37:55.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:17:0) after 4m5s
[04:42:00.708] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:17:0)
[04:42:00.762] Build status: STARTED
[04:42:00.762] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:18:0) after 2m34s
[04:44:34.835] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:18:0)
[04:44:34.863] Build status: STARTED
[04:44:34.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:19:0) after 3m35s
[04:48:09.921] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:19:0)
[04:48:09.952] Build status: STARTED
[04:48:09.952] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:20:0) after 3m27s
[04:51:37.103] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:20:0)
[04:51:37.134] Build status: STARTED
[04:51:37.134] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:21:0) after 6m40s
[04:58:17.294] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:21:0)
[04:58:17.430] Build status: STARTED
[04:58:17.430] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:22:0) after 3m55s
[05:02:12.453] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:22:0)
[05:02:12.481] Build status: STARTED
[05:02:12.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:23:0) after 1m30s
[05:03:42.511] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:23:0)
[05:03:42.540] Build status: STARTED
[05:03:42.540] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:24:0) after 9m52s
[05:13:34.585] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:24:0)
[05:13:34.615] Build status: STARTED
[05:13:34.615] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:25:0) after 6m53s
[05:20:27.872] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:25:0)
[05:20:27.915] Build status: STARTED
[05:20:27.915] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:26:0) after 3m32s
[05:23:59.954] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:26:0)
[05:23:59.984] Build status: STARTED
[05:23:59.984] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:27:0) after 3m14s
[05:27:14.073] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:27:0)
[05:27:14.103] Build status: STARTED
[05:27:14.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:28:0) after 6m5s
[05:33:19.127] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:28:0)
[05:33:19.176] Build status: STARTED
[05:33:19.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:29:0) after 2m25s
[05:35:44.215] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:29:0)
[05:35:44.251] Build status: STARTED
[05:35:44.251] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:30:0) after 8m23s
[05:46:58.287] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:30:0)
[05:46:58.349] Build status: STARTED
[05:46:58.349] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:31:0) after 7m50s
[05:54:48.391] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:31:0)
[05:54:48.424] Build status: STARTED
[05:54:48.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:32:0) after 3m28s
[05:58:16.446] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:32:0)
[05:58:16.484] Build status: STARTED
[05:58:16.484] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:33:0) after 8m2s
[06:06:18.511] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:33:0)
[06:06:18.545] Build status: STARTED
[06:06:18.545] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:34:0) after 1m6s
[06:07:24.576] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:34:0)
[06:07:24.613] Build status: STARTED
[06:07:24.613] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:35:0) after 8m42s
[06:16:06.728] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:35:0)
[06:16:06.792] Build status: STARTED
[06:16:06.792] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:36:0) after 1m58s
[06:18:04.916] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:36:0)
[06:18:04.944] Build status: STARTED
[06:18:04.944] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:37:0) after 9m56s
[06:28:01.067] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:37:0)
[06:28:01.098] Build status: STARTED
[06:28:01.098] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:38:0) after 3m44s
[06:31:46.911] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:38:0)
[06:31:46.945] Build status: STARTED
[06:31:46.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:39:0) after 1m6s
[06:32:52.969] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:39:0)
[06:32:53.001] Build status: STARTED
[06:32:53.001] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:40:0) after 1m54s
[06:34:47.026] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:40:0)
[06:34:47.058] Build status: STARTED
[06:34:47.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:41:0) after 6m56s
[06:41:43.181] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:41:0)
[06:41:43.219] Build status: STARTED
[06:41:43.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:42:0) after 6m5s
[06:47:48.347] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:42:0)
[06:47:48.699] Build status: STARTED
[06:47:48.699] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:43:0) after 9m12s
[06:57:00.765] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:43:0)
[06:57:00.824] Build status: STARTED
[06:57:00.824] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:44:0) after 4m28s
[07:01:28.879] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:44:0)
[07:01:28.931] Build status: STARTED
[07:01:28.931] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:45:0) after 9m54s
[07:11:23.284] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:45:0)
[07:11:23.342] Build status: STARTED
[07:11:23.342] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:46:0) after 1m47s
[07:13:10.915] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:46:0)
[07:13:10.947] Build status: STARTED
[07:13:10.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:47:0) after 3m7s
[07:16:17.972] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:47:0)
[07:16:18.003] Build status: STARTED
[07:16:18.003] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:48:0) after 1m18s
[07:17:36.079] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:48:0)
[07:17:36.112] Build status: STARTED
[07:17:36.112] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:49:0) after 7m51s
[07:25:27.306] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:49:0)
[07:25:27.338] Build status: STARTED
[07:25:27.338] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:50:0) after 2m33s
[07:28:00.367] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:50:0)
[07:28:00.401] Build status: STARTED
[07:28:00.401] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:51:0) after 9m49s
[07:37:49.806] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:51:0)
[07:37:49.868] Build status: STARTED
[07:37:49.868] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:52:0) after 7m32s
[07:45:21.916] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:52:0)
[07:45:21.952] Build status: STARTED
[07:45:21.952] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:53:0) after 1m41s
[07:47:03.191] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:53:0)
[07:47:03.219] Build status: STARTED
[07:47:03.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:54:0) after 6m42s
[07:53:45.244] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:54:0)
[07:53:45.279] Build status: STARTED
[07:53:45.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:55:0) after 1m0s
[07:54:45.303] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:55:0)
[07:54:45.335] Build status: STARTED
[07:54:45.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:56:0) after 8m54s
[08:03:39.490] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:56:0)
[08:03:39.527] Build status: STARTED
[08:03:39.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:57:0) after 9m29s
[08:13:08.749] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:57:0)
[08:13:08.783] Build status: STARTED
[08:13:08.783] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:58:0) after 1m13s
[08:14:21.777] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:58:0)
[08:14:21.820] Build status: STARTED
[08:14:21.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:59:0) after 9m23s
[08:23:44.953] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:59:0)
[08:23:44.984] Build status: STARTED
[08:23:44.984] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:60:0) after 7m52s
[08:31:37.038] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:60:0)
[08:31:37.192] Build status: STARTED
[08:31:37.192] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:61:0) after 9m15s
[08:40:52.307] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:61:0)
[08:40:52.336] Build status: STARTED
[08:40:52.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:62:0) after 1m48s
[08:42:40.403] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:62:0)
[08:42:40.541] Build status: STARTED
[08:42:40.541] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8996414237519863584:63:0) after 3m13s
[08:45:12.606] Received PubSub notification, asking Buildbucket for the build status
[08:45:12.634] Build:
{
"id": "8830984175562734977",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23534,
"createdBy": "project:chromium",
"createTime": "2021-11-10T03:17:05.383679536Z",
"startTime": "2021-11-10T03:19:34.643501Z",
"endTime": "2021-11-10T08:45:12.218682105Z",
"updateTime": "2021-11-10T08:45:12.218682105Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "931b1943ea667fd94d210e3a4f3e825e6d56a7cb",
"ref": "refs/heads/main"
}
}
}
[08:45:12.634] Invocation finished in 5h28m8.501485506s with status FAILED