[22:10:22.997] New invocation is queued and will start shortly
[22:10:25.356] Starting the invocation (attempt 1)
[22:10:25.406] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d6448db0e00b3f7934f6d4a40939fd562756a76c"
[22:10:25.406] Popped gitiles commit info from properties and tags
[22:10:25.406] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[22:10:25.406] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[22:10:25.406] Buildbucket request:
{
"requestId": "9003771886814301552",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d6448db0e00b3f7934f6d4a40939fd562756a76c",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "9003771886814301552"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[22:10:26.005] Scheduled build:
{
"id": "8838341823510736033",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 22981,
"createdBy": "project:chromium",
"createTime": "2021-08-20T22:10:25.506805803Z",
"updateTime": "2021-08-20T22:10:25.751416595Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d6448db0e00b3f7934f6d4a40939fd562756a76c",
"ref": "refs/heads/main"
}
},
"canary": true
}
[22:10:26.005] Task URL: https://cr-buildbucket.appspot.com/build/8838341823510736033
[22:10:26.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:2:0) after 3m5s
[22:13:18.286] Received PubSub notification, asking Buildbucket for the build status
[22:13:18.340] Build status: STARTED
[22:13:31.050] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:2:0)
[22:13:31.075] Build status: STARTED
[22:13:31.075] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:4:0) after 1m44s
[22:15:15.116] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:4:0)
[22:15:15.199] Build status: STARTED
[22:15:15.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:5:0) after 6m25s
[22:21:40.466] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:5:0)
[22:21:40.494] Build status: STARTED
[22:21:40.494] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:6:0) after 4m1s
[22:25:41.539] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:6:0)
[22:25:41.565] Build status: STARTED
[22:25:41.565] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:7:0) after 9m2s
[22:34:43.665] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:7:0)
[22:34:43.695] Build status: STARTED
[22:34:43.695] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:8:0) after 2m57s
[22:37:41.002] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:8:0)
[22:37:41.039] Build status: STARTED
[22:37:41.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:9:0) after 6m59s
[22:44:40.123] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:9:0)
[22:44:40.162] Build status: STARTED
[22:44:40.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:10:0) after 4m10s
[22:48:50.272] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:10:0)
[22:48:50.301] Build status: STARTED
[22:48:50.301] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:11:0) after 9m2s
[22:57:52.325] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:11:0)
[22:57:52.356] Build status: STARTED
[22:57:52.356] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:12:0) after 9m49s
[23:07:41.379] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:12:0)
[23:07:41.409] Build status: STARTED
[23:07:41.409] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:13:0) after 3m35s
[23:11:16.751] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:13:0)
[23:11:16.803] Build status: STARTED
[23:11:16.803] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:14:0) after 7m1s
[23:18:17.835] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:14:0)
[23:18:17.863] Build status: STARTED
[23:18:17.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:15:0) after 5m5s
[23:23:22.981] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:15:0)
[23:23:23.015] Build status: STARTED
[23:23:23.015] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:16:0) after 3m3s
[23:26:26.036] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:16:0)
[23:26:26.067] Build status: STARTED
[23:26:26.067] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:17:0) after 5m58s
[23:32:24.091] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:17:0)
[23:32:24.124] Build status: STARTED
[23:32:24.124] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:18:0) after 8m56s
[23:41:20.151] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:18:0)
[23:41:20.191] Build status: STARTED
[23:41:20.191] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:19:0) after 7m25s
[23:48:45.205] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:19:0)
[23:48:45.274] Build status: STARTED
[23:48:45.274] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:20:0) after 6m31s
[23:55:16.334] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:20:0)
[23:55:16.363] Build status: STARTED
[23:55:16.363] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:21:0) after 3m48s
[23:59:04.664] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:21:0)
[23:59:04.695] Build status: STARTED
[23:59:04.695] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:22:0) after 7m52s
[00:06:56.827] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:22:0)
[00:06:56.853] Build status: STARTED
[00:06:56.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:23:0) after 5m25s
[00:12:22.141] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:23:0)
[00:12:22.175] Build status: STARTED
[00:12:22.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:24:0) after 8m29s
[00:20:51.447] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:24:0)
[00:20:51.476] Build status: STARTED
[00:20:51.476] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:25:0) after 7m3s
[00:27:54.499] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:25:0)
[00:27:54.614] Build status: STARTED
[00:27:54.614] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:26:0) after 1m29s
[00:29:23.807] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:26:0)
[00:29:23.838] Build status: STARTED
[00:29:23.838] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:27:0) after 2m55s
[00:32:18.917] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:27:0)
[00:32:19.001] Build status: STARTED
[00:32:19.001] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:28:0) after 8m40s
[00:40:59.027] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:28:0)
[00:40:59.076] Build status: STARTED
[00:40:59.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:29:0) after 9m45s
[00:50:44.099] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:29:0)
[00:50:44.127] Build status: STARTED
[00:50:44.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:30:0) after 1m14s
[00:51:58.206] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:30:0)
[00:51:58.243] Build status: STARTED
[00:51:58.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:31:0) after 9m17s
[01:01:15.431] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:31:0)
[01:01:15.461] Build status: STARTED
[01:01:15.461] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:32:0) after 3m13s
[01:04:28.484] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:32:0)
[01:04:28.514] Build status: STARTED
[01:04:28.514] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:33:0) after 8m48s
[01:13:16.547] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:33:0)
[01:13:16.578] Build status: STARTED
[01:13:16.578] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:34:0) after 9m29s
[01:22:45.604] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:34:0)
[01:22:45.634] Build status: STARTED
[01:22:45.634] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:35:0) after 8m21s
[01:31:06.759] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:35:0)
[01:31:06.788] Build status: STARTED
[01:31:06.788] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:36:0) after 3m11s
[01:34:17.858] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:36:0)
[01:34:17.891] Build status: STARTED
[01:34:17.891] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:37:0) after 9m34s
[01:43:51.924] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:37:0)
[01:43:52.004] Build status: STARTED
[01:43:52.004] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:38:0) after 9m39s
[01:53:31.032] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:38:0)
[01:53:31.058] Build status: STARTED
[01:53:31.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:39:0) after 5m14s
[01:58:45.041] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:39:0)
[01:58:45.080] Build status: STARTED
[01:58:45.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:40:0) after 6m10s
[02:04:55.156] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:40:0)
[02:04:55.188] Build status: STARTED
[02:04:55.188] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:41:0) after 8m48s
[02:13:43.311] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:41:0)
[02:13:43.339] Build status: STARTED
[02:13:43.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:42:0) after 5m50s
[02:19:33.407] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:42:0)
[02:19:33.439] Build status: STARTED
[02:19:33.439] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:43:0) after 4m47s
[02:24:20.547] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:43:0)
[02:24:20.575] Build status: STARTED
[02:24:20.575] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:44:0) after 8m6s
[02:32:26.640] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:44:0)
[02:32:26.687] Build status: STARTED
[02:32:26.687] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:45:0) after 5m42s
[02:38:08.815] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:45:0)
[02:38:08.846] Build status: STARTED
[02:38:08.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:46:0) after 5m1s
[02:43:09.825] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:46:0)
[02:43:09.967] Build status: STARTED
[02:43:09.967] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:47:0) after 5m15s
[02:48:24.990] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:47:0)
[02:48:25.041] Build status: STARTED
[02:48:25.041] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:48:0) after 3m28s
[02:51:53.069] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:48:0)
[02:51:53.100] Build status: STARTED
[02:51:53.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:49:0) after 3m33s
[02:55:26.124] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:49:0)
[02:55:26.182] Build status: STARTED
[02:55:26.182] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:50:0) after 8m10s
[03:03:36.209] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:50:0)
[03:03:36.266] Build status: STARTED
[03:03:36.266] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:51:0) after 8m15s
[03:11:51.420] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:51:0)
[03:11:51.452] Build status: STARTED
[03:11:51.452] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:52:0) after 8m11s
[03:20:02.505] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:52:0)
[03:20:02.546] Build status: STARTED
[03:20:02.546] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:53:0) after 4m19s
[03:24:21.691] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:53:0)
[03:24:21.725] Build status: STARTED
[03:24:21.725] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:54:0) after 5m36s
[03:29:57.918] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:54:0)
[03:29:57.944] Build status: STARTED
[03:29:57.944] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:55:0) after 6m1s
[03:35:58.969] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:55:0)
[03:35:59.058] Build status: STARTED
[03:35:59.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:56:0) after 7m1s
[03:43:00.206] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:56:0)
[03:43:00.448] Build status: STARTED
[03:43:00.448] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:57:0) after 8m2s
[03:51:02.482] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:57:0)
[03:51:02.513] Build status: STARTED
[03:51:02.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:58:0) after 1m2s
[03:52:04.539] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:58:0)
[03:52:04.861] Build status: STARTED
[03:52:04.861] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:59:0) after 7m28s
[03:59:32.936] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:59:0)
[03:59:32.969] Build status: STARTED
[03:59:32.969] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:60:0) after 1m9s
[04:00:42.112] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:60:0)
[04:00:42.147] Build status: STARTED
[04:00:42.147] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:61:0) after 8m16s
[04:08:58.177] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:61:0)
[04:08:58.212] Build status: STARTED
[04:08:58.212] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:62:0) after 1m45s
[04:10:43.391] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:62:0)
[04:10:43.426] Build status: STARTED
[04:10:43.426] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:63:0) after 1m12s
[04:11:55.585] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:63:0)
[04:11:55.624] Build status: STARTED
[04:11:55.624] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:64:0) after 8m49s
[04:20:44.702] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:64:0)
[04:20:44.727] Build status: STARTED
[04:20:44.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:65:0) after 5m51s
[04:26:35.862] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:65:0)
[04:26:35.891] Build status: STARTED
[04:26:35.891] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9003771886814301552:66:0) after 7m26s
[04:30:27.041] Received PubSub notification, asking Buildbucket for the build status
[04:30:27.181] Build:
{
"id": "8838341823510736033",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 22981,
"createdBy": "project:chromium",
"createTime": "2021-08-20T22:10:25.506805803Z",
"startTime": "2021-08-20T22:13:06.982179Z",
"endTime": "2021-08-21T04:30:26.523776463Z",
"updateTime": "2021-08-21T04:30:26.524306603Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d6448db0e00b3f7934f6d4a40939fd562756a76c",
"ref": "refs/heads/main"
}
},
"canary": true
}
[04:30:27.181] Invocation finished in 6h20m4.208811337s with status FAILED