[23:20:35.400] New invocation is queued and will start shortly
[23:20:36.518] Starting the invocation (attempt 1)
[23:20:36.555] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/aafc5ec4af2b69ac0d72b4fe3caf79f9a5cc75ab"
[23:20:36.556] Popped gitiles commit info from properties and tags
[23:20:36.556] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[23:20:36.556] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[23:20:36.556] Buildbucket request:
{
"requestId": "9000052994156534096",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "aafc5ec4af2b69ac0d72b4fe3caf79f9a5cc75ab",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "9000052994156534096"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[23:20:37.110] Scheduled build:
{
"id": "8834622932219758049",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23270,
"createdBy": "project:chromium",
"createTime": "2021-09-30T23:20:36.616950926Z",
"updateTime": "2021-09-30T23:20:36.838612489Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "aafc5ec4af2b69ac0d72b4fe3caf79f9a5cc75ab",
"ref": "refs/heads/main"
}
}
}
[23:20:37.110] Task URL: https://cr-buildbucket.appspot.com/build/8834622932219758049
[23:20:37.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:2:0) after 9m59s
[23:23:59.988] Received PubSub notification, asking Buildbucket for the build status
[23:24:00.022] Build status: STARTED
[23:30:36.132] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:2:0)
[23:30:36.178] Build status: STARTED
[23:30:36.178] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:4:0) after 4m6s
[23:34:42.488] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:4:0)
[23:34:42.556] Build status: STARTED
[23:34:42.556] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:5:0) after 2m11s
[23:36:53.642] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:5:0)
[23:36:53.681] Build status: STARTED
[23:36:53.681] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:6:0) after 5m3s
[23:41:57.165] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:6:0)
[23:41:57.199] Build status: STARTED
[23:41:57.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:7:0) after 2m54s
[23:44:51.418] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:7:0)
[23:44:51.540] Build status: STARTED
[23:44:51.540] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:8:0) after 1m19s
[23:46:10.562] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:8:0)
[23:46:10.618] Build status: STARTED
[23:46:10.618] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:9:0) after 4m7s
[23:50:17.644] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:9:0)
[23:50:17.685] Build status: STARTED
[23:50:17.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:10:0) after 1m30s
[23:51:47.727] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:10:0)
[23:51:47.772] Build status: STARTED
[23:51:47.772] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:11:0) after 8m23s
[00:00:10.958] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:11:0)
[00:00:11.044] Build status: STARTED
[00:00:11.044] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:12:0) after 6m42s
[00:06:53.475] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:12:0)
[00:06:53.505] Build status: STARTED
[00:06:53.505] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:13:0) after 4m20s
[00:11:13.650] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:13:0)
[00:11:13.701] Build status: STARTED
[00:11:13.701] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:14:0) after 5m16s
[00:16:29.725] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:14:0)
[00:16:29.753] Build status: STARTED
[00:16:29.753] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:15:0) after 4m9s
[00:20:38.775] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:15:0)
[00:20:38.810] Build status: STARTED
[00:20:38.810] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:16:0) after 9m9s
[00:29:48.082] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:16:0)
[00:29:48.107] Build status: STARTED
[00:29:48.107] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:17:0) after 5m43s
[00:35:31.235] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:17:0)
[00:35:31.266] Build status: STARTED
[00:35:31.266] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:18:0) after 8m4s
[00:43:35.289] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:18:0)
[00:43:35.320] Build status: STARTED
[00:43:35.320] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:19:0) after 6m49s
[00:50:24.345] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:19:0)
[00:50:24.402] Build status: STARTED
[00:50:24.402] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:20:0) after 8m31s
[00:58:55.497] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:20:0)
[00:58:55.526] Build status: STARTED
[00:58:55.526] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:21:0) after 2m19s
[01:01:14.662] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:21:0)
[01:01:14.691] Build status: STARTED
[01:01:14.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:22:0) after 3m45s
[01:04:59.863] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:22:0)
[01:04:59.898] Build status: STARTED
[01:04:59.898] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:23:0) after 2m34s
[01:07:33.916] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:23:0)
[01:07:33.945] Build status: STARTED
[01:07:33.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:24:0) after 5m10s
[01:12:44.108] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:24:0)
[01:12:44.138] Build status: STARTED
[01:12:44.138] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:25:0) after 7m49s
[01:20:33.397] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:25:0)
[01:20:33.456] Build status: STARTED
[01:20:33.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:26:0) after 6m52s
[01:27:25.480] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:26:0)
[01:27:25.506] Build status: STARTED
[01:27:25.506] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:27:0) after 1m16s
[01:28:41.495] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:27:0)
[01:28:41.536] Build status: STARTED
[01:28:41.536] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:28:0) after 9m55s
[01:38:36.572] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:28:0)
[01:38:36.608] Build status: STARTED
[01:38:36.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:29:0) after 1m4s
[01:39:40.714] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:29:0)
[01:39:40.761] Build status: STARTED
[01:39:40.761] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:30:0) after 5m51s
[01:45:31.878] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:30:0)
[01:45:31.909] Build status: STARTED
[01:45:31.909] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:31:0) after 8m12s
[01:53:43.997] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:31:0)
[01:53:44.028] Build status: STARTED
[01:53:44.028] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:32:0) after 4m18s
[01:58:02.409] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:32:0)
[01:58:02.448] Build status: STARTED
[01:58:02.448] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:33:0) after 2m23s
[02:00:25.535] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:33:0)
[02:00:25.583] Build status: STARTED
[02:00:25.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:34:0) after 8m46s
[02:09:11.607] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:34:0)
[02:09:11.636] Build status: STARTED
[02:09:11.636] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:35:0) after 9m39s
[02:18:50.791] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:35:0)
[02:18:50.824] Build status: STARTED
[02:18:50.824] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:36:0) after 9m57s
[02:28:47.907] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:36:0)
[02:28:55.966] Build status: STARTED
[02:28:55.967] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:37:0) after 8m53s
[02:37:49.261] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:37:0)
[02:37:49.320] Build status: STARTED
[02:37:49.320] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:38:0) after 3m28s
[02:41:17.510] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:38:0)
[02:41:17.550] Build status: STARTED
[02:41:17.550] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:39:0) after 8m53s
[02:50:10.679] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:39:0)
[02:50:10.708] Build status: STARTED
[02:50:10.708] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:40:0) after 4m20s
[02:54:30.743] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:40:0)
[02:54:30.785] Build status: STARTED
[02:54:30.785] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:41:0) after 1m33s
[02:56:03.854] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:41:0)
[02:56:03.889] Build status: STARTED
[02:56:03.889] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:42:0) after 4m26s
[03:00:30.047] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:42:0)
[03:00:30.077] Build status: STARTED
[03:00:30.077] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:43:0) after 9m39s
[03:10:09.263] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:43:0)
[03:10:09.339] Build status: STARTED
[03:10:09.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:44:0) after 8m1s
[03:18:10.471] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:44:0)
[03:18:10.499] Build status: STARTED
[03:18:10.499] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:45:0) after 9m26s
[03:27:36.525] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:45:0)
[03:27:36.552] Build status: STARTED
[03:27:36.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:46:0) after 5m30s
[03:33:06.793] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:46:0)
[03:33:06.838] Build status: STARTED
[03:33:06.838] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:47:0) after 6m33s
[03:39:40.306] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:47:0)
[03:39:40.346] Build status: STARTED
[03:39:40.346] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:48:0) after 7m43s
[03:47:23.323] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:48:0)
[03:47:23.360] Build status: STARTED
[03:47:23.360] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:49:0) after 4m38s
[03:52:01.393] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:49:0)
[03:52:01.427] Build status: STARTED
[03:52:01.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:50:0) after 3m48s
[03:55:49.455] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:50:0)
[03:55:49.542] Build status: STARTED
[03:55:49.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:51:0) after 6m28s
[04:02:17.684] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:51:0)
[04:02:17.722] Build status: STARTED
[04:02:17.722] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:52:0) after 4m17s
[04:06:34.771] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:52:0)
[04:06:34.844] Build status: STARTED
[04:06:34.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:53:0) after 3m20s
[04:09:54.872] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:53:0)
[04:09:54.898] Build status: STARTED
[04:09:54.899] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:54:0) after 1m34s
[04:11:28.945] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:54:0)
[04:11:28.974] Build status: STARTED
[04:11:28.974] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:55:0) after 4m51s
[04:16:19.998] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:55:0)
[04:16:20.032] Build status: STARTED
[04:16:20.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:56:0) after 1m51s
[04:18:11.056] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:56:0)
[04:18:11.088] Build status: STARTED
[04:18:11.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:57:0) after 7m30s
[04:25:41.131] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:57:0)
[04:25:41.170] Build status: STARTED
[04:25:41.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:58:0) after 4m34s
[04:30:15.201] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:58:0)
[04:30:15.234] Build status: STARTED
[04:30:15.234] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:59:0) after 3m19s
[04:33:34.258] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:59:0)
[04:33:34.291] Build status: STARTED
[04:33:34.291] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:60:0) after 3m29s
[04:37:03.479] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:60:0)
[04:37:03.551] Build status: STARTED
[04:37:03.551] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:61:0) after 8m7s
[04:45:10.579] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:61:0)
[04:45:10.611] Build status: STARTED
[04:45:10.611] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:62:0) after 2m42s
[04:47:52.591] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:62:0)
[04:47:52.658] Build status: STARTED
[04:47:52.658] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:63:0) after 5m44s
[04:53:36.804] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:63:0)
[04:53:36.860] Build status: STARTED
[04:53:36.860] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:64:0) after 6m53s
[05:00:29.916] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:64:0)
[05:00:29.948] Build status: STARTED
[05:00:29.948] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000052994156534096:65:0) after 7m26s
[05:03:58.325] Received PubSub notification, asking Buildbucket for the build status
[05:03:58.356] Build:
{
"id": "8834622932219758049",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23270,
"createdBy": "project:chromium",
"createTime": "2021-09-30T23:20:36.616950926Z",
"startTime": "2021-09-30T23:23:59.181557Z",
"endTime": "2021-10-01T05:03:57.777549797Z",
"updateTime": "2021-10-01T05:03:57.778043234Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "aafc5ec4af2b69ac0d72b4fe3caf79f9a5cc75ab",
"ref": "refs/heads/main"
}
}
}
[05:03:58.356] Invocation finished in 5h43m22.970952388s with status FAILED