[16:13:03.062] New invocation is queued and will start shortly
[16:13:04.138] Starting the invocation (attempt 1)
[16:13:04.185] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/670685263a19b74d091fea46f00991b7c6232be9"
[16:13:04.185] Popped gitiles commit info from properties and tags
[16:13:04.185] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[16:13:04.185] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[16:13:04.185] Buildbucket request:
{
"requestId": "8953966036692726032",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "670685263a19b74d091fea46f00991b7c6232be9",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8953966036692726032"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[16:13:04.748] Scheduled build:
{
"id": "8788535974683499457",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 28977,
"createdBy": "project:chromium",
"createTime": "2023-02-21T16:13:04.339302751Z",
"updateTime": "2023-02-21T16:13:04.339302751Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "670685263a19b74d091fea46f00991b7c6232be9",
"ref": "refs/heads/main"
}
},
"canary": true
}
[16:13:04.748] Task URL: https://cr-buildbucket.appspot.com/build/8788535974683499457
[16:13:04.748] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:2:0) after 2m36s
[16:15:29.304] Received PubSub notification, asking Buildbucket for the build status
[16:15:29.329] Build status: STARTED
[16:15:40.731] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:2:0)
[16:15:40.759] Build status: STARTED
[16:15:40.759] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:4:0) after 8m15s
[16:23:55.874] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:4:0)
[16:23:55.895] Build status: STARTED
[16:23:55.895] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:5:0) after 7m47s
[16:31:42.916] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:5:0)
[16:31:42.937] Build status: STARTED
[16:31:42.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:6:0) after 1m35s
[16:33:18.197] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:6:0)
[16:33:18.228] Build status: STARTED
[16:33:18.228] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:7:0) after 9m9s
[16:42:27.937] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:7:0)
[16:42:27.964] Build status: STARTED
[16:42:27.964] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:8:0) after 3m45s
[16:46:13.448] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:8:0)
[16:46:13.494] Build status: STARTED
[16:46:13.494] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:9:0) after 2m24s
[16:48:37.513] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:9:0)
[16:48:37.534] Build status: STARTED
[16:48:37.534] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:10:0) after 3m1s
[16:51:38.558] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:10:0)
[16:51:38.593] Build status: STARTED
[16:51:38.593] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:11:0) after 9m0s
[17:00:38.776] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:11:0)
[17:00:38.807] Build status: STARTED
[17:00:38.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:12:0) after 1m46s
[17:02:25.093] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:12:0)
[17:02:25.127] Build status: STARTED
[17:02:25.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:13:0) after 5m50s
[17:08:15.147] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:13:0)
[17:08:15.171] Build status: STARTED
[17:08:15.171] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:14:0) after 6m31s
[17:14:46.247] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:14:0)
[17:14:46.282] Build status: STARTED
[17:14:46.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:15:0) after 3m24s
[17:18:10.528] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:15:0)
[17:18:10.548] Build status: STARTED
[17:18:10.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:16:0) after 4m26s
[17:22:36.758] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:16:0)
[17:22:36.781] Build status: STARTED
[17:22:36.781] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:17:0) after 4m6s
[17:26:43.013] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:17:0)
[17:26:43.178] Build status: STARTED
[17:26:43.178] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:18:0) after 7m23s
[17:34:06.204] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:18:0)
[17:34:06.229] Build status: STARTED
[17:34:06.229] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:19:0) after 1m38s
[17:35:44.361] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:19:0)
[17:35:44.381] Build status: STARTED
[17:35:44.381] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:20:0) after 2m1s
[17:37:45.651] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:20:0)
[17:37:45.672] Build status: STARTED
[17:37:45.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:21:0) after 6m58s
[17:44:43.796] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:21:0)
[17:44:43.822] Build status: STARTED
[17:44:43.822] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:22:0) after 4m16s
[17:48:59.859] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:22:0)
[17:48:59.880] Build status: STARTED
[17:48:59.880] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:23:0) after 6m7s
[17:55:07.233] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:23:0)
[17:55:07.286] Build status: STARTED
[17:55:07.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:24:0) after 7m58s
[18:04:12.427] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:24:0)
[18:04:12.449] Build status: STARTED
[18:04:12.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:25:0) after 9m9s
[18:13:21.532] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:25:0)
[18:13:21.556] Build status: STARTED
[18:13:21.556] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:26:0) after 7m9s
[18:20:30.669] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:26:0)
[18:20:30.696] Build status: STARTED
[18:20:30.696] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:27:0) after 6m31s
[18:27:01.722] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:27:0)
[18:27:01.747] Build status: STARTED
[18:27:01.747] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:28:0) after 7m44s
[18:34:45.779] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:28:0)
[18:34:45.827] Build status: STARTED
[18:34:45.827] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:29:0) after 6m25s
[18:41:10.844] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:29:0)
[18:41:10.867] Build status: STARTED
[18:41:10.867] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:30:0) after 5m22s
[18:46:32.891] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:30:0)
[18:46:32.914] Build status: STARTED
[18:46:32.914] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:31:0) after 3m54s
[18:50:26.939] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:31:0)
[18:50:26.968] Build status: STARTED
[18:50:26.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:32:0) after 1m50s
[18:52:17.083] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:32:0)
[18:52:17.106] Build status: STARTED
[18:52:17.106] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:33:0) after 8m37s
[19:00:54.163] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:33:0)
[19:00:54.194] Build status: STARTED
[19:00:54.194] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:34:0) after 6m48s
[19:07:42.312] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:34:0)
[19:07:42.353] Build status: STARTED
[19:07:42.353] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:35:0) after 6m40s
[19:14:22.388] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:35:0)
[19:14:22.415] Build status: STARTED
[19:14:22.415] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:36:0) after 2m4s
[19:16:26.436] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:36:0)
[19:16:26.468] Build status: STARTED
[19:16:26.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:37:0) after 4m45s
[19:21:11.494] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:37:0)
[19:21:11.513] Build status: STARTED
[19:21:11.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:38:0) after 7m5s
[19:28:16.918] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:38:0)
[19:28:16.943] Build status: STARTED
[19:28:16.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:39:0) after 8m52s
[19:37:08.966] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:39:0)
[19:37:08.995] Build status: STARTED
[19:37:08.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:40:0) after 7m57s
[19:45:06.022] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:40:0)
[19:45:06.048] Build status: STARTED
[19:45:06.048] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:41:0) after 3m15s
[19:48:21.108] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:41:0)
[19:48:21.137] Build status: STARTED
[19:48:21.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:42:0) after 1m10s
[19:49:31.165] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:42:0)
[19:49:31.195] Build status: STARTED
[19:49:31.195] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:43:0) after 1m15s
[19:50:46.258] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:43:0)
[19:50:46.284] Build status: STARTED
[19:50:46.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:44:0) after 7m0s
[19:57:46.552] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:44:0)
[19:57:46.571] Build status: STARTED
[19:57:46.571] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:45:0) after 4m40s
[20:02:26.849] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:45:0)
[20:02:26.871] Build status: STARTED
[20:02:26.871] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:46:0) after 2m44s
[20:05:10.978] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:46:0)
[20:05:11.009] Build status: STARTED
[20:05:11.009] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:47:0) after 3m38s
[20:08:49.055] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:47:0)
[20:08:49.091] Build status: STARTED
[20:08:49.091] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:48:0) after 3m7s
[20:11:56.191] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:48:0)
[20:11:56.213] Build status: STARTED
[20:11:56.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:49:0) after 4m32s
[20:16:28.301] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:49:0)
[20:16:28.334] Build status: STARTED
[20:16:28.334] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:50:0) after 9m57s
[20:26:25.354] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:50:0)
[20:26:25.379] Build status: STARTED
[20:26:25.379] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:51:0) after 4m6s
[20:30:31.442] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:51:0)
[20:30:31.470] Build status: STARTED
[20:30:31.470] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:52:0) after 4m43s
[20:35:14.511] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:52:0)
[20:35:14.573] Build status: STARTED
[20:35:14.573] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:53:0) after 4m37s
[20:39:51.608] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:53:0)
[20:39:51.638] Build status: STARTED
[20:39:51.638] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:54:0) after 7m6s
[20:46:57.712] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:54:0)
[20:46:57.748] Build status: STARTED
[20:46:57.748] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:55:0) after 1m41s
[20:48:38.770] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:55:0)
[20:48:38.874] Build status: STARTED
[20:48:38.874] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:56:0) after 3m32s
[20:52:10.945] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:56:0)
[20:52:10.968] Build status: STARTED
[20:52:10.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:57:0) after 7m34s
[20:59:44.980] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:57:0)
[20:59:45.006] Build status: STARTED
[20:59:45.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:58:0) after 3m37s
[21:03:22.060] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:58:0)
[21:03:22.082] Build status: STARTED
[21:03:22.082] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:59:0) after 4m17s
[21:07:39.116] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:59:0)
[21:07:39.168] Build status: STARTED
[21:07:39.168] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:60:0) after 1m41s
[21:09:20.417] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:60:0)
[21:09:20.449] Build status: STARTED
[21:09:20.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:61:0) after 1m29s
[21:10:49.723] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:61:0)
[21:10:49.868] Build status: STARTED
[21:10:49.868] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:62:0) after 1m9s
[21:11:59.094] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:62:0)
[21:11:59.126] Build status: STARTED
[21:11:59.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:63:0) after 1m53s
[21:13:52.433] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:63:0)
[21:13:52.467] Build status: STARTED
[21:13:52.467] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:64:0) after 6m5s
[21:19:57.639] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:64:0)
[21:19:57.661] Build status: STARTED
[21:19:57.661] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:65:0) after 5m10s
[21:25:07.686] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:65:0)
[21:25:07.708] Build status: STARTED
[21:25:07.708] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:66:0) after 1m7s
[21:26:14.821] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:66:0)
[21:26:14.843] Build status: STARTED
[21:26:14.843] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953966036692726032:67:0) after 2m30s
[21:26:57.076] Received PubSub notification, asking Buildbucket for the build status
[21:26:57.098] Build:
{
"id": "8788535974683499457",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 28977,
"createdBy": "project:chromium",
"createTime": "2023-02-21T16:13:04.339302751Z",
"startTime": "2023-02-21T16:15:28.758351Z",
"endTime": "2023-02-21T21:26:56.847760750Z",
"updateTime": "2023-02-21T21:26:56.847760750Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "670685263a19b74d091fea46f00991b7c6232be9",
"ref": "refs/heads/main"
}
},
"canary": true
}
[21:26:57.098] Invocation finished in 5h13m54.058704968s with status FAILED