[07:51:56.920] New invocation is queued and will start shortly
[07:51:58.011] Starting the invocation (attempt 1)
[07:51:58.042] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/23230717766c42967288899d5773d37fce260e2c"
[07:51:58.042] Popped gitiles commit info from properties and tags
[07:51:58.042] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[07:51:58.042] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[07:51:58.042] Buildbucket request:
{
"requestId": "8995219183031631328",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "23230717766c42967288899d5773d37fce260e2c",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8995219183031631328"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[07:51:58.682] Scheduled build:
{
"id": "8829789121118981873",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23595,
"createdBy": "project:chromium",
"createTime": "2021-11-23T07:51:58.113843356Z",
"updateTime": "2021-11-23T07:51:58.113843356Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "23230717766c42967288899d5773d37fce260e2c",
"ref": "refs/heads/main"
}
},
"canary": true
}
[07:51:58.682] Task URL: https://cr-buildbucket.appspot.com/build/8829789121118981873
[07:51:58.682] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:2:0) after 7m47s
[07:54:28.824] Received PubSub notification, asking Buildbucket for the build status
[07:54:28.860] Build status: STARTED
[07:59:45.703] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:2:0)
[07:59:45.761] Build status: STARTED
[07:59:45.761] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:4:0) after 5m50s
[08:05:35.782] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:4:0)
[08:05:35.830] Build status: STARTED
[08:05:35.830] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:5:0) after 7m19s
[08:12:55.002] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:5:0)
[08:12:55.032] Build status: STARTED
[08:12:55.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:6:0) after 1m36s
[08:14:31.057] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:6:0)
[08:14:31.098] Build status: STARTED
[08:14:31.098] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:7:0) after 2m35s
[08:17:06.185] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:7:0)
[08:17:06.214] Build status: STARTED
[08:17:06.214] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:8:0) after 8m2s
[08:25:08.322] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:8:0)
[08:25:08.698] Build status: STARTED
[08:25:08.698] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:9:0) after 7m29s
[08:32:37.747] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:9:0)
[08:32:37.813] Build status: STARTED
[08:32:37.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:10:0) after 4m46s
[08:37:24.308] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:10:0)
[08:37:24.338] Build status: STARTED
[08:37:24.338] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:11:0) after 6m46s
[08:44:10.653] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:11:0)
[08:44:10.685] Build status: STARTED
[08:44:10.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:12:0) after 2m12s
[08:46:22.707] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:12:0)
[08:46:22.751] Build status: STARTED
[08:46:22.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:13:0) after 9m34s
[08:55:56.824] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:13:0)
[08:55:56.852] Build status: STARTED
[08:55:56.852] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:14:0) after 2m0s
[08:57:57.033] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:14:0)
[08:57:57.076] Build status: STARTED
[08:57:57.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:15:0) after 6m38s
[09:04:35.160] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:15:0)
[09:04:35.254] Build status: STARTED
[09:04:35.254] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:16:0) after 9m26s
[09:14:01.321] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:16:0)
[09:14:01.376] Build status: STARTED
[09:14:01.376] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:17:0) after 1m38s
[09:15:39.440] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:17:0)
[09:15:39.475] Build status: STARTED
[09:15:39.475] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:18:0) after 4m37s
[09:20:16.596] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:18:0)
[09:20:16.629] Build status: STARTED
[09:20:16.629] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:19:0) after 6m3s
[09:26:19.651] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:19:0)
[09:26:19.684] Build status: STARTED
[09:26:19.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:20:0) after 2m4s
[09:28:23.787] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:20:0)
[09:28:23.880] Build status: STARTED
[09:28:23.880] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:21:0) after 7m41s
[09:36:04.900] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:21:0)
[09:36:04.961] Build status: STARTED
[09:36:04.961] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:22:0) after 4m41s
[09:40:45.981] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:22:0)
[09:40:46.015] Build status: STARTED
[09:40:46.015] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:23:0) after 9m57s
[09:50:43.041] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:23:0)
[09:50:43.079] Build status: STARTED
[09:50:43.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:24:0) after 3m20s
[09:54:03.163] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:24:0)
[09:54:03.206] Build status: STARTED
[09:54:03.206] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:25:0) after 5m54s
[09:59:57.292] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:25:0)
[09:59:57.332] Build status: STARTED
[09:59:57.332] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:26:0) after 1m15s
[10:01:12.355] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:26:0)
[10:01:12.391] Build status: STARTED
[10:01:12.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:27:0) after 8m4s
[10:09:16.454] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:27:0)
[10:09:16.518] Build status: STARTED
[10:09:16.519] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:28:0) after 8m31s
[10:17:47.814] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:28:0)
[10:17:47.843] Build status: STARTED
[10:17:47.843] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:29:0) after 4m20s
[10:22:08.011] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:29:0)
[10:22:08.040] Build status: STARTED
[10:22:08.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:30:0) after 2m6s
[10:24:14.064] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:30:0)
[10:24:14.101] Build status: STARTED
[10:24:14.101] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:31:0) after 8m31s
[10:32:45.156] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:31:0)
[10:32:45.187] Build status: STARTED
[10:32:45.187] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:32:0) after 1m30s
[10:34:15.339] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:32:0)
[10:34:15.371] Build status: STARTED
[10:34:15.371] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:33:0) after 5m13s
[10:39:28.490] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:33:0)
[10:39:28.527] Build status: STARTED
[10:39:28.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:34:0) after 2m2s
[10:41:30.519] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:34:0)
[10:41:30.547] Build status: STARTED
[10:41:30.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:35:0) after 3m59s
[10:45:29.570] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:35:0)
[10:45:29.602] Build status: STARTED
[10:45:29.602] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:36:0) after 4m46s
[10:50:15.622] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:36:0)
[10:50:15.799] Build status: STARTED
[10:50:15.799] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:37:0) after 5m17s
[10:55:32.914] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:37:0)
[10:55:32.944] Build status: STARTED
[10:55:32.944] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:38:0) after 9m6s
[11:04:38.996] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:38:0)
[11:04:39.031] Build status: STARTED
[11:04:39.031] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:39:0) after 6m43s
[11:11:22.127] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:39:0)
[11:11:22.165] Build status: STARTED
[11:11:22.165] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:40:0) after 6m21s
[11:17:43.295] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:40:0)
[11:17:43.333] Build status: STARTED
[11:17:43.333] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:41:0) after 5m10s
[11:22:53.357] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:41:0)
[11:22:53.394] Build status: STARTED
[11:22:53.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:42:0) after 4m31s
[11:27:24.499] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:42:0)
[11:27:24.538] Build status: STARTED
[11:27:24.538] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:43:0) after 9m52s
[11:37:16.541] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:43:0)
[11:37:16.572] Build status: STARTED
[11:37:16.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:44:0) after 2m52s
[11:40:08.613] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:44:0)
[11:40:08.655] Build status: STARTED
[11:40:08.655] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:45:0) after 5m51s
[11:45:59.990] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:45:0)
[11:46:00.037] Build status: STARTED
[11:46:00.037] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:46:0) after 1m24s
[11:47:24.059] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:46:0)
[11:47:24.098] Build status: STARTED
[11:47:24.098] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:47:0) after 9m59s
[11:57:23.120] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:47:0)
[11:57:23.156] Build status: STARTED
[11:57:23.156] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:48:0) after 8m24s
[12:05:47.330] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:48:0)
[12:05:47.373] Build status: STARTED
[12:05:47.373] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:49:0) after 6m21s
[12:12:08.395] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:49:0)
[12:12:08.486] Build status: STARTED
[12:12:08.486] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:50:0) after 8m1s
[12:20:09.676] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:50:0)
[12:20:09.704] Build status: STARTED
[12:20:09.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:51:0) after 4m14s
[12:24:23.801] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:51:0)
[12:24:23.832] Build status: STARTED
[12:24:23.832] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:52:0) after 2m17s
[12:26:40.851] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:52:0)
[12:26:40.888] Build status: STARTED
[12:26:40.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:53:0) after 6m18s
[12:32:58.958] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:53:0)
[12:32:58.988] Build status: STARTED
[12:32:58.988] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:54:0) after 1m37s
[12:34:36.009] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:54:0)
[12:34:36.038] Build status: STARTED
[12:34:36.038] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:55:0) after 5m58s
[12:40:34.057] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:55:0)
[12:40:34.085] Build status: STARTED
[12:40:34.085] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:56:0) after 9m29s
[12:50:03.115] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:56:0)
[12:50:03.151] Build status: STARTED
[12:50:03.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:57:0) after 7m57s
[12:58:00.274] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:57:0)
[12:58:00.318] Build status: STARTED
[12:58:00.318] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:58:0) after 5m4s
[13:03:04.540] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:58:0)
[13:03:04.570] Build status: STARTED
[13:03:04.570] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:59:0) after 9m55s
[13:12:59.593] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:59:0)
[13:12:59.624] Build status: STARTED
[13:12:59.624] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:60:0) after 8m14s
[13:21:13.660] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:60:0)
[13:21:13.711] Build status: STARTED
[13:21:13.711] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:61:0) after 1m1s
[13:22:14.739] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:61:0)
[13:22:14.769] Build status: STARTED
[13:22:14.769] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:62:0) after 3m48s
[13:26:02.927] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:62:0)
[13:26:03.058] Build status: STARTED
[13:26:03.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:63:0) after 1m29s
[13:27:32.194] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:63:0)
[13:27:32.236] Build status: STARTED
[13:27:32.236] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:64:0) after 5m40s
[13:33:12.226] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:64:0)
[13:33:12.257] Build status: STARTED
[13:33:12.257] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8995219183031631328:65:0) after 4m49s
[13:37:02.410] Received PubSub notification, asking Buildbucket for the build status
[13:37:02.444] Build:
{
"id": "8829789121118981873",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23595,
"createdBy": "project:chromium",
"createTime": "2021-11-23T07:51:58.113843356Z",
"startTime": "2021-11-23T07:54:28.159042Z",
"endTime": "2021-11-23T13:37:01.926226146Z",
"updateTime": "2021-11-23T13:37:01.926226146Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "23230717766c42967288899d5773d37fce260e2c",
"ref": "refs/heads/main"
}
},
"canary": true
}
[13:37:02.444] Invocation finished in 5h45m5.539274563s with status FAILED