[14:02:29.621] New invocation is queued and will start shortly
[14:02:30.692] Starting the invocation (attempt 1)
[14:02:30.736] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/99799edb591c7378842f5f9b9b6ab8c01dd16527"
[14:02:30.736] Popped gitiles commit info from properties and tags
[14:02:30.736] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[14:02:30.736] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[14:02:30.736] Buildbucket request:
{
"requestId": "9006429894193886688",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "99799edb591c7378842f5f9b9b6ab8c01dd16527",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "9006429894193886688"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[14:02:31.335] Scheduled build:
{
"id": "8840999832238543824",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 22582,
"createdBy": "project:chromium",
"createTime": "2021-07-22T14:02:30.850358Z",
"updateTime": "2021-07-22T14:02:31.247805Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "99799edb591c7378842f5f9b9b6ab8c01dd16527",
"ref": "refs/heads/main"
}
},
"canary": true
}
[14:02:31.335] Task URL: https://cr-buildbucket.appspot.com/build/8840999832238543824
[14:02:31.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:2:0) after 6m21s
[14:07:23.539] Received PubSub notification, asking Buildbucket for the build status
[14:07:23.570] Build status: STARTED
[14:08:52.390] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:2:0)
[14:08:52.424] Build status: STARTED
[14:08:52.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:4:0) after 1m16s
[14:10:08.536] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:4:0)
[14:10:08.572] Build status: STARTED
[14:10:08.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:5:0) after 5m12s
[14:15:20.608] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:5:0)
[14:15:20.640] Build status: STARTED
[14:15:20.640] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:6:0) after 3m54s
[14:19:14.804] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:6:0)
[14:19:14.832] Build status: STARTED
[14:19:14.832] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:7:0) after 9m10s
[14:28:24.902] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:7:0)
[14:28:24.931] Build status: STARTED
[14:28:24.931] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:8:0) after 2m47s
[14:31:11.952] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:8:0)
[14:31:11.990] Build status: STARTED
[14:31:11.990] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:9:0) after 9m11s
[14:40:23.032] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:9:0)
[14:40:23.078] Build status: STARTED
[14:40:23.078] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:10:0) after 2m47s
[14:43:10.139] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:10:0)
[14:43:10.173] Build status: STARTED
[14:43:10.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:11:0) after 7m54s
[14:51:04.506] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:11:0)
[14:51:04.534] Build status: STARTED
[14:51:04.534] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:12:0) after 6m38s
[14:57:42.734] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:12:0)
[14:57:42.763] Build status: STARTED
[14:57:42.763] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:13:0) after 9m44s
[15:07:26.779] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:13:0)
[15:07:26.807] Build status: STARTED
[15:07:26.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:14:0) after 9m16s
[15:16:42.833] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:14:0)
[15:16:42.860] Build status: STARTED
[15:16:42.860] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:15:0) after 7m5s
[15:23:47.866] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:15:0)
[15:23:47.893] Build status: STARTED
[15:23:47.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:16:0) after 5m13s
[15:29:00.940] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:16:0)
[15:29:00.995] Build status: STARTED
[15:29:00.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:17:0) after 6m20s
[15:35:21.205] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:17:0)
[15:35:21.236] Build status: STARTED
[15:35:21.236] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:18:0) after 5m17s
[15:40:38.256] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:18:0)
[15:40:38.285] Build status: STARTED
[15:40:38.285] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:19:0) after 5m37s
[15:46:15.474] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:19:0)
[15:46:15.527] Build status: STARTED
[15:46:15.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:20:0) after 1m0s
[15:47:15.824] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:20:0)
[15:47:15.855] Build status: STARTED
[15:47:15.855] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:21:0) after 5m47s
[15:53:02.885] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:21:0)
[15:53:02.916] Build status: STARTED
[15:53:02.916] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:22:0) after 8m15s
[16:01:18.078] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:22:0)
[16:01:18.126] Build status: STARTED
[16:01:18.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:23:0) after 4m13s
[16:05:31.270] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:23:0)
[16:05:31.325] Build status: STARTED
[16:05:31.325] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:24:0) after 5m16s
[16:10:47.317] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:24:0)
[16:10:47.345] Build status: STARTED
[16:10:47.345] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:25:0) after 6m27s
[16:17:14.372] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:25:0)
[16:17:14.399] Build status: STARTED
[16:17:14.399] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:26:0) after 1m56s
[16:19:10.703] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:26:0)
[16:19:10.791] Build status: STARTED
[16:19:10.791] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:27:0) after 6m29s
[16:25:39.857] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:27:0)
[16:25:39.890] Build status: STARTED
[16:25:39.890] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:28:0) after 5m51s
[16:31:31.180] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:28:0)
[16:31:31.228] Build status: STARTED
[16:31:31.229] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:29:0) after 4m24s
[16:35:55.284] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:29:0)
[16:35:55.372] Build status: STARTED
[16:35:55.372] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:30:0) after 6m15s
[16:42:10.405] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:30:0)
[16:42:10.456] Build status: STARTED
[16:42:10.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:31:0) after 1m23s
[16:43:33.591] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:31:0)
[16:43:33.622] Build status: STARTED
[16:43:33.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:32:0) after 8m59s
[16:52:32.725] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:32:0)
[16:52:32.795] Build status: STARTED
[16:52:32.795] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:33:0) after 2m4s
[16:54:36.856] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:33:0)
[16:54:36.896] Build status: STARTED
[16:54:36.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:34:0) after 7m29s
[17:02:06.037] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:34:0)
[17:02:06.983] Build status: STARTED
[17:02:06.983] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:35:0) after 7m35s
[17:09:42.025] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:35:0)
[17:09:42.058] Build status: STARTED
[17:09:42.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:36:0) after 5m38s
[17:15:20.084] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:36:0)
[17:15:20.119] Build status: STARTED
[17:15:20.119] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:37:0) after 5m39s
[17:20:59.258] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:37:0)
[17:20:59.289] Build status: STARTED
[17:20:59.289] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:38:0) after 9m9s
[17:30:08.527] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:38:0)
[17:30:08.567] Build status: STARTED
[17:30:08.567] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:39:0) after 3m0s
[17:33:08.600] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:39:0)
[17:33:08.646] Build status: STARTED
[17:33:08.647] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:40:0) after 9m47s
[17:42:55.673] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:40:0)
[17:42:55.701] Build status: STARTED
[17:42:55.701] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:41:0) after 6m52s
[17:49:47.836] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:41:0)
[17:49:47.870] Build status: STARTED
[17:49:47.870] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:42:0) after 6m41s
[17:56:29.031] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:42:0)
[17:56:29.059] Build status: STARTED
[17:56:29.059] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:43:0) after 9m55s
[18:06:24.137] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:43:0)
[18:06:24.173] Build status: STARTED
[18:06:24.173] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:44:0) after 1m17s
[18:07:41.258] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:44:0)
[18:07:41.343] Build status: STARTED
[18:07:41.343] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:45:0) after 7m3s
[18:14:44.556] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:45:0)
[18:14:44.587] Build status: STARTED
[18:14:44.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:46:0) after 7m51s
[18:22:35.637] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:46:0)
[18:22:35.704] Build status: STARTED
[18:22:35.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:47:0) after 4m50s
[18:27:25.728] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:47:0)
[18:27:25.759] Build status: STARTED
[18:27:25.759] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:48:0) after 5m19s
[18:32:45.014] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:48:0)
[18:32:45.050] Build status: STARTED
[18:32:45.050] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:49:0) after 3m19s
[18:36:04.193] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:49:0)
[18:36:04.253] Build status: STARTED
[18:36:04.253] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:50:0) after 5m6s
[18:41:10.281] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:50:0)
[18:41:10.318] Build status: STARTED
[18:41:10.319] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:51:0) after 4m4s
[18:45:14.977] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:51:0)
[18:45:15.022] Build status: STARTED
[18:45:15.022] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:52:0) after 3m49s
[18:49:04.205] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:52:0)
[18:49:04.232] Build status: STARTED
[18:49:04.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:53:0) after 4m49s
[18:53:53.268] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:53:0)
[18:53:53.298] Build status: STARTED
[18:53:53.298] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:54:0) after 8m6s
[19:01:59.324] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:54:0)
[19:01:59.364] Build status: STARTED
[19:01:59.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:55:0) after 7m50s
[19:09:49.448] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:55:0)
[19:09:49.479] Build status: STARTED
[19:09:49.479] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:56:0) after 3m25s
[19:13:14.661] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:56:0)
[19:13:14.699] Build status: STARTED
[19:13:14.699] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:57:0) after 9m17s
[19:22:31.726] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:57:0)
[19:22:31.752] Build status: STARTED
[19:22:31.752] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:58:0) after 7m14s
[19:29:45.824] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:58:0)
[19:29:45.858] Build status: STARTED
[19:29:45.859] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:59:0) after 5m28s
[19:35:13.979] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:59:0)
[19:35:14.011] Build status: STARTED
[19:35:14.011] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:60:0) after 6m26s
[19:41:40.717] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:60:0)
[19:41:42.015] Build status: STARTED
[19:41:42.015] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:61:0) after 3m15s
[19:44:57.138] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:61:0)
[19:44:57.169] Build status: STARTED
[19:44:57.169] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9006429894193886688:62:0) after 9m0s
[19:46:43.902] Received PubSub notification, asking Buildbucket for the build status
[19:46:43.930] Build:
{
"id": "8840999832238543824",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 22582,
"createdBy": "project:chromium",
"createTime": "2021-07-22T14:02:30.850358Z",
"startTime": "2021-07-22T14:07:22.364960Z",
"endTime": "2021-07-22T19:46:43.412825627Z",
"updateTime": "2021-07-22T19:46:43.413356219Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "99799edb591c7378842f5f9b9b6ab8c01dd16527",
"ref": "refs/heads/main"
}
},
"canary": true
}
[19:46:43.930] Invocation finished in 5h44m14.329307064s with status FAILED