[15:18:22.123] New invocation is queued and will start shortly
[15:18:23.276] Starting the invocation (attempt 1)
[15:18:23.334] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/19d8e06cccf1f845e697acfa1100ca5f53526a56"
[15:18:23.334] Popped gitiles commit info from properties and tags
[15:18:23.334] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:18:23.334] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:18:23.335] Buildbucket request:
{
"requestId": "9000355123797981184",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "19d8e06cccf1f845e697acfa1100ca5f53526a56",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "9000355123797981184"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:18:23.921] Scheduled build:
{
"id": "8834925061771618769",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23248,
"createdBy": "project:chromium",
"createTime": "2021-09-27T15:18:23.422855203Z",
"updateTime": "2021-09-27T15:18:23.746646825Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "19d8e06cccf1f845e697acfa1100ca5f53526a56",
"ref": "refs/heads/main"
}
}
}
[15:18:23.921] Task URL: https://cr-buildbucket.appspot.com/build/8834925061771618769
[15:18:23.921] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:2:0) after 8m10s
[15:20:54.567] Received PubSub notification, asking Buildbucket for the build status
[15:20:54.661] Build status: STARTED
[15:26:33.993] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:2:0)
[15:26:34.024] Build status: STARTED
[15:26:34.024] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:4:0) after 4m51s
[15:31:25.232] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:4:0)
[15:31:25.277] Build status: STARTED
[15:31:25.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:5:0) after 2m40s
[15:34:05.301] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:5:0)
[15:34:05.330] Build status: STARTED
[15:34:05.330] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:6:0) after 1m44s
[15:35:49.355] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:6:0)
[15:35:49.388] Build status: STARTED
[15:35:49.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:7:0) after 3m58s
[15:39:47.408] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:7:0)
[15:39:47.447] Build status: STARTED
[15:39:47.447] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:8:0) after 4m8s
[15:43:55.476] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:8:0)
[15:43:55.517] Build status: STARTED
[15:43:55.517] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:9:0) after 2m46s
[15:46:41.562] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:9:0)
[15:46:41.594] Build status: STARTED
[15:46:41.594] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:10:0) after 5m50s
[15:52:31.600] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:10:0)
[15:52:31.631] Build status: STARTED
[15:52:31.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:11:0) after 8m18s
[16:00:49.886] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:11:0)
[16:00:49.916] Build status: STARTED
[16:00:49.916] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:12:0) after 4m10s
[16:04:59.947] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:12:0)
[16:04:59.976] Build status: STARTED
[16:04:59.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:13:0) after 6m44s
[16:11:44.098] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:13:0)
[16:11:44.173] Build status: STARTED
[16:11:44.173] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:14:0) after 6m32s
[16:18:16.203] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:14:0)
[16:18:16.233] Build status: STARTED
[16:18:16.234] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:15:0) after 9m35s
[16:27:51.256] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:15:0)
[16:27:51.295] Build status: STARTED
[16:27:51.295] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:16:0) after 6m49s
[16:34:40.459] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:16:0)
[16:34:40.519] Build status: STARTED
[16:34:40.519] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:17:0) after 7m30s
[16:42:10.735] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:17:0)
[16:42:10.771] Build status: STARTED
[16:42:10.771] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:18:0) after 9m38s
[16:51:48.896] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:18:0)
[16:51:48.997] Build status: STARTED
[16:51:48.997] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:19:0) after 4m56s
[16:56:45.018] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:19:0)
[16:56:45.062] Build status: STARTED
[16:56:45.062] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:20:0) after 5m43s
[17:02:28.090] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:20:0)
[17:02:28.117] Build status: STARTED
[17:02:28.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:21:0) after 8m53s
[17:11:21.198] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:21:0)
[17:11:21.226] Build status: STARTED
[17:11:21.226] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:22:0) after 4m53s
[17:16:14.345] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:22:0)
[17:16:14.376] Build status: STARTED
[17:16:14.376] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:23:0) after 1m47s
[17:18:01.410] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:23:0)
[17:18:01.458] Build status: STARTED
[17:18:01.458] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:24:0) after 1m51s
[17:19:52.430] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:24:0)
[17:19:52.490] Build status: STARTED
[17:19:52.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:25:0) after 5m42s
[17:25:34.535] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:25:0)
[17:25:34.566] Build status: STARTED
[17:25:34.566] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:26:0) after 6m0s
[17:31:34.734] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:26:0)
[17:31:34.767] Build status: STARTED
[17:31:34.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:27:0) after 6m0s
[17:37:34.835] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:27:0)
[17:37:34.866] Build status: STARTED
[17:37:34.866] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:28:0) after 8m43s
[17:46:17.892] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:28:0)
[17:46:17.941] Build status: STARTED
[17:46:17.941] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:29:0) after 1m12s
[17:47:29.964] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:29:0)
[17:47:30.056] Build status: STARTED
[17:47:30.056] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:30:0) after 6m2s
[17:53:32.046] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:30:0)
[17:53:32.076] Build status: STARTED
[17:53:32.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:31:0) after 6m26s
[17:59:58.126] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:31:0)
[17:59:58.176] Build status: STARTED
[17:59:58.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:32:0) after 3m7s
[18:03:05.204] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:32:0)
[18:03:05.239] Build status: STARTED
[18:03:05.239] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:33:0) after 6m26s
[18:09:31.324] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:33:0)
[18:09:31.360] Build status: STARTED
[18:09:31.360] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:34:0) after 3m58s
[18:13:29.410] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:34:0)
[18:13:29.488] Build status: STARTED
[18:13:29.488] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:35:0) after 2m39s
[18:16:08.510] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:35:0)
[18:16:08.542] Build status: STARTED
[18:16:08.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:36:0) after 6m28s
[18:22:36.635] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:36:0)
[18:22:36.685] Build status: STARTED
[18:22:36.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:37:0) after 4m42s
[18:27:19.262] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:37:0)
[18:27:19.296] Build status: STARTED
[18:27:19.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:38:0) after 5m25s
[18:32:44.388] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:38:0)
[18:32:44.420] Build status: STARTED
[18:32:44.420] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:39:0) after 4m0s
[18:36:44.479] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:39:0)
[18:36:44.542] Build status: STARTED
[18:36:44.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:40:0) after 9m56s
[18:46:40.564] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:40:0)
[18:46:40.612] Build status: STARTED
[18:46:40.612] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:41:0) after 3m11s
[18:49:51.773] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:41:0)
[18:49:51.983] Build status: STARTED
[18:49:51.983] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:42:0) after 5m4s
[18:54:56.002] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:42:0)
[18:54:56.041] Build status: STARTED
[18:54:56.041] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:43:0) after 9m55s
[19:04:51.257] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:43:0)
[19:04:51.375] Build status: STARTED
[19:04:51.375] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:44:0) after 8m54s
[19:13:45.406] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:44:0)
[19:13:45.466] Build status: STARTED
[19:13:45.467] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:45:0) after 2m21s
[19:16:06.680] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:45:0)
[19:16:06.756] Build status: STARTED
[19:16:06.756] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:46:0) after 9m46s
[19:25:52.787] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:46:0)
[19:25:52.817] Build status: STARTED
[19:25:52.817] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:47:0) after 8m49s
[19:34:41.881] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:47:0)
[19:34:41.917] Build status: STARTED
[19:34:41.917] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:48:0) after 7m46s
[19:42:28.135] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:48:0)
[19:42:28.166] Build status: STARTED
[19:42:28.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:49:0) after 4m2s
[19:46:30.273] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:49:0)
[19:46:30.357] Build status: STARTED
[19:46:30.357] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:50:0) after 4m4s
[19:50:34.477] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:50:0)
[19:50:34.534] Build status: STARTED
[19:50:34.534] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:51:0) after 6m9s
[19:56:43.573] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:51:0)
[19:56:43.673] Build status: STARTED
[19:56:43.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:52:0) after 9m19s
[20:06:02.708] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:52:0)
[20:06:02.738] Build status: STARTED
[20:06:02.738] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:53:0) after 3m53s
[20:09:55.941] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:53:0)
[20:09:56.000] Build status: STARTED
[20:09:56.000] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:54:0) after 1m28s
[20:11:24.131] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:54:0)
[20:11:24.162] Build status: STARTED
[20:11:24.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:55:0) after 8m22s
[20:19:46.244] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:55:0)
[20:19:46.278] Build status: STARTED
[20:19:46.278] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:56:0) after 2m8s
[20:21:54.473] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:56:0)
[20:21:54.512] Build status: STARTED
[20:21:54.512] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:57:0) after 6m12s
[20:28:06.871] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:57:0)
[20:28:06.903] Build status: STARTED
[20:28:06.903] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:58:0) after 2m55s
[20:31:01.933] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:58:0)
[20:31:01.988] Build status: STARTED
[20:31:01.988] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:59:0) after 1m11s
[20:32:13.109] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:59:0)
[20:32:13.143] Build status: STARTED
[20:32:13.143] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:60:0) after 4m26s
[20:36:39.267] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:60:0)
[20:36:39.298] Build status: STARTED
[20:36:39.298] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:61:0) after 1m30s
[20:38:09.469] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:61:0)
[20:38:09.505] Build status: STARTED
[20:38:09.505] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:62:0) after 5m4s
[20:43:13.638] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:62:0)
[20:43:13.673] Build status: STARTED
[20:43:13.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:63:0) after 6m35s
[20:49:48.793] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:63:0)
[20:49:48.824] Build status: STARTED
[20:49:48.825] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:64:0) after 1m43s
[20:51:31.860] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:64:0)
[20:51:31.896] Build status: STARTED
[20:51:31.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:65:0) after 6m24s
[20:57:56.002] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:65:0)
[20:57:56.038] Build status: STARTED
[20:57:56.038] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:9000355123797981184:66:0) after 8m3s
[21:02:37.256] Received PubSub notification, asking Buildbucket for the build status
[21:02:37.292] Build:
{
"id": "8834925061771618769",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 23248,
"createdBy": "project:chromium",
"createTime": "2021-09-27T15:18:23.422855203Z",
"startTime": "2021-09-27T15:20:53.509966Z",
"endTime": "2021-09-27T21:02:36.941245858Z",
"updateTime": "2021-09-27T21:02:36.941557919Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "19d8e06cccf1f845e697acfa1100ca5f53526a56",
"ref": "refs/heads/main"
}
}
}
[21:02:37.292] Invocation finished in 5h44m15.18537799s with status FAILED