[22:09:06.882] New invocation is queued and will start shortly
[22:09:08.032] Starting the invocation (attempt 1)
[22:09:08.057] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/3c5451e8645886f27a77850fbc65855876e475ad"
[22:09:08.057] Popped gitiles commit info from properties and tags
[22:09:08.057] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[22:09:08.057] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[22:09:08.057] Buildbucket request:
{
"requestId": "8942166029460812688",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3c5451e8645886f27a77850fbc65855876e475ad",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8942166029460812688"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[22:09:08.527] Scheduled build:
{
"id": "8776735967473600417",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29850,
"createdBy": "project:chromium",
"createTime": "2023-07-01T22:09:08.149855277Z",
"updateTime": "2023-07-01T22:09:08.149855277Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3c5451e8645886f27a77850fbc65855876e475ad",
"ref": "refs/heads/main"
}
}
}
[22:09:08.527] Task URL: https://cr-buildbucket.appspot.com/build/8776735967473600417
[22:09:08.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:2:0) after 6m32s
[22:12:38.484] Received PubSub notification, asking Buildbucket for the build status
[22:12:38.512] Build status: STARTED
[22:15:40.552] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:2:0)
[22:15:40.573] Build status: STARTED
[22:15:40.573] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:4:0) after 1m45s
[22:17:25.642] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:4:0)
[22:17:25.663] Build status: STARTED
[22:17:25.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:5:0) after 2m27s
[22:19:52.878] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:5:0)
[22:19:52.905] Build status: STARTED
[22:19:52.905] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:6:0) after 6m13s
[22:26:06.447] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:6:0)
[22:26:06.468] Build status: STARTED
[22:26:06.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:7:0) after 5m27s
[22:31:33.495] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:7:0)
[22:31:33.586] Build status: STARTED
[22:31:33.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:8:0) after 9m2s
[22:40:35.713] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:8:0)
[22:40:35.732] Build status: STARTED
[22:40:35.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:9:0) after 9m32s
[22:50:07.789] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:9:0)
[22:50:07.808] Build status: STARTED
[22:50:07.808] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:10:0) after 3m43s
[22:53:50.823] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:10:0)
[22:53:50.848] Build status: STARTED
[22:53:50.848] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:11:0) after 8m21s
[23:02:11.867] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:11:0)
[23:02:11.903] Build status: STARTED
[23:02:11.903] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:12:0) after 9m4s
[23:11:15.933] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:12:0)
[23:11:15.956] Build status: STARTED
[23:11:15.956] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:13:0) after 1m18s
[23:12:33.979] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:13:0)
[23:12:33.998] Build status: STARTED
[23:12:33.998] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:14:0) after 3m43s
[23:16:17.019] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:14:0)
[23:16:17.035] Build status: STARTED
[23:16:17.035] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:15:0) after 4m10s
[23:20:27.021] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:15:0)
[23:20:27.047] Build status: STARTED
[23:20:27.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:16:0) after 9m23s
[23:29:50.065] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:16:0)
[23:29:50.084] Build status: STARTED
[23:29:50.084] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:17:0) after 4m42s
[23:34:32.341] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:17:0)
[23:34:32.361] Build status: STARTED
[23:34:32.361] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:18:0) after 5m17s
[23:39:49.340] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:18:0)
[23:39:49.359] Build status: STARTED
[23:39:49.359] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:19:0) after 3m13s
[23:43:02.397] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:19:0)
[23:43:02.415] Build status: STARTED
[23:43:02.415] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:20:0) after 5m44s
[23:48:47.332] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:20:0)
[23:48:47.349] Build status: STARTED
[23:48:47.349] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:21:0) after 4m25s
[23:53:12.367] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:21:0)
[23:53:12.388] Build status: STARTED
[23:53:12.389] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:22:0) after 2m49s
[23:56:01.703] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:22:0)
[23:56:01.723] Build status: STARTED
[23:56:01.723] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:23:0) after 3m55s
[23:59:57.087] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:23:0)
[23:59:57.109] Build status: STARTED
[23:59:57.109] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:24:0) after 6m1s
[00:05:59.082] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:24:0)
[00:05:59.103] Build status: STARTED
[00:05:59.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:25:0) after 8m32s
[00:14:31.258] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:25:0)
[00:14:31.288] Build status: STARTED
[00:14:31.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:26:0) after 1m26s
[00:15:57.265] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:26:0)
[00:15:57.281] Build status: STARTED
[00:15:57.281] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:27:0) after 8m52s
[00:24:49.327] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:27:0)
[00:24:49.483] Build status: STARTED
[00:24:49.483] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:28:0) after 3m8s
[00:27:57.656] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:28:0)
[00:27:57.677] Build status: STARTED
[00:27:57.677] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:29:0) after 1m13s
[00:29:10.694] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:29:0)
[00:29:10.713] Build status: STARTED
[00:29:10.713] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:30:0) after 1m40s
[00:30:50.693] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:30:0)
[00:30:50.714] Build status: STARTED
[00:30:50.714] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:31:0) after 5m46s
[00:36:36.739] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:31:0)
[00:36:36.754] Build status: STARTED
[00:36:36.754] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:32:0) after 1m24s
[00:38:00.797] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:32:0)
[00:38:00.870] Build status: STARTED
[00:38:00.870] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:33:0) after 3m32s
[00:41:32.951] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:33:0)
[00:41:33.004] Build status: STARTED
[00:41:33.004] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:34:0) after 7m35s
[00:49:08.276] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:34:0)
[00:49:08.299] Build status: STARTED
[00:49:08.299] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:35:0) after 6m21s
[00:55:29.317] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:35:0)
[00:55:29.338] Build status: STARTED
[00:55:29.338] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:36:0) after 5m55s
[01:01:24.362] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:36:0)
[01:01:24.386] Build status: STARTED
[01:01:24.386] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:37:0) after 4m15s
[01:05:39.521] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:37:0)
[01:05:39.547] Build status: STARTED
[01:05:39.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:38:0) after 7m19s
[01:12:58.657] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:38:0)
[01:12:58.679] Build status: STARTED
[01:12:58.679] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:39:0) after 9m9s
[01:22:07.699] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:39:0)
[01:22:07.718] Build status: STARTED
[01:22:07.718] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:40:0) after 8m10s
[01:30:17.841] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:40:0)
[01:30:17.858] Build status: STARTED
[01:30:17.858] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:41:0) after 3m39s
[01:33:56.869] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:41:0)
[01:33:56.888] Build status: STARTED
[01:33:56.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:42:0) after 6m11s
[01:40:07.909] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:42:0)
[01:40:07.966] Build status: STARTED
[01:40:07.966] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:43:0) after 4m52s
[01:45:00.590] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:43:0)
[01:45:00.611] Build status: STARTED
[01:45:00.611] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:44:0) after 7m43s
[01:52:43.837] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:44:0)
[01:52:43.855] Build status: STARTED
[01:52:43.855] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:45:0) after 4m28s
[01:57:11.865] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:45:0)
[01:57:11.889] Build status: STARTED
[01:57:11.889] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:46:0) after 9m15s
[02:06:27.475] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:46:0)
[02:06:27.560] Build status: STARTED
[02:06:27.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:47:0) after 1m40s
[02:08:07.579] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:47:0)
[02:08:07.617] Build status: STARTED
[02:08:07.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:48:0) after 2m1s
[02:10:08.648] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:48:0)
[02:10:08.666] Build status: STARTED
[02:10:08.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:49:0) after 6m32s
[02:16:40.701] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:49:0)
[02:16:40.721] Build status: STARTED
[02:16:40.721] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:50:0) after 3m32s
[02:20:12.740] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:50:0)
[02:20:12.758] Build status: STARTED
[02:20:12.758] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:51:0) after 7m45s
[02:27:57.808] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:51:0)
[02:27:57.835] Build status: STARTED
[02:27:57.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:52:0) after 2m7s
[02:30:04.863] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:52:0)
[02:30:04.914] Build status: STARTED
[02:30:04.914] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:53:0) after 1m10s
[02:31:15.033] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:53:0)
[02:31:15.059] Build status: STARTED
[02:31:15.059] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:54:0) after 9m7s
[02:40:22.186] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:54:0)
[02:40:22.202] Build status: STARTED
[02:40:22.202] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:55:0) after 5m53s
[02:46:15.283] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:55:0)
[02:46:15.312] Build status: STARTED
[02:46:15.312] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:56:0) after 1m33s
[02:47:48.336] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:56:0)
[02:47:48.360] Build status: STARTED
[02:47:48.360] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:57:0) after 6m3s
[02:53:51.399] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:57:0)
[02:53:51.414] Build status: STARTED
[02:53:51.414] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:58:0) after 9m22s
[03:03:13.439] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:58:0)
[03:03:13.461] Build status: STARTED
[03:03:13.461] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:59:0) after 6m30s
[03:09:43.478] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:59:0)
[03:09:43.497] Build status: STARTED
[03:09:43.497] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:60:0) after 4m44s
[03:14:27.481] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:60:0)
[03:14:27.502] Build status: STARTED
[03:14:27.502] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:61:0) after 3m22s
[03:17:49.521] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:61:0)
[03:17:49.541] Build status: STARTED
[03:17:49.541] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:62:0) after 9m13s
[03:27:02.562] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:62:0)
[03:27:02.580] Build status: STARTED
[03:27:02.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:63:0) after 9m6s
[03:36:08.608] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:63:0)
[03:36:08.627] Build status: STARTED
[03:36:08.627] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:64:0) after 6m57s
[03:43:05.737] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:64:0)
[03:43:05.752] Build status: STARTED
[03:43:05.752] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8942166029460812688:65:0) after 5m50s
[03:46:01.826] Received PubSub notification, asking Buildbucket for the build status
[03:46:01.842] Build:
{
"id": "8776735967473600417",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29850,
"createdBy": "project:chromium",
"createTime": "2023-07-01T22:09:08.149855277Z",
"startTime": "2023-07-01T22:12:37.842690Z",
"endTime": "2023-07-02T03:46:01.569821246Z",
"updateTime": "2023-07-02T03:46:01.569821246Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3c5451e8645886f27a77850fbc65855876e475ad",
"ref": "refs/heads/main"
}
}
}
[03:46:01.842] Invocation finished in 5h36m54.972628282s with status FAILED