[01:21:50.992] New invocation is queued and will start shortly
[01:21:52.060] Starting the invocation (attempt 1)
[01:21:52.088] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/cf3f29aedad7a52a9895e09ba2b61202d8709924"
[01:21:52.088] Popped gitiles commit info from properties and tags
[01:21:52.088] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[01:21:52.088] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[01:21:52.088] Buildbucket request:
{
"requestId": "8940613755183467584",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cf3f29aedad7a52a9895e09ba2b61202d8709924",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8940613755183467584"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[01:21:52.492] Scheduled build:
{
"id": "8775183693299146897",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 30011,
"createdBy": "project:chromium",
"createTime": "2023-07-19T01:21:52.161108877Z",
"updateTime": "2023-07-19T01:21:52.161108877Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cf3f29aedad7a52a9895e09ba2b61202d8709924",
"ref": "refs/heads/main"
}
},
"canary": true
}
[01:21:52.492] Task URL: https://cr-buildbucket.appspot.com/build/8775183693299146897
[01:21:52.493] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:2:0) after 4m1s
[01:25:19.685] Received PubSub notification, asking Buildbucket for the build status
[01:25:19.707] Build status: STARTED
[01:25:53.574] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:2:0)
[01:25:53.597] Build status: STARTED
[01:25:53.597] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:4:0) after 2m26s
[01:28:19.678] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:4:0)
[01:28:19.708] Build status: STARTED
[01:28:19.708] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:5:0) after 5m55s
[01:34:14.746] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:5:0)
[01:34:14.775] Build status: STARTED
[01:34:14.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:6:0) after 3m15s
[01:37:29.854] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:6:0)
[01:37:29.871] Build status: STARTED
[01:37:29.871] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:7:0) after 4m20s
[01:41:49.888] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:7:0)
[01:41:49.911] Build status: STARTED
[01:41:49.911] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:8:0) after 3m0s
[01:44:49.895] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:8:0)
[01:44:49.966] Build status: STARTED
[01:44:49.966] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:9:0) after 3m50s
[01:48:40.236] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:9:0)
[01:48:40.256] Build status: STARTED
[01:48:40.256] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:10:0) after 4m10s
[01:52:50.874] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:10:0)
[01:52:50.889] Build status: STARTED
[01:52:50.889] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:11:0) after 5m48s
[01:58:38.913] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:11:0)
[01:58:38.960] Build status: STARTED
[01:58:38.960] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:12:0) after 7m22s
[02:06:01.030] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:12:0)
[02:06:01.071] Build status: STARTED
[02:06:01.071] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:13:0) after 9m57s
[02:15:58.096] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:13:0)
[02:15:58.112] Build status: STARTED
[02:15:58.112] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:14:0) after 4m28s
[02:20:26.187] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:14:0)
[02:20:26.216] Build status: STARTED
[02:20:26.216] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:15:0) after 1m34s
[02:22:00.296] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:15:0)
[02:22:00.315] Build status: STARTED
[02:22:00.315] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:16:0) after 1m10s
[02:23:10.340] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:16:0)
[02:23:10.394] Build status: STARTED
[02:23:10.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:17:0) after 5m59s
[02:29:09.463] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:17:0)
[02:29:09.484] Build status: STARTED
[02:29:09.484] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:18:0) after 7m20s
[02:36:29.536] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:18:0)
[02:36:29.639] Build status: STARTED
[02:36:29.639] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:19:0) after 9m24s
[02:45:53.897] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:19:0)
[02:45:53.919] Build status: STARTED
[02:45:53.919] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:20:0) after 3m46s
[02:49:40.171] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:20:0)
[02:49:40.190] Build status: STARTED
[02:49:40.190] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:21:0) after 9m16s
[02:58:56.310] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:21:0)
[02:58:56.328] Build status: STARTED
[02:58:56.328] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:22:0) after 6m16s
[03:05:12.474] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:22:0)
[03:05:12.500] Build status: STARTED
[03:05:12.500] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:23:0) after 4m8s
[03:09:20.635] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:23:0)
[03:09:20.655] Build status: STARTED
[03:09:20.655] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:24:0) after 2m17s
[03:11:38.020] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:24:0)
[03:11:38.043] Build status: STARTED
[03:11:38.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:25:0) after 9m47s
[03:21:25.365] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:25:0)
[03:21:25.399] Build status: STARTED
[03:21:25.399] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:26:0) after 6m48s
[03:28:13.671] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:26:0)
[03:28:13.711] Build status: STARTED
[03:28:13.711] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:27:0) after 6m58s
[03:35:11.936] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:27:0)
[03:35:12.001] Build status: STARTED
[03:35:12.001] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:28:0) after 5m27s
[03:40:39.061] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:28:0)
[03:40:39.087] Build status: STARTED
[03:40:39.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:29:0) after 9m7s
[03:49:46.266] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:29:0)
[03:49:46.284] Build status: STARTED
[03:49:46.321] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:30:0) after 3m40s
[03:53:26.486] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:30:0)
[03:53:26.505] Build status: STARTED
[03:53:26.505] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:31:0) after 8m39s
[04:02:05.580] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:31:0)
[04:02:05.683] Build status: STARTED
[04:02:05.683] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:32:0) after 7m53s
[04:09:58.905] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:32:0)
[04:09:59.028] Build status: STARTED
[04:09:59.028] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:33:0) after 4m24s
[04:14:23.048] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:33:0)
[04:14:23.066] Build status: STARTED
[04:14:23.066] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:34:0) after 5m36s
[04:19:59.200] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:34:0)
[04:19:59.220] Build status: STARTED
[04:19:59.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:35:0) after 6m12s
[04:26:11.562] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:35:0)
[04:26:11.598] Build status: STARTED
[04:26:11.598] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:36:0) after 7m17s
[04:33:28.650] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:36:0)
[04:33:28.668] Build status: STARTED
[04:33:28.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:37:0) after 2m33s
[04:36:01.706] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:37:0)
[04:36:01.727] Build status: STARTED
[04:36:01.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:38:0) after 4m14s
[04:40:15.848] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:38:0)
[04:40:15.869] Build status: STARTED
[04:40:15.869] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:39:0) after 9m40s
[04:49:55.991] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:39:0)
[04:49:56.006] Build status: STARTED
[04:49:56.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:40:0) after 5m28s
[04:55:24.029] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:40:0)
[04:55:24.045] Build status: STARTED
[04:55:24.046] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:41:0) after 7m28s
[05:02:52.216] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:41:0)
[05:02:52.235] Build status: STARTED
[05:02:52.235] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:42:0) after 2m4s
[05:04:56.374] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:42:0)
[05:04:56.418] Build status: STARTED
[05:04:56.418] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:43:0) after 5m18s
[05:10:14.450] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:43:0)
[05:10:14.466] Build status: STARTED
[05:10:14.466] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:44:0) after 4m24s
[05:14:38.564] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:44:0)
[05:14:38.588] Build status: STARTED
[05:14:38.588] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:45:0) after 6m52s
[05:21:30.677] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:45:0)
[05:21:30.695] Build status: STARTED
[05:21:30.695] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:46:0) after 9m19s
[05:30:49.856] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:46:0)
[05:30:49.878] Build status: STARTED
[05:30:49.878] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:47:0) after 6m32s
[05:37:21.938] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:47:0)
[05:37:21.957] Build status: STARTED
[05:37:21.957] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:48:0) after 3m18s
[05:40:39.974] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:48:0)
[05:40:39.991] Build status: STARTED
[05:40:39.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:49:0) after 8m30s
[05:49:10.017] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:49:0)
[05:49:10.036] Build status: STARTED
[05:49:10.036] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:50:0) after 5m26s
[05:54:36.264] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:50:0)
[05:54:36.288] Build status: STARTED
[05:54:36.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:51:0) after 2m14s
[05:56:50.536] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:51:0)
[05:56:50.567] Build status: STARTED
[05:56:50.567] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:52:0) after 1m33s
[05:58:23.651] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:52:0)
[05:58:23.691] Build status: STARTED
[05:58:23.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:53:0) after 1m49s
[06:00:13.332] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:53:0)
[06:00:13.354] Build status: STARTED
[06:00:13.354] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:54:0) after 1m25s
[06:01:38.595] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:54:0)
[06:01:38.619] Build status: STARTED
[06:01:38.619] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:55:0) after 8m37s
[06:10:15.709] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:55:0)
[06:10:15.732] Build status: STARTED
[06:10:15.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:56:0) after 3m11s
[06:13:26.931] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:56:0)
[06:13:26.952] Build status: STARTED
[06:13:26.952] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:57:0) after 3m1s
[06:16:27.977] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:57:0)
[06:16:27.995] Build status: STARTED
[06:16:27.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:58:0) after 1m12s
[06:17:40.030] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:58:0)
[06:17:40.048] Build status: STARTED
[06:17:40.048] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:59:0) after 9m28s
[06:27:08.127] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:59:0)
[06:27:08.178] Build status: STARTED
[06:27:08.178] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:60:0) after 8m30s
[06:35:38.313] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:60:0)
[06:35:38.356] Build status: STARTED
[06:35:38.356] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:61:0) after 3m6s
[06:38:44.482] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:61:0)
[06:38:44.504] Build status: STARTED
[06:38:44.504] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:62:0) after 1m58s
[06:40:42.619] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:62:0)
[06:40:42.653] Build status: STARTED
[06:40:42.653] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:63:0) after 1m37s
[06:42:19.686] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:63:0)
[06:42:19.701] Build status: STARTED
[06:42:19.701] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:64:0) after 4m39s
[06:46:58.780] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:64:0)
[06:46:58.803] Build status: STARTED
[06:46:58.803] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:65:0) after 1m22s
[06:48:20.822] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:65:0)
[06:48:20.853] Build status: STARTED
[06:48:20.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:66:0) after 4m3s
[06:52:23.875] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:66:0)
[06:52:23.895] Build status: STARTED
[06:52:23.895] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:67:0) after 7m18s
[06:59:42.122] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:67:0)
[06:59:42.179] Build status: STARTED
[06:59:42.179] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8940613755183467584:68:0) after 4m55s
[07:02:11.863] Received PubSub notification, asking Buildbucket for the build status
[07:02:11.888] Build:
{
"id": "8775183693299146897",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 30011,
"createdBy": "project:chromium",
"createTime": "2023-07-19T01:21:52.161108877Z",
"startTime": "2023-07-19T01:25:19.056882Z",
"endTime": "2023-07-19T07:02:11.394605403Z",
"updateTime": "2023-07-19T07:02:11.394605403Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cf3f29aedad7a52a9895e09ba2b61202d8709924",
"ref": "refs/heads/main"
}
},
"canary": true
}
[07:02:11.888] Invocation finished in 5h40m20.906606676s with status FAILED