[01:53:52.226] New invocation is queued and will start shortly
[01:53:53.337] Starting the invocation (attempt 1)
[01:53:53.369] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/33ceb68b7ca58c4dd26ef2d68a30f6b64fc25c87"
[01:53:53.369] Popped gitiles commit info from properties and tags
[01:53:53.369] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[01:53:53.370] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[01:53:53.370] Buildbucket request:
{
"requestId": "8957553373336098368",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "33ceb68b7ca58c4dd26ef2d68a30f6b64fc25c87",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8957553373336098368"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[01:53:54.004] Scheduled build:
{
"id": "8792123311347895793",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 28759,
"createdBy": "project:chromium",
"createTime": "2023-01-13T01:53:53.498566189Z",
"updateTime": "2023-01-13T01:53:53.498566189Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "33ceb68b7ca58c4dd26ef2d68a30f6b64fc25c87",
"ref": "refs/heads/main"
}
}
}
[01:53:54.004] Task URL: https://cr-buildbucket.appspot.com/build/8792123311347895793
[01:53:54.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:2:0) after 2m58s
[01:56:33.722] Received PubSub notification, asking Buildbucket for the build status
[01:56:33.838] Build status: STARTED
[01:56:52.135] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:2:0)
[01:56:52.159] Build status: STARTED
[01:56:52.159] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:4:0) after 2m34s
[01:59:26.288] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:4:0)
[01:59:26.335] Build status: STARTED
[01:59:26.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:5:0) after 7m51s
[02:07:17.358] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:5:0)
[02:07:17.385] Build status: STARTED
[02:07:17.385] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:6:0) after 1m35s
[02:08:52.411] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:6:0)
[02:08:52.435] Build status: STARTED
[02:08:52.435] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:7:0) after 2m51s
[02:11:43.578] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:7:0)
[02:11:43.605] Build status: STARTED
[02:11:43.605] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:8:0) after 9m42s
[02:21:25.626] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:8:0)
[02:21:25.664] Build status: STARTED
[02:21:25.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:9:0) after 4m19s
[02:25:44.725] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:9:0)
[02:25:44.746] Build status: STARTED
[02:25:44.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:10:0) after 8m44s
[02:34:28.882] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:10:0)
[02:34:28.955] Build status: STARTED
[02:34:28.955] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:11:0) after 4m16s
[02:38:45.095] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:11:0)
[02:38:45.126] Build status: STARTED
[02:38:45.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:12:0) after 4m48s
[02:43:33.211] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:12:0)
[02:43:33.231] Build status: STARTED
[02:43:33.231] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:13:0) after 9m25s
[02:52:58.262] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:13:0)
[02:52:58.302] Build status: STARTED
[02:52:58.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:14:0) after 5m53s
[02:58:51.368] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:14:0)
[02:58:51.398] Build status: STARTED
[02:58:51.398] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:15:0) after 9m32s
[03:08:23.424] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:15:0)
[03:08:23.442] Build status: STARTED
[03:08:23.442] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:16:0) after 4m44s
[03:13:07.500] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:16:0)
[03:13:07.521] Build status: STARTED
[03:13:07.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:17:0) after 5m16s
[03:18:23.549] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:17:0)
[03:18:23.571] Build status: STARTED
[03:18:23.571] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:18:0) after 2m15s
[03:20:38.595] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:18:0)
[03:20:38.620] Build status: STARTED
[03:20:38.620] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:19:0) after 9m58s
[03:30:36.787] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:19:0)
[03:30:36.809] Build status: STARTED
[03:30:36.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:20:0) after 2m32s
[03:33:08.836] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:20:0)
[03:33:08.894] Build status: STARTED
[03:33:08.894] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:21:0) after 5m8s
[03:38:16.914] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:21:0)
[03:38:16.975] Build status: STARTED
[03:38:16.975] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:22:0) after 1m15s
[03:39:32.073] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:22:0)
[03:39:32.104] Build status: STARTED
[03:39:32.104] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:23:0) after 3m52s
[03:43:24.125] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:23:0)
[03:43:24.190] Build status: STARTED
[03:43:24.190] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:24:0) after 7m55s
[03:51:19.273] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:24:0)
[03:51:19.301] Build status: STARTED
[03:51:19.301] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:25:0) after 5m42s
[03:57:01.330] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:25:0)
[03:57:01.364] Build status: STARTED
[03:57:01.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:26:0) after 4m56s
[04:01:57.388] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:26:0)
[04:01:57.409] Build status: STARTED
[04:01:57.409] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:27:0) after 2m12s
[04:04:09.431] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:27:0)
[04:04:09.453] Build status: STARTED
[04:04:09.453] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:28:0) after 8m17s
[04:12:26.481] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:28:0)
[04:12:26.524] Build status: STARTED
[04:12:26.524] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:29:0) after 1m44s
[04:14:10.555] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:29:0)
[04:14:10.577] Build status: STARTED
[04:14:10.577] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:30:0) after 6m49s
[04:20:59.601] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:30:0)
[04:20:59.636] Build status: STARTED
[04:20:59.636] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:31:0) after 7m0s
[04:27:59.826] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:31:0)
[04:27:59.849] Build status: STARTED
[04:27:59.849] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:32:0) after 3m45s
[04:31:44.913] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:32:0)
[04:31:44.961] Build status: STARTED
[04:31:44.961] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:33:0) after 2m36s
[04:34:20.986] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:33:0)
[04:34:21.008] Build status: STARTED
[04:34:21.008] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:34:0) after 9m20s
[04:43:41.265] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:34:0)
[04:43:41.291] Build status: STARTED
[04:43:41.291] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:35:0) after 2m35s
[04:46:16.368] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:35:0)
[04:46:16.398] Build status: STARTED
[04:46:16.398] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:36:0) after 8m48s
[04:55:04.579] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:36:0)
[04:55:04.603] Build status: STARTED
[04:55:04.603] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:37:0) after 6m13s
[05:01:17.701] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:37:0)
[05:01:17.727] Build status: STARTED
[05:01:17.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:38:0) after 4m23s
[05:05:40.842] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:38:0)
[05:05:40.877] Build status: STARTED
[05:05:40.877] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:39:0) after 2m42s
[05:08:22.898] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:39:0)
[05:08:22.919] Build status: STARTED
[05:08:22.919] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:40:0) after 6m25s
[05:14:48.508] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:40:0)
[05:14:48.530] Build status: STARTED
[05:14:48.530] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:41:0) after 6m58s
[05:21:46.523] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:41:0)
[05:21:46.555] Build status: STARTED
[05:21:46.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:42:0) after 9m20s
[05:31:06.586] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:42:0)
[05:31:06.672] Build status: STARTED
[05:31:06.672] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:43:0) after 5m38s
[05:36:44.711] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:43:0)
[05:36:44.739] Build status: STARTED
[05:36:44.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:44:0) after 6m15s
[05:42:59.816] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:44:0)
[05:42:59.835] Build status: STARTED
[05:42:59.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:45:0) after 7m57s
[05:50:56.904] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:45:0)
[05:50:56.928] Build status: STARTED
[05:50:56.928] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:46:0) after 9m2s
[05:59:58.961] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:46:0)
[05:59:58.996] Build status: STARTED
[05:59:58.996] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:47:0) after 1m13s
[06:01:12.070] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:47:0)
[06:01:12.110] Build status: STARTED
[06:01:12.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:48:0) after 6m21s
[06:07:33.145] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:48:0)
[06:07:33.172] Build status: STARTED
[06:07:33.172] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:49:0) after 5m50s
[06:13:23.407] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:49:0)
[06:13:23.450] Build status: STARTED
[06:13:23.450] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:50:0) after 3m14s
[06:16:37.474] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:50:0)
[06:16:37.494] Build status: STARTED
[06:16:37.494] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:51:0) after 8m50s
[06:25:27.527] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:51:0)
[06:25:27.551] Build status: STARTED
[06:25:27.551] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:52:0) after 6m59s
[06:32:26.600] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:52:0)
[06:32:26.751] Build status: STARTED
[06:32:26.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:53:0) after 9m11s
[06:41:37.774] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:53:0)
[06:41:37.797] Build status: STARTED
[06:41:37.797] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:54:0) after 3m4s
[06:44:41.892] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:54:0)
[06:44:41.913] Build status: STARTED
[06:44:41.913] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:55:0) after 1m3s
[06:45:44.938] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:55:0)
[06:45:44.965] Build status: STARTED
[06:45:44.965] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:56:0) after 3m52s
[06:49:37.064] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:56:0)
[06:49:37.087] Build status: STARTED
[06:49:37.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:57:0) after 2m49s
[06:52:26.118] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:57:0)
[06:52:26.148] Build status: STARTED
[06:52:26.149] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:58:0) after 7m2s
[06:59:28.175] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:58:0)
[06:59:28.197] Build status: STARTED
[06:59:28.198] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:59:0) after 6m0s
[07:05:28.368] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:59:0)
[07:05:28.394] Build status: STARTED
[07:05:28.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:60:0) after 2m30s
[07:07:58.426] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:60:0)
[07:07:58.580] Build status: STARTED
[07:07:58.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:61:0) after 4m50s
[07:12:48.671] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:61:0)
[07:12:48.691] Build status: STARTED
[07:12:48.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:62:0) after 2m12s
[07:15:00.673] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:62:0)
[07:15:00.708] Build status: STARTED
[07:15:00.708] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:63:0) after 9m39s
[07:24:39.732] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:63:0)
[07:24:39.751] Build status: STARTED
[07:24:39.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:64:0) after 7m14s
[07:31:53.777] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:64:0)
[07:31:53.801] Build status: STARTED
[07:31:53.801] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:65:0) after 8m38s
[07:40:31.884] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:65:0)
[07:40:31.938] Build status: STARTED
[07:40:31.938] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:66:0) after 4m6s
[07:44:37.962] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:66:0)
[07:44:37.983] Build status: STARTED
[07:44:37.983] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:67:0) after 2m1s
[07:46:39.112] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:67:0)
[07:46:39.132] Build status: STARTED
[07:46:39.132] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8957553373336098368:68:0) after 7m7s
[07:51:49.920] Received PubSub notification, asking Buildbucket for the build status
[07:51:49.941] Build:
{
"id": "8792123311347895793",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 28759,
"createdBy": "project:chromium",
"createTime": "2023-01-13T01:53:53.498566189Z",
"startTime": "2023-01-13T01:56:33.117819Z",
"endTime": "2023-01-13T07:51:49.662527412Z",
"updateTime": "2023-01-13T07:51:49.662527412Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "33ceb68b7ca58c4dd26ef2d68a30f6b64fc25c87",
"ref": "refs/heads/main"
}
}
}
[07:51:49.941] Invocation finished in 5h57m57.722077575s with status FAILED