[15:36:54.988] New invocation is queued and will start shortly
[15:36:56.345] Starting the invocation (attempt 1)
[15:36:56.379] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d19e9392a0358e50832f9a040a7325429e109791"
[15:36:56.380] Popped gitiles commit info from properties and tags
[15:36:56.380] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:36:56.381] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:36:56.381] Buildbucket request:
{
"requestId": "8876689097723343584",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d19e9392a0358e50832f9a040a7325429e109791",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8876689097723343584"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:36:56.939] Scheduled build:
{
"id": "8711259035549630081",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 38630,
"createdBy": "project:chromium",
"createTime": "2025-06-23T15:36:56.442688398Z",
"updateTime": "2025-06-23T15:36:56.442688398Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d19e9392a0358e50832f9a040a7325429e109791",
"ref": "refs/heads/main"
}
},
"canary": true
}
[15:36:56.939] Task URL: https://cr-buildbucket.appspot.com/build/8711259035549630081
[15:36:56.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:2:0) after 1m44s
[15:36:57.932] Received PubSub notification, asking Buildbucket for the build status
[15:36:58.035] Build status: SCHEDULED
[15:37:00.333] Received PubSub notification, asking Buildbucket for the build status
[15:37:00.353] Build status: STARTED
[15:38:40.962] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:2:0)
[15:38:41.006] Build status: STARTED
[15:38:41.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:5:0) after 9m0s
[15:47:41.024] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:5:0)
[15:47:41.052] Build status: STARTED
[15:47:41.052] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:6:0) after 2m53s
[15:50:34.075] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:6:0)
[15:50:34.108] Build status: STARTED
[15:50:34.108] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:7:0) after 4m22s
[15:54:56.159] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:7:0)
[15:54:56.178] Build status: STARTED
[15:54:56.178] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:8:0) after 6m22s
[16:01:18.251] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:8:0)
[16:01:18.273] Build status: STARTED
[16:01:18.273] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:9:0) after 4m11s
[16:05:29.330] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:9:0)
[16:05:29.370] Build status: STARTED
[16:05:29.370] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:10:0) after 4m50s
[16:10:19.350] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:10:0)
[16:10:19.374] Build status: STARTED
[16:10:19.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:11:0) after 9m16s
[16:19:35.391] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:11:0)
[16:19:35.404] Build status: STARTED
[16:19:35.404] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:12:0) after 9m50s
[16:29:25.420] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:12:0)
[16:29:25.436] Build status: STARTED
[16:29:25.436] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:13:0) after 7m33s
[16:36:58.466] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:13:0)
[16:36:58.490] Build status: STARTED
[16:36:58.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:14:0) after 6m32s
[16:43:30.570] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:14:0)
[16:43:30.586] Build status: STARTED
[16:43:30.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:15:0) after 6m19s
[16:49:49.610] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:15:0)
[16:49:49.630] Build status: STARTED
[16:49:49.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:16:0) after 5m2s
[16:54:51.695] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:16:0)
[16:54:51.878] Build status: STARTED
[16:54:51.878] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:17:0) after 9m9s
[17:04:00.905] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:17:0)
[17:04:00.942] Build status: STARTED
[17:04:00.942] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:18:0) after 6m13s
[17:10:14.111] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:18:0)
[17:10:14.162] Build status: STARTED
[17:10:14.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:19:0) after 9m59s
[17:20:13.231] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:19:0)
[17:20:13.248] Build status: STARTED
[17:20:13.248] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:20:0) after 3m35s
[17:23:48.302] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:20:0)
[17:23:48.319] Build status: STARTED
[17:23:48.319] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:21:0) after 9m20s
[17:33:08.380] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:21:0)
[17:33:08.397] Build status: STARTED
[17:33:08.397] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:22:0) after 7m24s
[17:40:32.423] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:22:0)
[17:40:32.439] Build status: STARTED
[17:40:32.439] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:23:0) after 5m10s
[17:45:42.514] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:23:0)
[17:45:42.533] Build status: STARTED
[17:45:42.533] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:24:0) after 6m22s
[17:52:04.565] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:24:0)
[17:52:04.590] Build status: STARTED
[17:52:04.590] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:25:0) after 4m22s
[17:56:26.618] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:25:0)
[17:56:26.636] Build status: STARTED
[17:56:26.636] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:26:0) after 7m7s
[18:03:33.665] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:26:0)
[18:03:33.686] Build status: STARTED
[18:03:33.686] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:27:0) after 8m16s
[18:11:49.764] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:27:0)
[18:11:49.790] Build status: STARTED
[18:11:49.790] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:28:0) after 5m50s
[18:17:39.873] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:28:0)
[18:17:39.896] Build status: STARTED
[18:17:39.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:29:0) after 3m44s
[18:21:23.993] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:29:0)
[18:21:24.013] Build status: STARTED
[18:21:24.013] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:30:0) after 6m54s
[18:28:18.105] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:30:0)
[18:28:18.127] Build status: STARTED
[18:28:18.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:31:0) after 4m48s
[18:33:06.152] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:31:0)
[18:33:06.170] Build status: STARTED
[18:33:06.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:32:0) after 1m14s
[18:34:20.190] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:32:0)
[18:34:20.210] Build status: STARTED
[18:34:20.210] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:33:0) after 1m7s
[18:35:27.241] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:33:0)
[18:35:27.257] Build status: STARTED
[18:35:27.257] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:34:0) after 5m39s
[18:41:06.333] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:34:0)
[18:41:06.364] Build status: STARTED
[18:41:06.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:35:0) after 3m46s
[18:44:52.499] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:35:0)
[18:44:52.532] Build status: STARTED
[18:44:52.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:36:0) after 3m38s
[18:48:30.595] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:36:0)
[18:48:30.610] Build status: STARTED
[18:48:30.611] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:37:0) after 6m30s
[18:55:00.669] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:37:0)
[18:55:00.690] Build status: STARTED
[18:55:00.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:38:0) after 3m6s
[18:58:06.681] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:38:0)
[18:58:06.701] Build status: STARTED
[18:58:06.701] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:39:0) after 3m39s
[19:01:45.790] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:39:0)
[19:01:45.830] Build status: STARTED
[19:01:45.830] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:40:0) after 1m57s
[19:03:42.857] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:40:0)
[19:03:42.874] Build status: STARTED
[19:03:42.874] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:41:0) after 4m21s
[19:08:03.934] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:41:0)
[19:08:04.003] Build status: STARTED
[19:08:04.003] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:42:0) after 3m35s
[19:11:39.061] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:42:0)
[19:11:39.102] Build status: STARTED
[19:11:39.102] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:43:0) after 4m43s
[19:16:22.156] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:43:0)
[19:16:22.175] Build status: STARTED
[19:16:22.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:44:0) after 5m43s
[19:22:05.205] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:44:0)
[19:22:05.228] Build status: STARTED
[19:22:05.228] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8876689097723343584:45:0) after 9m30s
[19:22:39.904] Received PubSub notification, asking Buildbucket for the build status
[19:22:39.936] Build:
{
"id": "8711259035549630081",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 38630,
"createdBy": "project:chromium",
"createTime": "2025-06-23T15:36:56.442688398Z",
"startTime": "2025-06-23T15:37:00.089572408Z",
"endTime": "2025-06-23T19:22:39.422043272Z",
"updateTime": "2025-06-23T19:22:39.422043272Z",
"status": "INFRA_FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d19e9392a0358e50832f9a040a7325429e109791",
"ref": "refs/heads/main"
}
},
"canary": true
}
[19:22:39.936] Invocation finished in 3h45m44.951128565s with status FAILED