[21:32:22.551] New invocation is queued and will start shortly
[21:32:23.736] Starting the invocation (attempt 1)
[21:32:23.761] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/bb2acc2f1bd85bfab155abfc0e07d5761269f370"
[21:32:23.761] Popped gitiles commit info from properties and tags
[21:32:23.761] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:32:23.761] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:32:23.761] Buildbucket request:
{
"requestId": "8878841061347010656",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bb2acc2f1bd85bfab155abfc0e07d5761269f370",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8878841061347010656"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:32:24.246] Scheduled build:
{
"id": "8713410999382814353",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 38400,
"createdBy": "project:chromium",
"createTime": "2025-05-30T21:32:23.804603931Z",
"updateTime": "2025-05-30T21:32:23.804603931Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bb2acc2f1bd85bfab155abfc0e07d5761269f370",
"ref": "refs/heads/main"
}
}
}
[21:32:24.246] Task URL: https://cr-buildbucket.appspot.com/build/8713410999382814353
[21:32:24.246] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:2:0) after 1m28s
[21:32:24.467] Received PubSub notification, asking Buildbucket for the build status
[21:32:24.497] Build status: SCHEDULED
[21:33:01.167] Received PubSub notification, asking Buildbucket for the build status
[21:33:01.224] Build status: STARTED
[21:33:52.309] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:2:0)
[21:33:52.339] Build status: STARTED
[21:33:52.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:5:0) after 3m29s
[21:37:21.362] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:5:0)
[21:37:21.382] Build status: STARTED
[21:37:21.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:6:0) after 9m0s
[21:46:21.400] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:6:0)
[21:46:21.420] Build status: STARTED
[21:46:21.420] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:7:0) after 3m31s
[21:49:52.436] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:7:0)
[21:49:52.464] Build status: STARTED
[21:49:52.464] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:8:0) after 1m21s
[21:51:13.483] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:8:0)
[21:51:13.501] Build status: STARTED
[21:51:13.501] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:9:0) after 5m31s
[21:56:44.521] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:9:0)
[21:56:44.602] Build status: STARTED
[21:56:44.602] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:10:0) after 8m34s
[22:05:18.675] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:10:0)
[22:05:18.737] Build status: STARTED
[22:05:18.737] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:11:0) after 8m32s
[22:13:50.825] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:11:0)
[22:13:50.949] Build status: STARTED
[22:13:50.949] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:12:0) after 9m19s
[22:23:10.020] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:12:0)
[22:23:10.038] Build status: STARTED
[22:23:10.038] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:13:0) after 1m34s
[22:24:44.089] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:13:0)
[22:24:44.107] Build status: STARTED
[22:24:44.107] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:14:0) after 2m53s
[22:27:37.129] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:14:0)
[22:27:37.152] Build status: STARTED
[22:27:37.152] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:15:0) after 4m13s
[22:31:50.173] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:15:0)
[22:31:50.191] Build status: STARTED
[22:31:50.191] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:16:0) after 9m4s
[22:40:54.220] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:16:0)
[22:40:54.244] Build status: STARTED
[22:40:54.245] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:17:0) after 1m8s
[22:42:02.267] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:17:0)
[22:42:02.285] Build status: STARTED
[22:42:02.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:18:0) after 2m43s
[22:44:45.301] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:18:0)
[22:44:45.325] Build status: STARTED
[22:44:45.325] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:19:0) after 4m18s
[22:49:03.338] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:19:0)
[22:49:03.366] Build status: STARTED
[22:49:03.366] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:20:0) after 5m31s
[22:54:34.343] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:20:0)
[22:54:34.368] Build status: STARTED
[22:54:34.368] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:21:0) after 2m8s
[22:56:42.438] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:21:0)
[22:56:42.476] Build status: STARTED
[22:56:42.476] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:22:0) after 3m26s
[23:00:08.522] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:22:0)
[23:00:08.545] Build status: STARTED
[23:00:08.545] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:23:0) after 4m17s
[23:04:25.562] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:23:0)
[23:04:25.609] Build status: STARTED
[23:04:25.609] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:24:0) after 9m2s
[23:13:27.681] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:24:0)
[23:13:27.721] Build status: STARTED
[23:13:27.721] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:25:0) after 8m47s
[23:22:14.840] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:25:0)
[23:22:14.856] Build status: STARTED
[23:22:14.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:26:0) after 1m37s
[23:23:51.845] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:26:0)
[23:23:51.866] Build status: STARTED
[23:23:51.866] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:27:0) after 2m27s
[23:26:18.919] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:27:0)
[23:26:18.937] Build status: STARTED
[23:26:18.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:28:0) after 9m43s
[23:36:01.959] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:28:0)
[23:36:01.986] Build status: STARTED
[23:36:01.986] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:29:0) after 9m25s
[23:45:27.089] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:29:0)
[23:45:27.106] Build status: STARTED
[23:45:27.106] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:30:0) after 5m52s
[23:51:19.132] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:30:0)
[23:51:19.158] Build status: STARTED
[23:51:19.158] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:31:0) after 1m38s
[23:52:57.174] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:31:0)
[23:52:57.199] Build status: STARTED
[23:52:57.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:32:0) after 2m7s
[23:55:04.220] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:32:0)
[23:55:04.240] Build status: STARTED
[23:55:04.240] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:33:0) after 3m15s
[23:58:19.293] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:33:0)
[23:58:19.311] Build status: STARTED
[23:58:19.311] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:34:0) after 7m55s
[00:06:14.380] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:34:0)
[00:06:14.394] Build status: STARTED
[00:06:14.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:35:0) after 7m39s
[00:13:53.468] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:35:0)
[00:13:53.489] Build status: STARTED
[00:13:53.489] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:36:0) after 9m54s
[00:23:47.677] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:36:0)
[00:23:47.693] Build status: STARTED
[00:23:47.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:37:0) after 5m20s
[00:29:07.804] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:37:0)
[00:29:07.823] Build status: STARTED
[00:29:07.823] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:38:0) after 3m44s
[00:32:51.839] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:38:0)
[00:32:51.858] Build status: STARTED
[00:32:51.858] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:39:0) after 1m3s
[00:33:54.878] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:39:0)
[00:33:54.898] Build status: STARTED
[00:33:54.898] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:40:0) after 8m0s
[00:41:54.943] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:40:0)
[00:41:55.044] Build status: STARTED
[00:41:55.044] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:41:0) after 5m7s
[00:47:02.063] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:41:0)
[00:47:02.098] Build status: STARTED
[00:47:02.098] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:42:0) after 5m2s
[00:52:04.170] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:42:0)
[00:52:04.192] Build status: STARTED
[00:52:04.192] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:43:0) after 1m51s
[00:53:55.252] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:43:0)
[00:53:55.271] Build status: STARTED
[00:53:55.271] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:44:0) after 6m17s
[01:00:12.361] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:44:0)
[01:00:12.378] Build status: STARTED
[01:00:12.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:45:0) after 2m50s
[01:03:02.398] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:45:0)
[01:03:02.473] Build status: STARTED
[01:03:02.473] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:46:0) after 3m52s
[01:06:54.540] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:46:0)
[01:06:54.561] Build status: STARTED
[01:06:54.561] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:47:0) after 6m3s
[01:12:57.565] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:47:0)
[01:12:57.621] Build status: STARTED
[01:12:57.621] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:48:0) after 4m59s
[01:17:56.646] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:48:0)
[01:17:56.685] Build status: STARTED
[01:17:56.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8878841061347010656:49:0) after 5m48s
[01:20:28.015] Received PubSub notification, asking Buildbucket for the build status
[01:20:28.037] Build:
{
"id": "8713410999382814353",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 38400,
"createdBy": "project:chromium",
"createTime": "2025-05-30T21:32:23.804603931Z",
"startTime": "2025-05-30T21:33:00.820380511Z",
"endTime": "2025-05-31T01:20:27.697736171Z",
"updateTime": "2025-05-31T01:20:27.697736171Z",
"status": "INFRA_FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bb2acc2f1bd85bfab155abfc0e07d5761269f370",
"ref": "refs/heads/main"
}
}
}
[01:20:28.037] Invocation finished in 3h48m5.488612417s with status FAILED