[13:59:50.069] New invocation is queued and will start shortly
[13:59:51.156] Starting the invocation (attempt 1)
[13:59:51.182] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/070b42c6694b5dbe4487eb8e3b2d2fcb8531d5cd"
[13:59:51.182] Popped gitiles commit info from properties and tags
[13:59:51.182] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[13:59:51.182] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[13:59:51.182] Buildbucket request:
{
"requestId": "8851418651970223040",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "070b42c6694b5dbe4487eb8e3b2d2fcb8531d5cd",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851418651970223040"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[13:59:51.644] Scheduled build:
{
"id": "8685988590100796241",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41149,
"createdBy": "project:chromium",
"createTime": "2026-03-29T13:59:51.231375414Z",
"updateTime": "2026-03-29T13:59:51.231375414Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "070b42c6694b5dbe4487eb8e3b2d2fcb8531d5cd",
"ref": "refs/heads/main"
}
},
"canary": true
}
[13:59:51.644] Task URL: https://cr-buildbucket.appspot.com/build/8685988590100796241
[13:59:51.645] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:2:0) after 3m27s
[13:59:52.579] Received PubSub notification, asking Buildbucket for the build status
[13:59:52.597] Build status: SCHEDULED
[14:00:30.448] Received PubSub notification, asking Buildbucket for the build status
[14:00:30.473] Build status: STARTED
[14:03:18.774] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:2:0)
[14:03:18.787] Build status: STARTED
[14:03:18.787] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:5:0) after 7m46s
[14:11:04.981] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:5:0)
[14:11:05.002] Build status: STARTED
[14:11:05.002] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:6:0) after 9m53s
[14:20:57.992] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:6:0)
[14:20:58.020] Build status: STARTED
[14:20:58.020] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:7:0) after 1m33s
[14:22:31.704] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:7:0)
[14:22:31.728] Build status: STARTED
[14:22:31.728] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:8:0) after 9m58s
[14:32:29.895] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:8:0)
[14:32:29.917] Build status: STARTED
[14:32:29.917] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:9:0) after 1m39s
[14:34:09.469] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:9:0)
[14:34:09.488] Build status: STARTED
[14:34:09.488] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:10:0) after 4m24s
[14:38:33.759] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:10:0)
[14:38:33.779] Build status: STARTED
[14:38:33.779] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:11:0) after 6m26s
[14:44:59.891] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:11:0)
[14:44:59.914] Build status: STARTED
[14:44:59.914] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:12:0) after 1m22s
[14:46:22.057] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:12:0)
[14:46:22.075] Build status: STARTED
[14:46:22.075] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:13:0) after 2m58s
[14:49:20.191] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:13:0)
[14:49:20.241] Build status: STARTED
[14:49:20.241] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:14:0) after 7m6s
[14:56:26.290] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:14:0)
[14:56:26.382] Build status: STARTED
[14:56:26.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:15:0) after 8m2s
[15:04:28.534] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:15:0)
[15:04:28.574] Build status: STARTED
[15:04:28.574] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:16:0) after 6m50s
[15:11:18.586] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:16:0)
[15:11:18.608] Build status: STARTED
[15:11:18.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:17:0) after 6m13s
[15:17:31.945] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:17:0)
[15:17:31.963] Build status: STARTED
[15:17:31.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:18:0) after 7m26s
[15:24:58.076] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:18:0)
[15:24:58.090] Build status: STARTED
[15:24:58.090] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:19:0) after 3m58s
[15:28:56.204] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:19:0)
[15:28:56.223] Build status: STARTED
[15:28:56.223] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:20:0) after 6m27s
[15:35:23.804] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:20:0)
[15:35:23.821] Build status: STARTED
[15:35:23.821] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:21:0) after 7m34s
[15:42:57.859] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:21:0)
[15:42:57.874] Build status: STARTED
[15:42:57.874] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:22:0) after 4m24s
[15:47:21.944] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:22:0)
[15:47:21.958] Build status: STARTED
[15:47:21.958] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:23:0) after 7m4s
[15:54:25.972] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:23:0)
[15:54:25.990] Build status: STARTED
[15:54:25.990] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:24:0) after 3m39s
[15:58:05.050] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:24:0)
[15:58:05.079] Build status: STARTED
[15:58:05.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:25:0) after 6m41s
[16:04:46.322] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:25:0)
[16:04:46.342] Build status: STARTED
[16:04:46.342] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:26:0) after 1m7s
[16:05:53.354] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:26:0)
[16:05:53.376] Build status: STARTED
[16:05:53.376] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:27:0) after 5m58s
[16:11:51.390] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:27:0)
[16:11:51.412] Build status: STARTED
[16:11:51.412] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:28:0) after 7m25s
[16:19:16.815] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:28:0)
[16:19:16.831] Build status: STARTED
[16:19:16.831] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:29:0) after 5m49s
[16:25:05.910] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:29:0)
[16:25:05.928] Build status: STARTED
[16:25:05.928] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:30:0) after 7m39s
[16:32:44.897] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:30:0)
[16:32:44.923] Build status: STARTED
[16:32:44.923] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:31:0) after 5m57s
[16:38:42.461] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:31:0)
[16:38:42.481] Build status: STARTED
[16:38:42.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:32:0) after 9m39s
[16:48:21.489] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:32:0)
[16:48:21.503] Build status: STARTED
[16:48:21.503] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:33:0) after 6m45s
[16:55:06.516] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:33:0)
[16:55:06.531] Build status: STARTED
[16:55:06.531] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:34:0) after 6m37s
[17:01:44.038] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:34:0)
[17:01:44.077] Build status: STARTED
[17:01:44.077] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:35:0) after 4m13s
[17:05:57.190] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:35:0)
[17:05:57.210] Build status: STARTED
[17:05:57.210] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:36:0) after 5m33s
[17:11:30.227] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:36:0)
[17:11:30.245] Build status: STARTED
[17:11:30.245] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:37:0) after 4m17s
[17:15:47.309] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:37:0)
[17:15:47.324] Build status: STARTED
[17:15:47.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:38:0) after 4m50s
[17:20:37.416] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:38:0)
[17:20:37.440] Build status: STARTED
[17:20:37.440] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851418651970223040:39:0) after 9m50s
[17:22:10.654] Received PubSub notification, asking Buildbucket for the build status
[17:22:10.676] Build:
{
"id": "8685988590100796241",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41149,
"createdBy": "project:chromium",
"createTime": "2026-03-29T13:59:51.231375414Z",
"startTime": "2026-03-29T14:00:28.746400114Z",
"endTime": "2026-03-29T17:22:10.283039804Z",
"updateTime": "2026-03-29T17:22:10.283039804Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "070b42c6694b5dbe4487eb8e3b2d2fcb8531d5cd",
"ref": "refs/heads/main"
}
},
"canary": true
}
[17:22:10.676] Invocation finished in 3h22m20.612770639s with status FAILED