[15:38:22.437] New invocation is queued and will start shortly
[15:38:24.154] Starting the invocation (attempt 1)
[15:38:24.185] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/ebb761783b647b083cd675a67ec1cb52a8906c2c"
[15:38:24.185] Popped gitiles commit info from properties and tags
[15:38:24.185] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:38:24.185] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:38:24.185] Buildbucket request:
{
"requestId": "8848785140380789936",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ebb761783b647b083cd675a67ec1cb52a8906c2c",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8848785140380789936"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:38:24.981] Scheduled build:
{
"id": "8683355077664521233",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41411,
"createdBy": "project:chromium",
"createTime": "2026-04-27T15:38:24.403424328Z",
"updateTime": "2026-04-27T15:38:24.403424328Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ebb761783b647b083cd675a67ec1cb52a8906c2c",
"ref": "refs/heads/main"
}
}
}
[15:38:24.981] Task URL: https://cr-buildbucket.appspot.com/build/8683355077664521233
[15:38:24.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:2:0) after 4m38s
[15:38:25.861] Received PubSub notification, asking Buildbucket for the build status
[15:38:25.898] Build status: SCHEDULED
[15:39:01.851] Received PubSub notification, asking Buildbucket for the build status
[15:39:01.935] Build status: STARTED
[15:43:03.066] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:2:0)
[15:43:03.086] Build status: STARTED
[15:43:03.086] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:5:0) after 5m35s
[15:48:38.158] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:5:0)
[15:48:38.175] Build status: STARTED
[15:48:38.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:6:0) after 9m32s
[15:58:10.205] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:6:0)
[15:58:10.224] Build status: STARTED
[15:58:10.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:7:0) after 9m9s
[16:07:19.343] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:7:0)
[16:07:19.359] Build status: STARTED
[16:07:19.359] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:8:0) after 2m34s
[16:09:53.332] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:8:0)
[16:09:53.355] Build status: STARTED
[16:09:53.355] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:9:0) after 6m46s
[16:16:39.372] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:9:0)
[16:16:39.388] Build status: STARTED
[16:16:39.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:10:0) after 7m17s
[16:23:56.511] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:10:0)
[16:23:56.529] Build status: STARTED
[16:23:56.529] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:11:0) after 9m23s
[16:33:19.635] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:11:0)
[16:33:19.651] Build status: STARTED
[16:33:19.651] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:12:0) after 2m8s
[16:35:27.722] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:12:0)
[16:35:27.752] Build status: STARTED
[16:35:27.752] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:13:0) after 1m32s
[16:36:59.786] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:13:0)
[16:36:59.807] Build status: STARTED
[16:36:59.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:14:0) after 5m54s
[16:42:53.878] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:14:0)
[16:42:53.896] Build status: STARTED
[16:42:53.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:15:0) after 7m17s
[16:50:10.924] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:15:0)
[16:50:10.947] Build status: STARTED
[16:50:10.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:16:0) after 1m28s
[16:51:39.018] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:16:0)
[16:51:39.116] Build status: STARTED
[16:51:39.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:17:0) after 6m3s
[16:57:42.134] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:17:0)
[16:57:42.150] Build status: STARTED
[16:57:42.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:18:0) after 6m7s
[17:03:49.167] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:18:0)
[17:03:49.185] Build status: STARTED
[17:03:49.185] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:19:0) after 8m48s
[17:12:37.203] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:19:0)
[17:12:37.225] Build status: STARTED
[17:12:37.225] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:20:0) after 9m13s
[17:21:50.247] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:20:0)
[17:21:50.265] Build status: STARTED
[17:21:50.265] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:21:0) after 3m3s
[17:24:53.266] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:21:0)
[17:24:53.282] Build status: STARTED
[17:24:53.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:22:0) after 3m51s
[17:28:44.314] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:22:0)
[17:28:44.346] Build status: STARTED
[17:28:44.346] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:23:0) after 4m57s
[17:33:41.413] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:23:0)
[17:33:41.434] Build status: STARTED
[17:33:41.434] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:24:0) after 8m10s
[17:41:51.551] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:24:0)
[17:41:51.649] Build status: STARTED
[17:41:51.649] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:25:0) after 5m7s
[17:46:58.665] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:25:0)
[17:46:58.682] Build status: STARTED
[17:46:58.682] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:26:0) after 3m4s
[17:50:02.697] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:26:0)
[17:50:02.711] Build status: STARTED
[17:50:02.711] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:27:0) after 7m43s
[17:57:45.730] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:27:0)
[17:57:45.746] Build status: STARTED
[17:57:45.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:28:0) after 1m49s
[17:59:34.763] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:28:0)
[17:59:34.779] Build status: STARTED
[17:59:34.780] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:29:0) after 3m36s
[18:03:10.829] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:29:0)
[18:03:10.845] Build status: STARTED
[18:03:10.845] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:30:0) after 5m24s
[18:08:34.846] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:30:0)
[18:08:34.869] Build status: STARTED
[18:08:34.869] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:31:0) after 8m36s
[18:17:10.915] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:31:0)
[18:17:10.933] Build status: STARTED
[18:17:10.933] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:32:0) after 2m17s
[18:19:27.950] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:32:0)
[18:19:27.969] Build status: STARTED
[18:19:27.969] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:33:0) after 4m33s
[18:24:01.000] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:33:0)
[18:24:01.022] Build status: STARTED
[18:24:01.022] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:34:0) after 9m57s
[18:33:57.999] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:34:0)
[18:33:58.015] Build status: STARTED
[18:33:58.015] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:35:0) after 5m35s
[18:39:33.066] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:35:0)
[18:39:33.084] Build status: STARTED
[18:39:33.084] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:36:0) after 3m30s
[18:43:03.156] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:36:0)
[18:43:03.174] Build status: STARTED
[18:43:03.174] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:37:0) after 4m6s
[18:47:09.191] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:37:0)
[18:47:09.210] Build status: STARTED
[18:47:09.210] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:38:0) after 5m50s
[18:52:59.329] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:38:0)
[18:52:59.348] Build status: STARTED
[18:52:59.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:39:0) after 5m59s
[18:58:58.353] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:39:0)
[18:58:58.383] Build status: STARTED
[18:58:58.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:40:0) after 5m54s
[19:04:52.507] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:40:0)
[19:04:52.526] Build status: STARTED
[19:04:52.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:41:0) after 1m46s
[19:06:38.655] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:41:0)
[19:06:38.670] Build status: STARTED
[19:06:38.670] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:42:0) after 6m13s
[19:12:51.827] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:42:0)
[19:12:51.846] Build status: STARTED
[19:12:51.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:43:0) after 3m39s
[19:16:30.883] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:43:0)
[19:16:30.917] Build status: STARTED
[19:16:30.917] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:44:0) after 7m42s
[19:24:13.035] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:44:0)
[19:24:13.053] Build status: STARTED
[19:24:13.053] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:45:0) after 8m14s
[19:32:27.072] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:45:0)
[19:32:27.089] Build status: STARTED
[19:32:27.089] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:46:0) after 4m58s
[19:37:25.115] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:46:0)
[19:37:25.136] Build status: STARTED
[19:37:25.136] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:47:0) after 4m55s
[19:42:20.211] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:47:0)
[19:42:20.229] Build status: STARTED
[19:42:20.229] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:48:0) after 7m14s
[19:49:34.251] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:48:0)
[19:49:34.280] Build status: STARTED
[19:49:34.280] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:49:0) after 4m11s
[19:53:45.364] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:49:0)
[19:53:45.445] Build status: STARTED
[19:53:45.445] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:50:0) after 7m37s
[20:01:22.467] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:50:0)
[20:01:22.499] Build status: STARTED
[20:01:22.499] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:51:0) after 1m16s
[20:02:38.516] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:51:0)
[20:02:38.533] Build status: STARTED
[20:02:38.533] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:52:0) after 4m9s
[20:06:47.550] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:52:0)
[20:06:47.567] Build status: STARTED
[20:06:47.567] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:53:0) after 2m48s
[20:09:35.584] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:53:0)
[20:09:35.607] Build status: STARTED
[20:09:35.607] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:54:0) after 5m27s
[20:15:02.648] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:54:0)
[20:15:02.668] Build status: STARTED
[20:15:02.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8848785140380789936:55:0) after 1m50s
[20:16:48.041] Received PubSub notification, asking Buildbucket for the build status
[20:16:48.173] Build:
{
"id": "8683355077664521233",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41411,
"createdBy": "project:chromium",
"createTime": "2026-04-27T15:38:24.403424328Z",
"startTime": "2026-04-27T15:39:01.701259598Z",
"endTime": "2026-04-27T20:16:47.658485713Z",
"updateTime": "2026-04-27T20:16:47.658485713Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ebb761783b647b083cd675a67ec1cb52a8906c2c",
"ref": "refs/heads/main"
}
}
}
[20:16:48.173] Invocation finished in 4h38m25.743995994s with status FAILED