[03:08:34.074] New invocation is queued and will start shortly
[03:08:35.577] Starting the invocation (attempt 1)
[03:08:35.603] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/cf3c8441134331148737ba36d8a0bea29b707509"
[03:08:35.603] Popped gitiles commit info from properties and tags
[03:08:35.603] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[03:08:35.603] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[03:08:35.603] Buildbucket request:
{
"requestId": "8851187835220885568",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cf3c8441134331148737ba36d8a0bea29b707509",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851187835220885568"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[03:08:36.212] Scheduled build:
{
"id": "8685757772919173569",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41167,
"createdBy": "project:chromium",
"createTime": "2026-04-01T03:08:35.649747913Z",
"updateTime": "2026-04-01T03:08:35.649747913Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cf3c8441134331148737ba36d8a0bea29b707509",
"ref": "refs/heads/main"
}
}
}
[03:08:36.212] Task URL: https://cr-buildbucket.appspot.com/build/8685757772919173569
[03:08:36.212] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:2:0) after 1m45s
[03:08:37.030] Received PubSub notification, asking Buildbucket for the build status
[03:08:37.070] Build status: SCHEDULED
[03:09:22.003] Received PubSub notification, asking Buildbucket for the build status
[03:09:22.218] Build status: STARTED
[03:10:21.288] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:2:0)
[03:10:21.354] Build status: STARTED
[03:10:21.354] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:5:0) after 1m48s
[03:12:09.358] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:5:0)
[03:12:09.399] Build status: STARTED
[03:12:09.399] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:6:0) after 5m29s
[03:17:38.422] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:6:0)
[03:17:38.456] Build status: STARTED
[03:17:38.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:7:0) after 4m1s
[03:21:39.484] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:7:0)
[03:21:39.511] Build status: STARTED
[03:21:39.511] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:8:0) after 4m42s
[03:26:21.529] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:8:0)
[03:26:21.548] Build status: STARTED
[03:26:21.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:9:0) after 3m49s
[03:30:10.608] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:9:0)
[03:30:10.626] Build status: STARTED
[03:30:10.626] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:10:0) after 7m26s
[03:37:36.695] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:10:0)
[03:37:36.722] Build status: STARTED
[03:37:36.722] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:11:0) after 4m30s
[03:42:06.747] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:11:0)
[03:42:06.774] Build status: STARTED
[03:42:06.774] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:12:0) after 5m47s
[03:47:53.802] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:12:0)
[03:47:53.820] Build status: STARTED
[03:47:53.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:13:0) after 7m37s
[03:55:30.845] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:13:0)
[03:55:30.862] Build status: STARTED
[03:55:30.862] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:14:0) after 1m46s
[03:57:16.878] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:14:0)
[03:57:16.896] Build status: STARTED
[03:57:16.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:15:0) after 9m49s
[04:07:06.009] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:15:0)
[04:07:06.032] Build status: STARTED
[04:07:06.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:16:0) after 2m10s
[04:09:16.093] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:16:0)
[04:09:16.112] Build status: STARTED
[04:09:16.112] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:17:0) after 7m4s
[04:16:20.092] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:17:0)
[04:16:20.110] Build status: STARTED
[04:16:20.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:18:0) after 2m24s
[04:18:44.227] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:18:0)
[04:18:44.251] Build status: STARTED
[04:18:44.251] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:19:0) after 5m13s
[04:23:57.320] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:19:0)
[04:23:57.339] Build status: STARTED
[04:23:57.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:20:0) after 7m59s
[04:31:56.378] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:20:0)
[04:31:56.400] Build status: STARTED
[04:31:56.401] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:21:0) after 7m14s
[04:39:10.448] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:21:0)
[04:39:10.463] Build status: STARTED
[04:39:10.463] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:22:0) after 8m34s
[04:47:44.502] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:22:0)
[04:47:44.539] Build status: STARTED
[04:47:44.539] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:23:0) after 1m52s
[04:49:36.546] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:23:0)
[04:49:36.560] Build status: STARTED
[04:49:36.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:24:0) after 5m28s
[04:55:04.583] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:24:0)
[04:55:04.632] Build status: STARTED
[04:55:04.632] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:25:0) after 2m24s
[04:57:28.652] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:25:0)
[04:57:28.678] Build status: STARTED
[04:57:28.678] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:26:0) after 5m48s
[05:03:16.739] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:26:0)
[05:03:16.764] Build status: STARTED
[05:03:16.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:27:0) after 4m15s
[05:07:31.836] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:27:0)
[05:07:31.853] Build status: STARTED
[05:07:31.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:28:0) after 6m48s
[05:14:19.877] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:28:0)
[05:14:19.899] Build status: STARTED
[05:14:19.899] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:29:0) after 5m40s
[05:19:59.917] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:29:0)
[05:19:59.937] Build status: STARTED
[05:19:59.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:30:0) after 3m6s
[05:23:05.979] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:30:0)
[05:23:05.999] Build status: STARTED
[05:23:05.999] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:31:0) after 1m20s
[05:24:26.119] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:31:0)
[05:24:26.186] Build status: STARTED
[05:24:26.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:32:0) after 9m17s
[05:33:43.207] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:32:0)
[05:33:43.224] Build status: STARTED
[05:33:43.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:33:0) after 6m2s
[05:39:45.243] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:33:0)
[05:39:45.258] Build status: STARTED
[05:39:45.258] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:34:0) after 5m7s
[05:44:52.340] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:34:0)
[05:44:52.428] Build status: STARTED
[05:44:52.428] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:35:0) after 4m56s
[05:49:48.450] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:35:0)
[05:49:48.463] Build status: STARTED
[05:49:48.463] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:36:0) after 6m37s
[05:56:25.480] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:36:0)
[05:56:25.519] Build status: STARTED
[05:56:25.519] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:37:0) after 6m45s
[06:03:10.621] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:37:0)
[06:03:10.635] Build status: STARTED
[06:03:10.635] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:38:0) after 7m51s
[06:11:01.731] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:38:0)
[06:11:01.751] Build status: STARTED
[06:11:01.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:39:0) after 7m26s
[06:18:27.791] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:39:0)
[06:18:27.815] Build status: STARTED
[06:18:27.815] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:40:0) after 3m54s
[06:22:21.834] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:40:0)
[06:22:21.876] Build status: STARTED
[06:22:21.876] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:41:0) after 3m0s
[06:25:21.868] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:41:0)
[06:25:21.887] Build status: STARTED
[06:25:21.887] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:42:0) after 3m55s
[06:29:16.904] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:42:0)
[06:29:16.929] Build status: STARTED
[06:29:16.929] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:43:0) after 6m9s
[06:35:26.012] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:43:0)
[06:35:26.027] Build status: STARTED
[06:35:26.027] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:44:0) after 2m5s
[06:37:31.094] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:44:0)
[06:37:31.157] Build status: STARTED
[06:37:31.157] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8851187835220885568:45:0) after 7m1s
[06:39:41.626] Received PubSub notification, asking Buildbucket for the build status
[06:39:41.644] Build:
{
"id": "8685757772919173569",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 41167,
"createdBy": "project:chromium",
"createTime": "2026-04-01T03:08:35.649747913Z",
"startTime": "2026-04-01T03:09:21.725457495Z",
"endTime": "2026-04-01T06:39:41.374484993Z",
"updateTime": "2026-04-01T06:39:41.374484993Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cf3c8441134331148737ba36d8a0bea29b707509",
"ref": "refs/heads/main"
}
}
}
[06:39:41.644] Invocation finished in 3h31m7.573999661s with status FAILED