[04:00:07.449] New invocation is queued and will start shortly
[04:00:08.842] Starting the invocation (attempt 1)
[04:00:08.868] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/2466e9f5a58613ed4b17415fa68e6b09d4339595"
[04:00:08.868] Popped gitiles commit info from properties and tags
[04:00:08.868] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:00:08.869] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:00:08.869] Buildbucket request:
{
"requestId": "8865951897111675648",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "fuchsia-code-coverage"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2466e9f5a58613ed4b17415fa68e6b09d4339595",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8865951897111675648"
},
{
"key": "scheduler_job_id",
"value": "chromium/fuchsia-code-coverage"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:00:09.323] Scheduled build:
{
"id": "8700521834913172609",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "fuchsia-code-coverage"
},
"number": 3500,
"createdBy": "project:chromium",
"createTime": "2025-10-20T04:00:08.919717672Z",
"updateTime": "2025-10-20T04:00:08.919717672Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2466e9f5a58613ed4b17415fa68e6b09d4339595",
"ref": "refs/heads/main"
}
}
}
[04:00:09.323] Task URL: https://cr-buildbucket.appspot.com/build/8700521834913172609
[04:00:09.323] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:2:0) after 9m42s
[04:00:12.880] Received PubSub notification, asking Buildbucket for the build status
[04:00:12.916] Build status: SCHEDULED
[04:00:33.029] Received PubSub notification, asking Buildbucket for the build status
[04:00:33.044] Build status: STARTED
[04:09:51.292] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:2:0)
[04:09:51.320] Build status: STARTED
[04:09:51.320] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:5:0) after 7m4s
[04:16:55.481] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:5:0)
[04:16:55.495] Build status: STARTED
[04:16:55.496] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:6:0) after 4m18s
[04:21:13.474] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:6:0)
[04:21:13.505] Build status: STARTED
[04:21:13.505] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:7:0) after 3m46s
[04:24:59.661] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:7:0)
[04:24:59.783] Build status: STARTED
[04:24:59.783] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:8:0) after 3m44s
[04:28:43.798] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:8:0)
[04:28:43.817] Build status: STARTED
[04:28:43.817] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:9:0) after 6m51s
[04:35:34.831] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:9:0)
[04:35:34.919] Build status: STARTED
[04:35:34.919] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:10:0) after 4m12s
[04:39:46.983] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:10:0)
[04:39:46.997] Build status: STARTED
[04:39:46.997] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:11:0) after 9m42s
[04:49:29.020] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:11:0)
[04:49:29.039] Build status: STARTED
[04:49:29.039] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:12:0) after 6m36s
[04:56:06.125] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:12:0)
[04:56:06.143] Build status: STARTED
[04:56:06.143] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:13:0) after 6m58s
[05:03:04.186] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:13:0)
[05:03:04.208] Build status: STARTED
[05:03:04.208] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:14:0) after 1m49s
[05:04:53.231] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:14:0)
[05:04:53.249] Build status: STARTED
[05:04:53.249] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:15:0) after 4m43s
[05:09:36.531] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:15:0)
[05:09:36.589] Build status: STARTED
[05:09:36.589] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:16:0) after 6m45s
[05:16:22.167] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:16:0)
[05:16:22.182] Build status: STARTED
[05:16:22.182] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:17:0) after 9m12s
[05:25:34.268] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:17:0)
[05:25:34.283] Build status: STARTED
[05:25:34.283] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:18:0) after 3m26s
[05:29:00.295] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:18:0)
[05:29:00.311] Build status: STARTED
[05:29:00.311] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:19:0) after 9m41s
[05:38:42.177] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:19:0)
[05:38:42.230] Build status: STARTED
[05:38:42.230] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:20:0) after 5m38s
[05:44:20.246] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:20:0)
[05:44:20.318] Build status: STARTED
[05:44:20.318] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:21:0) after 2m27s
[05:46:47.395] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:21:0)
[05:46:47.414] Build status: STARTED
[05:46:47.414] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:22:0) after 6m53s
[05:53:40.443] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:22:0)
[05:53:40.486] Build status: STARTED
[05:53:40.486] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:23:0) after 8m1s
[06:01:41.588] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:23:0)
[06:01:41.639] Build status: STARTED
[06:01:41.639] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:24:0) after 6m17s
[06:07:58.753] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:24:0)
[06:07:58.771] Build status: STARTED
[06:07:58.771] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:25:0) after 9m27s
[06:17:25.852] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:25:0)
[06:17:25.864] Build status: STARTED
[06:17:25.864] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:26:0) after 8m51s
[06:26:17.489] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:26:0)
[06:26:17.507] Build status: STARTED
[06:26:17.507] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:27:0) after 1m38s
[06:27:55.565] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:27:0)
[06:27:55.580] Build status: STARTED
[06:27:55.580] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:28:0) after 7m21s
[06:35:16.766] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:28:0)
[06:35:16.806] Build status: STARTED
[06:35:16.806] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:29:0) after 6m55s
[06:42:11.824] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:29:0)
[06:42:11.844] Build status: STARTED
[06:42:11.844] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:30:0) after 5m54s
[06:48:06.073] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:30:0)
[06:48:06.086] Build status: STARTED
[06:48:06.086] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:31:0) after 9m17s
[06:57:23.248] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:31:0)
[06:57:23.263] Build status: STARTED
[06:57:23.263] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:32:0) after 7m44s
[07:05:07.459] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:32:0)
[07:05:07.488] Build status: STARTED
[07:05:07.488] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:33:0) after 5m23s
[07:10:30.665] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:33:0)
[07:10:30.684] Build status: STARTED
[07:10:30.684] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:34:0) after 1m24s
[07:11:54.890] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:34:0)
[07:11:54.904] Build status: STARTED
[07:11:54.904] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:35:0) after 2m42s
[07:14:36.916] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:35:0)
[07:14:36.955] Build status: STARTED
[07:14:36.955] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:36:0) after 3m49s
[07:18:26.175] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:36:0)
[07:18:26.190] Build status: STARTED
[07:18:26.190] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:37:0) after 3m4s
[07:21:30.211] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:37:0)
[07:21:30.237] Build status: STARTED
[07:21:30.237] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:38:0) after 6m8s
[07:27:38.351] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:38:0)
[07:27:38.371] Build status: STARTED
[07:27:38.371] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:39:0) after 6m56s
[07:34:34.744] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:39:0)
[07:34:34.759] Build status: STARTED
[07:34:34.759] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:40:0) after 5m12s
[07:39:47.301] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:40:0)
[07:39:47.350] Build status: STARTED
[07:39:47.350] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:41:0) after 1m58s
[07:41:45.611] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:41:0)
[07:41:45.626] Build status: STARTED
[07:41:45.626] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:42:0) after 6m22s
[07:48:07.743] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:42:0)
[07:48:07.761] Build status: STARTED
[07:48:07.761] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:43:0) after 9m53s
[07:58:00.780] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:43:0)
[07:58:00.796] Build status: STARTED
[07:58:00.796] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:44:0) after 2m37s
[08:00:37.813] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:44:0)
[08:00:37.830] Build status: STARTED
[08:00:37.830] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:45:0) after 9m3s
[08:09:40.845] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:45:0)
[08:09:40.858] Build status: STARTED
[08:09:40.858] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:46:0) after 7m12s
[08:16:53.020] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:46:0)
[08:16:53.034] Build status: STARTED
[08:16:53.034] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:47:0) after 6m58s
[08:23:51.085] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:47:0)
[08:23:51.097] Build status: STARTED
[08:23:51.097] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:48:0) after 4m41s
[08:28:32.110] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:48:0)
[08:28:32.149] Build status: STARTED
[08:28:32.149] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:49:0) after 6m43s
[08:35:15.213] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:49:0)
[08:35:15.230] Build status: STARTED
[08:35:15.230] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:50:0) after 8m8s
[08:43:23.331] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:50:0)
[08:43:23.350] Build status: STARTED
[08:43:23.350] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:51:0) after 3m12s
[08:46:35.394] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:51:0)
[08:46:35.413] Build status: STARTED
[08:46:35.413] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:52:0) after 3m21s
[08:49:56.530] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:52:0)
[08:49:56.546] Build status: STARTED
[08:49:56.546] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:53:0) after 2m51s
[08:52:47.567] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:53:0)
[08:52:47.581] Build status: STARTED
[08:52:47.581] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:54:0) after 4m12s
[08:56:59.847] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:54:0)
[08:56:59.861] Build status: STARTED
[08:56:59.861] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:55:0) after 4m11s
[09:01:10.885] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:55:0)
[09:01:10.954] Build status: STARTED
[09:01:10.954] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:56:0) after 1m46s
[09:02:56.970] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:56:0)
[09:02:56.987] Build status: STARTED
[09:02:56.987] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:57:0) after 5m48s
[09:08:45.230] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:57:0)
[09:08:45.248] Build status: STARTED
[09:08:45.248] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:58:0) after 8m17s
[09:17:02.378] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:58:0)
[09:17:02.395] Build status: STARTED
[09:17:02.395] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:59:0) after 8m36s
[09:25:38.417] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:59:0)
[09:25:38.501] Build status: STARTED
[09:25:38.501] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:60:0) after 4m12s
[09:29:50.518] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:60:0)
[09:29:50.549] Build status: STARTED
[09:29:50.549] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:61:0) after 2m51s
[09:32:41.674] Handling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:61:0)
[09:32:42.029] Build status: STARTED
[09:32:42.029] Scheduling timer "check-buildbucket-build-status" (chromium/fuchsia-code-coverage:8865951897111675648:62:0) after 2m23s
[09:35:03.734] Received PubSub notification, asking Buildbucket for the build status
[09:35:03.806] Build:
{
"id": "8700521834913172609",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "fuchsia-code-coverage"
},
"number": 3500,
"createdBy": "project:chromium",
"createTime": "2025-10-20T04:00:08.919717672Z",
"startTime": "2025-10-20T04:00:29.303058052Z",
"endTime": "2025-10-20T09:35:03.244103764Z",
"updateTime": "2025-10-20T09:35:03.244103764Z",
"status": "INFRA_FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2466e9f5a58613ed4b17415fa68e6b09d4339595",
"ref": "refs/heads/main"
}
}
}
[09:35:03.806] Invocation finished in 5h34m56.36670714s with status FAILED