[05:31:30.432] New invocation is queued and will start shortly
[05:31:31.530] Starting the invocation (attempt 1)
[05:31:31.556] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/80f9d80dec286fc35c4d4ca73278b906f556da64"
[05:31:31.556] Popped gitiles commit info from properties and tags
[05:31:31.556] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[05:31:31.557] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[05:31:31.557] Buildbucket request:
{
"requestId": "8851813021022015104",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "80f9d80dec286fc35c4d4ca73278b906f556da64",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851813021022015104"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMac"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[05:31:32.103] Scheduled build:
{
"id": "8686382959137522817",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 40038,
"createdBy": "project:chromium",
"createTime": "2026-03-25T05:31:31.609677557Z",
"updateTime": "2026-03-25T05:31:31.609677557Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "80f9d80dec286fc35c4d4ca73278b906f556da64",
"ref": "refs/heads/main"
}
}
}
[05:31:32.103] Task URL: https://cr-buildbucket.appspot.com/build/8686382959137522817
[05:31:32.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:2:0) after 7m35s
[05:31:33.040] Received PubSub notification, asking Buildbucket for the build status
[05:31:33.070] Build status: SCHEDULED
[05:32:17.537] Received PubSub notification, asking Buildbucket for the build status
[05:32:17.604] Build status: STARTED
[05:39:07.182] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:2:0)
[05:39:07.210] Build status: STARTED
[05:39:07.210] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:5:0) after 9m45s
[05:48:52.278] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:5:0)
[05:48:52.297] Build status: STARTED
[05:48:52.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:6:0) after 8m8s
[05:57:00.329] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:6:0)
[05:57:00.364] Build status: STARTED
[05:57:00.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:7:0) after 6m21s
[06:03:21.388] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:7:0)
[06:03:21.406] Build status: STARTED
[06:03:21.407] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:8:0) after 3m8s
[06:06:29.427] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:8:0)
[06:06:29.442] Build status: STARTED
[06:06:29.443] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:9:0) after 9m1s
[06:15:30.478] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:9:0)
[06:15:30.549] Build status: STARTED
[06:15:30.549] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:10:0) after 9m31s
[06:25:01.559] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:10:0)
[06:25:01.587] Build status: STARTED
[06:25:01.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:11:0) after 6m7s
[06:31:08.608] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:11:0)
[06:31:08.792] Build status: STARTED
[06:31:08.792] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:12:0) after 7m21s
[06:38:29.820] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:12:0)
[06:38:29.838] Build status: STARTED
[06:38:29.838] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:13:0) after 2m22s
[06:40:51.858] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:13:0)
[06:40:51.878] Build status: STARTED
[06:40:51.878] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:14:0) after 5m44s
[06:46:35.908] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:14:0)
[06:46:35.928] Build status: STARTED
[06:46:35.928] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:15:0) after 8m19s
[06:54:54.957] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:15:0)
[06:54:54.969] Build status: STARTED
[06:54:54.969] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:16:0) after 2m57s
[06:57:52.077] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:16:0)
[06:57:52.096] Build status: STARTED
[06:57:52.096] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:17:0) after 2m8s
[07:00:00.171] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:17:0)
[07:00:00.243] Build status: STARTED
[07:00:00.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:18:0) after 1m29s
[07:01:29.290] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:18:0)
[07:01:29.322] Build status: STARTED
[07:01:29.322] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:19:0) after 8m3s
[07:09:32.342] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:19:0)
[07:09:32.357] Build status: STARTED
[07:09:32.357] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:20:0) after 7m56s
[07:17:28.381] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:20:0)
[07:17:28.397] Build status: STARTED
[07:17:28.398] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:21:0) after 3m4s
[07:20:32.373] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:21:0)
[07:20:32.463] Build status: STARTED
[07:20:32.463] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:22:0) after 1m44s
[07:22:16.445] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:22:0)
[07:22:16.554] Build status: STARTED
[07:22:16.554] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:23:0) after 5m40s
[07:27:56.693] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:23:0)
[07:27:56.711] Build status: STARTED
[07:27:56.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:24:0) after 2m7s
[07:30:03.832] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:24:0)
[07:30:03.852] Build status: STARTED
[07:30:03.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:25:0) after 8m49s
[07:38:53.262] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:25:0)
[07:38:53.286] Build status: STARTED
[07:38:53.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:26:0) after 7m1s
[07:45:54.310] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:26:0)
[07:45:54.389] Build status: STARTED
[07:45:54.389] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:27:0) after 2m23s
[07:48:17.498] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:27:0)
[07:48:17.513] Build status: STARTED
[07:48:17.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:28:0) after 8m51s
[07:57:08.516] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:28:0)
[07:57:08.542] Build status: STARTED
[07:57:08.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:29:0) after 1m5s
[07:58:13.606] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:29:0)
[07:58:13.627] Build status: STARTED
[07:58:13.627] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:30:0) after 9m9s
[08:07:22.629] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:30:0)
[08:07:22.644] Build status: STARTED
[08:07:22.644] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:31:0) after 2m45s
[08:10:07.745] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:31:0)
[08:10:07.769] Build status: STARTED
[08:10:07.769] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:32:0) after 1m57s
[08:12:04.806] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:32:0)
[08:12:04.823] Build status: STARTED
[08:12:04.823] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:33:0) after 2m58s
[08:15:02.851] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:33:0)
[08:15:02.875] Build status: STARTED
[08:15:02.875] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:34:0) after 1m7s
[08:16:09.945] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:34:0)
[08:16:09.960] Build status: STARTED
[08:16:09.960] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:35:0) after 3m47s
[08:19:56.977] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:35:0)
[08:19:56.993] Build status: STARTED
[08:19:56.993] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:36:0) after 8m42s
[08:28:39.054] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:36:0)
[08:28:39.088] Build status: STARTED
[08:28:39.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:37:0) after 9m21s
[08:38:00.134] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:37:0)
[08:38:00.152] Build status: STARTED
[08:38:00.152] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:38:0) after 2m3s
[08:40:03.242] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:38:0)
[08:40:03.256] Build status: STARTED
[08:40:03.256] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:39:0) after 9m51s
[08:49:54.278] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:39:0)
[08:49:54.294] Build status: STARTED
[08:49:54.294] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:40:0) after 2m41s
[08:52:35.517] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:40:0)
[08:52:35.542] Build status: STARTED
[08:52:35.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:41:0) after 1m13s
[08:53:48.586] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:41:0)
[08:53:48.619] Build status: STARTED
[08:53:48.619] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:42:0) after 5m34s
[08:59:22.645] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:42:0)
[08:59:22.662] Build status: STARTED
[08:59:22.662] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:43:0) after 7m7s
[09:06:29.683] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:43:0)
[09:06:29.702] Build status: STARTED
[09:06:29.702] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:44:0) after 6m18s
[09:12:47.732] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:44:0)
[09:12:47.746] Build status: STARTED
[09:12:47.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:45:0) after 7m47s
[09:20:34.732] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:45:0)
[09:20:34.754] Build status: STARTED
[09:20:34.754] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:46:0) after 4m51s
[09:25:25.776] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:46:0)
[09:25:25.797] Build status: STARTED
[09:25:25.797] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:47:0) after 5m32s
[09:30:57.824] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:47:0)
[09:30:57.912] Build status: STARTED
[09:30:57.912] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:48:0) after 5m36s
[09:36:33.950] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:48:0)
[09:36:33.963] Build status: STARTED
[09:36:33.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:49:0) after 7m49s
[09:44:23.082] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:49:0)
[09:44:23.097] Build status: STARTED
[09:44:23.098] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:50:0) after 3m29s
[09:47:52.120] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:50:0)
[09:47:52.150] Build status: STARTED
[09:47:52.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:51:0) after 7m44s
[09:55:36.174] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:51:0)
[09:55:36.232] Build status: STARTED
[09:55:36.233] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:52:0) after 8m49s
[10:04:25.311] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:52:0)
[10:04:25.328] Build status: STARTED
[10:04:25.328] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:53:0) after 1m28s
[10:05:53.346] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:53:0)
[10:05:53.364] Build status: STARTED
[10:05:53.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:54:0) after 3m7s
[10:09:00.382] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:54:0)
[10:09:00.411] Build status: STARTED
[10:09:00.411] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:55:0) after 6m34s
[10:15:34.483] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:55:0)
[10:15:34.499] Build status: STARTED
[10:15:34.499] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:56:0) after 6m18s
[10:21:52.546] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:56:0)
[10:21:52.563] Build status: STARTED
[10:21:52.563] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:57:0) after 5m25s
[10:27:17.641] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:57:0)
[10:27:17.664] Build status: STARTED
[10:27:17.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:58:0) after 1m12s
[10:28:29.689] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:58:0)
[10:28:29.712] Build status: STARTED
[10:28:29.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:59:0) after 7m18s
[10:35:47.890] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:59:0)
[10:35:47.909] Build status: STARTED
[10:35:47.909] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:60:0) after 9m0s
[10:44:47.952] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:60:0)
[10:44:47.981] Build status: STARTED
[10:44:47.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:61:0) after 9m49s
[10:54:37.092] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:61:0)
[10:54:37.107] Build status: STARTED
[10:54:37.108] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:62:0) after 7m40s
[11:02:17.184] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:62:0)
[11:02:17.214] Build status: STARTED
[11:02:17.214] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:63:0) after 1m41s
[11:03:58.285] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:63:0)
[11:03:58.303] Build status: STARTED
[11:03:58.303] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:64:0) after 7m44s
[11:11:42.341] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:64:0)
[11:11:42.356] Build status: STARTED
[11:11:42.356] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:65:0) after 9m20s
[11:21:02.409] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:65:0)
[11:21:02.480] Build status: STARTED
[11:21:02.480] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:66:0) after 2m23s
[11:23:25.547] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:66:0)
[11:23:25.570] Build status: STARTED
[11:23:25.570] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:67:0) after 5m58s
[11:29:23.689] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:67:0)
[11:29:23.702] Build status: STARTED
[11:29:23.702] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851813021022015104:68:0) after 5m44s
[11:32:34.956] Received PubSub notification, asking Buildbucket for the build status
[11:32:34.976] Build:
{
"id": "8686382959137522817",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 40038,
"createdBy": "project:chromium",
"createTime": "2026-03-25T05:31:31.609677557Z",
"startTime": "2026-03-25T05:32:16.912432312Z",
"endTime": "2026-03-25T11:31:26.108462253Z",
"updateTime": "2026-03-25T11:31:26.108462253Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "80f9d80dec286fc35c4d4ca73278b906f556da64",
"ref": "refs/heads/main"
}
}
}
[11:32:34.976] Invocation finished in 6h1m4.547919658s with status FAILED