[21:54:07.970] New invocation is queued and will start shortly
[21:54:09.029] Starting the invocation (attempt 1)
[21:54:09.054] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/1d9b0d70a0f09526fcf4ca5d5fd0175924a9b248"
[21:54:09.054] Popped gitiles commit info from properties and tags
[21:54:09.054] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:54:09.054] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:54:09.054] Buildbucket request:
{
"requestId": "8850935826866189568",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1d9b0d70a0f09526fcf4ca5d5fd0175924a9b248",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850935826866189568"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMac"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:54:09.733] Scheduled build:
{
"id": "8685505765012587361",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 40124,
"createdBy": "project:chromium",
"createTime": "2026-04-03T21:54:09.117430011Z",
"updateTime": "2026-04-03T21:54:09.117430011Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1d9b0d70a0f09526fcf4ca5d5fd0175924a9b248",
"ref": "refs/heads/main"
}
}
}
[21:54:09.733] Task URL: https://cr-buildbucket.appspot.com/build/8685505765012587361
[21:54:09.733] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:2:0) after 6m29s
[21:54:10.629] Received PubSub notification, asking Buildbucket for the build status
[21:54:10.705] Build status: SCHEDULED
[21:54:54.520] Received PubSub notification, asking Buildbucket for the build status
[21:54:54.602] Build status: STARTED
[22:00:38.751] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:2:0)
[22:00:38.779] Build status: STARTED
[22:00:38.779] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:5:0) after 2m51s
[22:03:29.802] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:5:0)
[22:03:29.830] Build status: STARTED
[22:03:29.830] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:6:0) after 3m55s
[22:07:24.886] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:6:0)
[22:07:24.921] Build status: STARTED
[22:07:24.922] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:7:0) after 7m47s
[22:15:11.981] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:7:0)
[22:15:12.038] Build status: STARTED
[22:15:12.038] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:8:0) after 5m44s
[22:20:56.055] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:8:0)
[22:20:56.080] Build status: STARTED
[22:20:56.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:9:0) after 6m35s
[22:27:31.099] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:9:0)
[22:27:31.127] Build status: STARTED
[22:27:31.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:10:0) after 8m22s
[22:35:53.153] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:10:0)
[22:35:53.172] Build status: STARTED
[22:35:53.172] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:11:0) after 3m52s
[22:39:45.168] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:11:0)
[22:39:45.198] Build status: STARTED
[22:39:45.198] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:12:0) after 6m58s
[22:46:43.219] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:12:0)
[22:46:43.377] Build status: STARTED
[22:46:43.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:13:0) after 4m24s
[22:51:07.417] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:13:0)
[22:51:07.449] Build status: STARTED
[22:51:07.450] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:14:0) after 8m40s
[22:59:47.467] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:14:0)
[22:59:47.484] Build status: STARTED
[22:59:47.484] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:15:0) after 7m42s
[23:07:29.588] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:15:0)
[23:07:29.716] Build status: STARTED
[23:07:29.716] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:16:0) after 9m16s
[23:16:45.734] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:16:0)
[23:16:45.752] Build status: STARTED
[23:16:45.753] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:17:0) after 3m25s
[23:20:10.796] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:17:0)
[23:20:10.832] Build status: STARTED
[23:20:10.832] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:18:0) after 8m3s
[23:28:13.849] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:18:0)
[23:28:13.864] Build status: STARTED
[23:28:13.864] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:19:0) after 4m36s
[23:32:49.953] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:19:0)
[23:32:49.997] Build status: STARTED
[23:32:49.997] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:20:0) after 4m56s
[23:37:46.016] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:20:0)
[23:37:46.033] Build status: STARTED
[23:37:46.033] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:21:0) after 7m21s
[23:45:07.191] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:21:0)
[23:45:07.266] Build status: STARTED
[23:45:07.266] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:22:0) after 5m55s
[23:51:02.321] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:22:0)
[23:51:02.435] Build status: STARTED
[23:51:02.435] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:23:0) after 1m45s
[23:52:47.461] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:23:0)
[23:52:47.481] Build status: STARTED
[23:52:47.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:24:0) after 2m46s
[23:55:33.574] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:24:0)
[23:55:33.596] Build status: STARTED
[23:55:33.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:25:0) after 8m47s
[00:04:20.675] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:25:0)
[00:04:20.691] Build status: STARTED
[00:04:20.692] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:26:0) after 8m15s
[00:12:35.708] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:26:0)
[00:12:35.725] Build status: STARTED
[00:12:35.725] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:27:0) after 9m26s
[00:22:01.749] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:27:0)
[00:22:01.770] Build status: STARTED
[00:22:01.770] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:28:0) after 4m17s
[00:26:18.858] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:28:0)
[00:26:18.878] Build status: STARTED
[00:26:18.878] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:29:0) after 8m37s
[00:34:55.897] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:29:0)
[00:34:55.927] Build status: STARTED
[00:34:55.927] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:30:0) after 4m48s
[00:39:43.945] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:30:0)
[00:39:43.959] Build status: STARTED
[00:39:43.959] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:31:0) after 4m12s
[00:43:55.941] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:31:0)
[00:43:56.024] Build status: STARTED
[00:43:56.024] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:32:0) after 1m19s
[00:45:15.041] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:32:0)
[00:45:15.076] Build status: STARTED
[00:45:15.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:33:0) after 4m9s
[00:49:24.115] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:33:0)
[00:49:24.203] Build status: STARTED
[00:49:24.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:34:0) after 5m40s
[00:55:04.247] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:34:0)
[00:55:04.265] Build status: STARTED
[00:55:04.265] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:35:0) after 3m28s
[00:58:32.286] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:35:0)
[00:58:32.311] Build status: STARTED
[00:58:32.311] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:36:0) after 4m21s
[01:02:53.390] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:36:0)
[01:02:53.422] Build status: STARTED
[01:02:53.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:37:0) after 3m3s
[01:05:56.456] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:37:0)
[01:05:56.480] Build status: STARTED
[01:05:56.480] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:38:0) after 8m8s
[01:14:04.541] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:38:0)
[01:14:04.560] Build status: STARTED
[01:14:04.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:39:0) after 9m26s
[01:23:30.597] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:39:0)
[01:23:30.615] Build status: STARTED
[01:23:30.615] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:40:0) after 7m42s
[01:31:12.666] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:40:0)
[01:31:12.685] Build status: STARTED
[01:31:12.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:41:0) after 5m31s
[01:36:43.754] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:41:0)
[01:36:43.771] Build status: STARTED
[01:36:43.771] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:42:0) after 3m4s
[01:39:47.845] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:42:0)
[01:39:47.946] Build status: STARTED
[01:39:47.946] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:43:0) after 6m19s
[01:46:06.968] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:43:0)
[01:46:06.992] Build status: STARTED
[01:46:06.992] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:44:0) after 2m22s
[01:48:29.058] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:44:0)
[01:48:29.076] Build status: STARTED
[01:48:29.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:45:0) after 8m4s
[01:56:33.474] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:45:0)
[01:56:33.494] Build status: STARTED
[01:56:33.494] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:46:0) after 6m43s
[02:03:16.514] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:46:0)
[02:03:16.546] Build status: STARTED
[02:03:16.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:47:0) after 3m9s
[02:06:25.711] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:47:0)
[02:06:25.727] Build status: STARTED
[02:06:25.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:48:0) after 1m1s
[02:07:26.755] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:48:0)
[02:07:26.774] Build status: STARTED
[02:07:26.774] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:49:0) after 5m16s
[02:12:42.789] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:49:0)
[02:12:42.813] Build status: STARTED
[02:12:42.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:50:0) after 9m11s
[02:21:53.910] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:50:0)
[02:21:53.928] Build status: STARTED
[02:21:53.928] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:51:0) after 8m46s
[02:30:39.967] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:51:0)
[02:30:39.995] Build status: STARTED
[02:30:39.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:52:0) after 7m29s
[02:38:09.356] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:52:0)
[02:38:09.381] Build status: STARTED
[02:38:09.381] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:53:0) after 9m54s
[02:48:03.417] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:53:0)
[02:48:03.482] Build status: STARTED
[02:48:03.482] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:54:0) after 1m4s
[02:49:07.548] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:54:0)
[02:49:07.562] Build status: STARTED
[02:49:07.562] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:55:0) after 5m47s
[02:54:54.581] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:55:0)
[02:54:54.595] Build status: STARTED
[02:54:54.595] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:56:0) after 4m23s
[02:59:17.618] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:56:0)
[02:59:17.651] Build status: STARTED
[02:59:17.651] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:57:0) after 2m37s
[03:01:54.745] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:57:0)
[03:01:54.766] Build status: STARTED
[03:01:54.766] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:58:0) after 6m54s
[03:08:48.781] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:58:0)
[03:08:48.798] Build status: STARTED
[03:08:48.798] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:59:0) after 4m6s
[03:12:54.814] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:59:0)
[03:12:54.830] Build status: STARTED
[03:12:54.830] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:60:0) after 4m28s
[03:17:23.124] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:60:0)
[03:17:23.166] Build status: STARTED
[03:17:23.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:61:0) after 5m23s
[03:22:46.552] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:61:0)
[03:22:46.566] Build status: STARTED
[03:22:46.567] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:62:0) after 5m32s
[03:28:18.586] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:62:0)
[03:28:18.601] Build status: STARTED
[03:28:18.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:63:0) after 4m37s
[03:32:55.620] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:63:0)
[03:32:55.650] Build status: STARTED
[03:32:55.650] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:64:0) after 3m30s
[03:36:25.754] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:64:0)
[03:36:25.768] Build status: STARTED
[03:36:25.768] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:65:0) after 3m25s
[03:39:50.791] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:65:0)
[03:39:50.808] Build status: STARTED
[03:39:50.808] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:66:0) after 1m7s
[03:40:57.831] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:66:0)
[03:40:57.882] Build status: STARTED
[03:40:57.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:67:0) after 7m27s
[03:48:25.441] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:67:0)
[03:48:25.456] Build status: STARTED
[03:48:25.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:68:0) after 4m44s
[03:53:09.611] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:68:0)
[03:53:09.631] Build status: STARTED
[03:53:09.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:69:0) after 3m38s
[03:56:47.895] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:69:0)
[03:56:47.910] Build status: STARTED
[03:56:47.910] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850935826866189568:70:0) after 3m34s
[03:59:18.005] Received PubSub notification, asking Buildbucket for the build status
[03:59:18.053] Build:
{
"id": "8685505765012587361",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 40124,
"createdBy": "project:chromium",
"createTime": "2026-04-03T21:54:09.117430011Z",
"startTime": "2026-04-03T21:54:54.343417476Z",
"endTime": "2026-04-04T03:59:16.544059044Z",
"updateTime": "2026-04-04T03:59:16.544059044Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1d9b0d70a0f09526fcf4ca5d5fd0175924a9b248",
"ref": "refs/heads/main"
}
}
}
[03:59:18.053] Invocation finished in 6h5m10.087243492s with status FAILED