[22:32:27.111] New invocation is queued and will start shortly
[22:32:28.753] Starting the invocation (attempt 1)
[22:32:28.787] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0a61ac6913880631bb23a4798381363d1aa2a8b2"
[22:32:28.788] Popped gitiles commit info from properties and tags
[22:32:28.788] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[22:32:28.789] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[22:32:28.789] Buildbucket request:
{
"requestId": "8843051479965985024",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0a61ac6913880631bb23a4798381363d1aa2a8b2",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8843051479965985024"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMac"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[22:32:29.324] Scheduled build:
{
"id": "8677621417467164321",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 41543,
"createdBy": "project:chromium",
"createTime": "2026-06-29T22:32:28.872307752Z",
"updateTime": "2026-06-29T22:32:28.872307752Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0a61ac6913880631bb23a4798381363d1aa2a8b2",
"ref": "refs/heads/main"
}
}
}
[22:32:29.324] Task URL: https://cr-buildbucket.appspot.com/build/8677621417467164321
[22:32:29.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:2:0) after 7m31s
[22:32:30.296] Received PubSub notification, asking Buildbucket for the build status
[22:32:30.309] Build status: SCHEDULED
[22:33:10.368] Received PubSub notification, asking Buildbucket for the build status
[22:33:10.385] Build status: STARTED
[22:40:00.461] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:2:0)
[22:40:00.489] Build status: STARTED
[22:40:00.489] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:5:0) after 2m4s
[22:42:04.518] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:5:0)
[22:42:04.537] Build status: STARTED
[22:42:04.537] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:6:0) after 7m55s
[22:49:59.587] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:6:0)
[22:49:59.612] Build status: STARTED
[22:49:59.612] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:7:0) after 9m2s
[22:59:01.721] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:7:0)
[22:59:01.742] Build status: STARTED
[22:59:01.742] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:8:0) after 6m1s
[23:05:02.824] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:8:0)
[23:05:02.842] Build status: STARTED
[23:05:02.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:9:0) after 3m5s
[23:08:07.867] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:9:0)
[23:08:07.897] Build status: STARTED
[23:08:07.897] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:10:0) after 1m46s
[23:09:54.133] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:10:0)
[23:09:54.162] Build status: STARTED
[23:09:54.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:11:0) after 3m52s
[23:13:46.229] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:11:0)
[23:13:46.255] Build status: STARTED
[23:13:46.255] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:12:0) after 1m2s
[23:14:48.420] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:12:0)
[23:14:48.450] Build status: STARTED
[23:14:48.450] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:13:0) after 7m16s
[23:22:04.469] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:13:0)
[23:22:04.487] Build status: STARTED
[23:22:04.487] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:14:0) after 2m16s
[23:24:20.527] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:14:0)
[23:24:20.555] Build status: STARTED
[23:24:20.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:15:0) after 2m13s
[23:26:33.596] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:15:0)
[23:26:33.612] Build status: STARTED
[23:26:33.612] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:16:0) after 4m34s
[23:31:07.689] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:16:0)
[23:31:07.705] Build status: STARTED
[23:31:07.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:17:0) after 9m15s
[23:40:22.908] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:17:0)
[23:40:22.945] Build status: STARTED
[23:40:22.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:18:0) after 5m24s
[23:45:46.945] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:18:0)
[23:45:47.012] Build status: STARTED
[23:45:47.013] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:19:0) after 7m51s
[23:53:38.133] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:19:0)
[23:53:38.152] Build status: STARTED
[23:53:38.152] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:20:0) after 7m43s
[00:01:21.187] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:20:0)
[00:01:21.205] Build status: STARTED
[00:01:21.205] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:21:0) after 2m12s
[00:03:33.230] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:21:0)
[00:03:33.249] Build status: STARTED
[00:03:33.249] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:22:0) after 7m21s
[00:10:54.318] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:22:0)
[00:10:54.335] Build status: STARTED
[00:10:54.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:23:0) after 7m27s
[00:18:21.359] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:23:0)
[00:18:21.380] Build status: STARTED
[00:18:21.380] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:24:0) after 8m1s
[00:26:22.407] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:24:0)
[00:26:22.423] Build status: STARTED
[00:26:22.423] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:25:0) after 6m21s
[00:32:43.472] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:25:0)
[00:32:43.490] Build status: STARTED
[00:32:43.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:26:0) after 2m45s
[00:35:28.584] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:26:0)
[00:35:28.601] Build status: STARTED
[00:35:28.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:27:0) after 5m22s
[00:40:50.671] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:27:0)
[00:40:50.687] Build status: STARTED
[00:40:50.687] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:28:0) after 9m10s
[00:50:00.712] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:28:0)
[00:50:00.760] Build status: STARTED
[00:50:00.760] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:29:0) after 2m11s
[00:52:11.785] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:29:0)
[00:52:11.850] Build status: STARTED
[00:52:11.850] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:30:0) after 6m42s
[00:58:53.874] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:30:0)
[00:58:53.897] Build status: STARTED
[00:58:53.897] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:31:0) after 8m52s
[01:07:45.941] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:31:0)
[01:07:45.992] Build status: STARTED
[01:07:45.992] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:32:0) after 5m13s
[01:12:59.095] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:32:0)
[01:12:59.127] Build status: STARTED
[01:12:59.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:33:0) after 5m14s
[01:18:13.190] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:33:0)
[01:18:13.218] Build status: STARTED
[01:18:13.218] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:34:0) after 6m13s
[01:24:26.291] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:34:0)
[01:24:26.308] Build status: STARTED
[01:24:26.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:35:0) after 7m31s
[01:31:57.328] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:35:0)
[01:31:57.353] Build status: STARTED
[01:31:57.353] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:36:0) after 1m41s
[01:33:38.385] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:36:0)
[01:33:38.435] Build status: STARTED
[01:33:38.435] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:37:0) after 8m55s
[01:42:33.426] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:37:0)
[01:42:33.439] Build status: STARTED
[01:42:33.439] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:38:0) after 1m29s
[01:44:02.690] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:38:0)
[01:44:02.781] Build status: STARTED
[01:44:02.782] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:39:0) after 4m1s
[01:48:03.797] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:39:0)
[01:48:03.817] Build status: STARTED
[01:48:03.817] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:40:0) after 3m50s
[01:51:53.888] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:40:0)
[01:51:53.938] Build status: STARTED
[01:51:53.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:41:0) after 2m30s
[01:54:24.049] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:41:0)
[01:54:24.074] Build status: STARTED
[01:54:24.074] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:42:0) after 6m28s
[02:00:52.102] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:42:0)
[02:00:52.140] Build status: STARTED
[02:00:52.140] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:43:0) after 6m25s
[02:07:17.162] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:43:0)
[02:07:17.246] Build status: STARTED
[02:07:17.246] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:44:0) after 9m21s
[02:16:38.268] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:44:0)
[02:16:38.286] Build status: STARTED
[02:16:38.287] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:45:0) after 8m47s
[02:25:25.363] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:45:0)
[02:25:25.388] Build status: STARTED
[02:25:25.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:46:0) after 6m24s
[02:31:49.412] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:46:0)
[02:31:49.431] Build status: STARTED
[02:31:49.431] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:47:0) after 3m16s
[02:35:05.518] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:47:0)
[02:35:05.541] Build status: STARTED
[02:35:05.541] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:48:0) after 8m27s
[02:43:32.628] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:48:0)
[02:43:32.651] Build status: STARTED
[02:43:32.651] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:49:0) after 5m37s
[02:49:09.733] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:49:0)
[02:49:09.749] Build status: STARTED
[02:49:09.749] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:50:0) after 1m32s
[02:50:41.745] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:50:0)
[02:50:41.767] Build status: STARTED
[02:50:41.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:51:0) after 3m19s
[02:54:00.801] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:51:0)
[02:54:00.820] Build status: STARTED
[02:54:00.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:52:0) after 4m20s
[02:58:20.917] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:52:0)
[02:58:20.934] Build status: STARTED
[02:58:20.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:53:0) after 7m48s
[03:06:09.006] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:53:0)
[03:06:09.028] Build status: STARTED
[03:06:09.028] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:54:0) after 8m2s
[03:14:11.188] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:54:0)
[03:14:11.208] Build status: STARTED
[03:14:11.208] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:55:0) after 1m37s
[03:15:48.257] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:55:0)
[03:15:48.284] Build status: STARTED
[03:15:48.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:56:0) after 7m28s
[03:23:16.306] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:56:0)
[03:23:16.328] Build status: STARTED
[03:23:16.328] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:57:0) after 7m21s
[03:30:37.395] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:57:0)
[03:30:37.412] Build status: STARTED
[03:30:37.412] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:58:0) after 4m1s
[03:34:38.503] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:58:0)
[03:34:38.518] Build status: STARTED
[03:34:38.518] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:59:0) after 4m29s
[03:39:07.559] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:59:0)
[03:39:07.578] Build status: STARTED
[03:39:07.578] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8843051479965985024:60:0) after 9m8s
[03:47:48.947] Received PubSub notification, asking Buildbucket for the build status
[03:47:48.963] Build:
{
"id": "8677621417467164321",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 41543,
"createdBy": "project:chromium",
"createTime": "2026-06-29T22:32:28.872307752Z",
"startTime": "2026-06-29T22:33:08.972462921Z",
"endTime": "2026-06-30T03:47:40.824643811Z",
"updateTime": "2026-06-30T03:47:40.824643811Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0a61ac6913880631bb23a4798381363d1aa2a8b2",
"ref": "refs/heads/main"
}
}
}
[03:47:48.963] Invocation finished in 5h15m21.857674188s with status FAILED