[08:20:56.355] New invocation is queued and will start shortly
[08:20:57.639] Starting the invocation (attempt 1)
[08:20:57.663] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0237f07066e0344a8107da05d5ac804ab77e1cf1"
[08:20:57.663] Popped gitiles commit info from properties and tags
[08:20:57.663] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[08:20:57.663] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[08:20:57.664] Buildbucket request:
{
"requestId": "8850805794657715760",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0237f07066e0344a8107da05d5ac804ab77e1cf1",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850805794657715760"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[08:20:58.144] Scheduled build:
{
"id": "8685375732569157537",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2396,
"createdBy": "project:chromium",
"createTime": "2026-04-05T08:20:57.718944022Z",
"updateTime": "2026-04-05T08:20:57.718944022Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0237f07066e0344a8107da05d5ac804ab77e1cf1",
"ref": "refs/heads/main"
}
}
}
[08:20:58.144] Task URL: https://cr-buildbucket.appspot.com/build/8685375732569157537
[08:20:58.144] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:2:0) after 1m41s
[08:20:58.613] Received PubSub notification, asking Buildbucket for the build status
[08:20:58.650] Build status: SCHEDULED
[08:21:37.942] Received PubSub notification, asking Buildbucket for the build status
[08:21:37.958] Build status: STARTED
[08:22:39.156] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:2:0)
[08:22:39.183] Build status: STARTED
[08:22:39.183] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:5:0) after 2m21s
[08:25:00.355] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:5:0)
[08:25:00.467] Build status: STARTED
[08:25:00.467] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:6:0) after 7m22s
[08:32:22.569] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:6:0)
[08:32:22.589] Build status: STARTED
[08:32:22.589] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:7:0) after 2m58s
[08:35:20.603] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:7:0)
[08:35:20.680] Build status: STARTED
[08:35:20.680] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:8:0) after 4m53s
[08:40:13.787] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:8:0)
[08:40:13.810] Build status: STARTED
[08:40:13.810] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:9:0) after 1m52s
[08:42:05.865] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:9:0)
[08:42:05.881] Build status: STARTED
[08:42:05.881] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:10:0) after 6m43s
[08:48:48.987] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:10:0)
[08:48:49.004] Build status: STARTED
[08:48:49.004] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:11:0) after 4m25s
[08:53:14.002] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:11:0)
[08:53:14.028] Build status: STARTED
[08:53:14.028] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:12:0) after 3m47s
[08:57:01.010] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:12:0)
[08:57:01.030] Build status: STARTED
[08:57:01.030] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:13:0) after 8m34s
[09:05:35.043] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:13:0)
[09:05:35.152] Build status: STARTED
[09:05:35.152] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:14:0) after 5m55s
[09:11:30.267] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:14:0)
[09:11:30.290] Build status: STARTED
[09:11:30.290] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:15:0) after 1m33s
[09:13:03.516] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:15:0)
[09:13:03.532] Build status: STARTED
[09:13:03.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:16:0) after 2m32s
[09:15:35.616] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:16:0)
[09:15:35.634] Build status: STARTED
[09:15:35.634] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:17:0) after 7m14s
[09:22:49.648] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:17:0)
[09:22:49.667] Build status: STARTED
[09:22:49.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:18:0) after 5m30s
[09:28:19.812] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:18:0)
[09:28:19.837] Build status: STARTED
[09:28:19.837] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:19:0) after 8m45s
[09:37:04.949] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:19:0)
[09:37:04.975] Build status: STARTED
[09:37:04.975] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:20:0) after 1m59s
[09:39:04.231] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:20:0)
[09:39:04.252] Build status: STARTED
[09:39:04.252] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:21:0) after 2m40s
[09:41:44.315] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:21:0)
[09:41:44.339] Build status: STARTED
[09:41:44.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:22:0) after 9m2s
[09:50:46.395] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:22:0)
[09:50:46.429] Build status: STARTED
[09:50:46.429] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:23:0) after 2m31s
[09:53:17.589] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:23:0)
[09:53:17.693] Build status: STARTED
[09:53:17.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:24:0) after 3m14s
[09:56:31.712] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:24:0)
[09:56:31.736] Build status: STARTED
[09:56:31.736] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:25:0) after 4m16s
[10:00:47.750] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:25:0)
[10:00:47.768] Build status: STARTED
[10:00:47.768] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:26:0) after 9m26s
[10:10:13.839] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:26:0)
[10:10:14.289] Build status: STARTED
[10:10:14.289] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:27:0) after 5m48s
[10:16:02.433] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:27:0)
[10:16:02.603] Build status: STARTED
[10:16:02.603] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:28:0) after 1m50s
[10:17:52.617] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:28:0)
[10:17:52.688] Build status: STARTED
[10:17:52.688] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:29:0) after 8m25s
[10:26:17.801] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:29:0)
[10:26:17.823] Build status: STARTED
[10:26:17.823] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:30:0) after 4m15s
[10:30:32.838] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:30:0)
[10:30:32.854] Build status: STARTED
[10:30:32.854] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:31:0) after 7m10s
[10:37:42.866] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:31:0)
[10:37:42.907] Build status: STARTED
[10:37:42.907] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:32:0) after 5m7s
[10:42:50.022] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:32:0)
[10:42:50.038] Build status: STARTED
[10:42:50.038] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:33:0) after 6m57s
[10:49:47.001] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:33:0)
[10:49:47.019] Build status: STARTED
[10:49:47.019] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:34:0) after 9m44s
[10:59:31.014] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:34:0)
[10:59:31.032] Build status: STARTED
[10:59:31.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:35:0) after 9m58s
[11:09:29.374] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:35:0)
[11:09:29.400] Build status: STARTED
[11:09:29.400] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:36:0) after 4m50s
[11:14:19.557] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:36:0)
[11:14:19.597] Build status: STARTED
[11:14:19.597] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:37:0) after 1m53s
[11:16:12.656] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:37:0)
[11:16:12.675] Build status: STARTED
[11:16:12.675] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:38:0) after 7m2s
[11:23:14.687] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:38:0)
[11:23:14.725] Build status: STARTED
[11:23:14.725] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:39:0) after 4m41s
[11:27:56.159] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:39:0)
[11:27:56.176] Build status: STARTED
[11:27:56.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:40:0) after 4m50s
[11:32:46.295] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:40:0)
[11:32:46.383] Build status: STARTED
[11:32:46.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:41:0) after 4m46s
[11:37:32.587] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:41:0)
[11:37:32.604] Build status: STARTED
[11:37:32.604] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:42:0) after 8m51s
[11:46:23.618] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:42:0)
[11:46:23.634] Build status: STARTED
[11:46:23.634] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:43:0) after 8m33s
[11:54:56.630] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:43:0)
[11:54:56.649] Build status: STARTED
[11:54:56.649] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:44:0) after 4m21s
[11:59:17.818] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:44:0)
[11:59:17.833] Build status: STARTED
[11:59:17.833] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:45:0) after 4m40s
[12:03:58.003] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:45:0)
[12:03:58.018] Build status: STARTED
[12:03:58.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:46:0) after 6m25s
[12:10:23.245] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:46:0)
[12:10:23.321] Build status: STARTED
[12:10:23.321] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:47:0) after 6m48s
[12:17:11.430] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:47:0)
[12:17:11.453] Build status: STARTED
[12:17:11.453] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:48:0) after 1m39s
[12:18:50.506] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:48:0)
[12:18:50.525] Build status: STARTED
[12:18:50.525] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:49:0) after 3m23s
[12:22:13.543] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:49:0)
[12:22:13.576] Build status: STARTED
[12:22:13.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:50:0) after 5m6s
[12:27:19.629] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:50:0)
[12:27:19.648] Build status: STARTED
[12:27:19.648] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:51:0) after 5m17s
[12:32:36.807] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:51:0)
[12:32:36.826] Build status: STARTED
[12:32:36.826] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:52:0) after 1m24s
[12:34:00.864] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:52:0)
[12:34:00.962] Build status: STARTED
[12:34:00.964] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:53:0) after 8m18s
[12:42:19.239] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:53:0)
[12:42:19.254] Build status: STARTED
[12:42:19.254] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:54:0) after 7m47s
[12:50:06.444] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:54:0)
[12:50:06.460] Build status: STARTED
[12:50:06.460] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:55:0) after 4m45s
[12:54:51.764] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:55:0)
[12:54:51.823] Build status: STARTED
[12:54:51.823] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:56:0) after 7m19s
[13:02:10.841] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:56:0)
[13:02:10.867] Build status: STARTED
[13:02:10.867] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8850805794657715760:57:0) after 7m7s
[13:03:42.737] Received PubSub notification, asking Buildbucket for the build status
[13:03:42.774] Build:
{
"id": "8685375732569157537",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2396,
"createdBy": "project:chromium",
"createTime": "2026-04-05T08:20:57.718944022Z",
"startTime": "2026-04-05T08:21:37.774796371Z",
"endTime": "2026-04-05T13:03:42.391106850Z",
"updateTime": "2026-04-05T13:03:42.391106850Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0237f07066e0344a8107da05d5ac804ab77e1cf1",
"ref": "refs/heads/main"
}
}
}
[13:03:42.774] Invocation finished in 4h42m46.430820154s with status FAILED