[15:44:20.163] New invocation is queued and will start shortly
[15:44:21.173] Starting the invocation (attempt 1)
[15:44:21.198] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/ca0171bae24453d18a961674eed287a21b752b91"
[15:44:21.198] Popped gitiles commit info from properties and tags
[15:44:21.198] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:44:21.198] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:44:21.198] Buildbucket request:
{
"requestId": "8851412077299289008",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ca0171bae24453d18a961674eed287a21b752b91",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851412077299289008"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:44:21.741] Scheduled build:
{
"id": "8685982015504174241",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2362,
"createdBy": "project:chromium",
"createTime": "2026-03-29T15:44:21.255840957Z",
"updateTime": "2026-03-29T15:44:21.255840957Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ca0171bae24453d18a961674eed287a21b752b91",
"ref": "refs/heads/main"
}
}
}
[15:44:21.741] Task URL: https://cr-buildbucket.appspot.com/build/8685982015504174241
[15:44:21.741] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:2:0) after 1m19s
[15:44:23.198] Received PubSub notification, asking Buildbucket for the build status
[15:44:23.216] Build status: SCHEDULED
[15:45:12.254] Received PubSub notification, asking Buildbucket for the build status
[15:45:12.295] Build status: STARTED
[15:45:40.759] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:2:0)
[15:45:40.812] Build status: STARTED
[15:45:40.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:5:0) after 5m14s
[15:50:54.908] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:5:0)
[15:50:54.926] Build status: STARTED
[15:50:54.926] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:6:0) after 9m11s
[16:00:05.946] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:6:0)
[16:00:05.975] Build status: STARTED
[16:00:05.975] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:7:0) after 7m37s
[16:07:43.115] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:7:0)
[16:07:43.134] Build status: STARTED
[16:07:43.134] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:8:0) after 5m15s
[16:12:58.205] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:8:0)
[16:12:58.225] Build status: STARTED
[16:12:58.225] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:9:0) after 1m30s
[16:14:28.238] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:9:0)
[16:14:28.331] Build status: STARTED
[16:14:28.331] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:10:0) after 4m44s
[16:19:12.793] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:10:0)
[16:19:12.919] Build status: STARTED
[16:19:12.919] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:11:0) after 2m51s
[16:22:03.933] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:11:0)
[16:22:03.991] Build status: STARTED
[16:22:03.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:12:0) after 1m2s
[16:23:06.066] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:12:0)
[16:23:06.109] Build status: STARTED
[16:23:06.109] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:13:0) after 2m9s
[16:25:15.154] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:13:0)
[16:25:15.175] Build status: STARTED
[16:25:15.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:14:0) after 4m2s
[16:29:17.229] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:14:0)
[16:29:17.247] Build status: STARTED
[16:29:17.247] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:15:0) after 7m13s
[16:36:31.145] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:15:0)
[16:36:31.168] Build status: STARTED
[16:36:31.168] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:16:0) after 9m32s
[16:46:03.420] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:16:0)
[16:46:03.480] Build status: STARTED
[16:46:03.480] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:17:0) after 7m6s
[16:53:09.498] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:17:0)
[16:53:09.512] Build status: STARTED
[16:53:09.512] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:18:0) after 8m50s
[17:01:59.623] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:18:0)
[17:01:59.651] Build status: STARTED
[17:01:59.651] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:19:0) after 4m21s
[17:06:20.823] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:19:0)
[17:06:20.846] Build status: STARTED
[17:06:20.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:20:0) after 5m20s
[17:11:40.863] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:20:0)
[17:11:40.898] Build status: STARTED
[17:11:40.898] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:21:0) after 3m33s
[17:15:13.974] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:21:0)
[17:15:13.996] Build status: STARTED
[17:15:13.996] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:22:0) after 1m45s
[17:16:59.020] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:22:0)
[17:16:59.037] Build status: STARTED
[17:16:59.037] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:23:0) after 7m35s
[17:24:34.193] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:23:0)
[17:24:34.216] Build status: STARTED
[17:24:34.216] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:24:0) after 1m33s
[17:26:07.395] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:24:0)
[17:26:07.408] Build status: STARTED
[17:26:07.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:25:0) after 8m29s
[17:34:36.525] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:25:0)
[17:34:36.543] Build status: STARTED
[17:34:36.544] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:26:0) after 8m47s
[17:43:23.870] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:26:0)
[17:43:23.885] Build status: STARTED
[17:43:23.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:27:0) after 1m58s
[17:45:21.901] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:27:0)
[17:45:21.925] Build status: STARTED
[17:45:21.925] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:28:0) after 9m34s
[17:54:55.938] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:28:0)
[17:54:55.975] Build status: STARTED
[17:54:55.975] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:29:0) after 9m37s
[18:04:33.075] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:29:0)
[18:04:33.097] Build status: STARTED
[18:04:33.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:30:0) after 5m56s
[18:10:29.113] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:30:0)
[18:10:29.136] Build status: STARTED
[18:10:29.136] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:31:0) after 4m11s
[18:14:40.153] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:31:0)
[18:14:40.170] Build status: STARTED
[18:14:40.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:32:0) after 2m41s
[18:17:21.208] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:32:0)
[18:17:21.241] Build status: STARTED
[18:17:21.241] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:33:0) after 1m34s
[18:18:55.256] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:33:0)
[18:18:55.273] Build status: STARTED
[18:18:55.273] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:34:0) after 9m40s
[18:28:35.619] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:34:0)
[18:28:35.641] Build status: STARTED
[18:28:35.641] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:35:0) after 6m19s
[18:34:54.659] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:35:0)
[18:34:54.675] Build status: STARTED
[18:34:54.675] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:36:0) after 3m10s
[18:38:04.977] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:36:0)
[18:38:05.018] Build status: STARTED
[18:38:05.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:37:0) after 2m5s
[18:40:10.250] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:37:0)
[18:40:10.279] Build status: STARTED
[18:40:10.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:38:0) after 8m53s
[18:49:03.355] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:38:0)
[18:49:03.428] Build status: STARTED
[18:49:03.428] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:39:0) after 7m41s
[18:56:44.743] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:39:0)
[18:56:44.764] Build status: STARTED
[18:56:44.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:40:0) after 6m57s
[19:03:41.779] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:40:0)
[19:03:41.808] Build status: STARTED
[19:03:41.808] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:41:0) after 4m19s
[19:08:01.069] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:41:0)
[19:08:01.090] Build status: STARTED
[19:08:01.090] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:42:0) after 7m42s
[19:15:43.108] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:42:0)
[19:15:43.170] Build status: STARTED
[19:15:43.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:43:0) after 5m25s
[19:21:08.773] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:43:0)
[19:21:08.797] Build status: STARTED
[19:21:08.797] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:44:0) after 5m26s
[19:26:34.905] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:44:0)
[19:26:34.924] Build status: STARTED
[19:26:34.924] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:45:0) after 4m56s
[19:31:30.925] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:45:0)
[19:31:30.945] Build status: STARTED
[19:31:30.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:46:0) after 1m58s
[19:33:29.022] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:46:0)
[19:33:29.047] Build status: STARTED
[19:33:29.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:47:0) after 1m9s
[19:34:38.402] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:47:0)
[19:34:38.424] Build status: STARTED
[19:34:38.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:48:0) after 4m23s
[19:39:01.442] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:48:0)
[19:39:01.472] Build status: STARTED
[19:39:01.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:49:0) after 5m2s
[19:44:03.875] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:49:0)
[19:44:04.037] Build status: STARTED
[19:44:04.037] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:50:0) after 8m43s
[19:52:47.051] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:50:0)
[19:52:47.099] Build status: STARTED
[19:52:47.099] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:51:0) after 2m26s
[19:55:13.088] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:51:0)
[19:55:13.224] Build status: STARTED
[19:55:13.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:52:0) after 9m54s
[20:05:07.341] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:52:0)
[20:05:07.438] Build status: STARTED
[20:05:07.438] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:53:0) after 6m18s
[20:11:25.516] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:53:0)
[20:11:25.533] Build status: STARTED
[20:11:25.533] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:54:0) after 3m44s
[20:15:09.552] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:54:0)
[20:15:09.580] Build status: STARTED
[20:15:09.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:55:0) after 4m27s
[20:19:36.598] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:55:0)
[20:19:36.655] Build status: STARTED
[20:19:36.655] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:56:0) after 2m6s
[20:21:42.972] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:56:0)
[20:21:42.990] Build status: STARTED
[20:21:42.990] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851412077299289008:57:0) after 7m0s
[20:25:38.978] Received PubSub notification, asking Buildbucket for the build status
[20:25:38.999] Build:
{
"id": "8685982015504174241",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2362,
"createdBy": "project:chromium",
"createTime": "2026-03-29T15:44:21.255840957Z",
"startTime": "2026-03-29T15:45:06.464930848Z",
"endTime": "2026-03-29T20:25:38.646865592Z",
"updateTime": "2026-03-29T20:25:38.646865592Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ca0171bae24453d18a961674eed287a21b752b91",
"ref": "refs/heads/main"
}
}
}
[20:25:38.999] Invocation finished in 4h41m18.839815978s with status FAILED