[04:21:45.553] New invocation is queued and will start shortly
[04:21:46.737] Starting the invocation (attempt 1)
[04:21:46.764] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/fb8f547ac133cb6da9c3b9bb12fbd9f4891dd650"
[04:21:46.764] Popped gitiles commit info from properties and tags
[04:21:46.764] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:21:46.764] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:21:46.764] Buildbucket request:
{
"requestId": "8851002036498099744",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "fb8f547ac133cb6da9c3b9bb12fbd9f4891dd650",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851002036498099744"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:21:47.282] Scheduled build:
{
"id": "8685571974513620369",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2385,
"createdBy": "project:chromium",
"createTime": "2026-04-03T04:21:46.816272505Z",
"updateTime": "2026-04-03T04:21:46.816272505Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "fb8f547ac133cb6da9c3b9bb12fbd9f4891dd650",
"ref": "refs/heads/main"
}
},
"canary": true
}
[04:21:47.282] Task URL: https://cr-buildbucket.appspot.com/build/8685571974513620369
[04:21:47.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:2:0) after 7m56s
[04:21:48.245] Received PubSub notification, asking Buildbucket for the build status
[04:21:48.267] Build status: SCHEDULED
[04:22:32.972] Received PubSub notification, asking Buildbucket for the build status
[04:22:33.036] Build status: STARTED
[04:29:43.321] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:2:0)
[04:29:43.336] Build status: STARTED
[04:29:43.337] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:5:0) after 1m42s
[04:31:25.379] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:5:0)
[04:31:25.399] Build status: STARTED
[04:31:25.399] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:6:0) after 1m7s
[04:32:32.427] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:6:0)
[04:32:32.444] Build status: STARTED
[04:32:32.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:7:0) after 6m4s
[04:38:36.587] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:7:0)
[04:38:36.610] Build status: STARTED
[04:38:36.610] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:8:0) after 7m34s
[04:46:11.088] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:8:0)
[04:46:11.106] Build status: STARTED
[04:46:11.106] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:9:0) after 2m49s
[04:49:00.149] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:9:0)
[04:49:00.166] Build status: STARTED
[04:49:00.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:10:0) after 6m25s
[04:55:25.189] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:10:0)
[04:55:25.210] Build status: STARTED
[04:55:25.211] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:11:0) after 2m54s
[04:58:19.238] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:11:0)
[04:58:19.276] Build status: STARTED
[04:58:19.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:12:0) after 4m6s
[05:02:25.720] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:12:0)
[05:02:25.751] Build status: STARTED
[05:02:25.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:13:0) after 6m36s
[05:09:01.767] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:13:0)
[05:09:01.786] Build status: STARTED
[05:09:01.786] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:14:0) after 6m41s
[05:15:43.079] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:14:0)
[05:15:43.100] Build status: STARTED
[05:15:43.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:15:0) after 8m58s
[05:24:41.156] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:15:0)
[05:24:41.173] Build status: STARTED
[05:24:41.174] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:16:0) after 5m36s
[05:30:17.243] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:16:0)
[05:30:17.267] Build status: STARTED
[05:30:17.267] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:17:0) after 4m44s
[05:35:01.289] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:17:0)
[05:35:01.306] Build status: STARTED
[05:35:01.306] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:18:0) after 2m57s
[05:37:58.382] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:18:0)
[05:37:58.402] Build status: STARTED
[05:37:58.402] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:19:0) after 1m42s
[05:39:40.542] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:19:0)
[05:39:40.571] Build status: STARTED
[05:39:40.571] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:20:0) after 7m4s
[05:46:44.542] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:20:0)
[05:46:44.562] Build status: STARTED
[05:46:44.562] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:21:0) after 5m12s
[05:51:56.662] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:21:0)
[05:51:56.680] Build status: STARTED
[05:51:56.681] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:22:0) after 5m2s
[05:56:58.696] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:22:0)
[05:56:58.710] Build status: STARTED
[05:56:58.710] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:23:0) after 9m37s
[06:06:35.791] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:23:0)
[06:06:35.811] Build status: STARTED
[06:06:35.811] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:24:0) after 2m20s
[06:08:55.830] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:24:0)
[06:08:55.915] Build status: STARTED
[06:08:55.915] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:25:0) after 7m28s
[06:16:23.937] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:25:0)
[06:16:23.956] Build status: STARTED
[06:16:23.956] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:26:0) after 1m53s
[06:18:16.933] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:26:0)
[06:18:16.957] Build status: STARTED
[06:18:16.957] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:27:0) after 9m1s
[06:27:18.387] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:27:0)
[06:27:18.403] Build status: STARTED
[06:27:18.403] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:28:0) after 3m10s
[06:30:28.474] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:28:0)
[06:30:28.520] Build status: STARTED
[06:30:28.520] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:29:0) after 2m56s
[06:33:24.538] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:29:0)
[06:33:24.578] Build status: STARTED
[06:33:24.578] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:30:0) after 4m51s
[06:38:15.708] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:30:0)
[06:38:15.727] Build status: STARTED
[06:38:15.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:31:0) after 4m11s
[06:42:26.826] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:31:0)
[06:42:26.840] Build status: STARTED
[06:42:26.840] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:32:0) after 5m44s
[06:48:10.956] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:32:0)
[06:48:10.971] Build status: STARTED
[06:48:10.971] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:33:0) after 4m58s
[06:53:09.161] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:33:0)
[06:53:09.183] Build status: STARTED
[06:53:09.183] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:34:0) after 2m25s
[06:55:34.199] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:34:0)
[06:55:34.217] Build status: STARTED
[06:55:34.217] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:35:0) after 7m10s
[07:02:44.348] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:35:0)
[07:02:44.365] Build status: STARTED
[07:02:44.365] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:36:0) after 7m35s
[07:10:20.335] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:36:0)
[07:10:20.433] Build status: STARTED
[07:10:20.433] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:37:0) after 3m2s
[07:13:22.484] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:37:0)
[07:13:22.504] Build status: STARTED
[07:13:22.504] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:38:0) after 6m23s
[07:19:45.519] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:38:0)
[07:19:45.538] Build status: STARTED
[07:19:45.538] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:39:0) after 8m53s
[07:28:38.534] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:39:0)
[07:28:38.560] Build status: STARTED
[07:28:38.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:40:0) after 2m31s
[07:31:09.579] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:40:0)
[07:31:09.596] Build status: STARTED
[07:31:09.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:41:0) after 8m5s
[07:39:14.615] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:41:0)
[07:39:14.634] Build status: STARTED
[07:39:14.634] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:42:0) after 1m41s
[07:40:56.230] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:42:0)
[07:40:56.267] Build status: STARTED
[07:40:56.267] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:43:0) after 3m49s
[07:44:45.360] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:43:0)
[07:44:45.388] Build status: STARTED
[07:44:45.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:44:0) after 1m37s
[07:46:22.418] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:44:0)
[07:46:22.447] Build status: STARTED
[07:46:22.447] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:45:0) after 1m16s
[07:47:38.466] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:45:0)
[07:47:38.486] Build status: STARTED
[07:47:38.486] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:46:0) after 6m19s
[07:53:57.506] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:46:0)
[07:53:57.526] Build status: STARTED
[07:53:57.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:47:0) after 5m8s
[07:59:05.816] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:47:0)
[07:59:05.865] Build status: STARTED
[07:59:05.865] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:48:0) after 2m43s
[08:01:49.209] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:48:0)
[08:01:49.227] Build status: STARTED
[08:01:49.227] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:49:0) after 8m17s
[08:10:07.121] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:49:0)
[08:10:07.153] Build status: STARTED
[08:10:07.153] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:50:0) after 2m10s
[08:12:17.189] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:50:0)
[08:12:17.279] Build status: STARTED
[08:12:17.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:51:0) after 9m2s
[08:21:19.254] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:51:0)
[08:21:19.272] Build status: STARTED
[08:21:19.272] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:52:0) after 8m39s
[08:29:58.804] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:52:0)
[08:29:58.821] Build status: STARTED
[08:29:58.821] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:53:0) after 6m45s
[08:36:43.845] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:53:0)
[08:36:43.866] Build status: STARTED
[08:36:43.866] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:54:0) after 2m44s
[08:39:27.893] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:54:0)
[08:39:27.913] Build status: STARTED
[08:39:27.913] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:55:0) after 8m48s
[08:48:15.976] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:55:0)
[08:48:16.037] Build status: STARTED
[08:48:16.037] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:56:0) after 4m59s
[08:53:15.074] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:56:0)
[08:53:15.095] Build status: STARTED
[08:53:15.095] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:57:0) after 5m24s
[08:58:39.163] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:57:0)
[08:58:39.219] Build status: STARTED
[08:58:39.219] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8851002036498099744:58:0) after 5m37s
[09:04:03.015] Received PubSub notification, asking Buildbucket for the build status
[09:04:03.030] Build:
{
"id": "8685571974513620369",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 2385,
"createdBy": "project:chromium",
"createTime": "2026-04-03T04:21:46.816272505Z",
"startTime": "2026-04-03T04:22:32.768448387Z",
"endTime": "2026-04-03T09:04:02.717037994Z",
"updateTime": "2026-04-03T09:04:02.717037994Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "fb8f547ac133cb6da9c3b9bb12fbd9f4891dd650",
"ref": "refs/heads/main"
}
},
"canary": true
}
[09:04:03.030] Invocation finished in 4h42m17.490126176s with status FAILED