[06:07:01.376] New invocation is queued and will start shortly
[06:07:02.450] Starting the invocation (attempt 1)
[06:07:02.481] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d18517157412063bca6f301d173ca55e7a3ce0bb"
[06:07:02.481] Popped gitiles commit info from properties and tags
[06:07:02.481] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[06:07:02.482] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[06:07:02.482] Buildbucket request:
{
"requestId": "8858514962081646624",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d18517157412063bca6f301d173ca55e7a3ce0bb",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8858514962081646624"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[06:07:02.889] Scheduled build:
{
"id": "8693084900208636817",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 1325,
"createdBy": "project:chromium",
"createTime": "2026-01-10T06:07:02.540814780Z",
"updateTime": "2026-01-10T06:07:02.540814780Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d18517157412063bca6f301d173ca55e7a3ce0bb",
"ref": "refs/heads/main"
}
},
"canary": true
}
[06:07:02.889] Task URL: https://cr-buildbucket.appspot.com/build/8693084900208636817
[06:07:02.889] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:2:0) after 3m4s
[06:07:03.868] Received PubSub notification, asking Buildbucket for the build status
[06:07:04.139] Build status: SCHEDULED
[06:07:05.990] Received PubSub notification, asking Buildbucket for the build status
[06:07:06.018] Build status: STARTED
[06:10:06.911] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:2:0)
[06:10:06.947] Build status: STARTED
[06:10:06.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:5:0) after 1m53s
[06:12:00.364] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:5:0)
[06:12:00.382] Build status: STARTED
[06:12:00.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:6:0) after 4m32s
[06:16:32.398] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:6:0)
[06:16:32.490] Build status: STARTED
[06:16:32.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:7:0) after 3m8s
[06:19:40.548] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:7:0)
[06:19:40.568] Build status: STARTED
[06:19:40.568] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:8:0) after 7m43s
[06:27:23.586] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:8:0)
[06:27:23.661] Build status: STARTED
[06:27:23.661] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:9:0) after 4m7s
[06:31:30.769] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:9:0)
[06:31:30.784] Build status: STARTED
[06:31:30.784] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:10:0) after 5m29s
[06:36:59.917] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:10:0)
[06:36:59.940] Build status: STARTED
[06:36:59.940] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:11:0) after 4m40s
[06:41:40.022] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:11:0)
[06:41:40.039] Build status: STARTED
[06:41:40.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:12:0) after 1m29s
[06:43:09.229] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:12:0)
[06:43:09.303] Build status: STARTED
[06:43:09.303] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:13:0) after 9m19s
[06:52:28.390] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:13:0)
[06:52:28.405] Build status: STARTED
[06:52:28.405] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:14:0) after 1m2s
[06:53:30.653] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:14:0)
[06:53:30.675] Build status: STARTED
[06:53:30.675] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:15:0) after 3m10s
[06:56:40.730] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:15:0)
[06:56:40.768] Build status: STARTED
[06:56:40.768] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:16:0) after 5m49s
[07:02:30.019] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:16:0)
[07:02:30.035] Build status: STARTED
[07:02:30.035] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:17:0) after 2m28s
[07:04:58.134] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:17:0)
[07:04:58.159] Build status: STARTED
[07:04:58.159] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:18:0) after 3m12s
[07:08:10.280] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:18:0)
[07:08:10.332] Build status: STARTED
[07:08:10.332] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:19:0) after 8m56s
[07:17:06.356] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:19:0)
[07:17:06.373] Build status: STARTED
[07:17:06.373] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:20:0) after 4m18s
[07:21:24.361] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:20:0)
[07:21:24.372] Build status: STARTED
[07:21:24.372] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:21:0) after 6m16s
[07:27:40.401] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:21:0)
[07:27:40.427] Build status: STARTED
[07:27:40.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:22:0) after 2m56s
[07:30:36.474] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:22:0)
[07:30:36.523] Build status: STARTED
[07:30:36.523] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:23:0) after 4m53s
[07:35:30.065] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:23:0)
[07:35:30.076] Build status: STARTED
[07:35:30.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:24:0) after 2m52s
[07:38:22.094] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:24:0)
[07:38:22.115] Build status: STARTED
[07:38:22.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:25:0) after 3m27s
[07:41:49.130] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:25:0)
[07:41:49.143] Build status: STARTED
[07:41:49.143] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:26:0) after 4m52s
[07:46:41.162] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:26:0)
[07:46:41.176] Build status: STARTED
[07:46:41.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:27:0) after 4m14s
[07:50:55.227] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:27:0)
[07:50:55.242] Build status: STARTED
[07:50:55.242] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:28:0) after 3m21s
[07:54:16.270] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:28:0)
[07:54:16.286] Build status: STARTED
[07:54:16.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:29:0) after 8m42s
[08:02:58.299] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:29:0)
[08:02:58.344] Build status: STARTED
[08:02:58.344] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:30:0) after 9m13s
[08:12:11.387] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:30:0)
[08:12:11.405] Build status: STARTED
[08:12:11.405] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:31:0) after 3m28s
[08:15:39.717] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:31:0)
[08:15:39.732] Build status: STARTED
[08:15:39.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:32:0) after 2m21s
[08:18:01.135] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:32:0)
[08:18:01.296] Build status: STARTED
[08:18:01.296] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:33:0) after 2m53s
[08:20:54.415] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:33:0)
[08:20:54.428] Build status: STARTED
[08:20:54.428] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:34:0) after 8m8s
[08:29:02.451] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:34:0)
[08:29:02.483] Build status: STARTED
[08:29:02.483] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:35:0) after 2m1s
[08:31:03.576] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:35:0)
[08:31:03.602] Build status: STARTED
[08:31:03.602] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:36:0) after 5m54s
[08:36:57.582] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:36:0)
[08:36:57.596] Build status: STARTED
[08:36:57.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:37:0) after 8m35s
[08:45:32.610] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:37:0)
[08:45:32.627] Build status: STARTED
[08:45:32.627] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:38:0) after 1m18s
[08:46:50.644] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:38:0)
[08:46:50.666] Build status: STARTED
[08:46:50.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:39:0) after 4m27s
[08:51:17.682] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:39:0)
[08:51:17.697] Build status: STARTED
[08:51:17.697] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:40:0) after 6m2s
[08:57:19.692] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:40:0)
[08:57:19.705] Build status: STARTED
[08:57:19.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:41:0) after 2m2s
[08:59:22.357] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:41:0)
[08:59:22.392] Build status: STARTED
[08:59:22.392] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:42:0) after 5m52s
[09:05:14.445] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:42:0)
[09:05:14.505] Build status: STARTED
[09:05:14.505] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:43:0) after 1m20s
[09:06:34.519] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:43:0)
[09:06:34.533] Build status: STARTED
[09:06:34.533] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:44:0) after 8m45s
[09:15:19.548] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:44:0)
[09:15:19.593] Build status: STARTED
[09:15:19.593] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:45:0) after 4m53s
[09:20:12.593] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:45:0)
[09:20:12.667] Build status: STARTED
[09:20:12.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:46:0) after 1m56s
[09:22:08.696] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:46:0)
[09:22:08.709] Build status: STARTED
[09:22:08.709] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:47:0) after 4m52s
[09:27:00.727] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:47:0)
[09:27:00.741] Build status: STARTED
[09:27:00.741] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:48:0) after 7m28s
[09:34:28.870] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:48:0)
[09:34:28.890] Build status: STARTED
[09:34:28.890] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:49:0) after 9m30s
[09:43:58.906] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:49:0)
[09:43:58.924] Build status: STARTED
[09:43:58.924] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:50:0) after 3m24s
[09:47:22.998] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:50:0)
[09:47:23.011] Build status: STARTED
[09:47:23.011] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:51:0) after 3m28s
[09:50:51.026] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:51:0)
[09:50:51.038] Build status: STARTED
[09:50:51.038] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:52:0) after 8m10s
[09:59:01.119] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:52:0)
[09:59:01.149] Build status: STARTED
[09:59:01.149] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:53:0) after 2m45s
[10:01:46.316] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:53:0)
[10:01:46.331] Build status: STARTED
[10:01:46.331] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:54:0) after 2m17s
[10:04:03.443] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:54:0)
[10:04:03.520] Build status: STARTED
[10:04:03.520] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:55:0) after 9m15s
[10:13:18.582] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:55:0)
[10:13:18.596] Build status: STARTED
[10:13:18.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:56:0) after 3m20s
[10:16:38.610] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:56:0)
[10:16:38.625] Build status: STARTED
[10:16:38.625] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:57:0) after 2m17s
[10:18:55.649] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:57:0)
[10:18:55.664] Build status: STARTED
[10:18:55.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:58:0) after 3m34s
[10:22:29.692] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:58:0)
[10:22:29.705] Build status: STARTED
[10:22:29.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:59:0) after 1m18s
[10:23:47.824] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:59:0)
[10:23:47.846] Build status: STARTED
[10:23:47.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64:8858514962081646624:60:0) after 5m13s
[10:26:22.878] Received PubSub notification, asking Buildbucket for the build status
[10:26:22.894] Build:
{
"id": "8693084900208636817",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64"
},
"number": 1325,
"createdBy": "project:chromium",
"createTime": "2026-01-10T06:07:02.540814780Z",
"startTime": "2026-01-10T06:07:05.590328168Z",
"endTime": "2026-01-10T10:26:21.517543184Z",
"updateTime": "2026-01-10T10:26:21.517543184Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d18517157412063bca6f301d173ca55e7a3ce0bb",
"ref": "refs/heads/main"
}
},
"canary": true
}
[10:26:22.894] Invocation finished in 4h19m21.524044393s with status FAILED