[17:44:57.455] New invocation is queued and will start shortly
[17:44:58.548] Starting the invocation (attempt 1)
[17:44:58.577] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/718d95be88f2f727fd0bf34c0f63958a5d8735ba"
[17:44:58.577] Popped gitiles commit info from properties and tags
[17:44:58.577] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[17:44:58.577] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[17:44:58.577] Buildbucket request:
{
"requestId": "8858289857903672640",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "718d95be88f2f727fd0bf34c0f63958a5d8735ba",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8858289857903672640"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[17:44:59.537] Scheduled build:
{
"id": "8692859795901560945",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1166,
"createdBy": "project:chromium",
"createTime": "2026-01-12T17:44:58.736085191Z",
"updateTime": "2026-01-12T17:44:58.736085191Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "718d95be88f2f727fd0bf34c0f63958a5d8735ba",
"ref": "refs/heads/main"
}
}
}
[17:44:59.537] Task URL: https://cr-buildbucket.appspot.com/build/8692859795901560945
[17:44:59.537] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:2:0) after 7m39s
[17:45:14.860] Received PubSub notification, asking Buildbucket for the build status
[17:45:14.915] Build status: STARTED
[17:45:25.647] Received PubSub notification, asking Buildbucket for the build status
[17:45:25.674] Build status: STARTED
[17:52:38.533] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:2:0)
[17:52:38.550] Build status: STARTED
[17:52:38.550] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:5:0) after 4m26s
[17:57:04.723] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:5:0)
[17:57:04.798] Build status: STARTED
[17:57:04.798] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:6:0) after 9m57s
[18:07:01.810] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:6:0)
[18:07:01.833] Build status: STARTED
[18:07:01.833] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:7:0) after 8m56s
[18:15:57.962] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:7:0)
[18:15:57.975] Build status: STARTED
[18:15:57.975] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:8:0) after 4m49s
[18:20:46.996] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:8:0)
[18:20:47.063] Build status: STARTED
[18:20:47.064] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:9:0) after 3m52s
[18:24:39.086] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:9:0)
[18:24:39.122] Build status: STARTED
[18:24:39.122] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:10:0) after 4m17s
[18:28:56.139] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:10:0)
[18:28:56.165] Build status: STARTED
[18:28:56.165] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:11:0) after 8m34s
[18:37:30.160] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:11:0)
[18:37:30.180] Build status: STARTED
[18:37:30.180] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:12:0) after 1m13s
[18:38:43.322] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:12:0)
[18:38:43.336] Build status: STARTED
[18:38:43.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:13:0) after 9m22s
[18:48:05.365] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:13:0)
[18:48:05.414] Build status: STARTED
[18:48:05.414] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:14:0) after 8m21s
[18:56:26.484] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:14:0)
[18:56:26.498] Build status: STARTED
[18:56:26.498] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:15:0) after 9m9s
[19:05:35.586] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:15:0)
[19:05:35.605] Build status: STARTED
[19:05:35.605] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:16:0) after 8m19s
[19:13:54.625] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:16:0)
[19:13:54.641] Build status: STARTED
[19:13:54.641] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:17:0) after 6m20s
[19:20:14.661] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:17:0)
[19:20:14.675] Build status: STARTED
[19:20:14.675] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:18:0) after 4m36s
[19:24:50.678] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:18:0)
[19:24:50.703] Build status: STARTED
[19:24:50.703] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:19:0) after 7m46s
[19:32:36.738] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:19:0)
[19:32:36.756] Build status: STARTED
[19:32:36.756] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:20:0) after 9m26s
[19:42:02.850] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:20:0)
[19:42:02.863] Build status: STARTED
[19:42:02.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:21:0) after 6m20s
[19:48:22.918] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:21:0)
[19:48:22.933] Build status: STARTED
[19:48:22.933] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:22:0) after 9m22s
[19:57:44.956] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:22:0)
[19:57:44.977] Build status: STARTED
[19:57:44.977] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:23:0) after 8m37s
[20:06:21.997] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:23:0)
[20:06:22.027] Build status: STARTED
[20:06:22.027] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:24:0) after 2m25s
[20:08:47.167] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:24:0)
[20:08:47.194] Build status: STARTED
[20:08:47.195] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:25:0) after 5m35s
[20:14:22.236] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:25:0)
[20:14:22.250] Build status: STARTED
[20:14:22.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:26:0) after 9m47s
[20:24:09.344] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:26:0)
[20:24:09.386] Build status: STARTED
[20:24:09.386] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:27:0) after 1m11s
[20:25:20.468] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:27:0)
[20:25:20.500] Build status: STARTED
[20:25:20.500] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:28:0) after 1m7s
[20:26:27.649] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:28:0)
[20:26:27.670] Build status: STARTED
[20:26:27.670] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:29:0) after 1m0s
[20:27:27.673] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:29:0)
[20:27:27.690] Build status: STARTED
[20:27:27.690] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:30:0) after 6m38s
[20:34:05.711] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:30:0)
[20:34:05.814] Build status: STARTED
[20:34:05.814] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:31:0) after 8m54s
[20:42:59.835] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:31:0)
[20:42:59.888] Build status: STARTED
[20:42:59.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:32:0) after 8m41s
[20:51:41.007] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:32:0)
[20:51:41.024] Build status: STARTED
[20:51:41.024] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:33:0) after 6m10s
[20:57:51.077] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:33:0)
[20:57:51.093] Build status: STARTED
[20:57:51.093] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:34:0) after 3m9s
[21:01:00.156] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:34:0)
[21:01:00.228] Build status: STARTED
[21:01:00.228] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:35:0) after 3m19s
[21:04:19.244] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:35:0)
[21:04:19.263] Build status: STARTED
[21:04:19.263] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:36:0) after 8m16s
[21:12:35.281] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:36:0)
[21:12:35.300] Build status: STARTED
[21:12:35.300] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:37:0) after 5m35s
[21:18:10.428] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:37:0)
[21:18:10.441] Build status: STARTED
[21:18:10.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:38:0) after 1m54s
[21:20:04.460] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:38:0)
[21:20:04.477] Build status: STARTED
[21:20:04.477] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:39:0) after 6m19s
[21:26:23.497] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:39:0)
[21:26:23.515] Build status: STARTED
[21:26:23.515] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:40:0) after 7m19s
[21:33:42.536] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:40:0)
[21:33:42.553] Build status: STARTED
[21:33:42.553] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:41:0) after 3m39s
[21:37:21.575] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:41:0)
[21:37:21.591] Build status: STARTED
[21:37:21.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:42:0) after 5m22s
[21:42:43.636] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:42:0)
[21:42:43.699] Build status: STARTED
[21:42:43.699] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:43:0) after 6m40s
[21:49:23.680] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:43:0)
[21:49:23.695] Build status: STARTED
[21:49:23.695] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:44:0) after 4m15s
[21:53:38.719] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:44:0)
[21:53:38.739] Build status: STARTED
[21:53:38.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:45:0) after 4m12s
[21:57:50.755] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:45:0)
[21:57:50.774] Build status: STARTED
[21:57:50.774] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8858289857903672640:46:0) after 4m54s
[21:58:04.386] Received PubSub notification, asking Buildbucket for the build status
[21:58:04.413] Build:
{
"id": "8692859795901560945",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1166,
"createdBy": "project:chromium",
"createTime": "2026-01-12T17:44:58.736085191Z",
"startTime": "2026-01-12T17:45:07.984088135Z",
"endTime": "2026-01-12T21:58:00.045124071Z",
"updateTime": "2026-01-12T21:58:00.045124071Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "718d95be88f2f727fd0bf34c0f63958a5d8735ba",
"ref": "refs/heads/main"
}
}
}
[21:58:04.413] Invocation finished in 4h13m6.969475285s with status FAILED