[21:38:10.046] New invocation is queued and will start shortly
[21:38:11.152] Starting the invocation (attempt 1)
[21:38:11.179] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/3780acfef44b27f9cfaff9de0dca116302887b73"
[21:38:11.179] Popped gitiles commit info from properties and tags
[21:38:11.179] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:38:11.180] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:38:11.180] Buildbucket request:
{
"requestId": "8851933397952293024",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3780acfef44b27f9cfaff9de0dca116302887b73",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851933397952293024"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:38:11.820] Scheduled build:
{
"id": "8686503336017858097",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1996,
"createdBy": "project:chromium",
"createTime": "2026-03-23T21:38:11.270456715Z",
"updateTime": "2026-03-23T21:38:11.270456715Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3780acfef44b27f9cfaff9de0dca116302887b73",
"ref": "refs/heads/main"
}
}
}
[21:38:11.820] Task URL: https://cr-buildbucket.appspot.com/build/8686503336017858097
[21:38:11.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:2:0) after 5m57s
[21:38:18.775] Received PubSub notification, asking Buildbucket for the build status
[21:38:18.791] Build status: STARTED
[21:38:20.565] Received PubSub notification, asking Buildbucket for the build status
[21:38:20.586] Build status: STARTED
[21:44:09.007] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:2:0)
[21:44:09.028] Build status: STARTED
[21:44:09.028] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:5:0) after 4m9s
[21:48:18.086] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:5:0)
[21:48:18.101] Build status: STARTED
[21:48:18.102] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:6:0) after 4m54s
[21:53:12.118] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:6:0)
[21:53:12.135] Build status: STARTED
[21:53:12.135] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:7:0) after 4m51s
[21:58:03.239] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:7:0)
[21:58:03.288] Build status: STARTED
[21:58:03.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:8:0) after 9m7s
[22:07:10.348] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:8:0)
[22:07:10.367] Build status: STARTED
[22:07:10.367] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:9:0) after 6m42s
[22:13:52.403] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:9:0)
[22:13:52.423] Build status: STARTED
[22:13:52.423] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:10:0) after 6m2s
[22:19:54.447] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:10:0)
[22:19:54.465] Build status: STARTED
[22:19:54.465] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:11:0) after 4m34s
[22:24:28.536] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:11:0)
[22:24:28.555] Build status: STARTED
[22:24:28.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:12:0) after 1m43s
[22:26:11.628] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:12:0)
[22:26:11.644] Build status: STARTED
[22:26:11.644] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:13:0) after 2m14s
[22:28:25.666] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:13:0)
[22:28:25.695] Build status: STARTED
[22:28:25.695] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:14:0) after 1m18s
[22:29:43.763] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:14:0)
[22:29:43.793] Build status: STARTED
[22:29:43.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:15:0) after 5m41s
[22:35:24.811] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:15:0)
[22:35:24.826] Build status: STARTED
[22:35:24.826] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:16:0) after 9m30s
[22:44:54.941] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:16:0)
[22:44:54.963] Build status: STARTED
[22:44:54.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:17:0) after 7m39s
[22:52:33.991] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:17:0)
[22:52:34.007] Build status: STARTED
[22:52:34.007] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:18:0) after 9m38s
[23:02:12.030] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:18:0)
[23:02:12.054] Build status: STARTED
[23:02:12.054] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:19:0) after 5m22s
[23:07:34.098] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:19:0)
[23:07:34.114] Build status: STARTED
[23:07:34.114] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:20:0) after 4m4s
[23:11:38.161] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:20:0)
[23:11:38.214] Build status: STARTED
[23:11:38.214] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:21:0) after 1m50s
[23:13:28.401] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:21:0)
[23:13:28.417] Build status: STARTED
[23:13:28.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:22:0) after 7m57s
[23:21:25.446] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:22:0)
[23:21:25.463] Build status: STARTED
[23:21:25.463] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:23:0) after 4m47s
[23:26:12.480] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:23:0)
[23:26:12.508] Build status: STARTED
[23:26:12.508] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:24:0) after 5m37s
[23:31:49.545] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:24:0)
[23:31:49.563] Build status: STARTED
[23:31:49.563] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:25:0) after 7m35s
[23:39:24.682] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:25:0)
[23:39:24.697] Build status: STARTED
[23:39:24.698] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:26:0) after 8m29s
[23:47:53.701] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:26:0)
[23:47:53.718] Build status: STARTED
[23:47:53.718] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:27:0) after 1m22s
[23:49:15.789] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:27:0)
[23:49:15.812] Build status: STARTED
[23:49:15.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:28:0) after 3m8s
[23:52:23.841] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:28:0)
[23:52:23.860] Build status: STARTED
[23:52:23.860] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:29:0) after 2m9s
[23:54:32.973] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:29:0)
[23:54:33.055] Build status: STARTED
[23:54:33.055] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:30:0) after 5m13s
[23:59:46.075] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:30:0)
[23:59:46.096] Build status: STARTED
[23:59:46.096] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:31:0) after 1m6s
[00:00:52.163] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:31:0)
[00:00:52.181] Build status: STARTED
[00:00:52.181] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:32:0) after 4m54s
[00:05:46.212] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:32:0)
[00:05:46.229] Build status: STARTED
[00:05:46.229] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:33:0) after 8m0s
[00:13:46.272] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:33:0)
[00:13:46.291] Build status: STARTED
[00:13:46.292] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:34:0) after 8m53s
[00:22:39.488] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:34:0)
[00:22:39.517] Build status: STARTED
[00:22:39.517] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:35:0) after 4m46s
[00:27:25.522] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:35:0)
[00:27:25.539] Build status: STARTED
[00:27:25.539] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:36:0) after 7m50s
[00:35:15.645] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:36:0)
[00:35:15.665] Build status: STARTED
[00:35:15.665] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:37:0) after 7m26s
[00:42:41.689] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:37:0)
[00:42:41.704] Build status: STARTED
[00:42:41.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:38:0) after 3m24s
[00:46:05.755] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:38:0)
[00:46:05.786] Build status: STARTED
[00:46:05.787] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:39:0) after 1m45s
[00:47:50.822] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:39:0)
[00:47:50.845] Build status: STARTED
[00:47:50.845] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:40:0) after 8m25s
[00:56:15.913] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:40:0)
[00:56:15.937] Build status: STARTED
[00:56:15.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:41:0) after 5m40s
[01:01:55.980] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:41:0)
[01:01:56.002] Build status: STARTED
[01:01:56.002] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:42:0) after 6m1s
[01:07:57.091] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:42:0)
[01:07:57.110] Build status: STARTED
[01:07:57.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:43:0) after 1m10s
[01:09:07.133] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:43:0)
[01:09:07.176] Build status: STARTED
[01:09:07.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:44:0) after 5m27s
[01:14:34.281] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:44:0)
[01:14:34.297] Build status: STARTED
[01:14:34.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:45:0) after 6m24s
[01:20:58.334] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:45:0)
[01:20:58.383] Build status: STARTED
[01:20:58.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:46:0) after 7m31s
[01:28:29.442] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:46:0)
[01:28:29.461] Build status: STARTED
[01:28:29.461] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:47:0) after 3m49s
[01:32:18.495] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:47:0)
[01:32:18.523] Build status: STARTED
[01:32:18.523] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:48:0) after 8m24s
[01:40:42.528] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:48:0)
[01:40:42.548] Build status: STARTED
[01:40:42.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:49:0) after 8m42s
[01:49:24.654] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:49:0)
[01:49:24.671] Build status: STARTED
[01:49:24.672] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:50:0) after 8m42s
[01:58:06.696] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:50:0)
[01:58:06.712] Build status: STARTED
[01:58:06.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:51:0) after 8m10s
[02:06:16.743] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:51:0)
[02:06:16.762] Build status: STARTED
[02:06:16.762] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:52:0) after 5m56s
[02:12:12.793] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:52:0)
[02:12:12.810] Build status: STARTED
[02:12:12.810] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:53:0) after 5m57s
[02:18:09.889] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:53:0)
[02:18:09.911] Build status: STARTED
[02:18:09.911] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:54:0) after 7m4s
[02:25:14.110] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:54:0)
[02:25:14.126] Build status: STARTED
[02:25:14.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:55:0) after 5m16s
[02:30:30.158] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:55:0)
[02:30:30.206] Build status: STARTED
[02:30:30.206] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:56:0) after 9m25s
[02:39:55.229] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:56:0)
[02:39:55.244] Build status: STARTED
[02:39:55.244] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:57:0) after 8m16s
[02:48:11.312] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:57:0)
[02:48:11.326] Build status: STARTED
[02:48:11.326] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:58:0) after 8m10s
[02:56:21.460] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:58:0)
[02:56:21.632] Build status: STARTED
[02:56:21.632] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:59:0) after 8m53s
[03:05:14.729] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:59:0)
[03:05:14.757] Build status: STARTED
[03:05:14.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:60:0) after 4m1s
[03:09:15.776] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:60:0)
[03:09:15.793] Build status: STARTED
[03:09:15.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8851933397952293024:61:0) after 7m8s
[03:10:19.157] Received PubSub notification, asking Buildbucket for the build status
[03:10:19.176] Build:
{
"id": "8686503336017858097",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 1996,
"createdBy": "project:chromium",
"createTime": "2026-03-23T21:38:11.270456715Z",
"startTime": "2026-03-23T21:38:15.284385246Z",
"endTime": "2026-03-24T03:10:18.853309173Z",
"updateTime": "2026-03-24T03:10:18.853309173Z",
"status": "SUCCESS",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3780acfef44b27f9cfaff9de0dca116302887b73",
"ref": "refs/heads/main"
}
}
}
[03:10:19.176] Invocation finished in 5h32m9.134229156s with status SUCCEEDED