[07:24:59.283] New invocation is queued and will start shortly
[07:25:00.550] Starting the invocation (attempt 1)
[07:25:00.576] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/b196a4a0d36a873cfb95f83a9edc93f1483d2ac9"
[07:25:00.576] Popped gitiles commit info from properties and tags
[07:25:00.576] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[07:25:00.576] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[07:25:00.576] Buildbucket request:
{
"requestId": "8845373496811655728",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b196a4a0d36a873cfb95f83a9edc93f1483d2ac9",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8845373496811655728"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[07:25:01.105] Scheduled build:
{
"id": "8679943434739983121",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3007,
"createdBy": "project:chromium",
"createTime": "2026-06-04T07:25:00.637584981Z",
"updateTime": "2026-06-04T07:25:00.637584981Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b196a4a0d36a873cfb95f83a9edc93f1483d2ac9",
"ref": "refs/heads/main"
}
}
}
[07:25:01.105] Task URL: https://cr-buildbucket.appspot.com/build/8679943434739983121
[07:25:01.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:2:0) after 2m49s
[07:25:02.031] Received PubSub notification, asking Buildbucket for the build status
[07:25:02.050] Build status: SCHEDULED
[07:25:04.926] Received PubSub notification, asking Buildbucket for the build status
[07:25:04.947] Build status: STARTED
[07:27:50.130] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:2:0)
[07:27:50.149] Build status: STARTED
[07:27:50.149] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:5:0) after 4m23s
[07:32:13.301] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:5:0)
[07:32:13.327] Build status: STARTED
[07:32:13.327] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:6:0) after 5m15s
[07:37:28.445] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:6:0)
[07:37:28.461] Build status: STARTED
[07:37:28.461] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:7:0) after 5m27s
[07:42:55.513] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:7:0)
[07:42:55.535] Build status: STARTED
[07:42:55.535] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:8:0) after 8m47s
[07:51:42.557] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:8:0)
[07:51:42.575] Build status: STARTED
[07:51:42.575] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:9:0) after 4m55s
[07:56:37.591] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:9:0)
[07:56:37.608] Build status: STARTED
[07:56:37.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:10:0) after 7m56s
[08:04:33.821] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:10:0)
[08:04:33.838] Build status: STARTED
[08:04:33.838] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:11:0) after 7m53s
[08:12:26.861] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:11:0)
[08:12:26.890] Build status: STARTED
[08:12:26.890] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:12:0) after 1m43s
[08:14:09.902] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:12:0)
[08:14:09.919] Build status: STARTED
[08:14:09.919] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:13:0) after 2m35s
[08:16:44.936] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:13:0)
[08:16:44.981] Build status: STARTED
[08:16:44.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:14:0) after 8m50s
[08:25:34.986] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:14:0)
[08:25:35.006] Build status: STARTED
[08:25:35.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:15:0) after 7m24s
[08:32:59.022] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:15:0)
[08:32:59.042] Build status: STARTED
[08:32:59.042] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:16:0) after 2m46s
[08:35:45.067] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:16:0)
[08:35:45.087] Build status: STARTED
[08:35:45.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:17:0) after 5m8s
[08:40:53.137] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:17:0)
[08:40:53.153] Build status: STARTED
[08:40:53.154] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:18:0) after 9m52s
[08:50:45.218] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:18:0)
[08:50:45.347] Build status: STARTED
[08:50:45.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:19:0) after 5m58s
[08:56:43.364] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:19:0)
[08:56:43.386] Build status: STARTED
[08:56:43.386] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:20:0) after 3m2s
[08:59:45.405] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:20:0)
[08:59:45.426] Build status: STARTED
[08:59:45.426] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:21:0) after 5m0s
[09:04:45.438] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:21:0)
[09:04:45.534] Build status: STARTED
[09:04:45.534] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:22:0) after 2m37s
[09:07:22.545] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:22:0)
[09:07:22.564] Build status: STARTED
[09:07:22.564] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:23:0) after 9m33s
[09:16:55.631] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:23:0)
[09:16:55.649] Build status: STARTED
[09:16:55.649] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:24:0) after 8m39s
[09:25:34.697] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:24:0)
[09:25:34.722] Build status: STARTED
[09:25:34.722] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:25:0) after 6m3s
[09:31:37.827] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:25:0)
[09:31:37.849] Build status: STARTED
[09:31:37.850] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:26:0) after 9m34s
[09:41:11.915] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:26:0)
[09:41:11.931] Build status: STARTED
[09:41:11.931] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:27:0) after 2m13s
[09:43:24.939] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:27:0)
[09:43:24.981] Build status: STARTED
[09:43:24.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:28:0) after 1m48s
[09:45:13.055] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:28:0)
[09:45:13.076] Build status: STARTED
[09:45:13.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:29:0) after 5m7s
[09:50:20.196] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:29:0)
[09:50:20.215] Build status: STARTED
[09:50:20.215] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:30:0) after 8m49s
[09:59:09.242] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:30:0)
[09:59:09.258] Build status: STARTED
[09:59:09.258] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:31:0) after 5m1s
[10:04:10.276] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:31:0)
[10:04:10.295] Build status: STARTED
[10:04:10.295] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:32:0) after 8m27s
[10:12:37.319] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:32:0)
[10:12:37.340] Build status: STARTED
[10:12:37.340] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:33:0) after 9m9s
[10:21:46.410] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:33:0)
[10:21:46.436] Build status: STARTED
[10:21:46.436] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:34:0) after 8m12s
[10:29:58.456] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:34:0)
[10:29:58.467] Build status: STARTED
[10:29:58.467] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:35:0) after 4m50s
[10:34:48.494] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:35:0)
[10:34:48.512] Build status: STARTED
[10:34:48.512] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:36:0) after 4m58s
[10:39:46.622] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:36:0)
[10:39:46.677] Build status: STARTED
[10:39:46.677] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:37:0) after 3m14s
[10:43:00.906] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:37:0)
[10:43:00.923] Build status: STARTED
[10:43:00.923] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:38:0) after 4m30s
[10:47:30.972] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:38:0)
[10:47:30.985] Build status: STARTED
[10:47:30.985] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:39:0) after 1m30s
[10:49:01.020] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:39:0)
[10:49:01.043] Build status: STARTED
[10:49:01.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:40:0) after 3m30s
[10:52:31.062] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:40:0)
[10:52:31.081] Build status: STARTED
[10:52:31.081] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:41:0) after 5m28s
[10:57:59.184] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:41:0)
[10:57:59.201] Build status: STARTED
[10:57:59.201] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:42:0) after 4m53s
[11:02:52.219] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:42:0)
[11:02:52.245] Build status: STARTED
[11:02:52.245] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:43:0) after 9m54s
[11:12:46.479] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:43:0)
[11:12:46.496] Build status: STARTED
[11:12:46.496] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:44:0) after 5m6s
[11:17:52.512] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:44:0)
[11:17:52.527] Build status: STARTED
[11:17:52.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:45:0) after 2m6s
[11:19:58.672] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:45:0)
[11:19:58.713] Build status: STARTED
[11:19:58.713] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:46:0) after 3m30s
[11:23:28.682] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:46:0)
[11:23:28.703] Build status: STARTED
[11:23:28.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:47:0) after 7m49s
[11:31:17.719] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:47:0)
[11:31:17.741] Build status: STARTED
[11:31:17.741] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:48:0) after 1m59s
[11:33:16.814] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:48:0)
[11:33:16.847] Build status: STARTED
[11:33:16.847] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:49:0) after 2m7s
[11:35:23.863] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:49:0)
[11:35:23.884] Build status: STARTED
[11:35:23.884] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:50:0) after 3m39s
[11:39:02.937] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:50:0)
[11:39:02.960] Build status: STARTED
[11:39:02.960] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:51:0) after 1m3s
[11:40:05.965] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:51:0)
[11:40:05.981] Build status: STARTED
[11:40:05.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:52:0) after 2m14s
[11:42:20.018] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:52:0)
[11:42:20.036] Build status: STARTED
[11:42:20.036] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:53:0) after 1m41s
[11:44:01.057] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:53:0)
[11:44:01.131] Build status: STARTED
[11:44:01.131] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:54:0) after 5m30s
[11:49:31.166] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:54:0)
[11:49:31.462] Build status: STARTED
[11:49:31.462] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:55:0) after 1m9s
[11:50:40.623] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:55:0)
[11:50:40.643] Build status: STARTED
[11:50:40.643] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:56:0) after 1m42s
[11:52:22.659] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:56:0)
[11:52:22.674] Build status: STARTED
[11:52:22.674] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:57:0) after 8m40s
[12:01:02.692] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:57:0)
[12:01:02.712] Build status: STARTED
[12:01:02.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:58:0) after 7m7s
[12:08:09.724] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:58:0)
[12:08:09.744] Build status: STARTED
[12:08:09.744] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8845373496811655728:59:0) after 5m23s
[12:11:12.668] Received PubSub notification, asking Buildbucket for the build status
[12:11:12.693] Build:
{
"id": "8679943434739983121",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3007,
"createdBy": "project:chromium",
"createTime": "2026-06-04T07:25:00.637584981Z",
"startTime": "2026-06-04T07:25:04.532932004Z",
"endTime": "2026-06-04T12:11:12.331553940Z",
"updateTime": "2026-06-04T12:11:12.331553940Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b196a4a0d36a873cfb95f83a9edc93f1483d2ac9",
"ref": "refs/heads/main"
}
}
}
[12:11:12.693] Invocation finished in 4h46m13.415220846s with status FAILED