[05:46:44.375] New invocation is queued and will start shortly
[05:46:45.557] Starting the invocation (attempt 1)
[05:46:45.578] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/9ca3cfa098cdfe7cc4c946f4436e747254e6288f"
[05:46:45.578] Popped gitiles commit info from properties and tags
[05:46:45.578] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[05:46:45.578] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[05:46:45.578] Buildbucket request:
{
"requestId": "8852174450552677088",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWinArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9ca3cfa098cdfe7cc4c946f4436e747254e6288f",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8852174450552677088"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTWinArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[05:46:46.139] Scheduled build:
{
"id": "8686744388572072673",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWinArm64PGO"
},
"number": 7745,
"createdBy": "project:chromium",
"createTime": "2026-03-21T05:46:45.640778046Z",
"updateTime": "2026-03-21T05:46:45.640778046Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9ca3cfa098cdfe7cc4c946f4436e747254e6288f",
"ref": "refs/heads/main"
}
}
}
[05:46:46.139] Task URL: https://cr-buildbucket.appspot.com/build/8686744388572072673
[05:46:46.139] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:2:0) after 1m12s
[05:47:30.781] Received PubSub notification, asking Buildbucket for the build status
[05:47:30.818] Build status: STARTED
[05:47:32.893] Received PubSub notification, asking Buildbucket for the build status
[05:47:32.906] Build status: STARTED
[05:47:58.156] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:2:0)
[05:47:58.185] Build status: STARTED
[05:47:58.185] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:5:0) after 7m17s
[05:55:15.233] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:5:0)
[05:55:15.249] Build status: STARTED
[05:55:15.249] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:6:0) after 6m18s
[06:01:33.368] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:6:0)
[06:01:33.391] Build status: STARTED
[06:01:33.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:7:0) after 8m21s
[06:09:54.407] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:7:0)
[06:09:54.511] Build status: STARTED
[06:09:54.511] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:8:0) after 4m27s
[06:14:21.582] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:8:0)
[06:14:21.608] Build status: STARTED
[06:14:21.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:9:0) after 1m23s
[06:15:44.600] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:9:0)
[06:15:44.659] Build status: STARTED
[06:15:44.659] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:10:0) after 5m23s
[06:21:07.935] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:10:0)
[06:21:07.976] Build status: STARTED
[06:21:07.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:11:0) after 3m16s
[06:24:24.437] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:11:0)
[06:24:24.481] Build status: STARTED
[06:24:24.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:12:0) after 6m31s
[06:30:55.494] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:12:0)
[06:30:55.512] Build status: STARTED
[06:30:55.512] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:13:0) after 9m37s
[06:40:32.539] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:13:0)
[06:40:32.553] Build status: STARTED
[06:40:32.553] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:14:0) after 9m1s
[06:49:33.796] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:14:0)
[06:49:33.813] Build status: STARTED
[06:49:33.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:15:0) after 3m18s
[06:52:51.829] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:15:0)
[06:52:51.849] Build status: STARTED
[06:52:51.849] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:16:0) after 5m10s
[06:58:01.867] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:16:0)
[06:58:01.893] Build status: STARTED
[06:58:01.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:17:0) after 9m20s
[07:07:21.943] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:17:0)
[07:07:21.960] Build status: STARTED
[07:07:21.960] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:18:0) after 9m25s
[07:16:46.976] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:18:0)
[07:16:46.992] Build status: STARTED
[07:16:46.992] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:19:0) after 6m55s
[07:23:42.008] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:19:0)
[07:23:42.024] Build status: STARTED
[07:23:42.024] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:20:0) after 1m39s
[07:25:21.036] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:20:0)
[07:25:21.063] Build status: STARTED
[07:25:21.063] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:21:0) after 6m44s
[07:32:05.105] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:21:0)
[07:32:05.125] Build status: STARTED
[07:32:05.125] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:22:0) after 3m28s
[07:35:33.190] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:22:0)
[07:35:33.215] Build status: STARTED
[07:35:33.215] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:23:0) after 5m28s
[07:41:01.232] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:23:0)
[07:41:01.245] Build status: STARTED
[07:41:01.246] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:24:0) after 6m47s
[07:47:48.437] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:24:0)
[07:47:48.469] Build status: STARTED
[07:47:48.469] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:25:0) after 5m53s
[07:53:41.484] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:25:0)
[07:53:41.508] Build status: STARTED
[07:53:41.508] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:26:0) after 2m45s
[07:56:26.664] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:26:0)
[07:56:26.688] Build status: STARTED
[07:56:26.688] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:27:0) after 7m46s
[08:04:12.939] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:27:0)
[08:04:13.019] Build status: STARTED
[08:04:13.019] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:28:0) after 8m13s
[08:12:26.263] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:28:0)
[08:12:26.303] Build status: STARTED
[08:12:26.303] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:29:0) after 9m34s
[08:22:00.334] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:29:0)
[08:22:00.350] Build status: STARTED
[08:22:00.351] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:30:0) after 5m43s
[08:27:43.481] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:30:0)
[08:27:43.499] Build status: STARTED
[08:27:43.499] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:31:0) after 2m47s
[08:30:30.613] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:31:0)
[08:30:30.726] Build status: STARTED
[08:30:30.726] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:32:0) after 3m27s
[08:33:57.799] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:32:0)
[08:33:57.813] Build status: STARTED
[08:33:57.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:33:0) after 1m59s
[08:35:56.928] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:33:0)
[08:35:56.943] Build status: STARTED
[08:35:56.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:34:0) after 8m10s
[08:44:07.262] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:34:0)
[08:44:07.277] Build status: STARTED
[08:44:07.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:35:0) after 7m43s
[08:51:50.291] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:35:0)
[08:51:50.384] Build status: STARTED
[08:51:50.384] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:36:0) after 4m35s
[08:56:25.399] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:36:0)
[08:56:25.417] Build status: STARTED
[08:56:25.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:37:0) after 8m27s
[09:04:52.615] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:37:0)
[09:04:52.644] Build status: STARTED
[09:04:52.644] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:38:0) after 7m27s
[09:12:19.658] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:38:0)
[09:12:19.685] Build status: STARTED
[09:12:19.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:39:0) after 2m16s
[09:14:35.774] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:39:0)
[09:14:35.808] Build status: STARTED
[09:14:35.808] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:40:0) after 5m48s
[09:20:23.825] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:40:0)
[09:20:23.852] Build status: STARTED
[09:20:23.852] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:41:0) after 9m43s
[09:30:06.930] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:41:0)
[09:30:06.974] Build status: STARTED
[09:30:06.974] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:42:0) after 3m21s
[09:33:28.001] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:42:0)
[09:33:28.042] Build status: STARTED
[09:33:28.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:43:0) after 6m1s
[09:39:29.066] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:43:0)
[09:39:29.083] Build status: STARTED
[09:39:29.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:44:0) after 1m58s
[09:41:27.098] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:44:0)
[09:41:27.109] Build status: STARTED
[09:41:27.109] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:45:0) after 4m2s
[09:45:29.236] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:45:0)
[09:45:29.289] Build status: STARTED
[09:45:29.289] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:46:0) after 6m22s
[09:51:51.489] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:46:0)
[09:51:51.504] Build status: STARTED
[09:51:51.504] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:47:0) after 9m14s
[10:01:05.618] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:47:0)
[10:01:05.711] Build status: STARTED
[10:01:05.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:48:0) after 3m23s
[10:04:28.888] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:48:0)
[10:04:28.991] Build status: STARTED
[10:04:28.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:49:0) after 6m0s
[10:10:29.098] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:49:0)
[10:10:29.125] Build status: STARTED
[10:10:29.125] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:50:0) after 1m14s
[10:11:43.144] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:50:0)
[10:11:43.162] Build status: STARTED
[10:11:43.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:51:0) after 5m18s
[10:17:01.178] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:51:0)
[10:17:01.218] Build status: STARTED
[10:17:01.218] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:52:0) after 9m40s
[10:26:41.238] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:52:0)
[10:26:41.265] Build status: STARTED
[10:26:41.265] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:53:0) after 9m44s
[10:36:25.363] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:53:0)
[10:36:25.386] Build status: STARTED
[10:36:25.387] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:54:0) after 7m35s
[10:44:00.413] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:54:0)
[10:44:00.430] Build status: STARTED
[10:44:00.430] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:55:0) after 2m15s
[10:46:15.501] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:55:0)
[10:46:15.526] Build status: STARTED
[10:46:15.526] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:56:0) after 9m47s
[10:56:02.543] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:56:0)
[10:56:02.587] Build status: STARTED
[10:56:02.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:57:0) after 2m17s
[10:58:19.601] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:57:0)
[10:58:19.617] Build status: STARTED
[10:58:19.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:58:0) after 2m12s
[11:00:31.682] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:58:0)
[11:00:31.697] Build status: STARTED
[11:00:31.697] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:59:0) after 2m12s
[11:02:43.712] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:59:0)
[11:02:43.726] Build status: STARTED
[11:02:43.726] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:60:0) after 9m28s
[11:12:11.818] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:60:0)
[11:12:11.887] Build status: STARTED
[11:12:11.887] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:61:0) after 2m37s
[11:14:49.161] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:61:0)
[11:14:49.176] Build status: STARTED
[11:14:49.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:62:0) after 4m33s
[11:19:22.193] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:62:0)
[11:19:22.248] Build status: STARTED
[11:19:22.248] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:63:0) after 3m35s
[11:22:57.386] Handling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:63:0)
[11:22:57.403] Build status: STARTED
[11:22:57.403] Scheduling timer "check-buildbucket-build-status" (chromium/ToTWinArm64PGO:8852174450552677088:64:0) after 8m7s
[11:25:13.436] Received PubSub notification, asking Buildbucket for the build status
[11:25:13.453] Build:
{
"id": "8686744388572072673",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTWinArm64PGO"
},
"number": 7745,
"createdBy": "project:chromium",
"createTime": "2026-03-21T05:46:45.640778046Z",
"startTime": "2026-03-21T05:46:52.288707951Z",
"endTime": "2026-03-21T11:25:12.843878590Z",
"updateTime": "2026-03-21T11:25:12.843878590Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9ca3cfa098cdfe7cc4c946f4436e747254e6288f",
"ref": "refs/heads/main"
}
}
}
[11:25:13.453] Invocation finished in 5h38m29.08477428s with status FAILED