[14:19:31.030] New invocation is queued and will start shortly
[14:19:32.217] Starting the invocation (attempt 1)
[14:19:32.241] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/befcbbf59b7c8c7f67701dcd3e8c4f948a545e4e"
[14:19:32.241] Popped gitiles commit info from properties and tags
[14:19:32.241] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[14:19:32.242] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[14:19:32.242] Buildbucket request:
{
"requestId": "8856400246802767600",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "befcbbf59b7c8c7f67701dcd3e8c4f948a545e4e",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8856400246802767600"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[14:19:32.770] Scheduled build:
{
"id": "8690970184805663921",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 1295,
"createdBy": "project:chromium",
"createTime": "2026-02-02T14:19:32.306694573Z",
"updateTime": "2026-02-02T14:19:32.306694573Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "befcbbf59b7c8c7f67701dcd3e8c4f948a545e4e",
"ref": "refs/heads/main"
}
},
"canary": true
}
[14:19:32.770] Task URL: https://cr-buildbucket.appspot.com/build/8690970184805663921
[14:19:32.770] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:2:0) after 9m20s
[14:19:33.814] Received PubSub notification, asking Buildbucket for the build status
[14:19:33.875] Build status: SCHEDULED
[14:19:36.403] Received PubSub notification, asking Buildbucket for the build status
[14:19:36.688] Build status: STARTED
[14:28:52.821] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:2:0)
[14:28:52.837] Build status: STARTED
[14:28:52.837] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:5:0) after 7m54s
[14:36:46.918] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:5:0)
[14:36:46.947] Build status: STARTED
[14:36:46.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:6:0) after 7m45s
[14:44:31.944] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:6:0)
[14:44:31.960] Build status: STARTED
[14:44:31.960] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:7:0) after 6m3s
[14:50:35.000] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:7:0)
[14:50:35.027] Build status: STARTED
[14:50:35.027] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:8:0) after 5m48s
[14:56:23.016] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:8:0)
[14:56:23.040] Build status: STARTED
[14:56:23.041] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:9:0) after 9m16s
[15:05:39.098] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:9:0)
[15:05:39.115] Build status: STARTED
[15:05:39.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:10:0) after 7m7s
[15:12:46.232] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:10:0)
[15:12:46.247] Build status: STARTED
[15:12:46.248] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:11:0) after 9m8s
[15:21:54.334] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:11:0)
[15:21:54.371] Build status: STARTED
[15:21:54.371] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:12:0) after 7m35s
[15:29:29.436] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:12:0)
[15:29:29.454] Build status: STARTED
[15:29:29.454] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:13:0) after 8m32s
[15:38:01.562] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:13:0)
[15:38:01.624] Build status: STARTED
[15:38:01.625] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:14:0) after 9m18s
[15:47:19.657] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:14:0)
[15:47:19.700] Build status: STARTED
[15:47:19.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:15:0) after 5m40s
[15:52:59.724] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:15:0)
[15:52:59.737] Build status: STARTED
[15:52:59.737] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:16:0) after 8m4s
[16:01:03.768] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:16:0)
[16:01:03.845] Build status: STARTED
[16:01:03.845] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:17:0) after 5m23s
[16:06:26.913] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:17:0)
[16:06:26.927] Build status: STARTED
[16:06:26.927] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:18:0) after 3m32s
[16:09:58.978] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:18:0)
[16:09:59.069] Build status: STARTED
[16:09:59.069] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:19:0) after 3m8s
[16:13:07.167] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:19:0)
[16:13:07.181] Build status: STARTED
[16:13:07.182] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:20:0) after 5m0s
[16:18:07.200] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:20:0)
[16:18:07.215] Build status: STARTED
[16:18:07.215] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:21:0) after 9m8s
[16:27:15.314] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:21:0)
[16:27:15.413] Build status: STARTED
[16:27:15.413] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:22:0) after 2m3s
[16:29:18.428] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:22:0)
[16:29:18.440] Build status: STARTED
[16:29:18.440] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:23:0) after 1m53s
[16:31:11.458] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:23:0)
[16:31:11.477] Build status: STARTED
[16:31:11.477] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:24:0) after 1m12s
[16:32:23.545] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:24:0)
[16:32:23.624] Build status: STARTED
[16:32:23.624] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:25:0) after 4m34s
[16:36:57.666] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:25:0)
[16:36:57.698] Build status: STARTED
[16:36:57.698] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:26:0) after 6m16s
[16:43:13.721] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:26:0)
[16:43:13.754] Build status: STARTED
[16:43:13.754] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:27:0) after 4m47s
[16:48:00.904] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:27:0)
[16:48:00.924] Build status: STARTED
[16:48:00.924] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:28:0) after 5m57s
[16:53:57.909] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:28:0)
[16:53:57.923] Build status: STARTED
[16:53:57.923] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:29:0) after 5m28s
[16:59:25.894] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:29:0)
[16:59:25.909] Build status: STARTED
[16:59:25.909] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:30:0) after 6m29s
[17:05:54.976] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:30:0)
[17:05:54.994] Build status: STARTED
[17:05:54.994] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:31:0) after 3m39s
[17:09:34.016] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:31:0)
[17:09:34.030] Build status: STARTED
[17:09:34.030] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:32:0) after 8m1s
[17:17:35.116] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:32:0)
[17:17:35.134] Build status: STARTED
[17:17:35.134] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:33:0) after 1m32s
[17:19:07.209] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:33:0)
[17:19:07.233] Build status: STARTED
[17:19:07.233] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:34:0) after 8m27s
[17:27:34.252] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:34:0)
[17:27:34.320] Build status: STARTED
[17:27:34.320] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:35:0) after 1m57s
[17:29:31.397] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:35:0)
[17:29:31.409] Build status: STARTED
[17:29:31.409] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:36:0) after 8m17s
[17:37:48.498] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:36:0)
[17:37:48.512] Build status: STARTED
[17:37:48.512] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:37:0) after 6m43s
[17:44:31.532] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:37:0)
[17:44:31.596] Build status: STARTED
[17:44:31.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:38:0) after 3m15s
[17:47:46.619] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:38:0)
[17:47:46.641] Build status: STARTED
[17:47:46.641] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:39:0) after 9m23s
[17:57:09.690] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:39:0)
[17:57:09.777] Build status: STARTED
[17:57:09.777] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:40:0) after 4m21s
[18:01:30.767] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:40:0)
[18:01:30.798] Build status: STARTED
[18:01:30.798] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:41:0) after 9m32s
[18:11:02.822] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:41:0)
[18:11:02.842] Build status: STARTED
[18:11:02.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:42:0) after 9m10s
[18:20:12.894] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:42:0)
[18:20:12.925] Build status: STARTED
[18:20:12.925] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:43:0) after 3m40s
[18:23:52.975] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:43:0)
[18:23:52.991] Build status: STARTED
[18:23:52.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:44:0) after 9m48s
[18:33:41.061] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:44:0)
[18:33:41.097] Build status: STARTED
[18:33:41.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:45:0) after 5m58s
[18:39:39.130] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:45:0)
[18:39:39.153] Build status: STARTED
[18:39:39.153] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:46:0) after 3m56s
[18:43:35.228] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:46:0)
[18:43:35.243] Build status: STARTED
[18:43:35.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:47:0) after 2m28s
[18:46:03.262] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:47:0)
[18:46:03.274] Build status: STARTED
[18:46:03.274] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:48:0) after 4m50s
[18:50:53.353] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:48:0)
[18:50:53.388] Build status: STARTED
[18:50:53.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:49:0) after 6m56s
[18:57:49.421] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:49:0)
[18:57:49.439] Build status: STARTED
[18:57:49.439] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:50:0) after 6m46s
[19:04:35.505] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:50:0)
[19:04:35.522] Build status: STARTED
[19:04:35.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:51:0) after 9m51s
[19:14:26.694] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:51:0)
[19:14:26.746] Build status: STARTED
[19:14:26.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:52:0) after 3m59s
[19:18:25.814] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:52:0)
[19:18:25.845] Build status: STARTED
[19:18:25.845] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:53:0) after 6m41s
[19:25:06.924] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:53:0)
[19:25:06.939] Build status: STARTED
[19:25:06.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:54:0) after 4m16s
[19:29:23.060] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:54:0)
[19:29:23.074] Build status: STARTED
[19:29:23.074] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:55:0) after 7m5s
[19:36:28.200] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:55:0)
[19:36:28.220] Build status: STARTED
[19:36:28.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:56:0) after 6m17s
[19:42:45.213] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:56:0)
[19:42:45.288] Build status: STARTED
[19:42:45.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:57:0) after 6m50s
[19:49:35.322] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:57:0)
[19:49:35.343] Build status: STARTED
[19:49:35.343] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:58:0) after 2m47s
[19:52:22.350] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:58:0)
[19:52:22.421] Build status: STARTED
[19:52:22.421] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8856400246802767600:59:0) after 7m13s
[19:54:41.345] Received PubSub notification, asking Buildbucket for the build status
[19:54:41.361] Build:
{
"id": "8690970184805663921",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 1295,
"createdBy": "project:chromium",
"createTime": "2026-02-02T14:19:32.306694573Z",
"startTime": "2026-02-02T14:19:36.080931622Z",
"endTime": "2026-02-02T19:54:41.065952867Z",
"updateTime": "2026-02-02T19:54:41.065952867Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "befcbbf59b7c8c7f67701dcd3e8c4f948a545e4e",
"ref": "refs/heads/main"
}
},
"canary": true
}
[19:54:41.361] Invocation finished in 5h35m10.343868126s with status FAILED