[22:56:34.759] New invocation is queued and will start shortly
[22:56:36.165] Starting the invocation (attempt 1)
[22:56:36.188] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/58a8c057b2b3b113e41ef8d8fb1228eb05d5a10c"
[22:56:36.188] Popped gitiles commit info from properties and tags
[22:56:36.188] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[22:56:36.188] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[22:56:36.188] Buildbucket request:
{
"requestId": "8857545476620129712",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "58a8c057b2b3b113e41ef8d8fb1228eb05d5a10c",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8857545476620129712"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[22:56:36.828] Scheduled build:
{
"id": "8692115414370353361",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 1183,
"createdBy": "project:chromium",
"createTime": "2026-01-20T22:56:36.284036577Z",
"updateTime": "2026-01-20T22:56:36.284036577Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "58a8c057b2b3b113e41ef8d8fb1228eb05d5a10c",
"ref": "refs/heads/main"
}
}
}
[22:56:36.828] Task URL: https://cr-buildbucket.appspot.com/build/8692115414370353361
[22:56:36.828] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:2:0) after 4m40s
[22:56:36.935] Received PubSub notification, asking Buildbucket for the build status
[22:56:37.009] Build status: SCHEDULED
[22:57:34.723] Received PubSub notification, asking Buildbucket for the build status
[22:57:34.794] Build status: STARTED
[23:01:16.837] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:2:0)
[23:01:16.893] Build status: STARTED
[23:01:16.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:5:0) after 1m40s
[23:02:56.872] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:5:0)
[23:02:56.907] Build status: STARTED
[23:02:56.907] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:6:0) after 4m43s
[23:07:39.927] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:6:0)
[23:07:39.947] Build status: STARTED
[23:07:39.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:7:0) after 8m34s
[23:16:14.023] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:7:0)
[23:16:14.048] Build status: STARTED
[23:16:14.048] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:8:0) after 6m46s
[23:23:00.073] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:8:0)
[23:23:00.087] Build status: STARTED
[23:23:00.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:9:0) after 4m40s
[23:27:40.110] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:9:0)
[23:27:40.129] Build status: STARTED
[23:27:40.129] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:10:0) after 5m57s
[23:33:37.202] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:10:0)
[23:33:37.224] Build status: STARTED
[23:33:37.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:11:0) after 7m43s
[23:41:20.408] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:11:0)
[23:41:20.424] Build status: STARTED
[23:41:20.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:12:0) after 2m58s
[23:44:18.450] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:12:0)
[23:44:18.467] Build status: STARTED
[23:44:18.467] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:13:0) after 1m13s
[23:45:31.501] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:13:0)
[23:45:31.521] Build status: STARTED
[23:45:31.521] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:14:0) after 4m48s
[23:50:19.545] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:14:0)
[23:50:19.560] Build status: STARTED
[23:50:19.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:15:0) after 8m15s
[23:58:34.580] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:15:0)
[23:58:34.774] Build status: STARTED
[23:58:34.775] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:16:0) after 8m1s
[00:06:35.801] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:16:0)
[00:06:35.830] Build status: STARTED
[00:06:35.830] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:17:0) after 3m46s
[00:10:21.894] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:17:0)
[00:10:21.911] Build status: STARTED
[00:10:21.911] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:18:0) after 3m14s
[00:13:35.986] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:18:0)
[00:13:35.998] Build status: STARTED
[00:13:35.998] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:19:0) after 2m13s
[00:15:49.021] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:19:0)
[00:15:49.088] Build status: STARTED
[00:15:49.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:20:0) after 4m28s
[00:20:17.147] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:20:0)
[00:20:17.184] Build status: STARTED
[00:20:17.184] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:21:0) after 5m0s
[00:25:17.318] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:21:0)
[00:25:17.332] Build status: STARTED
[00:25:17.332] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:22:0) after 4m32s
[00:29:49.408] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:22:0)
[00:29:49.424] Build status: STARTED
[00:29:49.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:23:0) after 6m22s
[00:36:11.540] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:23:0)
[00:36:11.593] Build status: STARTED
[00:36:11.593] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:24:0) after 5m11s
[00:41:22.600] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:24:0)
[00:41:22.615] Build status: STARTED
[00:41:22.615] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:25:0) after 6m56s
[00:48:18.640] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:25:0)
[00:48:18.659] Build status: STARTED
[00:48:18.659] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:26:0) after 2m0s
[00:50:18.729] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:26:0)
[00:50:18.757] Build status: STARTED
[00:50:18.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:27:0) after 9m3s
[00:59:21.786] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:27:0)
[00:59:21.802] Build status: STARTED
[00:59:21.802] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:28:0) after 6m56s
[01:06:17.830] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:28:0)
[01:06:17.861] Build status: STARTED
[01:06:17.861] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:29:0) after 8m0s
[01:14:17.884] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:29:0)
[01:14:17.899] Build status: STARTED
[01:14:17.899] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:30:0) after 1m6s
[01:15:24.031] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:30:0)
[01:15:24.046] Build status: STARTED
[01:15:24.046] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:31:0) after 1m15s
[01:16:39.119] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:31:0)
[01:16:39.139] Build status: STARTED
[01:16:39.139] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:32:0) after 5m8s
[01:21:47.212] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:32:0)
[01:21:47.232] Build status: STARTED
[01:21:47.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:33:0) after 9m18s
[01:31:05.337] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:33:0)
[01:31:05.389] Build status: STARTED
[01:31:05.389] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:34:0) after 4m26s
[01:35:31.514] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:34:0)
[01:35:31.551] Build status: STARTED
[01:35:31.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:35:0) after 4m29s
[01:40:00.627] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:35:0)
[01:40:00.642] Build status: STARTED
[01:40:00.642] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:36:0) after 3m44s
[01:43:44.651] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:36:0)
[01:43:44.690] Build status: STARTED
[01:43:44.690] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:37:0) after 6m53s
[01:50:37.712] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:37:0)
[01:50:37.726] Build status: STARTED
[01:50:37.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:38:0) after 4m3s
[01:54:40.750] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:38:0)
[01:54:40.764] Build status: STARTED
[01:54:40.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:39:0) after 7m50s
[02:02:30.792] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:39:0)
[02:02:30.810] Build status: STARTED
[02:02:30.810] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:40:0) after 9m39s
[02:12:09.834] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:40:0)
[02:12:09.856] Build status: STARTED
[02:12:09.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:41:0) after 6m0s
[02:18:09.923] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:41:0)
[02:18:09.937] Build status: STARTED
[02:18:09.938] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:42:0) after 5m37s
[02:23:46.981] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:42:0)
[02:23:46.996] Build status: STARTED
[02:23:46.996] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:43:0) after 1m51s
[02:25:38.042] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:43:0)
[02:25:38.056] Build status: STARTED
[02:25:38.056] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:44:0) after 6m8s
[02:31:46.088] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:44:0)
[02:31:46.149] Build status: STARTED
[02:31:46.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:45:0) after 9m2s
[02:40:48.171] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:45:0)
[02:40:48.187] Build status: STARTED
[02:40:48.187] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:46:0) after 3m59s
[02:44:47.251] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:46:0)
[02:44:47.269] Build status: STARTED
[02:44:47.269] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:47:0) after 5m10s
[02:49:57.293] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:47:0)
[02:49:57.305] Build status: STARTED
[02:49:57.305] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:48:0) after 7m23s
[02:57:20.474] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:48:0)
[02:57:20.502] Build status: STARTED
[02:57:20.503] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:49:0) after 5m42s
[03:03:02.563] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:49:0)
[03:03:02.580] Build status: STARTED
[03:03:02.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:50:0) after 1m3s
[03:04:05.617] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:50:0)
[03:04:05.632] Build status: STARTED
[03:04:05.632] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:51:0) after 8m44s
[03:12:49.705] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:51:0)
[03:12:49.720] Build status: STARTED
[03:12:49.720] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:52:0) after 8m3s
[03:20:52.780] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:52:0)
[03:20:52.841] Build status: STARTED
[03:20:52.841] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:53:0) after 5m33s
[03:26:25.919] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:53:0)
[03:26:25.934] Build status: STARTED
[03:26:25.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:54:0) after 1m25s
[03:27:50.985] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:54:0)
[03:27:50.998] Build status: STARTED
[03:27:50.998] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:55:0) after 7m27s
[03:35:18.024] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:55:0)
[03:35:18.037] Build status: STARTED
[03:35:18.037] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:56:0) after 2m55s
[03:38:13.140] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:56:0)
[03:38:13.154] Build status: STARTED
[03:38:13.155] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:57:0) after 2m59s
[03:41:12.191] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:57:0)
[03:41:12.216] Build status: STARTED
[03:41:12.216] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:58:0) after 6m19s
[03:47:31.236] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:58:0)
[03:47:31.278] Build status: STARTED
[03:47:31.278] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:59:0) after 3m33s
[03:51:04.379] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:59:0)
[03:51:04.474] Build status: STARTED
[03:51:04.474] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:60:0) after 8m46s
[03:59:50.523] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:60:0)
[03:59:50.537] Build status: STARTED
[03:59:50.537] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:61:0) after 9m14s
[04:09:04.592] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:61:0)
[04:09:04.608] Build status: STARTED
[04:09:04.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:62:0) after 9m17s
[04:18:22.761] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:62:0)
[04:18:22.777] Build status: STARTED
[04:18:22.777] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:63:0) after 5m30s
[04:23:52.794] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:63:0)
[04:23:52.807] Build status: STARTED
[04:23:52.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857545476620129712:64:0) after 4m2s
[04:25:52.820] Received PubSub notification, asking Buildbucket for the build status
[04:25:52.834] Build:
{
"id": "8692115414370353361",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 1183,
"createdBy": "project:chromium",
"createTime": "2026-01-20T22:56:36.284036577Z",
"startTime": "2026-01-20T22:57:34.481085181Z",
"endTime": "2026-01-21T04:25:52.570691426Z",
"updateTime": "2026-01-21T04:25:52.570691426Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "58a8c057b2b3b113e41ef8d8fb1228eb05d5a10c",
"ref": "refs/heads/main"
}
}
}
[04:25:52.834] Invocation finished in 5h29m18.079166314s with status FAILED