[02:32:10.706] New invocation is queued and will start shortly
[02:32:12.017] Starting the invocation (attempt 1)
[02:32:12.043] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/b690e337da85b3ac4ebb293d2376186efe8f017b"
[02:32:12.043] Popped gitiles commit info from properties and tags
[02:32:12.043] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[02:32:12.046] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[02:32:12.046] Buildbucket request:
{
"requestId": "8858256688035441904",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b690e337da85b3ac4ebb293d2376186efe8f017b",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8858256688035441904"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[02:32:13.354] Scheduled build:
{
"id": "8692826625850488145",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 1136,
"createdBy": "project:chromium",
"createTime": "2026-01-13T02:32:12.161684454Z",
"updateTime": "2026-01-13T02:32:12.161684454Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b690e337da85b3ac4ebb293d2376186efe8f017b",
"ref": "refs/heads/main"
}
}
}
[02:32:13.354] Task URL: https://cr-buildbucket.appspot.com/build/8692826625850488145
[02:32:13.354] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:2:0) after 7m54s
[02:32:14.208] Received PubSub notification, asking Buildbucket for the build status
[02:32:14.230] Build status: SCHEDULED
[02:33:33.130] Received PubSub notification, asking Buildbucket for the build status
[02:33:33.158] Build status: STARTED
[02:40:07.382] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:2:0)
[02:40:07.415] Build status: STARTED
[02:40:07.416] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:5:0) after 2m50s
[02:42:57.538] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:5:0)
[02:42:57.562] Build status: STARTED
[02:42:57.562] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:6:0) after 6m22s
[02:49:19.583] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:6:0)
[02:49:19.609] Build status: STARTED
[02:49:19.609] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:7:0) after 5m49s
[02:55:08.627] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:7:0)
[02:55:08.643] Build status: STARTED
[02:55:08.643] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:8:0) after 5m27s
[03:00:35.666] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:8:0)
[03:00:35.910] Build status: STARTED
[03:00:35.910] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:9:0) after 5m8s
[03:05:43.977] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:9:0)
[03:05:43.990] Build status: STARTED
[03:05:43.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:10:0) after 1m3s
[03:06:47.022] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:10:0)
[03:06:47.082] Build status: STARTED
[03:06:47.082] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:11:0) after 6m53s
[03:13:40.105] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:11:0)
[03:13:40.121] Build status: STARTED
[03:13:40.121] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:12:0) after 6m53s
[03:20:33.141] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:12:0)
[03:20:33.302] Build status: STARTED
[03:20:33.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:13:0) after 7m48s
[03:28:21.461] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:13:0)
[03:28:21.481] Build status: STARTED
[03:28:21.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:14:0) after 8m2s
[03:36:23.552] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:14:0)
[03:36:23.572] Build status: STARTED
[03:36:23.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:15:0) after 3m26s
[03:39:49.654] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:15:0)
[03:39:49.678] Build status: STARTED
[03:39:49.678] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:16:0) after 4m37s
[03:44:26.651] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:16:0)
[03:44:26.671] Build status: STARTED
[03:44:26.671] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:17:0) after 3m45s
[03:48:11.713] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:17:0)
[03:48:11.809] Build status: STARTED
[03:48:11.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:18:0) after 6m19s
[03:54:30.831] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:18:0)
[03:54:30.846] Build status: STARTED
[03:54:30.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:19:0) after 6m8s
[04:00:38.922] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:19:0)
[04:00:38.938] Build status: STARTED
[04:00:38.938] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:20:0) after 4m50s
[04:05:29.021] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:20:0)
[04:05:29.034] Build status: STARTED
[04:05:29.034] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:21:0) after 7m34s
[04:13:03.025] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:21:0)
[04:13:03.041] Build status: STARTED
[04:13:03.041] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:22:0) after 1m52s
[04:14:55.065] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:22:0)
[04:14:55.081] Build status: STARTED
[04:14:55.081] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:23:0) after 9m40s
[04:24:35.101] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:23:0)
[04:24:35.119] Build status: STARTED
[04:24:35.119] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:24:0) after 5m34s
[04:30:09.197] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:24:0)
[04:30:09.213] Build status: STARTED
[04:30:09.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:25:0) after 1m56s
[04:32:05.191] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:25:0)
[04:32:05.205] Build status: STARTED
[04:32:05.205] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:26:0) after 7m3s
[04:39:08.223] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:26:0)
[04:39:08.239] Build status: STARTED
[04:39:08.239] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:27:0) after 5m39s
[04:44:47.308] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:27:0)
[04:44:47.323] Build status: STARTED
[04:44:47.323] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:28:0) after 9m7s
[04:53:54.343] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:28:0)
[04:53:54.360] Build status: STARTED
[04:53:54.360] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:29:0) after 5m32s
[04:59:26.380] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:29:0)
[04:59:26.397] Build status: STARTED
[04:59:26.397] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:30:0) after 1m2s
[05:00:28.487] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:30:0)
[05:00:28.518] Build status: STARTED
[05:00:28.518] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:31:0) after 5m31s
[05:05:59.537] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:31:0)
[05:05:59.557] Build status: STARTED
[05:05:59.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:32:0) after 5m49s
[05:11:48.578] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:32:0)
[05:11:48.592] Build status: STARTED
[05:11:48.592] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:33:0) after 9m32s
[05:21:20.691] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:33:0)
[05:21:20.883] Build status: STARTED
[05:21:20.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:34:0) after 8m14s
[05:29:34.908] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:34:0)
[05:29:34.925] Build status: STARTED
[05:29:34.926] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:35:0) after 2m14s
[05:31:48.981] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:35:0)
[05:31:48.999] Build status: STARTED
[05:31:48.999] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:36:0) after 4m53s
[05:36:42.033] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:36:0)
[05:36:42.056] Build status: STARTED
[05:36:42.056] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:37:0) after 7m11s
[05:43:53.097] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:37:0)
[05:43:53.142] Build status: STARTED
[05:43:53.142] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:38:0) after 6m45s
[05:50:38.164] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:38:0)
[05:50:38.183] Build status: STARTED
[05:50:38.183] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:39:0) after 6m59s
[05:57:37.252] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:39:0)
[05:57:37.285] Build status: STARTED
[05:57:37.285] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:40:0) after 9m38s
[06:07:15.353] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:40:0)
[06:07:15.388] Build status: STARTED
[06:07:15.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:41:0) after 3m44s
[06:10:59.442] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:41:0)
[06:10:59.460] Build status: STARTED
[06:10:59.461] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:42:0) after 3m18s
[06:14:17.579] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:42:0)
[06:14:17.600] Build status: STARTED
[06:14:17.600] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:43:0) after 3m41s
[06:17:58.666] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:43:0)
[06:17:58.689] Build status: STARTED
[06:17:58.689] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:44:0) after 4m30s
[06:22:28.709] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:44:0)
[06:22:28.748] Build status: STARTED
[06:22:28.748] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:45:0) after 4m14s
[06:26:42.770] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:45:0)
[06:26:42.789] Build status: STARTED
[06:26:42.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:46:0) after 3m34s
[06:30:16.861] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:46:0)
[06:30:16.894] Build status: STARTED
[06:30:16.894] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:47:0) after 5m1s
[06:35:17.868] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:47:0)
[06:35:17.902] Build status: STARTED
[06:35:17.902] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:48:0) after 8m31s
[06:43:48.923] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:48:0)
[06:43:49.001] Build status: STARTED
[06:43:49.001] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:49:0) after 5m4s
[06:48:53.100] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:49:0)
[06:48:53.115] Build status: STARTED
[06:48:53.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:50:0) after 4m0s
[06:52:53.140] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:50:0)
[06:52:53.162] Build status: STARTED
[06:52:53.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:51:0) after 3m41s
[06:56:34.180] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:51:0)
[06:56:34.197] Build status: STARTED
[06:56:34.197] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:52:0) after 5m23s
[07:01:57.262] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:52:0)
[07:01:57.287] Build status: STARTED
[07:01:57.287] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:53:0) after 5m32s
[07:07:29.359] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:53:0)
[07:07:29.374] Build status: STARTED
[07:07:29.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:54:0) after 2m56s
[07:10:25.395] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:54:0)
[07:10:25.415] Build status: STARTED
[07:10:25.415] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:55:0) after 7m16s
[07:17:41.464] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:55:0)
[07:17:41.477] Build status: STARTED
[07:17:41.477] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:56:0) after 3m13s
[07:20:54.498] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:56:0)
[07:20:54.512] Build status: STARTED
[07:20:54.512] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:57:0) after 1m6s
[07:22:00.532] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:57:0)
[07:22:00.550] Build status: STARTED
[07:22:00.550] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:58:0) after 6m12s
[07:28:12.592] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:58:0)
[07:28:12.657] Build status: STARTED
[07:28:12.657] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:59:0) after 6m40s
[07:34:52.678] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:59:0)
[07:34:52.696] Build status: STARTED
[07:34:52.696] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:60:0) after 8m31s
[07:43:23.718] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:60:0)
[07:43:23.736] Build status: STARTED
[07:43:23.736] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:61:0) after 7m3s
[07:50:27.298] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:61:0)
[07:50:27.325] Build status: STARTED
[07:50:27.325] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8858256688035441904:62:0) after 5m9s
[07:51:51.701] Received PubSub notification, asking Buildbucket for the build status
[07:51:51.799] Build:
{
"id": "8692826625850488145",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 1136,
"createdBy": "project:chromium",
"createTime": "2026-01-13T02:32:12.161684454Z",
"startTime": "2026-01-13T02:33:27.641138385Z",
"endTime": "2026-01-13T07:51:51.502205329Z",
"updateTime": "2026-01-13T07:51:51.502205329Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b690e337da85b3ac4ebb293d2376186efe8f017b",
"ref": "refs/heads/main"
}
}
}
[07:51:51.799] Invocation finished in 5h19m41.105115091s with status FAILED