[14:18:08.307] New invocation is queued and will start shortly
[14:18:09.710] Starting the invocation (attempt 1)
[14:18:09.740] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/b8a45207967cf81ac1228ffb6734ed113f7509ef"
[14:18:09.741] Popped gitiles commit info from properties and tags
[14:18:09.741] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[14:18:09.741] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[14:18:09.741] Buildbucket request:
{
"requestId": "8857668691074193056",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b8a45207967cf81ac1228ffb6734ed113f7509ef",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8857668691074193056"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[14:18:10.938] Scheduled build:
{
"id": "8692238628803398049",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 1177,
"createdBy": "project:chromium",
"createTime": "2026-01-19T14:18:09.843507035Z",
"updateTime": "2026-01-19T14:18:09.843507035Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b8a45207967cf81ac1228ffb6734ed113f7509ef",
"ref": "refs/heads/main"
}
}
}
[14:18:10.938] Task URL: https://cr-buildbucket.appspot.com/build/8692238628803398049
[14:18:10.938] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:2:0) after 3m17s
[14:18:11.963] Received PubSub notification, asking Buildbucket for the build status
[14:18:12.038] Build status: SCHEDULED
[14:19:11.126] Received PubSub notification, asking Buildbucket for the build status
[14:19:11.144] Build status: STARTED
[14:21:27.957] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:2:0)
[14:21:27.991] Build status: STARTED
[14:21:27.991] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:5:0) after 6m5s
[14:27:33.063] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:5:0)
[14:27:33.079] Build status: STARTED
[14:27:33.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:6:0) after 4m20s
[14:31:53.096] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:6:0)
[14:31:53.113] Build status: STARTED
[14:31:53.113] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:7:0) after 6m30s
[14:38:23.136] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:7:0)
[14:38:23.154] Build status: STARTED
[14:38:23.154] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:8:0) after 5m26s
[14:43:49.177] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:8:0)
[14:43:49.191] Build status: STARTED
[14:43:49.191] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:9:0) after 5m38s
[14:49:27.208] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:9:0)
[14:49:27.227] Build status: STARTED
[14:49:27.227] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:10:0) after 8m49s
[14:58:16.244] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:10:0)
[14:58:16.259] Build status: STARTED
[14:58:16.259] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:11:0) after 3m26s
[15:01:42.289] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:11:0)
[15:01:42.304] Build status: STARTED
[15:01:42.304] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:12:0) after 8m41s
[15:10:23.337] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:12:0)
[15:10:23.352] Build status: STARTED
[15:10:23.352] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:13:0) after 2m11s
[15:12:34.373] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:13:0)
[15:12:35.816] Build status: STARTED
[15:12:35.816] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:14:0) after 1m23s
[15:13:58.836] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:14:0)
[15:13:58.855] Build status: STARTED
[15:13:58.855] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:15:0) after 1m26s
[15:15:24.915] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:15:0)
[15:15:24.927] Build status: STARTED
[15:15:24.927] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:16:0) after 6m12s
[15:21:36.947] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:16:0)
[15:21:36.961] Build status: STARTED
[15:21:36.961] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:17:0) after 6m37s
[15:28:14.038] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:17:0)
[15:28:14.052] Build status: STARTED
[15:28:14.052] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:18:0) after 6m21s
[15:34:35.217] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:18:0)
[15:34:35.264] Build status: STARTED
[15:34:35.264] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:19:0) after 5m9s
[15:39:44.286] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:19:0)
[15:39:44.309] Build status: STARTED
[15:39:44.309] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:20:0) after 1m2s
[15:40:46.384] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:20:0)
[15:40:46.395] Build status: STARTED
[15:40:46.395] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:21:0) after 7m22s
[15:48:08.409] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:21:0)
[15:48:08.454] Build status: STARTED
[15:48:08.454] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:22:0) after 9m24s
[15:57:32.478] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:22:0)
[15:57:32.495] Build status: STARTED
[15:57:32.495] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:23:0) after 8m32s
[16:06:04.643] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:23:0)
[16:06:04.710] Build status: STARTED
[16:06:04.710] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:24:0) after 7m7s
[16:13:11.778] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:24:0)
[16:13:11.806] Build status: STARTED
[16:13:11.806] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:25:0) after 2m30s
[16:15:41.827] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:25:0)
[16:15:41.844] Build status: STARTED
[16:15:41.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:26:0) after 1m46s
[16:17:27.815] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:26:0)
[16:17:27.833] Build status: STARTED
[16:17:27.833] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:27:0) after 3m42s
[16:21:09.929] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:27:0)
[16:21:09.944] Build status: STARTED
[16:21:09.944] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:28:0) after 4m52s
[16:26:01.971] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:28:0)
[16:26:02.087] Build status: STARTED
[16:26:02.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:29:0) after 4m25s
[16:30:27.074] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:29:0)
[16:30:27.089] Build status: STARTED
[16:30:27.089] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:30:0) after 8m50s
[16:39:17.111] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:30:0)
[16:39:17.126] Build status: STARTED
[16:39:17.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:31:0) after 7m49s
[16:47:06.148] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:31:0)
[16:47:06.164] Build status: STARTED
[16:47:06.164] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:32:0) after 2m13s
[16:49:19.185] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:32:0)
[16:49:19.202] Build status: STARTED
[16:49:19.202] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:33:0) after 6m7s
[16:55:26.320] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:33:0)
[16:55:26.338] Build status: STARTED
[16:55:26.338] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:34:0) after 2m50s
[16:58:16.402] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:34:0)
[16:58:16.417] Build status: STARTED
[16:58:16.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:35:0) after 4m31s
[17:02:47.471] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:35:0)
[17:02:47.522] Build status: STARTED
[17:02:47.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:36:0) after 7m16s
[17:10:03.598] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:36:0)
[17:10:03.659] Build status: STARTED
[17:10:03.659] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:37:0) after 4m15s
[17:14:18.736] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:37:0)
[17:14:18.757] Build status: STARTED
[17:14:18.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:38:0) after 3m31s
[17:17:49.755] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:38:0)
[17:17:49.770] Build status: STARTED
[17:17:49.771] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:39:0) after 7m20s
[17:25:09.810] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:39:0)
[17:25:09.839] Build status: STARTED
[17:25:09.839] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:40:0) after 5m1s
[17:30:10.894] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:40:0)
[17:30:10.910] Build status: STARTED
[17:30:10.910] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:41:0) after 3m2s
[17:33:12.917] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:41:0)
[17:33:12.987] Build status: STARTED
[17:33:12.987] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:42:0) after 1m59s
[17:35:12.043] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:42:0)
[17:35:12.066] Build status: STARTED
[17:35:12.066] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:43:0) after 6m27s
[17:41:39.098] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:43:0)
[17:41:39.119] Build status: STARTED
[17:41:39.119] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:44:0) after 4m6s
[17:45:45.145] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:44:0)
[17:45:45.161] Build status: STARTED
[17:45:45.161] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:45:0) after 3m21s
[17:49:06.499] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:45:0)
[17:49:06.518] Build status: STARTED
[17:49:06.518] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:46:0) after 4m42s
[17:53:48.541] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:46:0)
[17:53:48.558] Build status: STARTED
[17:53:48.558] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:47:0) after 5m40s
[17:59:28.715] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:47:0)
[17:59:28.729] Build status: STARTED
[17:59:28.729] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:48:0) after 6m38s
[18:06:07.438] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:48:0)
[18:06:07.455] Build status: STARTED
[18:06:07.455] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:49:0) after 7m10s
[18:13:17.529] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:49:0)
[18:13:17.556] Build status: STARTED
[18:13:17.556] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:50:0) after 7m57s
[18:21:14.597] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:50:0)
[18:21:14.619] Build status: STARTED
[18:21:14.619] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:51:0) after 7m8s
[18:28:22.630] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:51:0)
[18:28:22.641] Build status: STARTED
[18:28:22.641] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:52:0) after 5m11s
[18:33:33.718] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:52:0)
[18:33:33.740] Build status: STARTED
[18:33:33.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:53:0) after 1m29s
[18:35:02.830] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:53:0)
[18:35:02.875] Build status: STARTED
[18:35:02.875] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:54:0) after 5m52s
[18:40:54.893] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:54:0)
[18:40:54.934] Build status: STARTED
[18:40:54.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:55:0) after 7m5s
[18:47:59.984] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:55:0)
[18:48:00.213] Build status: STARTED
[18:48:00.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:56:0) after 1m46s
[18:49:46.319] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:56:0)
[18:49:46.336] Build status: STARTED
[18:49:46.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:57:0) after 5m5s
[18:54:51.355] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:57:0)
[18:54:51.370] Build status: STARTED
[18:54:51.370] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:58:0) after 4m52s
[18:59:43.509] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:58:0)
[18:59:43.595] Build status: STARTED
[18:59:43.595] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:59:0) after 2m49s
[19:02:32.616] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:59:0)
[19:02:32.640] Build status: STARTED
[19:02:32.640] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:60:0) after 9m51s
[19:12:23.682] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:60:0)
[19:12:23.698] Build status: STARTED
[19:12:23.698] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:61:0) after 5m18s
[19:17:41.722] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:61:0)
[19:17:41.807] Build status: STARTED
[19:17:41.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:62:0) after 8m31s
[19:26:13.146] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:62:0)
[19:26:13.263] Build status: STARTED
[19:26:13.263] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:63:0) after 6m37s
[19:32:50.464] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:63:0)
[19:32:50.503] Build status: STARTED
[19:32:50.503] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8857668691074193056:64:0) after 9m23s
[19:39:33.093] Received PubSub notification, asking Buildbucket for the build status
[19:39:33.111] Build:
{
"id": "8692238628803398049",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 1177,
"createdBy": "project:chromium",
"createTime": "2026-01-19T14:18:09.843507035Z",
"startTime": "2026-01-19T14:19:10.834686148Z",
"endTime": "2026-01-19T19:39:32.405992446Z",
"updateTime": "2026-01-19T19:39:32.405992446Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b8a45207967cf81ac1228ffb6734ed113f7509ef",
"ref": "refs/heads/main"
}
}
}
[19:39:33.111] Invocation finished in 5h21m24.816585673s with status FAILED