[13:49:30.744] New invocation is queued and will start shortly
[13:49:31.946] Starting the invocation (attempt 1)
[13:49:31.980] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/90885c74834f1409e7f8fb7eea08c8f3d7707206"
[13:49:31.980] Popped gitiles commit info from properties and tags
[13:49:31.980] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[13:49:31.980] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[13:49:31.980] Buildbucket request:
{
"requestId": "8861837952513711488",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "90885c74834f1409e7f8fb7eea08c8f3d7707206",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8861837952513711488"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[13:49:32.418] Scheduled build:
{
"id": "8696407890518484193",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 531,
"createdBy": "project:chromium",
"createTime": "2025-12-04T13:49:32.027164463Z",
"updateTime": "2025-12-04T13:49:32.027164463Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "90885c74834f1409e7f8fb7eea08c8f3d7707206",
"ref": "refs/heads/main"
}
}
}
[13:49:32.418] Task URL: https://cr-buildbucket.appspot.com/build/8696407890518484193
[13:49:32.418] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:2:0) after 1m48s
[13:49:33.425] Received PubSub notification, asking Buildbucket for the build status
[13:49:33.451] Build status: SCHEDULED
[13:49:36.592] Received PubSub notification, asking Buildbucket for the build status
[13:49:36.642] Build status: STARTED
[13:51:20.449] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:2:0)
[13:51:20.467] Build status: STARTED
[13:51:20.467] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:5:0) after 2m58s
[13:54:18.515] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:5:0)
[13:54:18.539] Build status: STARTED
[13:54:18.539] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:6:0) after 6m24s
[14:00:42.563] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:6:0)
[14:00:42.577] Build status: STARTED
[14:00:42.577] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:7:0) after 7m11s
[14:07:53.569] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:7:0)
[14:07:53.601] Build status: STARTED
[14:07:53.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:8:0) after 6m11s
[14:14:04.588] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:8:0)
[14:14:04.603] Build status: STARTED
[14:14:04.603] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:9:0) after 3m56s
[14:18:00.619] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:9:0)
[14:18:00.633] Build status: STARTED
[14:18:00.633] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:10:0) after 8m46s
[14:26:46.657] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:10:0)
[14:26:46.673] Build status: STARTED
[14:26:46.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:11:0) after 3m44s
[14:30:30.739] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:11:0)
[14:30:30.756] Build status: STARTED
[14:30:30.756] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:12:0) after 6m11s
[14:36:41.793] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:12:0)
[14:36:41.806] Build status: STARTED
[14:36:41.806] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:13:0) after 2m27s
[14:39:08.824] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:13:0)
[14:39:08.850] Build status: STARTED
[14:39:08.850] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:14:0) after 4m48s
[14:43:56.907] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:14:0)
[14:43:56.927] Build status: STARTED
[14:43:56.927] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:15:0) after 3m35s
[14:47:31.940] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:15:0)
[14:47:31.966] Build status: STARTED
[14:47:31.966] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:16:0) after 1m15s
[14:48:46.981] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:16:0)
[14:48:47.002] Build status: STARTED
[14:48:47.002] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:17:0) after 8m37s
[14:57:24.018] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:17:0)
[14:57:24.036] Build status: STARTED
[14:57:24.036] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:18:0) after 3m57s
[15:01:21.065] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:18:0)
[15:01:21.085] Build status: STARTED
[15:01:21.085] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:19:0) after 9m29s
[15:10:50.101] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:19:0)
[15:10:50.120] Build status: STARTED
[15:10:50.120] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:20:0) after 5m41s
[15:16:31.137] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:20:0)
[15:16:31.154] Build status: STARTED
[15:16:31.154] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:21:0) after 7m1s
[15:23:32.230] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:21:0)
[15:23:32.250] Build status: STARTED
[15:23:32.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:22:0) after 8m51s
[15:32:23.233] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:22:0)
[15:32:23.250] Build status: STARTED
[15:32:23.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:23:0) after 9m25s
[15:41:48.359] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:23:0)
[15:41:48.383] Build status: STARTED
[15:41:48.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:24:0) after 2m54s
[15:44:42.474] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:24:0)
[15:44:42.490] Build status: STARTED
[15:44:42.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:25:0) after 9m57s
[15:54:39.624] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:25:0)
[15:54:39.646] Build status: STARTED
[15:54:39.646] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:26:0) after 4m38s
[15:59:17.759] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:26:0)
[15:59:17.774] Build status: STARTED
[15:59:17.774] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:27:0) after 9m42s
[16:08:59.795] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:27:0)
[16:08:59.830] Build status: STARTED
[16:08:59.830] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:28:0) after 9m17s
[16:18:16.848] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:28:0)
[16:18:16.863] Build status: STARTED
[16:18:16.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:29:0) after 6m27s
[16:24:43.929] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:29:0)
[16:24:43.945] Build status: STARTED
[16:24:43.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:30:0) after 1m33s
[16:26:16.961] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:30:0)
[16:26:16.972] Build status: STARTED
[16:26:16.972] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:31:0) after 7m2s
[16:33:19.020] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:31:0)
[16:33:19.039] Build status: STARTED
[16:33:19.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:32:0) after 8m13s
[16:41:32.072] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:32:0)
[16:41:32.088] Build status: STARTED
[16:41:32.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:33:0) after 2m51s
[16:44:23.109] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:33:0)
[16:44:23.140] Build status: STARTED
[16:44:23.140] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:34:0) after 5m21s
[16:49:44.217] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:34:0)
[16:49:44.234] Build status: STARTED
[16:49:44.234] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:35:0) after 4m38s
[16:54:22.251] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:35:0)
[16:54:22.275] Build status: STARTED
[16:54:22.275] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:36:0) after 8m58s
[17:03:20.293] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:36:0)
[17:03:20.309] Build status: STARTED
[17:03:20.309] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:37:0) after 4m46s
[17:08:06.424] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:37:0)
[17:08:06.437] Build status: STARTED
[17:08:06.437] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:38:0) after 2m11s
[17:10:17.500] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:38:0)
[17:10:17.521] Build status: STARTED
[17:10:17.521] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:39:0) after 9m15s
[17:19:32.620] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:39:0)
[17:19:32.636] Build status: STARTED
[17:19:32.636] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:40:0) after 5m57s
[17:25:29.624] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:40:0)
[17:25:29.640] Build status: STARTED
[17:25:29.640] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:41:0) after 7m1s
[17:32:30.682] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:41:0)
[17:32:31.459] Build status: STARTED
[17:32:31.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:42:0) after 1m17s
[17:33:48.527] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:42:0)
[17:33:48.541] Build status: STARTED
[17:33:48.541] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:43:0) after 9m13s
[17:43:01.558] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:43:0)
[17:43:01.593] Build status: STARTED
[17:43:01.593] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:44:0) after 2m51s
[17:45:52.636] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:44:0)
[17:45:52.658] Build status: STARTED
[17:45:52.658] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:45:0) after 3m39s
[17:49:31.723] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:45:0)
[17:49:31.737] Build status: STARTED
[17:49:31.737] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:46:0) after 2m39s
[17:52:10.877] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:46:0)
[17:52:10.892] Build status: STARTED
[17:52:10.892] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:47:0) after 6m44s
[17:58:54.904] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:47:0)
[17:58:54.928] Build status: STARTED
[17:58:54.928] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:48:0) after 9m51s
[18:08:45.984] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:48:0)
[18:08:45.998] Build status: STARTED
[18:08:45.998] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:49:0) after 7m42s
[18:16:28.150] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:49:0)
[18:16:28.178] Build status: STARTED
[18:16:28.179] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:50:0) after 6m38s
[18:23:06.205] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:50:0)
[18:23:06.218] Build status: STARTED
[18:23:06.218] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:51:0) after 4m40s
[18:27:46.251] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:51:0)
[18:27:46.271] Build status: STARTED
[18:27:46.271] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:52:0) after 7m53s
[18:35:39.641] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:52:0)
[18:35:39.732] Build status: STARTED
[18:35:39.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:53:0) after 5m6s
[18:40:45.781] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:53:0)
[18:40:45.797] Build status: STARTED
[18:40:45.797] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:54:0) after 9m48s
[18:50:33.825] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:54:0)
[18:50:33.844] Build status: STARTED
[18:50:33.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:55:0) after 7m8s
[18:57:41.929] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:55:0)
[18:57:41.945] Build status: STARTED
[18:57:41.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:56:0) after 9m7s
[19:06:49.108] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:56:0)
[19:06:49.123] Build status: STARTED
[19:06:49.124] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:57:0) after 8m25s
[19:15:14.369] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:57:0)
[19:15:14.394] Build status: STARTED
[19:15:14.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8861837952513711488:58:0) after 9m19s
[19:17:49.062] Received PubSub notification, asking Buildbucket for the build status
[19:17:49.079] Build:
{
"id": "8696407890518484193",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 531,
"createdBy": "project:chromium",
"createTime": "2025-12-04T13:49:32.027164463Z",
"startTime": "2025-12-04T13:49:36.348056934Z",
"endTime": "2025-12-04T19:17:48.819528186Z",
"updateTime": "2025-12-04T19:17:48.819528186Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "90885c74834f1409e7f8fb7eea08c8f3d7707206",
"ref": "refs/heads/main"
}
}
}
[19:17:49.079] Invocation finished in 5h28m18.33717671s with status FAILED