[20:32:57.863] New invocation is queued and will start shortly
[20:32:59.521] Starting the invocation (attempt 1)
[20:32:59.548] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/849740784748e0d1188b07c00babee9868e26a73"
[20:32:59.548] Popped gitiles commit info from properties and tags
[20:32:59.548] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[20:32:59.548] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[20:32:59.548] Buildbucket request:
{
"requestId": "8862899733113509104",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "849740784748e0d1188b07c00babee9868e26a73",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8862899733113509104"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[20:33:00.056] Scheduled build:
{
"id": "8697469670619392737",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 268,
"createdBy": "project:chromium",
"createTime": "2025-11-22T20:32:59.615639260Z",
"updateTime": "2025-11-22T20:32:59.615639260Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "849740784748e0d1188b07c00babee9868e26a73",
"ref": "refs/heads/main"
}
},
"canary": true
}
[20:33:00.056] Task URL: https://cr-buildbucket.appspot.com/build/8697469670619392737
[20:33:00.056] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:2:0) after 8m27s
[20:33:00.569] Received PubSub notification, asking Buildbucket for the build status
[20:33:00.585] Build status: SCHEDULED
[20:33:03.056] Received PubSub notification, asking Buildbucket for the build status
[20:33:03.107] Build status: STARTED
[20:41:27.290] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:2:0)
[20:41:27.420] Build status: STARTED
[20:41:27.420] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:5:0) after 2m32s
[20:43:59.437] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:5:0)
[20:43:59.451] Build status: STARTED
[20:43:59.451] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:6:0) after 9m12s
[20:53:11.558] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:6:0)
[20:53:11.593] Build status: STARTED
[20:53:11.594] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:7:0) after 5m57s
[20:59:08.638] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:7:0)
[20:59:08.652] Build status: STARTED
[20:59:08.652] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:8:0) after 6m44s
[21:05:53.076] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:8:0)
[21:05:53.091] Build status: STARTED
[21:05:53.092] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:9:0) after 9m9s
[21:15:02.296] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:9:0)
[21:15:02.371] Build status: STARTED
[21:15:02.371] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:10:0) after 9m7s
[21:24:09.386] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:10:0)
[21:24:09.502] Build status: STARTED
[21:24:09.502] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:11:0) after 7m30s
[21:31:39.518] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:11:0)
[21:31:39.538] Build status: STARTED
[21:31:39.538] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:12:0) after 4m44s
[21:36:23.560] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:12:0)
[21:36:23.575] Build status: STARTED
[21:36:23.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:13:0) after 6m38s
[21:43:01.677] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:13:0)
[21:43:01.724] Build status: STARTED
[21:43:01.724] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:14:0) after 6m0s
[21:49:01.739] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:14:0)
[21:49:01.758] Build status: STARTED
[21:49:01.759] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:15:0) after 3m41s
[21:52:42.781] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:15:0)
[21:52:42.803] Build status: STARTED
[21:52:42.803] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:16:0) after 2m45s
[21:55:28.116] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:16:0)
[21:55:28.141] Build status: STARTED
[21:55:28.141] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:17:0) after 9m40s
[22:05:08.220] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:17:0)
[22:05:08.234] Build status: STARTED
[22:05:08.234] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:18:0) after 4m9s
[22:09:17.316] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:18:0)
[22:09:17.332] Build status: STARTED
[22:09:17.333] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:19:0) after 8m39s
[22:17:56.347] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:19:0)
[22:17:56.364] Build status: STARTED
[22:17:56.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:20:0) after 8m14s
[22:26:10.359] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:20:0)
[22:26:10.377] Build status: STARTED
[22:26:10.377] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:21:0) after 3m23s
[22:29:33.428] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:21:0)
[22:29:33.444] Build status: STARTED
[22:29:33.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:22:0) after 6m14s
[22:35:47.566] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:22:0)
[22:35:47.582] Build status: STARTED
[22:35:47.582] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:23:0) after 4m43s
[22:40:30.597] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:23:0)
[22:40:30.623] Build status: STARTED
[22:40:30.623] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:24:0) after 6m21s
[22:46:51.641] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:24:0)
[22:46:51.658] Build status: STARTED
[22:46:51.658] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:25:0) after 5m49s
[22:52:40.772] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:25:0)
[22:52:40.787] Build status: STARTED
[22:52:40.787] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:26:0) after 6m57s
[22:59:37.879] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:26:0)
[22:59:37.896] Build status: STARTED
[22:59:37.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:27:0) after 6m12s
[23:05:49.910] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:27:0)
[23:05:49.922] Build status: STARTED
[23:05:49.922] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:28:0) after 6m42s
[23:12:32.037] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:28:0)
[23:12:32.053] Build status: STARTED
[23:12:32.053] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:29:0) after 4m8s
[23:16:40.065] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:29:0)
[23:16:40.081] Build status: STARTED
[23:16:40.081] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:30:0) after 9m6s
[23:25:46.218] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:30:0)
[23:25:46.238] Build status: STARTED
[23:25:46.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:31:0) after 9m8s
[23:34:54.357] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:31:0)
[23:34:54.371] Build status: STARTED
[23:34:54.371] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:32:0) after 9m6s
[23:44:00.386] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:32:0)
[23:44:00.409] Build status: STARTED
[23:44:00.409] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:33:0) after 4m41s
[23:48:41.513] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:33:0)
[23:48:41.607] Build status: STARTED
[23:48:41.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:34:0) after 6m22s
[23:55:03.801] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:34:0)
[23:55:03.978] Build status: STARTED
[23:55:03.978] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:35:0) after 5m27s
[00:00:31.113] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:35:0)
[00:00:31.133] Build status: STARTED
[00:00:31.133] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:36:0) after 7m52s
[00:08:23.160] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:36:0)
[00:08:23.177] Build status: STARTED
[00:08:23.177] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:37:0) after 3m44s
[00:12:07.295] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:37:0)
[00:12:07.309] Build status: STARTED
[00:12:07.309] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:38:0) after 8m44s
[00:20:51.323] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:38:0)
[00:20:51.339] Build status: STARTED
[00:20:51.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:39:0) after 8m24s
[00:29:15.551] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:39:0)
[00:29:15.604] Build status: STARTED
[00:29:15.605] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:40:0) after 7m51s
[00:37:06.967] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:40:0)
[00:37:06.985] Build status: STARTED
[00:37:06.985] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:41:0) after 4m48s
[00:41:54.998] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:41:0)
[00:41:55.014] Build status: STARTED
[00:41:55.014] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:42:0) after 1m8s
[00:43:03.029] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:42:0)
[00:43:03.043] Build status: STARTED
[00:43:03.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:43:0) after 8m10s
[00:51:13.280] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:43:0)
[00:51:13.297] Build status: STARTED
[00:51:13.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:44:0) after 4m14s
[00:55:27.489] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:44:0)
[00:55:27.617] Build status: STARTED
[00:55:27.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:45:0) after 3m52s
[00:59:19.823] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:45:0)
[00:59:19.843] Build status: STARTED
[00:59:19.843] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:46:0) after 9m9s
[01:08:29.370] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:46:0)
[01:08:29.392] Build status: STARTED
[01:08:29.392] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:47:0) after 5m21s
[01:13:50.480] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:47:0)
[01:13:50.497] Build status: STARTED
[01:13:50.497] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:48:0) after 7m27s
[01:21:17.532] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:48:0)
[01:21:17.550] Build status: STARTED
[01:21:17.550] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:49:0) after 7m25s
[01:28:42.636] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:49:0)
[01:28:42.821] Build status: STARTED
[01:28:42.821] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:50:0) after 2m31s
[01:31:13.872] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:50:0)
[01:31:13.892] Build status: STARTED
[01:31:13.892] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:51:0) after 6m24s
[01:37:38.053] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:51:0)
[01:37:38.068] Build status: STARTED
[01:37:38.068] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:52:0) after 6m12s
[01:43:50.186] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:52:0)
[01:43:50.202] Build status: STARTED
[01:43:50.202] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:53:0) after 6m55s
[01:50:45.548] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:53:0)
[01:50:45.562] Build status: STARTED
[01:50:45.562] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:54:0) after 9m51s
[02:00:36.652] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:54:0)
[02:00:36.676] Build status: STARTED
[02:00:36.676] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:55:0) after 5m31s
[02:06:07.887] Handling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:55:0)
[02:06:07.905] Build status: STARTED
[02:06:07.906] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacPGO:8862899733113509104:56:0) after 7m0s
[02:11:37.676] Received PubSub notification, asking Buildbucket for the build status
[02:11:37.756] Build:
{
"id": "8697469670619392737",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacPGO"
},
"number": 268,
"createdBy": "project:chromium",
"createTime": "2025-11-22T20:32:59.615639260Z",
"startTime": "2025-11-22T20:33:02.762862169Z",
"endTime": "2025-11-23T02:11:36.044540609Z",
"updateTime": "2025-11-23T02:11:36.044540609Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "849740784748e0d1188b07c00babee9868e26a73",
"ref": "refs/heads/main"
}
},
"canary": true
}
[02:11:37.756] Invocation finished in 5h38m39.903081334s with status FAILED