[17:03:06.197] New invocation is queued and will start shortly
[17:03:07.422] Starting the invocation (attempt 1)
[17:03:07.449] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0bc27a53cc5eb04c10119416e5736708fb6c0109"
[17:03:07.449] Popped gitiles commit info from properties and tags
[17:03:07.449] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[17:03:07.450] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[17:03:07.450] Buildbucket request:
{
"requestId": "8843615782587273728",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0bc27a53cc5eb04c10119416e5736708fb6c0109",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8843615782587273728"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacArm64PGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[17:03:08.030] Scheduled build:
{
"id": "8678185720492736193",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3542,
"createdBy": "project:chromium",
"createTime": "2026-06-23T17:03:07.570965936Z",
"updateTime": "2026-06-23T17:03:07.570965936Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0bc27a53cc5eb04c10119416e5736708fb6c0109",
"ref": "refs/heads/main"
}
}
}
[17:03:08.030] Task URL: https://cr-buildbucket.appspot.com/build/8678185720492736193
[17:03:08.030] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:2:0) after 5m26s
[17:03:09.008] Received PubSub notification, asking Buildbucket for the build status
[17:03:09.064] Build status: SCHEDULED
[17:03:12.521] Received PubSub notification, asking Buildbucket for the build status
[17:03:12.545] Build status: STARTED
[17:08:34.023] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:2:0)
[17:08:34.050] Build status: STARTED
[17:08:34.050] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:5:0) after 2m24s
[17:10:58.048] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:5:0)
[17:10:58.144] Build status: STARTED
[17:10:58.144] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:6:0) after 5m33s
[17:16:31.228] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:6:0)
[17:16:31.246] Build status: STARTED
[17:16:31.246] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:7:0) after 2m43s
[17:19:14.402] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:7:0)
[17:19:14.420] Build status: STARTED
[17:19:14.420] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:8:0) after 5m4s
[17:24:18.505] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:8:0)
[17:24:18.612] Build status: STARTED
[17:24:18.612] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:9:0) after 2m41s
[17:26:59.649] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:9:0)
[17:26:59.665] Build status: STARTED
[17:26:59.665] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:10:0) after 2m15s
[17:29:14.720] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:10:0)
[17:29:14.739] Build status: STARTED
[17:29:14.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:11:0) after 4m3s
[17:33:17.769] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:11:0)
[17:33:17.793] Build status: STARTED
[17:33:17.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:12:0) after 3m16s
[17:36:33.860] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:12:0)
[17:36:33.884] Build status: STARTED
[17:36:33.884] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:13:0) after 6m23s
[17:42:56.994] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:13:0)
[17:42:57.072] Build status: STARTED
[17:42:57.072] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:14:0) after 2m32s
[17:45:29.088] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:14:0)
[17:45:29.105] Build status: STARTED
[17:45:29.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:15:0) after 6m29s
[17:51:58.121] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:15:0)
[17:51:58.151] Build status: STARTED
[17:51:58.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:16:0) after 5m19s
[17:57:17.164] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:16:0)
[17:57:17.184] Build status: STARTED
[17:57:17.184] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:17:0) after 7m24s
[18:04:41.200] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:17:0)
[18:04:41.216] Build status: STARTED
[18:04:41.216] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:18:0) after 1m32s
[18:06:13.269] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:18:0)
[18:06:13.309] Build status: STARTED
[18:06:13.310] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:19:0) after 2m23s
[18:08:36.415] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:19:0)
[18:08:36.460] Build status: STARTED
[18:08:36.460] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:20:0) after 1m19s
[18:09:55.490] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:20:0)
[18:09:55.513] Build status: STARTED
[18:09:55.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:21:0) after 2m30s
[18:12:25.566] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:21:0)
[18:12:25.652] Build status: STARTED
[18:12:25.652] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:22:0) after 2m36s
[18:15:01.698] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:22:0)
[18:15:01.776] Build status: STARTED
[18:15:01.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:23:0) after 1m32s
[18:16:33.780] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:23:0)
[18:16:33.827] Build status: STARTED
[18:16:33.827] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:24:0) after 3m56s
[18:20:29.907] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:24:0)
[18:20:29.925] Build status: STARTED
[18:20:29.925] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:25:0) after 8m21s
[18:28:50.954] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:25:0)
[18:28:50.972] Build status: STARTED
[18:28:50.972] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:26:0) after 3m59s
[18:32:49.999] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:26:0)
[18:32:50.079] Build status: STARTED
[18:32:50.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:27:0) after 1m21s
[18:34:11.192] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:27:0)
[18:34:11.251] Build status: STARTED
[18:34:11.251] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:28:0) after 2m3s
[18:36:14.394] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:28:0)
[18:36:14.414] Build status: STARTED
[18:36:14.414] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:29:0) after 3m23s
[18:39:37.434] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:29:0)
[18:39:37.465] Build status: STARTED
[18:39:37.465] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:30:0) after 3m41s
[18:43:18.530] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:30:0)
[18:43:18.554] Build status: STARTED
[18:43:18.554] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:31:0) after 6m4s
[18:49:22.573] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:31:0)
[18:49:22.589] Build status: STARTED
[18:49:22.589] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:32:0) after 6m32s
[18:55:54.617] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:32:0)
[18:55:54.634] Build status: STARTED
[18:55:54.634] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:33:0) after 5m34s
[19:01:28.687] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:33:0)
[19:01:28.733] Build status: STARTED
[19:01:28.733] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:34:0) after 8m15s
[19:09:43.852] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:34:0)
[19:09:43.875] Build status: STARTED
[19:09:43.875] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:35:0) after 9m19s
[19:19:02.964] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:35:0)
[19:19:02.979] Build status: STARTED
[19:19:02.979] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:36:0) after 6m49s
[19:25:52.082] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:36:0)
[19:25:52.100] Build status: STARTED
[19:25:52.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:37:0) after 8m23s
[19:34:15.125] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:37:0)
[19:34:15.166] Build status: STARTED
[19:34:15.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:38:0) after 1m48s
[19:36:03.193] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:38:0)
[19:36:03.214] Build status: STARTED
[19:36:03.214] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:39:0) after 2m29s
[19:38:32.233] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:39:0)
[19:38:32.253] Build status: STARTED
[19:38:32.253] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:40:0) after 2m35s
[19:41:07.279] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:40:0)
[19:41:07.296] Build status: STARTED
[19:41:07.296] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:41:0) after 9m58s
[19:51:05.409] Handling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:41:0)
[19:51:05.428] Build status: STARTED
[19:51:05.428] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacArm64PGO:8843615782587273728:42:0) after 8m10s
[19:54:24.672] Received PubSub notification, asking Buildbucket for the build status
[19:54:24.695] Build:
{
"id": "8678185720492736193",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacArm64PGO"
},
"number": 3542,
"createdBy": "project:chromium",
"createTime": "2026-06-23T17:03:07.570965936Z",
"startTime": "2026-06-23T17:03:12.305614205Z",
"endTime": "2026-06-23T19:54:24.133140076Z",
"updateTime": "2026-06-23T19:54:24.133140076Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0bc27a53cc5eb04c10119416e5736708fb6c0109",
"ref": "refs/heads/main"
}
}
}
[19:54:24.695] Invocation finished in 2h51m18.505503081s with status FAILED