[20:46:02.659] New invocation is queued and will start shortly
[20:46:03.932] Starting the invocation (attempt 1)
[20:46:03.955] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/541efb45b0c9e5b1571c016a97ba593f0862da9e"
[20:46:03.955] Popped gitiles commit info from properties and tags
[20:46:03.955] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[20:46:03.956] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[20:46:03.956] Buildbucket request:
{
"requestId": "8858459658841475872",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "541efb45b0c9e5b1571c016a97ba593f0862da9e",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8858459658841475872"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxPGO"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[20:46:04.434] Scheduled build:
{
"id": "8693029596751173137",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 19344,
"createdBy": "project:chromium",
"createTime": "2026-01-10T20:46:04.027985283Z",
"updateTime": "2026-01-10T20:46:04.027985283Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "541efb45b0c9e5b1571c016a97ba593f0862da9e",
"ref": "refs/heads/main"
}
}
}
[20:46:04.434] Task URL: https://cr-buildbucket.appspot.com/build/8693029596751173137
[20:46:04.434] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:2:0) after 4m40s
[20:46:48.358] Received PubSub notification, asking Buildbucket for the build status
[20:46:48.377] Build status: STARTED
[20:47:20.586] Received PubSub notification, asking Buildbucket for the build status
[20:47:20.610] Build status: STARTED
[20:50:44.646] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:2:0)
[20:50:44.666] Build status: STARTED
[20:50:44.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:5:0) after 6m33s
[20:57:17.700] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:5:0)
[20:57:17.743] Build status: STARTED
[20:57:17.743] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:6:0) after 1m20s
[20:58:37.929] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:6:0)
[20:58:37.943] Build status: STARTED
[20:58:37.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:7:0) after 8m42s
[21:07:20.134] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:7:0)
[21:07:20.151] Build status: STARTED
[21:07:20.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:8:0) after 3m26s
[21:10:46.127] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:8:0)
[21:10:46.142] Build status: STARTED
[21:10:46.142] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:9:0) after 5m2s
[21:15:48.206] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:9:0)
[21:15:48.227] Build status: STARTED
[21:15:48.227] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:10:0) after 6m56s
[21:22:44.245] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:10:0)
[21:22:44.265] Build status: STARTED
[21:22:44.265] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:11:0) after 5m25s
[21:28:09.455] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:11:0)
[21:28:09.475] Build status: STARTED
[21:28:09.476] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:12:0) after 8m18s
[21:36:27.499] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:12:0)
[21:36:27.532] Build status: STARTED
[21:36:27.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:13:0) after 6m59s
[21:43:26.731] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:13:0)
[21:43:26.750] Build status: STARTED
[21:43:26.750] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:14:0) after 9m22s
[21:52:48.839] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:14:0)
[21:52:48.863] Build status: STARTED
[21:52:48.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:15:0) after 8m57s
[22:01:45.878] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:15:0)
[22:01:45.890] Build status: STARTED
[22:01:45.890] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:16:0) after 4m34s
[22:06:20.006] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:16:0)
[22:06:20.027] Build status: STARTED
[22:06:20.027] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:17:0) after 6m23s
[22:12:43.067] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:17:0)
[22:12:43.087] Build status: STARTED
[22:12:43.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:18:0) after 3m47s
[22:16:30.100] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:18:0)
[22:16:30.113] Build status: STARTED
[22:16:30.113] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:19:0) after 4m8s
[22:20:38.156] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:19:0)
[22:20:38.170] Build status: STARTED
[22:20:38.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:20:0) after 4m47s
[22:25:25.193] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:20:0)
[22:25:25.238] Build status: STARTED
[22:25:25.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:21:0) after 1m23s
[22:26:48.253] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:21:0)
[22:26:48.265] Build status: STARTED
[22:26:48.265] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:22:0) after 8m31s
[22:35:19.283] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:22:0)
[22:35:19.376] Build status: STARTED
[22:35:19.376] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:23:0) after 5m38s
[22:40:57.716] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:23:0)
[22:40:57.754] Build status: STARTED
[22:40:57.754] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:24:0) after 6m4s
[22:47:02.082] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:24:0)
[22:47:02.101] Build status: STARTED
[22:47:02.101] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:25:0) after 8m28s
[22:55:30.507] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:25:0)
[22:55:30.528] Build status: STARTED
[22:55:30.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:26:0) after 1m56s
[22:57:26.785] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:26:0)
[22:57:26.809] Build status: STARTED
[22:57:26.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:27:0) after 2m8s
[22:59:34.969] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:27:0)
[22:59:34.992] Build status: STARTED
[22:59:34.992] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:28:0) after 5m2s
[23:04:37.005] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:28:0)
[23:04:37.019] Build status: STARTED
[23:04:37.020] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:29:0) after 1m47s
[23:06:24.031] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:29:0)
[23:06:24.102] Build status: STARTED
[23:06:24.102] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:30:0) after 8m35s
[23:14:59.481] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:30:0)
[23:14:59.496] Build status: STARTED
[23:14:59.496] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:31:0) after 1m24s
[23:16:23.509] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:31:0)
[23:16:23.530] Build status: STARTED
[23:16:23.530] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:32:0) after 4m17s
[23:20:40.546] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:32:0)
[23:20:40.567] Build status: STARTED
[23:20:40.567] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:33:0) after 9m40s
[23:30:20.604] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:33:0)
[23:30:20.629] Build status: STARTED
[23:30:20.629] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:34:0) after 1m8s
[23:31:28.645] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:34:0)
[23:31:28.662] Build status: STARTED
[23:31:28.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:35:0) after 4m42s
[23:36:10.736] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:35:0)
[23:36:10.798] Build status: STARTED
[23:36:10.798] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:36:0) after 4m17s
[23:40:27.763] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:36:0)
[23:40:27.782] Build status: STARTED
[23:40:27.782] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:37:0) after 9m18s
[23:49:45.799] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:37:0)
[23:49:45.816] Build status: STARTED
[23:49:45.816] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:38:0) after 6m51s
[23:56:36.897] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:38:0)
[23:56:36.915] Build status: STARTED
[23:56:36.915] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:39:0) after 1m31s
[23:58:07.938] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:39:0)
[23:58:08.089] Build status: STARTED
[23:58:08.089] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:40:0) after 4m16s
[00:02:24.172] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:40:0)
[00:02:24.199] Build status: STARTED
[00:02:24.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:41:0) after 2m0s
[00:04:24.314] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:41:0)
[00:04:24.349] Build status: STARTED
[00:04:24.349] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:42:0) after 4m29s
[00:08:53.570] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:42:0)
[00:08:53.586] Build status: STARTED
[00:08:53.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:43:0) after 7m26s
[00:16:19.599] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:43:0)
[00:16:19.651] Build status: STARTED
[00:16:19.651] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:44:0) after 3m54s
[00:20:13.676] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:44:0)
[00:20:13.774] Build status: STARTED
[00:20:13.774] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:45:0) after 8m6s
[00:28:19.791] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:45:0)
[00:28:19.808] Build status: STARTED
[00:28:19.808] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:46:0) after 9m23s
[00:37:43.006] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:46:0)
[00:37:43.027] Build status: STARTED
[00:37:43.027] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:47:0) after 3m1s
[00:40:44.149] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:47:0)
[00:40:44.161] Build status: STARTED
[00:40:44.161] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:48:0) after 6m9s
[00:46:53.279] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:48:0)
[00:46:53.292] Build status: STARTED
[00:46:53.292] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:49:0) after 9m30s
[00:56:23.393] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:49:0)
[00:56:23.430] Build status: STARTED
[00:56:23.430] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:50:0) after 4m24s
[01:00:47.538] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:50:0)
[01:00:47.578] Build status: STARTED
[01:00:47.578] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:51:0) after 3m14s
[01:04:01.615] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:51:0)
[01:04:01.641] Build status: STARTED
[01:04:01.642] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:52:0) after 5m47s
[01:09:48.660] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:52:0)
[01:09:48.674] Build status: STARTED
[01:09:48.674] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxPGO:8858459658841475872:53:0) after 4m56s
[01:11:07.441] Received PubSub notification, asking Buildbucket for the build status
[01:11:07.458] Build:
{
"id": "8693029596751173137",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxPGO"
},
"number": 19344,
"createdBy": "project:chromium",
"createTime": "2026-01-10T20:46:04.027985283Z",
"startTime": "2026-01-10T20:46:31.929814845Z",
"endTime": "2026-01-11T01:11:07.170636329Z",
"updateTime": "2026-01-11T01:11:07.170636329Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "541efb45b0c9e5b1571c016a97ba593f0862da9e",
"ref": "refs/heads/main"
}
}
}
[01:11:07.458] Invocation finished in 4h25m4.808001655s with status FAILED