[04:49:23.325] New invocation is queued and will start shortly
[04:49:24.591] Starting the invocation (attempt 1)
[04:49:24.616] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/82940a75f07be85cb6b9dc2bf440e973ccdbcf2f"
[04:49:24.616] Popped gitiles commit info from properties and tags
[04:49:24.616] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:49:24.616] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:49:24.616] Buildbucket request:
{
"requestId": "8851815670887730624",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "82940a75f07be85cb6b9dc2bf440e973ccdbcf2f",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851815670887730624"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:49:25.590] Scheduled build:
{
"id": "8686385608655145969",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29867,
"createdBy": "project:chromium",
"createTime": "2026-03-25T04:49:24.832225182Z",
"updateTime": "2026-03-25T04:49:24.832225182Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "82940a75f07be85cb6b9dc2bf440e973ccdbcf2f",
"ref": "refs/heads/main"
}
}
}
[04:49:25.590] Task URL: https://cr-buildbucket.appspot.com/build/8686385608655145969
[04:49:25.590] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:2:0) after 6m24s
[04:49:26.521] Received PubSub notification, asking Buildbucket for the build status
[04:49:26.538] Build status: SCHEDULED
[04:53:22.112] Received PubSub notification, asking Buildbucket for the build status
[04:53:22.130] Build status: STARTED
[04:55:49.703] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:2:0)
[04:55:49.750] Build status: STARTED
[04:55:49.750] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:5:0) after 7m4s
[05:02:53.773] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:5:0)
[05:02:53.796] Build status: STARTED
[05:02:53.796] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:6:0) after 7m50s
[05:10:43.863] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:6:0)
[05:10:43.898] Build status: STARTED
[05:10:43.898] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:7:0) after 8m29s
[05:19:12.923] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:7:0)
[05:19:12.954] Build status: STARTED
[05:19:12.954] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:8:0) after 3m9s
[05:22:21.973] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:8:0)
[05:22:21.996] Build status: STARTED
[05:22:21.996] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:9:0) after 9m40s
[05:32:02.015] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:9:0)
[05:32:02.042] Build status: STARTED
[05:32:02.042] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:10:0) after 1m23s
[05:33:25.085] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:10:0)
[05:33:25.108] Build status: STARTED
[05:33:25.109] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:11:0) after 6m6s
[05:39:31.179] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:11:0)
[05:39:31.198] Build status: STARTED
[05:39:31.198] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:12:0) after 4m13s
[05:43:44.277] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:12:0)
[05:43:44.296] Build status: STARTED
[05:43:44.296] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:13:0) after 4m41s
[05:48:25.426] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:13:0)
[05:48:25.444] Build status: STARTED
[05:48:25.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:14:0) after 1m53s
[05:50:18.465] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:14:0)
[05:50:18.499] Build status: STARTED
[05:50:18.499] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:15:0) after 4m42s
[05:55:00.989] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:15:0)
[05:55:01.012] Build status: STARTED
[05:55:01.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:16:0) after 7m12s
[06:02:13.063] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:16:0)
[06:02:13.089] Build status: STARTED
[06:02:13.089] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:17:0) after 5m24s
[06:07:37.128] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:17:0)
[06:07:37.150] Build status: STARTED
[06:07:37.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:18:0) after 7m34s
[06:15:11.168] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:18:0)
[06:15:11.226] Build status: STARTED
[06:15:11.226] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:19:0) after 6m12s
[06:21:23.248] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:19:0)
[06:21:23.330] Build status: STARTED
[06:21:23.330] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:20:0) after 7m25s
[06:28:48.352] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:20:0)
[06:28:48.422] Build status: STARTED
[06:28:48.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:21:0) after 7m57s
[06:36:45.452] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:21:0)
[06:36:45.534] Build status: STARTED
[06:36:45.534] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:22:0) after 8m49s
[06:45:34.554] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:22:0)
[06:45:34.572] Build status: STARTED
[06:45:34.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:23:0) after 7m0s
[06:52:34.662] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:23:0)
[06:52:34.704] Build status: STARTED
[06:52:34.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:24:0) after 7m30s
[07:00:04.718] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:24:0)
[07:00:04.753] Build status: STARTED
[07:00:04.753] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:25:0) after 9m0s
[07:09:04.754] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:25:0)
[07:09:04.834] Build status: STARTED
[07:09:04.834] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:26:0) after 1m1s
[07:10:05.853] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:26:0)
[07:10:05.870] Build status: STARTED
[07:10:05.871] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:27:0) after 3m48s
[07:13:53.992] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:27:0)
[07:13:54.014] Build status: STARTED
[07:13:54.014] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:28:0) after 1m28s
[07:15:22.035] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:28:0)
[07:15:22.081] Build status: STARTED
[07:15:22.081] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:29:0) after 5m20s
[07:20:42.148] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:29:0)
[07:20:42.165] Build status: STARTED
[07:20:42.165] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:30:0) after 9m31s
[07:30:13.182] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:30:0)
[07:30:13.215] Build status: STARTED
[07:30:13.215] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:31:0) after 4m44s
[07:34:57.233] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:31:0)
[07:34:57.251] Build status: STARTED
[07:34:57.251] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:32:0) after 4m35s
[07:39:32.652] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:32:0)
[07:39:32.666] Build status: STARTED
[07:39:32.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:33:0) after 9m26s
[07:48:58.738] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:33:0)
[07:48:58.754] Build status: STARTED
[07:48:58.754] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:34:0) after 8m22s
[07:57:20.810] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:34:0)
[07:57:20.826] Build status: STARTED
[07:57:20.826] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:35:0) after 7m29s
[08:04:49.849] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:35:0)
[08:04:49.868] Build status: STARTED
[08:04:49.868] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:36:0) after 2m7s
[08:06:56.910] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:36:0)
[08:06:56.951] Build status: STARTED
[08:06:56.951] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:37:0) after 8m11s
[08:15:07.971] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:37:0)
[08:15:07.990] Build status: STARTED
[08:15:07.990] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8851815670887730624:38:0) after 7m21s
[08:18:02.033] Received PubSub notification, asking Buildbucket for the build status
[08:18:02.059] Build:
{
"id": "8686385608655145969",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29867,
"createdBy": "project:chromium",
"createTime": "2026-03-25T04:49:24.832225182Z",
"startTime": "2026-03-25T04:53:21.971290275Z",
"endTime": "2026-03-25T08:18:01.769681608Z",
"updateTime": "2026-03-25T08:18:01.769681608Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "82940a75f07be85cb6b9dc2bf440e973ccdbcf2f",
"ref": "refs/heads/main"
}
}
}
[08:18:02.059] Invocation finished in 3h28m38.739117772s with status FAILED