[13:51:46.142] New invocation is queued and will start shortly
[13:51:48.440] Starting the invocation (attempt 1)
[13:51:48.479] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d6c5c65df374ab3cd4f025a8f57e8a3e4381dc63"
[13:51:48.479] Popped gitiles commit info from properties and tags
[13:51:48.479] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[13:51:48.479] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[13:51:48.479] Buildbucket request:
{
"requestId": "8853049904800542816",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d6c5c65df374ab3cd4f025a8f57e8a3e4381dc63",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8853049904800542816"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[13:51:48.979] Scheduled build:
{
"id": "8687619841579892577",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29733,
"createdBy": "project:chromium",
"createTime": "2026-03-11T13:51:48.591788443Z",
"updateTime": "2026-03-11T13:51:48.591788443Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d6c5c65df374ab3cd4f025a8f57e8a3e4381dc63",
"ref": "refs/heads/main"
}
}
}
[13:51:48.979] Task URL: https://cr-buildbucket.appspot.com/build/8687619841579892577
[13:51:48.979] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:2:0) after 7m40s
[13:51:49.983] Received PubSub notification, asking Buildbucket for the build status
[13:51:50.001] Build status: SCHEDULED
[13:51:54.260] Received PubSub notification, asking Buildbucket for the build status
[13:51:54.285] Build status: STARTED
[13:59:28.997] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:2:0)
[13:59:29.013] Build status: STARTED
[13:59:29.013] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:5:0) after 8m9s
[14:07:38.051] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:5:0)
[14:07:38.070] Build status: STARTED
[14:07:38.071] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:6:0) after 8m54s
[14:16:32.042] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:6:0)
[14:16:32.061] Build status: STARTED
[14:16:32.061] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:7:0) after 6m10s
[14:22:42.108] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:7:0)
[14:22:42.135] Build status: STARTED
[14:22:42.135] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:8:0) after 3m35s
[14:26:17.198] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:8:0)
[14:26:17.256] Build status: STARTED
[14:26:17.256] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:9:0) after 8m49s
[14:35:06.278] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:9:0)
[14:35:06.301] Build status: STARTED
[14:35:06.301] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:10:0) after 6m24s
[14:41:30.336] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:10:0)
[14:41:30.501] Build status: STARTED
[14:41:30.501] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:11:0) after 1m1s
[14:42:31.523] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:11:0)
[14:42:31.588] Build status: STARTED
[14:42:31.588] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:12:0) after 8m54s
[14:51:25.614] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:12:0)
[14:51:25.653] Build status: STARTED
[14:51:25.653] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:13:0) after 2m0s
[14:53:25.669] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:13:0)
[14:53:25.693] Build status: STARTED
[14:53:25.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:14:0) after 3m1s
[14:56:26.777] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:14:0)
[14:56:26.794] Build status: STARTED
[14:56:26.795] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:15:0) after 4m53s
[15:01:19.813] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:15:0)
[15:01:19.841] Build status: STARTED
[15:01:19.841] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:16:0) after 7m34s
[15:08:53.821] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:16:0)
[15:08:53.896] Build status: STARTED
[15:08:53.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:17:0) after 6m44s
[15:15:37.913] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:17:0)
[15:15:37.942] Build status: STARTED
[15:15:37.942] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:18:0) after 6m12s
[15:21:49.959] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:18:0)
[15:21:49.977] Build status: STARTED
[15:21:49.977] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:19:0) after 5m14s
[15:27:03.992] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:19:0)
[15:27:04.054] Build status: STARTED
[15:27:04.054] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:20:0) after 5m1s
[15:32:05.076] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:20:0)
[15:32:05.116] Build status: STARTED
[15:32:05.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:21:0) after 7m15s
[15:39:20.123] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:21:0)
[15:39:20.158] Build status: STARTED
[15:39:20.158] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:22:0) after 3m5s
[15:42:25.185] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:22:0)
[15:42:25.209] Build status: STARTED
[15:42:25.209] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:23:0) after 5m24s
[15:47:49.228] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:23:0)
[15:47:49.244] Build status: STARTED
[15:47:49.244] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:24:0) after 3m29s
[15:51:18.259] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:24:0)
[15:51:18.280] Build status: STARTED
[15:51:18.280] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:25:0) after 3m13s
[15:54:31.306] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:25:0)
[15:54:31.324] Build status: STARTED
[15:54:31.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:26:0) after 2m25s
[15:56:56.390] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:26:0)
[15:56:56.428] Build status: STARTED
[15:56:56.428] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:27:0) after 7m50s
[16:04:46.422] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:27:0)
[16:04:46.441] Build status: STARTED
[16:04:46.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:28:0) after 8m49s
[16:13:35.521] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:28:0)
[16:13:35.537] Build status: STARTED
[16:13:35.537] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:29:0) after 5m13s
[16:18:48.663] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:29:0)
[16:18:48.679] Build status: STARTED
[16:18:48.679] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:30:0) after 9m26s
[16:28:14.720] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:30:0)
[16:28:14.735] Build status: STARTED
[16:28:14.735] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:31:0) after 6m57s
[16:35:11.756] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:31:0)
[16:35:11.802] Build status: STARTED
[16:35:11.802] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:32:0) after 7m41s
[16:42:52.851] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:32:0)
[16:42:52.882] Build status: STARTED
[16:42:52.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:33:0) after 7m47s
[16:50:40.101] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:33:0)
[16:50:40.137] Build status: STARTED
[16:50:40.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:34:0) after 2m25s
[16:53:05.155] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:34:0)
[16:53:05.174] Build status: STARTED
[16:53:05.174] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:35:0) after 8m17s
[17:01:22.295] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:35:0)
[17:01:22.318] Build status: STARTED
[17:01:22.318] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:36:0) after 6m56s
[17:08:18.385] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:36:0)
[17:08:18.401] Build status: STARTED
[17:08:18.401] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853049904800542816:37:0) after 9m53s
[17:17:22.386] Received PubSub notification, asking Buildbucket for the build status
[17:17:22.403] Build:
{
"id": "8687619841579892577",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29733,
"createdBy": "project:chromium",
"createTime": "2026-03-11T13:51:48.591788443Z",
"startTime": "2026-03-11T13:51:54.072711885Z",
"endTime": "2026-03-11T17:17:21.908839250Z",
"updateTime": "2026-03-11T17:17:21.908839250Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d6c5c65df374ab3cd4f025a8f57e8a3e4381dc63",
"ref": "refs/heads/main"
}
}
}
[17:17:22.403] Invocation finished in 3h25m36.267107144s with status FAILED