[07:27:24.555] New invocation is queued and will start shortly
[07:27:25.600] Starting the invocation (attempt 1)
[07:27:25.625] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/eec0cdffee5b284dd0426bf45d984ab5d95b9482"
[07:27:25.626] Popped gitiles commit info from properties and tags
[07:27:25.626] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[07:27:25.626] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[07:27:25.626] Buildbucket request:
{
"requestId": "8853074086628695472",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "eec0cdffee5b284dd0426bf45d984ab5d95b9482",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8853074086628695472"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[07:27:26.075] Scheduled build:
{
"id": "8687644024790892609",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29731,
"createdBy": "project:chromium",
"createTime": "2026-03-11T07:27:25.684967584Z",
"updateTime": "2026-03-11T07:27:25.684967584Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "eec0cdffee5b284dd0426bf45d984ab5d95b9482",
"ref": "refs/heads/main"
}
}
}
[07:27:26.075] Task URL: https://cr-buildbucket.appspot.com/build/8687644024790892609
[07:27:26.075] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:2:0) after 5m38s
[07:27:27.025] Received PubSub notification, asking Buildbucket for the build status
[07:27:27.043] Build status: SCHEDULED
[07:28:23.751] Received PubSub notification, asking Buildbucket for the build status
[07:28:23.774] Build status: STARTED
[07:33:04.095] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:2:0)
[07:33:04.119] Build status: STARTED
[07:33:04.119] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:5:0) after 1m51s
[07:34:55.141] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:5:0)
[07:34:55.156] Build status: STARTED
[07:34:55.156] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:6:0) after 8m6s
[07:43:01.221] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:6:0)
[07:43:01.237] Build status: STARTED
[07:43:01.237] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:7:0) after 9m49s
[07:52:50.257] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:7:0)
[07:52:50.284] Build status: STARTED
[07:52:50.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:8:0) after 6m53s
[07:59:43.302] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:8:0)
[07:59:43.332] Build status: STARTED
[07:59:43.332] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:9:0) after 8m49s
[08:08:32.549] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:9:0)
[08:08:32.567] Build status: STARTED
[08:08:32.567] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:10:0) after 4m1s
[08:12:33.584] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:10:0)
[08:12:33.678] Build status: STARTED
[08:12:33.678] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:11:0) after 3m54s
[08:16:27.747] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:11:0)
[08:16:27.764] Build status: STARTED
[08:16:27.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:12:0) after 4m25s
[08:20:52.783] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:12:0)
[08:20:52.851] Build status: STARTED
[08:20:52.851] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:13:0) after 7m18s
[08:28:10.865] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:13:0)
[08:28:10.950] Build status: STARTED
[08:28:10.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:14:0) after 1m27s
[08:29:37.976] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:14:0)
[08:29:37.993] Build status: STARTED
[08:29:37.993] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:15:0) after 7m38s
[08:37:16.010] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:15:0)
[08:37:16.028] Build status: STARTED
[08:37:16.028] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:16:0) after 8m45s
[08:46:01.039] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:16:0)
[08:46:01.062] Build status: STARTED
[08:46:01.062] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:17:0) after 6m43s
[08:52:44.188] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:17:0)
[08:52:44.213] Build status: STARTED
[08:52:44.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:18:0) after 3m31s
[08:56:15.299] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:18:0)
[08:56:15.330] Build status: STARTED
[08:56:15.330] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:19:0) after 6m0s
[09:02:15.347] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:19:0)
[09:02:15.444] Build status: STARTED
[09:02:15.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:20:0) after 5m39s
[09:07:54.458] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:20:0)
[09:07:54.558] Build status: STARTED
[09:07:54.558] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:21:0) after 2m16s
[09:10:10.668] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:21:0)
[09:10:10.686] Build status: STARTED
[09:10:10.686] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:22:0) after 4m30s
[09:14:40.708] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:22:0)
[09:14:40.746] Build status: STARTED
[09:14:40.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:23:0) after 3m30s
[09:18:10.810] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:23:0)
[09:18:10.838] Build status: STARTED
[09:18:10.838] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:24:0) after 3m55s
[09:22:05.859] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:24:0)
[09:22:05.873] Build status: STARTED
[09:22:05.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:25:0) after 6m20s
[09:28:25.904] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:25:0)
[09:28:25.923] Build status: STARTED
[09:28:25.923] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:26:0) after 2m31s
[09:30:56.991] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:26:0)
[09:30:57.006] Build status: STARTED
[09:30:57.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:27:0) after 1m54s
[09:32:51.026] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:27:0)
[09:32:51.047] Build status: STARTED
[09:32:51.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:28:0) after 3m26s
[09:36:17.064] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:28:0)
[09:36:17.080] Build status: STARTED
[09:36:17.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:29:0) after 8m44s
[09:45:01.119] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:29:0)
[09:45:01.141] Build status: STARTED
[09:45:01.141] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:30:0) after 2m59s
[09:48:00.159] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:30:0)
[09:48:00.243] Build status: STARTED
[09:48:00.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:31:0) after 6m14s
[09:54:14.207] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:31:0)
[09:54:14.223] Build status: STARTED
[09:54:14.223] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:32:0) after 1m5s
[09:55:19.239] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:32:0)
[09:55:19.266] Build status: STARTED
[09:55:19.266] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:33:0) after 8m39s
[10:03:58.330] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:33:0)
[10:03:58.379] Build status: STARTED
[10:03:58.379] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:34:0) after 3m15s
[10:07:13.398] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:34:0)
[10:07:13.448] Build status: STARTED
[10:07:13.448] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:35:0) after 2m59s
[10:10:12.464] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:35:0)
[10:10:12.481] Build status: STARTED
[10:10:12.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:36:0) after 4m49s
[10:15:01.528] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:36:0)
[10:15:01.580] Build status: STARTED
[10:15:01.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:37:0) after 6m52s
[10:21:53.643] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:37:0)
[10:21:53.669] Build status: STARTED
[10:21:53.669] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:38:0) after 4m23s
[10:26:16.911] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:38:0)
[10:26:16.937] Build status: STARTED
[10:26:16.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:39:0) after 6m24s
[10:32:40.960] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:39:0)
[10:32:40.977] Build status: STARTED
[10:32:40.977] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853074086628695472:40:0) after 3m52s
[10:35:48.837] Received PubSub notification, asking Buildbucket for the build status
[10:35:48.854] Build:
{
"id": "8687644024790892609",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29731,
"createdBy": "project:chromium",
"createTime": "2026-03-11T07:27:25.684967584Z",
"startTime": "2026-03-11T07:28:23.551503677Z",
"endTime": "2026-03-11T10:35:47.909616592Z",
"updateTime": "2026-03-11T10:35:47.909616592Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "eec0cdffee5b284dd0426bf45d984ab5d95b9482",
"ref": "refs/heads/main"
}
}
}
[10:35:48.854] Invocation finished in 3h8m24.306044164s with status FAILED