[21:29:26.338] New invocation is queued and will start shortly
[21:29:27.461] Starting the invocation (attempt 1)
[21:29:27.550] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/69c36be0405b163183ec25e93e8f320ef0cacf71"
[21:29:27.551] Popped gitiles commit info from properties and tags
[21:29:27.551] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:29:27.551] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:29:27.551] Buildbucket request:
{
"requestId": "8853021110698567152",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "69c36be0405b163183ec25e93e8f320ef0cacf71",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8853021110698567152"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:29:28.170] Scheduled build:
{
"id": "8687591048631988945",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29735,
"createdBy": "project:chromium",
"createTime": "2026-03-11T21:29:27.686082704Z",
"updateTime": "2026-03-11T21:29:27.686082704Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "69c36be0405b163183ec25e93e8f320ef0cacf71",
"ref": "refs/heads/main"
}
}
}
[21:29:28.170] Task URL: https://cr-buildbucket.appspot.com/build/8687591048631988945
[21:29:28.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:2:0) after 8m13s
[21:29:28.255] Received PubSub notification, asking Buildbucket for the build status
[21:29:28.290] Build status: SCHEDULED
[21:30:27.251] Received PubSub notification, asking Buildbucket for the build status
[21:30:27.272] Build status: STARTED
[21:37:41.191] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:2:0)
[21:37:41.221] Build status: STARTED
[21:37:41.221] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:5:0) after 9m44s
[21:47:25.226] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:5:0)
[21:47:25.242] Build status: STARTED
[21:47:25.242] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:6:0) after 5m9s
[21:52:34.310] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:6:0)
[21:52:34.393] Build status: STARTED
[21:52:34.393] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:7:0) after 3m43s
[21:56:17.412] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:7:0)
[21:56:17.426] Build status: STARTED
[21:56:17.426] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:8:0) after 4m8s
[22:00:25.478] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:8:0)
[22:00:25.497] Build status: STARTED
[22:00:25.497] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:9:0) after 8m20s
[22:08:45.643] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:9:0)
[22:08:45.668] Build status: STARTED
[22:08:45.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:10:0) after 5m25s
[22:14:10.912] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:10:0)
[22:14:10.934] Build status: STARTED
[22:14:10.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:11:0) after 5m5s
[22:19:16.048] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:11:0)
[22:19:16.066] Build status: STARTED
[22:19:16.066] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:12:0) after 6m18s
[22:25:34.132] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:12:0)
[22:25:34.147] Build status: STARTED
[22:25:34.147] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:13:0) after 6m22s
[22:31:56.264] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:13:0)
[22:31:56.280] Build status: STARTED
[22:31:56.280] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:14:0) after 8m55s
[22:40:51.305] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:14:0)
[22:40:51.337] Build status: STARTED
[22:40:51.337] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:15:0) after 3m30s
[22:44:21.361] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:15:0)
[22:44:21.412] Build status: STARTED
[22:44:21.412] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:16:0) after 4m51s
[22:49:12.427] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:16:0)
[22:49:12.465] Build status: STARTED
[22:49:12.465] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:17:0) after 1m54s
[22:51:06.581] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:17:0)
[22:51:06.596] Build status: STARTED
[22:51:06.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:18:0) after 8m29s
[22:59:35.614] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:18:0)
[22:59:35.640] Build status: STARTED
[22:59:35.640] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:19:0) after 8m41s
[23:08:16.665] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:19:0)
[23:08:16.682] Build status: STARTED
[23:08:16.683] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:20:0) after 2m43s
[23:10:59.800] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:20:0)
[23:10:59.822] Build status: STARTED
[23:10:59.822] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:21:0) after 8m51s
[23:19:50.816] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:21:0)
[23:19:50.833] Build status: STARTED
[23:19:50.833] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:22:0) after 5m31s
[23:25:21.893] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:22:0)
[23:25:21.909] Build status: STARTED
[23:25:21.909] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:23:0) after 1m12s
[23:26:33.929] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:23:0)
[23:26:33.948] Build status: STARTED
[23:26:33.948] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:24:0) after 1m6s
[23:27:39.970] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:24:0)
[23:27:40.012] Build status: STARTED
[23:27:40.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:25:0) after 8m29s
[23:36:09.097] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:25:0)
[23:36:09.118] Build status: STARTED
[23:36:09.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:26:0) after 2m27s
[23:38:36.120] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:26:0)
[23:38:36.136] Build status: STARTED
[23:38:36.136] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:27:0) after 1m46s
[23:40:22.148] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:27:0)
[23:40:22.169] Build status: STARTED
[23:40:22.169] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:28:0) after 7m25s
[23:47:47.499] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:28:0)
[23:47:47.514] Build status: STARTED
[23:47:47.514] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:29:0) after 2m32s
[23:50:19.630] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:29:0)
[23:50:19.678] Build status: STARTED
[23:50:19.678] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:30:0) after 6m43s
[23:57:02.785] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:30:0)
[23:57:02.830] Build status: STARTED
[23:57:02.830] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:31:0) after 5m36s
[00:02:38.930] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:31:0)
[00:02:38.946] Build status: STARTED
[00:02:38.946] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:32:0) after 5m58s
[00:08:37.043] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:32:0)
[00:08:37.059] Build status: STARTED
[00:08:37.059] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:33:0) after 9m3s
[00:17:40.260] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:33:0)
[00:17:40.285] Build status: STARTED
[00:17:40.285] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:34:0) after 9m30s
[00:27:10.357] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:34:0)
[00:27:10.419] Build status: STARTED
[00:27:10.419] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:35:0) after 7m21s
[00:34:31.542] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:35:0)
[00:34:31.607] Build status: STARTED
[00:34:31.607] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:36:0) after 7m38s
[00:42:09.623] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:36:0)
[00:42:09.650] Build status: STARTED
[00:42:09.650] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:37:0) after 1m51s
[00:44:00.671] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:37:0)
[00:44:00.731] Build status: STARTED
[00:44:00.731] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:38:0) after 4m23s
[00:48:23.794] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:38:0)
[00:48:23.816] Build status: STARTED
[00:48:23.816] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:39:0) after 4m31s
[00:52:54.867] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:39:0)
[00:52:54.883] Build status: STARTED
[00:52:54.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:40:0) after 1m19s
[00:54:13.905] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:40:0)
[00:54:13.978] Build status: STARTED
[00:54:13.979] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853021110698567152:41:0) after 5m18s
[00:57:33.304] Received PubSub notification, asking Buildbucket for the build status
[00:57:33.371] Build:
{
"id": "8687591048631988945",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29735,
"createdBy": "project:chromium",
"createTime": "2026-03-11T21:29:27.686082704Z",
"startTime": "2026-03-11T21:30:25.818182711Z",
"endTime": "2026-03-12T00:57:32.895365873Z",
"updateTime": "2026-03-12T00:57:32.895365873Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "69c36be0405b163183ec25e93e8f320ef0cacf71",
"ref": "refs/heads/main"
}
}
}
[00:57:33.371] Invocation finished in 3h28m7.039283572s with status FAILED