[11:52:20.318] New invocation is queued and will start shortly
[11:52:21.352] Starting the invocation (attempt 1)
[11:52:21.375] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/caaf30631cd02fd109aa81916cf48bbdd5b01090"
[11:52:21.375] Popped gitiles commit info from properties and tags
[11:52:21.375] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[11:52:21.375] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[11:52:21.376] Buildbucket request:
{
"requestId": "8835119219364428416",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "caaf30631cd02fd109aa81916cf48bbdd5b01090",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8835119219364428416"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[11:52:21.895] Scheduled build:
{
"id": "8669689157526451905",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 32778,
"createdBy": "project:chromium",
"createTime": "2026-09-25T11:52:21.449096267Z",
"updateTime": "2026-09-25T11:52:21.449096267Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "caaf30631cd02fd109aa81916cf48bbdd5b01090",
"ref": "refs/heads/main"
}
}
}
[11:52:21.895] Task URL: https://cr-buildbucket.appspot.com/build/8669689157526451905
[11:52:21.895] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:2:0) after 9m42s
[11:52:21.983] Received PubSub notification, asking Buildbucket for the build status
[11:52:22.000] Build status: SCHEDULED
[12:02:03.913] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:2:0)
[12:02:03.931] Build status: SCHEDULED
[12:02:03.931] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:4:0) after 4m12s
[12:06:15.951] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:4:0)
[12:06:16.202] Build status: SCHEDULED
[12:06:16.202] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:5:0) after 4m43s
[12:10:59.224] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:5:0)
[12:10:59.245] Build status: SCHEDULED
[12:10:59.245] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:6:0) after 1m43s
[12:12:42.263] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:6:0)
[12:12:42.282] Build status: SCHEDULED
[12:12:42.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:7:0) after 5m7s
[12:17:49.303] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:7:0)
[12:17:49.337] Build status: SCHEDULED
[12:17:49.337] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:8:0) after 9m30s
[12:19:00.854] Received PubSub notification, asking Buildbucket for the build status
[12:19:00.879] Build status: STARTED
[12:27:19.354] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:8:0)
[12:27:19.371] Build status: STARTED
[12:27:19.371] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:10:0) after 5m4s
[12:32:23.389] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:10:0)
[12:32:23.411] Build status: STARTED
[12:32:23.411] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:11:0) after 3m38s
[12:36:01.529] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:11:0)
[12:36:01.546] Build status: STARTED
[12:36:01.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:12:0) after 2m14s
[12:38:15.567] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:12:0)
[12:38:15.591] Build status: STARTED
[12:38:15.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:13:0) after 3m52s
[12:42:07.649] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:13:0)
[12:42:07.666] Build status: STARTED
[12:42:07.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:14:0) after 6m27s
[12:48:34.692] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:14:0)
[12:48:34.710] Build status: STARTED
[12:48:34.710] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:15:0) after 9m26s
[12:58:00.727] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:15:0)
[12:58:00.745] Build status: STARTED
[12:58:00.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:16:0) after 6m59s
[13:04:59.939] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:16:0)
[13:04:59.959] Build status: STARTED
[13:04:59.959] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:17:0) after 3m14s
[13:08:14.076] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:17:0)
[13:08:14.098] Build status: STARTED
[13:08:14.098] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:18:0) after 9m36s
[13:17:50.364] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:18:0)
[13:17:50.399] Build status: STARTED
[13:17:50.400] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:19:0) after 1m59s
[13:19:49.386] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:19:0)
[13:19:49.447] Build status: STARTED
[13:19:49.447] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:20:0) after 1m7s
[13:20:56.516] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:20:0)
[13:20:56.537] Build status: STARTED
[13:20:56.537] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:21:0) after 4m47s
[13:25:43.739] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:21:0)
[13:25:43.801] Build status: STARTED
[13:25:43.801] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:22:0) after 3m45s
[13:29:28.821] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:22:0)
[13:29:28.839] Build status: STARTED
[13:29:28.839] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:23:0) after 5m16s
[13:34:44.819] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:23:0)
[13:34:44.850] Build status: STARTED
[13:34:44.850] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:24:0) after 1m11s
[13:35:55.824] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:24:0)
[13:35:55.844] Build status: STARTED
[13:35:55.845] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:25:0) after 2m33s
[13:38:28.860] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:25:0)
[13:38:28.884] Build status: STARTED
[13:38:28.884] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:26:0) after 7m29s
[13:45:58.072] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:26:0)
[13:45:58.109] Build status: STARTED
[13:45:58.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:27:0) after 2m46s
[13:48:44.346] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:27:0)
[13:48:44.380] Build status: STARTED
[13:48:44.380] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:28:0) after 3m50s
[13:52:34.425] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:28:0)
[13:52:34.655] Build status: STARTED
[13:52:34.655] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:29:0) after 2m36s
[13:55:10.674] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:29:0)
[13:55:10.700] Build status: STARTED
[13:55:10.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:30:0) after 4m7s
[13:59:17.981] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:30:0)
[13:59:18.039] Build status: STARTED
[13:59:18.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:31:0) after 1m18s
[14:00:36.057] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:31:0)
[14:00:36.084] Build status: STARTED
[14:00:36.084] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:32:0) after 9m19s
[14:09:55.102] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:32:0)
[14:09:55.123] Build status: STARTED
[14:09:55.123] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:33:0) after 7m4s
[14:16:59.536] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:33:0)
[14:16:59.560] Build status: STARTED
[14:16:59.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:34:0) after 7m53s
[14:24:52.590] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:34:0)
[14:24:52.653] Build status: STARTED
[14:24:52.653] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:35:0) after 5m27s
[14:30:19.680] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:35:0)
[14:30:19.708] Build status: STARTED
[14:30:19.708] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:36:0) after 1m35s
[14:31:54.758] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:36:0)
[14:31:54.789] Build status: STARTED
[14:31:54.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835119219364428416:37:0) after 5m36s
[14:33:15.748] Received PubSub notification, asking Buildbucket for the build status
[14:33:15.768] Build:
{
"id": "8669689157526451905",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 32778,
"createdBy": "project:chromium",
"createTime": "2026-09-25T11:52:21.449096267Z",
"startTime": "2026-09-25T12:19:00.667690738Z",
"endTime": "2026-09-25T14:33:09.410668006Z",
"updateTime": "2026-09-25T14:33:09.410668006Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "caaf30631cd02fd109aa81916cf48bbdd5b01090",
"ref": "refs/heads/main"
}
}
}
[14:33:15.768] Invocation finished in 2h40m55.456211943s with status FAILED