[05:10:59.328] New invocation is queued and will start shortly
[05:11:00.434] Starting the invocation (attempt 1)
[05:11:00.456] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/bd28a7e7b9dc083684b77d9a842f801b6e9da0c0"
[05:11:00.457] Popped gitiles commit info from properties and tags
[05:11:00.457] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[05:11:00.457] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[05:11:00.457] Buildbucket request:
{
"requestId": "8857521920815147248",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bd28a7e7b9dc083684b77d9a842f801b6e9da0c0",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8857521920815147248"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[05:11:01.026] Scheduled build:
{
"id": "8692091858912815345",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 28974,
"createdBy": "project:chromium",
"createTime": "2026-01-21T05:11:00.518802835Z",
"updateTime": "2026-01-21T05:11:00.518802835Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bd28a7e7b9dc083684b77d9a842f801b6e9da0c0",
"ref": "refs/heads/main"
}
}
}
[05:11:01.026] Task URL: https://cr-buildbucket.appspot.com/build/8692091858912815345
[05:11:01.026] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:2:0) after 5m42s
[05:11:01.228] Received PubSub notification, asking Buildbucket for the build status
[05:11:01.253] Build status: SCHEDULED
[05:11:59.784] Received PubSub notification, asking Buildbucket for the build status
[05:11:59.809] Build status: STARTED
[05:16:43.040] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:2:0)
[05:16:43.063] Build status: STARTED
[05:16:43.063] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:5:0) after 4m48s
[05:21:31.085] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:5:0)
[05:21:31.109] Build status: STARTED
[05:21:31.109] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:6:0) after 4m56s
[05:26:27.129] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:6:0)
[05:26:27.145] Build status: STARTED
[05:26:27.145] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:7:0) after 5m56s
[05:32:23.232] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:7:0)
[05:32:23.247] Build status: STARTED
[05:32:23.247] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:8:0) after 5m44s
[05:38:07.263] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:8:0)
[05:38:07.298] Build status: STARTED
[05:38:07.298] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:9:0) after 6m39s
[05:44:46.311] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:9:0)
[05:44:46.439] Build status: STARTED
[05:44:46.439] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:10:0) after 3m14s
[05:48:00.534] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:10:0)
[05:48:00.547] Build status: STARTED
[05:48:00.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:11:0) after 3m20s
[05:51:20.560] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:11:0)
[05:51:20.576] Build status: STARTED
[05:51:20.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:12:0) after 4m47s
[05:56:07.594] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:12:0)
[05:56:07.612] Build status: STARTED
[05:56:07.612] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:13:0) after 6m15s
[06:02:22.628] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:13:0)
[06:02:22.645] Build status: STARTED
[06:02:22.645] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:14:0) after 4m27s
[06:06:49.661] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:14:0)
[06:06:49.693] Build status: STARTED
[06:06:49.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:15:0) after 8m35s
[06:15:24.714] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:15:0)
[06:15:24.735] Build status: STARTED
[06:15:24.735] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:16:0) after 1m53s
[06:17:17.798] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:16:0)
[06:17:17.819] Build status: STARTED
[06:17:17.819] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:17:0) after 4m9s
[06:21:26.824] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:17:0)
[06:21:26.861] Build status: STARTED
[06:21:26.861] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:18:0) after 1m18s
[06:22:44.958] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:18:0)
[06:22:44.990] Build status: STARTED
[06:22:44.990] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:19:0) after 2m39s
[06:25:24.012] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:19:0)
[06:25:24.063] Build status: STARTED
[06:25:24.063] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:20:0) after 4m57s
[06:30:21.134] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:20:0)
[06:30:21.150] Build status: STARTED
[06:30:21.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:21:0) after 4m42s
[06:35:03.145] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:21:0)
[06:35:03.198] Build status: STARTED
[06:35:03.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:22:0) after 3m45s
[06:38:48.289] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:22:0)
[06:38:48.313] Build status: STARTED
[06:38:48.313] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:23:0) after 4m45s
[06:43:33.334] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:23:0)
[06:43:33.364] Build status: STARTED
[06:43:33.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:24:0) after 3m10s
[06:46:43.546] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:24:0)
[06:46:43.560] Build status: STARTED
[06:46:43.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:25:0) after 9m3s
[06:55:46.553] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:25:0)
[06:55:46.569] Build status: STARTED
[06:55:46.569] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:26:0) after 7m21s
[07:03:07.636] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:26:0)
[07:03:07.673] Build status: STARTED
[07:03:07.674] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:27:0) after 7m59s
[07:11:06.703] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:27:0)
[07:11:06.720] Build status: STARTED
[07:11:06.721] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:28:0) after 5m44s
[07:16:50.784] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:28:0)
[07:16:50.809] Build status: STARTED
[07:16:50.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:29:0) after 5m6s
[07:21:56.826] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:29:0)
[07:21:56.844] Build status: STARTED
[07:21:56.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:30:0) after 9m26s
[07:31:22.860] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:30:0)
[07:31:22.876] Build status: STARTED
[07:31:22.876] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:31:0) after 4m56s
[07:36:18.944] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:31:0)
[07:36:18.993] Build status: STARTED
[07:36:18.993] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:32:0) after 4m45s
[07:41:04.009] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:32:0)
[07:41:04.022] Build status: STARTED
[07:41:04.022] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:33:0) after 9m6s
[07:50:10.131] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:33:0)
[07:50:10.147] Build status: STARTED
[07:50:10.147] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:34:0) after 7m46s
[07:57:56.283] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:34:0)
[07:57:56.299] Build status: STARTED
[07:57:56.299] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:35:0) after 8m47s
[08:06:43.357] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:35:0)
[08:06:43.480] Build status: STARTED
[08:06:43.480] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:36:0) after 9m28s
[08:16:11.615] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:36:0)
[08:16:11.632] Build status: STARTED
[08:16:11.632] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:37:0) after 5m13s
[08:21:24.651] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:37:0)
[08:21:24.673] Build status: STARTED
[08:21:24.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:38:0) after 6m58s
[08:28:22.834] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:38:0)
[08:28:22.849] Build status: STARTED
[08:28:22.849] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:39:0) after 6m18s
[08:34:40.950] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:39:0)
[08:34:40.967] Build status: STARTED
[08:34:40.967] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:40:0) after 4m34s
[08:39:14.983] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:40:0)
[08:39:14.998] Build status: STARTED
[08:39:14.998] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8857521920815147248:41:0) after 5m57s
[08:42:31.782] Received PubSub notification, asking Buildbucket for the build status
[08:42:31.798] Build:
{
"id": "8692091858912815345",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 28974,
"createdBy": "project:chromium",
"createTime": "2026-01-21T05:11:00.518802835Z",
"startTime": "2026-01-21T05:11:59.646666226Z",
"endTime": "2026-01-21T08:42:31.503382557Z",
"updateTime": "2026-01-21T08:42:31.503382557Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bd28a7e7b9dc083684b77d9a842f801b6e9da0c0",
"ref": "refs/heads/main"
}
}
}
[08:42:31.798] Invocation finished in 3h31m32.477776934s with status FAILED