[08:13:49.444] New invocation is queued and will start shortly
[08:13:50.521] Starting the invocation (attempt 1)
[08:13:50.553] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/1ec73ec3cdc03549a90155a5170c95b14d71d5b9"
[08:13:50.553] Popped gitiles commit info from properties and tags
[08:13:50.553] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[08:13:50.553] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[08:13:50.553] Buildbucket request:
{
"requestId": "8850625048365476192",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1ec73ec3cdc03549a90155a5170c95b14d71d5b9",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850625048365476192"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[08:13:51.090] Scheduled build:
{
"id": "8685194986494323201",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 30021,
"createdBy": "project:chromium",
"createTime": "2026-04-07T08:13:50.608203047Z",
"updateTime": "2026-04-07T08:13:50.608203047Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1ec73ec3cdc03549a90155a5170c95b14d71d5b9",
"ref": "refs/heads/main"
}
}
}
[08:13:51.090] Task URL: https://cr-buildbucket.appspot.com/build/8685194986494323201
[08:13:51.090] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:2:0) after 5m10s
[08:13:52.084] Received PubSub notification, asking Buildbucket for the build status
[08:13:52.145] Build status: SCHEDULED
[08:14:56.079] Received PubSub notification, asking Buildbucket for the build status
[08:14:56.097] Build status: STARTED
[08:19:01.104] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:2:0)
[08:19:01.309] Build status: STARTED
[08:19:01.309] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:5:0) after 9m23s
[08:28:24.344] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:5:0)
[08:28:24.364] Build status: STARTED
[08:28:24.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:6:0) after 6m16s
[08:34:40.405] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:6:0)
[08:34:40.444] Build status: STARTED
[08:34:40.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:7:0) after 8m30s
[08:43:10.461] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:7:0)
[08:43:10.481] Build status: STARTED
[08:43:10.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:8:0) after 6m23s
[08:49:33.545] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:8:0)
[08:49:33.562] Build status: STARTED
[08:49:33.562] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:9:0) after 7m22s
[08:56:55.668] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:9:0)
[08:56:55.740] Build status: STARTED
[08:56:55.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:10:0) after 2m1s
[08:58:56.756] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:10:0)
[08:58:56.819] Build status: STARTED
[08:58:56.819] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:11:0) after 1m1s
[08:59:57.838] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:11:0)
[08:59:57.940] Build status: STARTED
[08:59:57.940] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:12:0) after 3m47s
[09:03:44.957] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:12:0)
[09:03:44.979] Build status: STARTED
[09:03:44.979] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:13:0) after 7m37s
[09:11:21.995] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:13:0)
[09:11:22.017] Build status: STARTED
[09:11:22.017] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:14:0) after 1m16s
[09:12:38.049] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:14:0)
[09:12:38.101] Build status: STARTED
[09:12:38.101] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:15:0) after 1m35s
[09:14:13.156] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:15:0)
[09:14:13.179] Build status: STARTED
[09:14:13.179] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:16:0) after 5m23s
[09:19:36.238] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:16:0)
[09:19:36.268] Build status: STARTED
[09:19:36.268] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:17:0) after 3m29s
[09:23:05.291] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:17:0)
[09:23:05.308] Build status: STARTED
[09:23:05.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:18:0) after 5m15s
[09:28:20.331] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:18:0)
[09:28:20.353] Build status: STARTED
[09:28:20.353] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:19:0) after 7m20s
[09:35:40.400] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:19:0)
[09:35:40.457] Build status: STARTED
[09:35:40.457] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:20:0) after 1m2s
[09:36:42.556] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:20:0)
[09:36:42.576] Build status: STARTED
[09:36:42.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:21:0) after 9m24s
[09:46:06.607] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:21:0)
[09:46:06.632] Build status: STARTED
[09:46:06.632] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:22:0) after 1m6s
[09:47:12.647] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:22:0)
[09:47:12.664] Build status: STARTED
[09:47:12.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:23:0) after 7m18s
[09:54:30.727] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:23:0)
[09:54:30.746] Build status: STARTED
[09:54:30.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:24:0) after 5m26s
[09:59:56.770] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:24:0)
[09:59:56.788] Build status: STARTED
[09:59:56.788] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:25:0) after 9m17s
[10:09:13.825] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:25:0)
[10:09:13.883] Build status: STARTED
[10:09:13.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:26:0) after 5m24s
[10:14:37.924] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:26:0)
[10:14:37.940] Build status: STARTED
[10:14:37.940] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:27:0) after 9m12s
[10:23:50.006] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:27:0)
[10:23:50.023] Build status: STARTED
[10:23:50.023] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:28:0) after 1m22s
[10:25:12.045] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:28:0)
[10:25:12.118] Build status: STARTED
[10:25:12.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:29:0) after 9m7s
[10:34:19.133] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:29:0)
[10:34:19.159] Build status: STARTED
[10:34:19.159] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:30:0) after 7m12s
[10:41:31.211] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:30:0)
[10:41:31.231] Build status: STARTED
[10:41:31.231] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:31:0) after 5m59s
[10:47:30.303] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:31:0)
[10:47:30.338] Build status: STARTED
[10:47:30.338] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:32:0) after 8m23s
[10:55:53.347] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:32:0)
[10:55:53.361] Build status: STARTED
[10:55:53.361] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:33:0) after 5m44s
[11:01:37.386] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:33:0)
[11:01:37.414] Build status: STARTED
[11:01:37.414] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:34:0) after 9m48s
[11:11:25.435] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:34:0)
[11:11:25.454] Build status: STARTED
[11:11:25.454] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:35:0) after 1m46s
[11:13:11.425] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:35:0)
[11:13:11.443] Build status: STARTED
[11:13:11.443] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:36:0) after 4m55s
[11:18:06.509] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:36:0)
[11:18:06.547] Build status: STARTED
[11:18:06.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:37:0) after 9m20s
[11:27:26.584] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:37:0)
[11:27:26.602] Build status: STARTED
[11:27:26.602] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:38:0) after 7m33s
[11:34:59.699] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:38:0)
[11:34:59.716] Build status: STARTED
[11:34:59.716] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850625048365476192:39:0) after 8m0s
[11:37:19.857] Received PubSub notification, asking Buildbucket for the build status
[11:37:19.895] Build:
{
"id": "8685194986494323201",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 30021,
"createdBy": "project:chromium",
"createTime": "2026-04-07T08:13:50.608203047Z",
"startTime": "2026-04-07T08:14:55.888854163Z",
"endTime": "2026-04-07T11:37:05.863706175Z",
"updateTime": "2026-04-07T11:37:05.863706175Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "1ec73ec3cdc03549a90155a5170c95b14d71d5b9",
"ref": "refs/heads/main"
}
}
}
[11:37:19.895] Invocation finished in 3h23m30.455873567s with status FAILED