[07:48:46.022] New invocation is queued and will start shortly
[07:48:47.306] Starting the invocation (attempt 1)
[07:48:47.332] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/cfd29e16496ad38cc4717fe7774df231043b3379"
[07:48:47.332] Popped gitiles commit info from properties and tags
[07:48:47.332] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[07:48:47.333] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[07:48:47.333] Buildbucket request:
{
"requestId": "8848542894591004080",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cfd29e16496ad38cc4717fe7774df231043b3379",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8848542894591004080"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[07:48:47.771] Scheduled build:
{
"id": "8683112832493824449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 30358,
"createdBy": "project:chromium",
"createTime": "2026-04-30T07:48:47.401926139Z",
"updateTime": "2026-04-30T07:48:47.401926139Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cfd29e16496ad38cc4717fe7774df231043b3379",
"ref": "refs/heads/main"
}
}
}
[07:48:47.771] Task URL: https://cr-buildbucket.appspot.com/build/8683112832493824449
[07:48:47.771] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:2:0) after 1m44s
[07:48:48.738] Received PubSub notification, asking Buildbucket for the build status
[07:48:48.760] Build status: SCHEDULED
[07:49:25.732] Received PubSub notification, asking Buildbucket for the build status
[07:49:25.752] Build status: STARTED
[07:50:31.912] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:2:0)
[07:50:31.929] Build status: STARTED
[07:50:31.929] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:5:0) after 4m36s
[07:55:07.990] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:5:0)
[07:55:08.009] Build status: STARTED
[07:55:08.009] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:6:0) after 7m18s
[08:02:26.107] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:6:0)
[08:02:26.130] Build status: STARTED
[08:02:26.130] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:7:0) after 3m12s
[08:05:38.247] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:7:0)
[08:05:38.270] Build status: STARTED
[08:05:38.271] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:8:0) after 9m35s
[08:15:13.283] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:8:0)
[08:15:13.300] Build status: STARTED
[08:15:13.300] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:9:0) after 8m4s
[08:23:17.304] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:9:0)
[08:23:17.326] Build status: STARTED
[08:23:17.326] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:10:0) after 7m59s
[08:31:16.799] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:10:0)
[08:31:16.823] Build status: STARTED
[08:31:16.823] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:11:0) after 3m31s
[08:34:47.839] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:11:0)
[08:34:47.859] Build status: STARTED
[08:34:47.859] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:12:0) after 6m59s
[08:41:46.990] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:12:0)
[08:41:47.011] Build status: STARTED
[08:41:47.011] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:13:0) after 1m29s
[08:43:16.109] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:13:0)
[08:43:16.127] Build status: STARTED
[08:43:16.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:14:0) after 3m14s
[08:46:30.141] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:14:0)
[08:46:30.166] Build status: STARTED
[08:46:30.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:15:0) after 2m28s
[08:48:58.358] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:15:0)
[08:48:58.375] Build status: STARTED
[08:48:58.375] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:16:0) after 4m17s
[08:53:15.442] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:16:0)
[08:53:15.484] Build status: STARTED
[08:53:15.484] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:17:0) after 1m37s
[08:54:52.548] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:17:0)
[08:54:52.667] Build status: STARTED
[08:54:52.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:18:0) after 1m11s
[08:56:03.729] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:18:0)
[08:56:03.760] Build status: STARTED
[08:56:03.760] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:19:0) after 1m43s
[08:57:46.778] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:19:0)
[08:57:46.794] Build status: STARTED
[08:57:46.795] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:20:0) after 4m47s
[09:02:33.808] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:20:0)
[09:02:33.835] Build status: STARTED
[09:02:33.837] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:21:0) after 1m29s
[09:04:02.851] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:21:0)
[09:04:02.867] Build status: STARTED
[09:04:02.867] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:22:0) after 8m4s
[09:12:06.932] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:22:0)
[09:12:06.973] Build status: STARTED
[09:12:06.973] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:23:0) after 6m4s
[09:18:10.960] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:23:0)
[09:18:10.985] Build status: STARTED
[09:18:10.985] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:24:0) after 8m25s
[09:26:35.999] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:24:0)
[09:26:36.016] Build status: STARTED
[09:26:36.016] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:25:0) after 2m53s
[09:29:29.043] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:25:0)
[09:29:29.066] Build status: STARTED
[09:29:29.066] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:26:0) after 4m32s
[09:34:01.222] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:26:0)
[09:34:01.259] Build status: STARTED
[09:34:01.260] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:27:0) after 4m47s
[09:38:48.328] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:27:0)
[09:38:48.363] Build status: STARTED
[09:38:48.363] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:28:0) after 8m22s
[09:47:10.473] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:28:0)
[09:47:10.493] Build status: STARTED
[09:47:10.493] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:29:0) after 5m28s
[09:52:38.604] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:29:0)
[09:52:38.623] Build status: STARTED
[09:52:38.623] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:30:0) after 8m18s
[10:00:56.693] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:30:0)
[10:00:56.750] Build status: STARTED
[10:00:56.750] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:31:0) after 9m22s
[10:10:18.884] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:31:0)
[10:10:18.914] Build status: STARTED
[10:10:18.914] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:32:0) after 2m36s
[10:12:54.916] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:32:0)
[10:12:54.958] Build status: STARTED
[10:12:54.958] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:33:0) after 6m6s
[10:19:01.096] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:33:0)
[10:19:01.166] Build status: STARTED
[10:19:01.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:34:0) after 6m3s
[10:25:04.133] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:34:0)
[10:25:04.203] Build status: STARTED
[10:25:04.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:35:0) after 4m46s
[10:29:50.206] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:35:0)
[10:29:50.223] Build status: STARTED
[10:29:50.223] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:36:0) after 7m25s
[10:37:15.341] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:36:0)
[10:37:15.367] Build status: STARTED
[10:37:15.367] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:37:0) after 9m23s
[10:46:38.472] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:37:0)
[10:46:38.521] Build status: STARTED
[10:46:38.521] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:38:0) after 2m29s
[10:49:07.599] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:38:0)
[10:49:07.616] Build status: STARTED
[10:49:07.616] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:39:0) after 3m37s
[10:52:44.700] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:39:0)
[10:52:44.731] Build status: STARTED
[10:52:44.731] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:40:0) after 2m59s
[10:55:43.747] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:40:0)
[10:55:43.809] Build status: STARTED
[10:55:43.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:41:0) after 6m23s
[11:02:06.931] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:41:0)
[11:02:06.952] Build status: STARTED
[11:02:06.952] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:42:0) after 7m36s
[11:09:43.030] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:42:0)
[11:09:43.058] Build status: STARTED
[11:09:43.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:43:0) after 3m29s
[11:13:12.125] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:43:0)
[11:13:12.150] Build status: STARTED
[11:13:12.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:44:0) after 9m58s
[11:23:10.170] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:44:0)
[11:23:10.187] Build status: STARTED
[11:23:10.187] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8848542894591004080:45:0) after 6m52s
[11:29:02.307] Received PubSub notification, asking Buildbucket for the build status
[11:29:02.325] Build:
{
"id": "8683112832493824449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 30358,
"createdBy": "project:chromium",
"createTime": "2026-04-30T07:48:47.401926139Z",
"startTime": "2026-04-30T07:49:25.423961282Z",
"endTime": "2026-04-30T11:29:02.048359918Z",
"updateTime": "2026-04-30T11:29:02.048359918Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "cfd29e16496ad38cc4717fe7774df231043b3379",
"ref": "refs/heads/main"
}
}
}
[11:29:02.325] Invocation finished in 3h40m16.307775727s with status FAILED