[21:25:36.627] New invocation is queued and will start shortly
[21:25:37.869] Starting the invocation (attempt 1)
[21:25:37.897] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/e02848f25c737b8528b58daec4581c8ff2aae85e"
[21:25:37.898] Popped gitiles commit info from properties and tags
[21:25:37.898] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:25:37.898] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:25:37.899] Buildbucket request:
{
"requestId": "8841515536837189728",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e02848f25c737b8528b58daec4581c8ff2aae85e",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8841515536837189728"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:25:38.388] Scheduled build:
{
"id": "8676085474775779873",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31513,
"createdBy": "project:chromium",
"createTime": "2026-07-16T21:25:37.969602475Z",
"updateTime": "2026-07-16T21:25:37.969602475Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e02848f25c737b8528b58daec4581c8ff2aae85e",
"ref": "refs/heads/main"
}
},
"canary": true
}
[21:25:38.388] Task URL: https://cr-buildbucket.appspot.com/build/8676085474775779873
[21:25:38.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:2:0) after 4m31s
[21:25:39.344] Received PubSub notification, asking Buildbucket for the build status
[21:25:39.364] Build status: SCHEDULED
[21:25:45.915] Received PubSub notification, asking Buildbucket for the build status
[21:25:45.933] Build status: STARTED
[21:30:09.459] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:2:0)
[21:30:09.599] Build status: STARTED
[21:30:09.599] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:5:0) after 6m57s
[21:37:06.704] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:5:0)
[21:37:06.721] Build status: STARTED
[21:37:06.726] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:6:0) after 1m52s
[21:38:58.803] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:6:0)
[21:38:58.824] Build status: STARTED
[21:38:58.824] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:7:0) after 8m35s
[21:47:33.894] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:7:0)
[21:47:33.911] Build status: STARTED
[21:47:33.911] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:8:0) after 2m6s
[21:49:40.018] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:8:0)
[21:49:40.036] Build status: STARTED
[21:49:40.036] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:9:0) after 6m36s
[21:56:16.012] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:9:0)
[21:56:16.039] Build status: STARTED
[21:56:16.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:10:0) after 8m28s
[22:04:44.069] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:10:0)
[22:04:44.086] Build status: STARTED
[22:04:44.086] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:11:0) after 9m23s
[22:14:07.190] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:11:0)
[22:14:07.283] Build status: STARTED
[22:14:07.283] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:12:0) after 6m53s
[22:21:00.375] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:12:0)
[22:21:00.444] Build status: STARTED
[22:21:00.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:13:0) after 2m18s
[22:23:18.495] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:13:0)
[22:23:18.517] Build status: STARTED
[22:23:18.517] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:14:0) after 6m46s
[22:30:04.604] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:14:0)
[22:30:04.624] Build status: STARTED
[22:30:04.624] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:15:0) after 3m19s
[22:33:23.650] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:15:0)
[22:33:23.675] Build status: STARTED
[22:33:23.675] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:16:0) after 7m44s
[22:41:07.961] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:16:0)
[22:41:07.981] Build status: STARTED
[22:41:07.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:17:0) after 8m3s
[22:49:11.243] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:17:0)
[22:49:11.263] Build status: STARTED
[22:49:11.263] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:18:0) after 1m45s
[22:50:56.301] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:18:0)
[22:50:56.318] Build status: STARTED
[22:50:56.318] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:19:0) after 8m55s
[22:59:52.664] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:19:0)
[22:59:52.714] Build status: STARTED
[22:59:52.714] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:20:0) after 3m28s
[23:03:20.851] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:20:0)
[23:03:20.875] Build status: STARTED
[23:03:20.875] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:21:0) after 8m54s
[23:12:15.011] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:21:0)
[23:12:15.078] Build status: STARTED
[23:12:15.078] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:22:0) after 7m5s
[23:19:20.167] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:22:0)
[23:19:20.192] Build status: STARTED
[23:19:20.192] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:23:0) after 4m30s
[23:23:50.233] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:23:0)
[23:23:50.249] Build status: STARTED
[23:23:50.249] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:24:0) after 4m16s
[23:28:06.332] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:24:0)
[23:28:06.353] Build status: STARTED
[23:28:06.353] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:25:0) after 4m6s
[23:32:12.453] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:25:0)
[23:32:12.468] Build status: STARTED
[23:32:12.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:26:0) after 1m9s
[23:33:21.552] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:26:0)
[23:33:21.573] Build status: STARTED
[23:33:21.573] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:27:0) after 7m51s
[23:41:12.793] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:27:0)
[23:41:12.811] Build status: STARTED
[23:41:12.811] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:28:0) after 4m36s
[23:45:48.849] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:28:0)
[23:45:48.875] Build status: STARTED
[23:45:48.875] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:29:0) after 6m5s
[23:51:54.004] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:29:0)
[23:51:54.052] Build status: STARTED
[23:51:54.052] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:30:0) after 5m38s
[23:57:32.200] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:30:0)
[23:57:32.258] Build status: STARTED
[23:57:32.258] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:31:0) after 2m48s
[00:00:20.319] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:31:0)
[00:00:20.336] Build status: STARTED
[00:00:20.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:32:0) after 6m29s
[00:06:49.359] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:32:0)
[00:06:49.391] Build status: STARTED
[00:06:49.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:33:0) after 8m53s
[00:15:42.447] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:33:0)
[00:15:42.464] Build status: STARTED
[00:15:42.464] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:34:0) after 9m31s
[00:25:13.538] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:34:0)
[00:25:13.862] Build status: STARTED
[00:25:13.862] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:35:0) after 1m46s
[00:27:00.142] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:35:0)
[00:27:00.166] Build status: STARTED
[00:27:00.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:36:0) after 8m27s
[00:35:27.221] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:36:0)
[00:35:27.269] Build status: STARTED
[00:35:27.269] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:37:0) after 7m4s
[00:42:31.345] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:37:0)
[00:42:31.360] Build status: STARTED
[00:42:31.360] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:38:0) after 3m22s
[00:45:53.437] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:38:0)
[00:45:53.462] Build status: STARTED
[00:45:53.462] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:39:0) after 7m51s
[00:53:44.496] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:39:0)
[00:53:44.517] Build status: STARTED
[00:53:44.517] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8841515536837189728:40:0) after 6m12s
[00:55:31.895] Received PubSub notification, asking Buildbucket for the build status
[00:55:31.940] Build:
{
"id": "8676085474775779873",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31513,
"createdBy": "project:chromium",
"createTime": "2026-07-16T21:25:37.969602475Z",
"startTime": "2026-07-16T21:25:45.607430747Z",
"endTime": "2026-07-17T00:55:31.544140491Z",
"updateTime": "2026-07-17T00:55:31.544140491Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e02848f25c737b8528b58daec4581c8ff2aae85e",
"ref": "refs/heads/main"
}
},
"canary": true
}
[00:55:31.941] Invocation finished in 3h29m55.321992418s with status FAILED