[13:41:35.167] New invocation is queued and will start shortly
[13:41:36.737] Starting the invocation (attempt 1)
[13:41:36.766] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/a1ec5523bfcd3a33f2ecf14eee78fca39d43d733"
[13:41:36.766] Popped gitiles commit info from properties and tags
[13:41:36.766] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[13:41:36.766] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[13:41:36.766] Buildbucket request:
{
"requestId": "8839732791395102272",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a1ec5523bfcd3a33f2ecf14eee78fca39d43d733",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8839732791395102272"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[13:41:37.593] Scheduled build:
{
"id": "8674302728791845745",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 32028,
"createdBy": "project:chromium",
"createTime": "2026-08-05T13:41:37.026434505Z",
"updateTime": "2026-08-05T13:41:37.026434505Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a1ec5523bfcd3a33f2ecf14eee78fca39d43d733",
"ref": "refs/heads/main"
}
}
}
[13:41:37.593] Task URL: https://cr-buildbucket.appspot.com/build/8674302728791845745
[13:41:37.593] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:2:0) after 6m48s
[13:41:38.558] Received PubSub notification, asking Buildbucket for the build status
[13:41:38.579] Build status: SCHEDULED
[13:42:49.846] Received PubSub notification, asking Buildbucket for the build status
[13:42:49.863] Build status: STARTED
[13:48:25.659] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:2:0)
[13:48:25.686] Build status: STARTED
[13:48:25.686] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:5:0) after 9m2s
[13:57:27.710] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:5:0)
[13:57:27.745] Build status: STARTED
[13:57:27.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:6:0) after 7m28s
[14:04:55.765] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:6:0)
[14:04:55.783] Build status: STARTED
[14:04:55.783] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:7:0) after 2m15s
[14:07:11.005] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:7:0)
[14:07:11.024] Build status: STARTED
[14:07:11.024] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:8:0) after 5m11s
[14:12:22.075] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:8:0)
[14:12:22.098] Build status: STARTED
[14:12:22.098] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:9:0) after 5m12s
[14:17:34.116] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:9:0)
[14:17:34.135] Build status: STARTED
[14:17:34.135] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:10:0) after 3m46s
[14:21:20.187] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:10:0)
[14:21:20.215] Build status: STARTED
[14:21:20.215] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:11:0) after 1m46s
[14:23:06.235] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:11:0)
[14:23:06.252] Build status: STARTED
[14:23:06.252] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:12:0) after 8m12s
[14:31:18.373] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:12:0)
[14:31:18.393] Build status: STARTED
[14:31:18.393] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:13:0) after 5m38s
[14:36:56.408] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:13:0)
[14:36:56.429] Build status: STARTED
[14:36:56.429] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:14:0) after 2m8s
[14:39:04.545] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:14:0)
[14:39:04.581] Build status: STARTED
[14:39:04.581] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:15:0) after 6m2s
[14:45:06.706] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:15:0)
[14:45:06.740] Build status: STARTED
[14:45:06.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:16:0) after 6m32s
[14:51:38.957] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:16:0)
[14:51:38.993] Build status: STARTED
[14:51:38.993] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:17:0) after 8m55s
[15:00:34.090] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:17:0)
[15:00:34.116] Build status: STARTED
[15:00:34.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:18:0) after 7m43s
[15:08:17.218] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:18:0)
[15:08:17.241] Build status: STARTED
[15:08:17.241] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:19:0) after 6m0s
[15:14:17.265] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:19:0)
[15:14:17.284] Build status: STARTED
[15:14:17.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:20:0) after 8m4s
[15:22:21.310] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:20:0)
[15:22:21.329] Build status: STARTED
[15:22:21.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:21:0) after 6m19s
[15:28:40.390] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:21:0)
[15:28:40.414] Build status: STARTED
[15:28:40.414] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:22:0) after 2m55s
[15:31:35.641] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:22:0)
[15:31:35.658] Build status: STARTED
[15:31:35.659] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:23:0) after 4m9s
[15:35:44.730] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:23:0)
[15:35:44.749] Build status: STARTED
[15:35:44.749] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:24:0) after 2m41s
[15:38:25.748] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:24:0)
[15:38:25.768] Build status: STARTED
[15:38:25.768] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:25:0) after 2m23s
[15:40:48.890] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:25:0)
[15:40:48.908] Build status: STARTED
[15:40:48.908] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:26:0) after 7m41s
[15:48:30.077] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:26:0)
[15:48:30.097] Build status: STARTED
[15:48:30.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:27:0) after 9m30s
[15:58:00.118] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:27:0)
[15:58:00.150] Build status: STARTED
[15:58:00.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:28:0) after 8m11s
[16:06:11.211] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:28:0)
[16:06:11.255] Build status: STARTED
[16:06:11.255] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:29:0) after 6m8s
[16:12:19.276] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:29:0)
[16:12:19.293] Build status: STARTED
[16:12:19.293] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:30:0) after 3m28s
[16:15:47.760] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:30:0)
[16:15:48.160] Build status: STARTED
[16:15:48.160] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:31:0) after 4m24s
[16:20:12.148] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:31:0)
[16:20:12.166] Build status: STARTED
[16:20:12.166] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:32:0) after 7m2s
[16:27:14.215] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:32:0)
[16:27:14.280] Build status: STARTED
[16:27:14.280] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:33:0) after 8m59s
[16:36:13.297] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:33:0)
[16:36:13.337] Build status: STARTED
[16:36:13.338] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:34:0) after 2m56s
[16:39:09.395] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:34:0)
[16:39:09.617] Build status: STARTED
[16:39:09.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:35:0) after 9m13s
[16:48:22.643] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:35:0)
[16:48:22.662] Build status: STARTED
[16:48:22.662] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:36:0) after 2m24s
[16:50:46.682] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:36:0)
[16:50:46.702] Build status: STARTED
[16:50:46.702] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:37:0) after 8m35s
[16:59:21.815] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:37:0)
[16:59:21.899] Build status: STARTED
[16:59:21.899] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:38:0) after 5m13s
[17:04:34.918] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:38:0)
[17:04:34.937] Build status: STARTED
[17:04:34.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8839732791395102272:39:0) after 6m13s
[17:06:52.110] Received PubSub notification, asking Buildbucket for the build status
[17:06:52.150] Build:
{
"id": "8674302728791845745",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 32028,
"createdBy": "project:chromium",
"createTime": "2026-08-05T13:41:37.026434505Z",
"startTime": "2026-08-05T13:42:48.368980503Z",
"endTime": "2026-08-05T17:06:51.851021365Z",
"updateTime": "2026-08-05T17:06:51.851021365Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a1ec5523bfcd3a33f2ecf14eee78fca39d43d733",
"ref": "refs/heads/main"
}
}
}
[17:06:52.150] Invocation finished in 3h25m16.989686139s with status FAILED