[18:13:03.320] New invocation is queued and will start shortly
[18:13:04.689] Starting the invocation (attempt 1)
[18:13:04.741] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/4f939d4646c8dfa5152183760bf2d2bf9428946c"
[18:13:04.741] Popped gitiles commit info from properties and tags
[18:13:04.741] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[18:13:04.741] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[18:13:04.741] Buildbucket request:
{
"requestId": "8843067799784515920",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4f939d4646c8dfa5152183760bf2d2bf9428946c",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8843067799784515920"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[18:13:05.329] Scheduled build:
{
"id": "8677637737559177153",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31187,
"createdBy": "project:chromium",
"createTime": "2026-06-29T18:13:04.820143529Z",
"updateTime": "2026-06-29T18:13:04.820143529Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4f939d4646c8dfa5152183760bf2d2bf9428946c",
"ref": "refs/heads/main"
}
}
}
[18:13:05.329] Task URL: https://cr-buildbucket.appspot.com/build/8677637737559177153
[18:13:05.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:2:0) after 1m48s
[18:13:06.342] Received PubSub notification, asking Buildbucket for the build status
[18:13:06.372] Build status: SCHEDULED
[18:14:08.176] Received PubSub notification, asking Buildbucket for the build status
[18:14:08.212] Build status: STARTED
[18:14:53.365] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:2:0)
[18:14:53.383] Build status: STARTED
[18:14:53.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:5:0) after 6m57s
[18:21:50.384] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:5:0)
[18:21:50.408] Build status: STARTED
[18:21:50.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:6:0) after 8m54s
[18:30:44.429] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:6:0)
[18:30:44.452] Build status: STARTED
[18:30:44.452] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:7:0) after 1m23s
[18:32:07.514] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:7:0)
[18:32:07.553] Build status: STARTED
[18:32:07.553] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:8:0) after 6m12s
[18:38:19.561] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:8:0)
[18:38:19.578] Build status: STARTED
[18:38:19.578] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:9:0) after 8m21s
[18:46:40.554] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:9:0)
[18:46:40.572] Build status: STARTED
[18:46:40.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:10:0) after 5m32s
[18:52:12.627] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:10:0)
[18:52:12.666] Build status: STARTED
[18:52:12.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:11:0) after 9m33s
[19:01:45.782] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:11:0)
[19:01:45.796] Build status: STARTED
[19:01:45.797] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:12:0) after 4m37s
[19:06:22.819] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:12:0)
[19:06:22.836] Build status: STARTED
[19:06:22.836] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:13:0) after 6m40s
[19:13:02.837] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:13:0)
[19:13:02.861] Build status: STARTED
[19:13:02.862] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:14:0) after 2m24s
[19:15:26.848] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:14:0)
[19:15:26.867] Build status: STARTED
[19:15:26.867] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:15:0) after 4m59s
[19:20:25.886] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:15:0)
[19:20:25.904] Build status: STARTED
[19:20:25.904] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:16:0) after 4m32s
[19:24:57.959] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:16:0)
[19:24:57.976] Build status: STARTED
[19:24:57.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:17:0) after 3m56s
[19:28:54.077] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:17:0)
[19:28:54.099] Build status: STARTED
[19:28:54.099] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:18:0) after 9m0s
[19:37:54.196] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:18:0)
[19:37:54.214] Build status: STARTED
[19:37:54.215] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:19:0) after 9m37s
[19:47:31.247] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:19:0)
[19:47:31.264] Build status: STARTED
[19:47:31.264] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:20:0) after 9m8s
[19:56:39.312] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:20:0)
[19:56:39.339] Build status: STARTED
[19:56:39.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:21:0) after 7m21s
[20:04:00.362] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:21:0)
[20:04:00.436] Build status: STARTED
[20:04:00.436] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:22:0) after 9m51s
[20:13:51.513] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:22:0)
[20:13:51.532] Build status: STARTED
[20:13:51.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:23:0) after 3m43s
[20:17:34.589] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:23:0)
[20:17:34.607] Build status: STARTED
[20:17:34.607] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:24:0) after 7m31s
[20:25:05.627] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:24:0)
[20:25:05.645] Build status: STARTED
[20:25:05.645] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:25:0) after 4m46s
[20:29:51.667] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:25:0)
[20:29:51.689] Build status: STARTED
[20:29:51.689] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:26:0) after 5m47s
[20:35:38.744] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:26:0)
[20:35:39.032] Build status: STARTED
[20:35:39.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:27:0) after 2m8s
[20:37:47.058] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:27:0)
[20:37:47.086] Build status: STARTED
[20:37:47.086] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:28:0) after 2m36s
[20:40:23.295] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:28:0)
[20:40:23.315] Build status: STARTED
[20:40:23.315] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:29:0) after 5m40s
[20:46:03.373] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:29:0)
[20:46:03.412] Build status: STARTED
[20:46:03.412] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:30:0) after 3m30s
[20:49:33.541] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:30:0)
[20:49:33.556] Build status: STARTED
[20:49:33.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:31:0) after 9m3s
[20:58:36.577] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:31:0)
[20:58:36.594] Build status: STARTED
[20:58:36.594] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:32:0) after 4m20s
[21:02:56.695] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:32:0)
[21:02:56.712] Build status: STARTED
[21:02:56.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:33:0) after 8m49s
[21:11:45.735] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:33:0)
[21:11:45.759] Build status: STARTED
[21:11:45.759] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:34:0) after 9m42s
[21:21:27.827] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:34:0)
[21:21:27.843] Build status: STARTED
[21:21:27.843] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:35:0) after 6m49s
[21:28:16.916] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:35:0)
[21:28:16.932] Build status: STARTED
[21:28:16.932] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:36:0) after 5m9s
[21:33:25.959] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:36:0)
[21:33:25.976] Build status: STARTED
[21:33:25.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:37:0) after 9m57s
[21:43:22.996] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:37:0)
[21:43:23.066] Build status: STARTED
[21:43:23.066] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:38:0) after 6m25s
[21:49:48.167] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:38:0)
[21:49:48.186] Build status: STARTED
[21:49:48.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843067799784515920:39:0) after 9m3s
[21:52:05.581] Received PubSub notification, asking Buildbucket for the build status
[21:52:05.669] Build:
{
"id": "8677637737559177153",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31187,
"createdBy": "project:chromium",
"createTime": "2026-06-29T18:13:04.820143529Z",
"startTime": "2026-06-29T18:14:07.933437599Z",
"endTime": "2026-06-29T21:52:05.288962784Z",
"updateTime": "2026-06-29T21:52:05.288962784Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4f939d4646c8dfa5152183760bf2d2bf9428946c",
"ref": "refs/heads/main"
}
}
}
[21:52:05.669] Invocation finished in 3h39m2.354757179s with status FAILED