[20:48:21.432] New invocation is queued and will start shortly
[20:48:22.990] Starting the invocation (attempt 1)
[20:48:23.018] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/546d8aa3f1f5f9873d99deb49f4fe8d925adce7e"
[20:48:23.018] Popped gitiles commit info from properties and tags
[20:48:23.018] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[20:48:23.018] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[20:48:23.018] Buildbucket request:
{
"requestId": "8842423850268371088",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "546d8aa3f1f5f9873d99deb49f4fe8d925adce7e",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8842423850268371088"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[20:48:23.728] Scheduled build:
{
"id": "8676993787866340177",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31248,
"createdBy": "project:chromium",
"createTime": "2026-07-06T20:48:23.103485440Z",
"updateTime": "2026-07-06T20:48:23.103485440Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "546d8aa3f1f5f9873d99deb49f4fe8d925adce7e",
"ref": "refs/heads/main"
}
},
"canary": true
}
[20:48:23.728] Task URL: https://cr-buildbucket.appspot.com/build/8676993787866340177
[20:48:23.728] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:2:0) after 4m12s
[20:48:24.657] Received PubSub notification, asking Buildbucket for the build status
[20:48:24.808] Build status: SCHEDULED
[20:49:46.243] Received PubSub notification, asking Buildbucket for the build status
[20:49:46.269] Build status: STARTED
[20:52:36.075] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:2:0)
[20:52:36.106] Build status: STARTED
[20:52:36.106] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:5:0) after 9m18s
[21:01:54.166] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:5:0)
[21:01:54.195] Build status: STARTED
[21:01:54.195] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:6:0) after 1m36s
[21:03:30.285] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:6:0)
[21:03:30.390] Build status: STARTED
[21:03:30.390] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:7:0) after 4m59s
[21:08:29.414] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:7:0)
[21:08:29.431] Build status: STARTED
[21:08:29.431] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:8:0) after 6m56s
[21:15:25.490] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:8:0)
[21:15:25.513] Build status: STARTED
[21:15:25.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:9:0) after 4m48s
[21:20:13.657] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:9:0)
[21:20:13.689] Build status: STARTED
[21:20:13.689] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:10:0) after 2m48s
[21:23:01.721] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:10:0)
[21:23:01.776] Build status: STARTED
[21:23:01.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:11:0) after 5m55s
[21:28:56.840] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:11:0)
[21:28:56.863] Build status: STARTED
[21:28:56.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:12:0) after 1m40s
[21:30:36.885] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:12:0)
[21:30:36.908] Build status: STARTED
[21:30:36.908] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:13:0) after 3m28s
[21:34:04.937] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:13:0)
[21:34:04.970] Build status: STARTED
[21:34:04.970] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:14:0) after 1m57s
[21:36:01.964] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:14:0)
[21:36:02.032] Build status: STARTED
[21:36:02.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:15:0) after 1m22s
[21:37:24.102] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:15:0)
[21:37:24.172] Build status: STARTED
[21:37:24.172] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:16:0) after 2m3s
[21:39:27.195] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:16:0)
[21:39:27.211] Build status: STARTED
[21:39:27.211] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:17:0) after 7m38s
[21:47:05.326] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:17:0)
[21:47:05.352] Build status: STARTED
[21:47:05.352] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:18:0) after 7m49s
[21:54:54.418] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:18:0)
[21:54:54.458] Build status: STARTED
[21:54:54.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:19:0) after 6m34s
[22:01:28.477] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:19:0)
[22:01:28.508] Build status: STARTED
[22:01:28.508] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:20:0) after 9m41s
[22:11:09.570] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:20:0)
[22:11:09.594] Build status: STARTED
[22:11:09.594] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:21:0) after 9m15s
[22:20:24.668] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:21:0)
[22:20:24.713] Build status: STARTED
[22:20:24.713] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:22:0) after 4m13s
[22:24:37.732] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:22:0)
[22:24:37.749] Build status: STARTED
[22:24:37.749] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:23:0) after 2m38s
[22:27:15.752] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:23:0)
[22:27:15.776] Build status: STARTED
[22:27:15.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:24:0) after 8m56s
[22:36:11.854] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:24:0)
[22:36:11.910] Build status: STARTED
[22:36:11.910] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:25:0) after 2m34s
[22:38:45.961] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:25:0)
[22:38:45.986] Build status: STARTED
[22:38:45.986] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:26:0) after 7m48s
[22:46:34.008] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:26:0)
[22:46:34.032] Build status: STARTED
[22:46:34.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:27:0) after 8m31s
[22:55:05.070] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:27:0)
[22:55:05.088] Build status: STARTED
[22:55:05.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:28:0) after 8m5s
[23:03:10.113] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:28:0)
[23:03:10.131] Build status: STARTED
[23:03:10.131] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:29:0) after 2m7s
[23:05:17.182] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:29:0)
[23:05:17.204] Build status: STARTED
[23:05:17.204] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:30:0) after 9m59s
[23:15:16.244] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:30:0)
[23:15:16.271] Build status: STARTED
[23:15:16.271] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:31:0) after 9m21s
[23:24:37.337] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:31:0)
[23:24:37.354] Build status: STARTED
[23:24:37.354] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:32:0) after 3m30s
[23:28:07.405] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:32:0)
[23:28:07.421] Build status: STARTED
[23:28:07.421] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:33:0) after 3m4s
[23:31:11.448] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:33:0)
[23:31:11.465] Build status: STARTED
[23:31:11.465] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:34:0) after 1m48s
[23:32:59.486] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:34:0)
[23:32:59.512] Build status: STARTED
[23:32:59.512] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:35:0) after 6m46s
[23:39:45.559] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:35:0)
[23:39:45.582] Build status: STARTED
[23:39:45.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:36:0) after 7m36s
[23:47:21.617] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:36:0)
[23:47:21.634] Build status: STARTED
[23:47:21.634] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:37:0) after 7m51s
[23:55:12.979] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:37:0)
[23:55:12.996] Build status: STARTED
[23:55:12.997] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:38:0) after 9m52s
[00:05:05.017] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:38:0)
[00:05:05.042] Build status: STARTED
[00:05:05.042] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:39:0) after 4m35s
[00:09:40.102] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:39:0)
[00:09:40.159] Build status: STARTED
[00:09:40.160] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:40:0) after 5m20s
[00:15:00.219] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:40:0)
[00:15:00.242] Build status: STARTED
[00:15:00.242] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:41:0) after 5m41s
[00:20:41.261] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:41:0)
[00:20:41.277] Build status: STARTED
[00:20:41.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:42:0) after 1m7s
[00:21:48.382] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:42:0)
[00:21:48.407] Build status: STARTED
[00:21:48.407] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842423850268371088:43:0) after 7m24s
[00:25:44.881] Received PubSub notification, asking Buildbucket for the build status
[00:25:44.911] Build:
{
"id": "8676993787866340177",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31248,
"createdBy": "project:chromium",
"createTime": "2026-07-06T20:48:23.103485440Z",
"startTime": "2026-07-06T20:49:44.885934192Z",
"endTime": "2026-07-07T00:25:44.284805162Z",
"updateTime": "2026-07-07T00:25:44.284805162Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "546d8aa3f1f5f9873d99deb49f4fe8d925adce7e",
"ref": "refs/heads/main"
}
},
"canary": true
}
[00:25:44.911] Invocation finished in 3h37m23.482890268s with status FAILED