[16:57:07.502] New invocation is queued and will start shortly
[16:57:09.073] Starting the invocation (attempt 1)
[16:57:09.104] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/697a5f208a535bbf974a1535d25b8dc055a5edd9"
[16:57:09.104] Popped gitiles commit info from properties and tags
[16:57:09.104] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[16:57:09.105] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[16:57:09.105] Buildbucket request:
{
"requestId": "8842438398142615760",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "697a5f208a535bbf974a1535d25b8dc055a5edd9",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8842438398142615760"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[16:57:09.619] Scheduled build:
{
"id": "8677008335732562081",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31247,
"createdBy": "project:chromium",
"createTime": "2026-07-06T16:57:09.176662669Z",
"updateTime": "2026-07-06T16:57:09.176662669Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "697a5f208a535bbf974a1535d25b8dc055a5edd9",
"ref": "refs/heads/main"
}
}
}
[16:57:09.619] Task URL: https://cr-buildbucket.appspot.com/build/8677008335732562081
[16:57:09.619] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:2:0) after 4m53s
[16:57:10.013] Received PubSub notification, asking Buildbucket for the build status
[16:57:10.035] Build status: SCHEDULED
[16:58:27.559] Received PubSub notification, asking Buildbucket for the build status
[16:58:27.584] Build status: STARTED
[17:02:02.650] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:2:0)
[17:02:02.704] Build status: STARTED
[17:02:02.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:5:0) after 5m49s
[17:07:51.859] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:5:0)
[17:07:51.909] Build status: STARTED
[17:07:51.909] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:6:0) after 2m7s
[17:09:58.921] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:6:0)
[17:09:58.937] Build status: STARTED
[17:09:58.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:7:0) after 7m7s
[17:17:06.006] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:7:0)
[17:17:06.024] Build status: STARTED
[17:17:06.024] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:8:0) after 4m43s
[17:21:49.025] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:8:0)
[17:21:49.052] Build status: STARTED
[17:21:49.052] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:9:0) after 3m53s
[17:25:42.070] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:9:0)
[17:25:42.088] Build status: STARTED
[17:25:42.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:10:0) after 4m17s
[17:29:59.188] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:10:0)
[17:29:59.206] Build status: STARTED
[17:29:59.206] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:11:0) after 2m47s
[17:32:46.250] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:11:0)
[17:32:46.277] Build status: STARTED
[17:32:46.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:12:0) after 8m3s
[17:40:49.416] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:12:0)
[17:40:49.453] Build status: STARTED
[17:40:49.453] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:13:0) after 9m50s
[17:50:39.495] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:13:0)
[17:50:39.526] Build status: STARTED
[17:50:39.526] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:14:0) after 4m39s
[17:55:18.670] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:14:0)
[17:55:18.696] Build status: STARTED
[17:55:18.696] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:15:0) after 1m10s
[17:56:28.758] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:15:0)
[17:56:28.779] Build status: STARTED
[17:56:28.779] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:16:0) after 4m28s
[18:00:56.826] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:16:0)
[18:00:57.010] Build status: STARTED
[18:00:57.010] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:17:0) after 5m0s
[18:05:57.029] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:17:0)
[18:05:57.050] Build status: STARTED
[18:05:57.050] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:18:0) after 3m29s
[18:09:26.079] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:18:0)
[18:09:26.097] Build status: STARTED
[18:09:26.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:19:0) after 3m2s
[18:12:28.166] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:19:0)
[18:12:28.186] Build status: STARTED
[18:12:28.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:20:0) after 8m44s
[18:21:12.212] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:20:0)
[18:21:12.229] Build status: STARTED
[18:21:12.230] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:21:0) after 8m10s
[18:29:22.356] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:21:0)
[18:29:22.391] Build status: STARTED
[18:29:22.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:22:0) after 5m53s
[18:35:15.423] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:22:0)
[18:35:15.446] Build status: STARTED
[18:35:15.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:23:0) after 2m6s
[18:37:21.558] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:23:0)
[18:37:21.577] Build status: STARTED
[18:37:21.577] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:24:0) after 1m54s
[18:39:15.595] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:24:0)
[18:39:15.619] Build status: STARTED
[18:39:15.619] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:25:0) after 6m56s
[18:46:11.712] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:25:0)
[18:46:11.755] Build status: STARTED
[18:46:11.755] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:26:0) after 6m45s
[18:52:56.816] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:26:0)
[18:52:56.836] Build status: STARTED
[18:52:56.836] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:27:0) after 8m10s
[19:01:06.924] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:27:0)
[19:01:07.203] Build status: STARTED
[19:01:07.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:28:0) after 8m55s
[19:10:02.253] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:28:0)
[19:10:02.272] Build status: STARTED
[19:10:02.272] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:29:0) after 1m19s
[19:11:21.465] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:29:0)
[19:11:21.508] Build status: STARTED
[19:11:21.508] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:30:0) after 5m14s
[19:16:35.528] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:30:0)
[19:16:35.547] Build status: STARTED
[19:16:35.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:31:0) after 1m0s
[19:17:35.648] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:31:0)
[19:17:35.670] Build status: STARTED
[19:17:35.670] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:32:0) after 9m18s
[19:26:53.735] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:32:0)
[19:26:53.772] Build status: STARTED
[19:26:53.772] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:33:0) after 6m52s
[19:33:45.818] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:33:0)
[19:33:45.844] Build status: STARTED
[19:33:45.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:34:0) after 3m31s
[19:37:16.869] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:34:0)
[19:37:16.886] Build status: STARTED
[19:37:16.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:35:0) after 5m51s
[19:43:07.954] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:35:0)
[19:43:07.980] Build status: STARTED
[19:43:07.980] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:36:0) after 8m18s
[19:51:25.999] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:36:0)
[19:51:26.014] Build status: STARTED
[19:51:26.014] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:37:0) after 5m56s
[19:57:22.103] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:37:0)
[19:57:22.122] Build status: STARTED
[19:57:22.122] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:38:0) after 7m36s
[20:04:58.204] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:38:0)
[20:04:58.218] Build status: STARTED
[20:04:58.218] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:39:0) after 3m58s
[20:08:56.243] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:39:0)
[20:08:56.261] Build status: STARTED
[20:08:56.261] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:40:0) after 7m34s
[20:16:30.280] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:40:0)
[20:16:30.307] Build status: STARTED
[20:16:30.307] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:41:0) after 2m46s
[20:19:16.334] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:41:0)
[20:19:16.362] Build status: STARTED
[20:19:16.362] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:42:0) after 2m30s
[20:21:46.389] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:42:0)
[20:21:46.473] Build status: STARTED
[20:21:46.473] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:43:0) after 7m13s
[20:28:59.492] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:43:0)
[20:28:59.618] Build status: STARTED
[20:28:59.618] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:44:0) after 7m15s
[20:36:14.640] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:44:0)
[20:36:14.667] Build status: STARTED
[20:36:14.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:45:0) after 5m47s
[20:42:01.699] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:45:0)
[20:42:01.719] Build status: STARTED
[20:42:01.719] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842438398142615760:46:0) after 8m47s
[20:48:19.544] Received PubSub notification, asking Buildbucket for the build status
[20:48:19.564] Build:
{
"id": "8677008335732562081",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31247,
"createdBy": "project:chromium",
"createTime": "2026-07-06T16:57:09.176662669Z",
"startTime": "2026-07-06T16:58:26.760521554Z",
"endTime": "2026-07-06T20:48:18.890116361Z",
"updateTime": "2026-07-06T20:48:18.890116361Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "697a5f208a535bbf974a1535d25b8dc055a5edd9",
"ref": "refs/heads/main"
}
}
}
[20:48:19.564] Invocation finished in 3h51m12.069632144s with status FAILED