[09:48:32.621] New invocation is queued and will start shortly
[09:48:33.731] Starting the invocation (attempt 1)
[09:48:33.755] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/6fb1f4315499853f4f7b6cba18ffefbed05c1b0e"
[09:48:33.755] Popped gitiles commit info from properties and tags
[09:48:33.755] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[09:48:33.755] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[09:48:33.756] Buildbucket request:
{
"requestId": "8850890880021835328",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "6fb1f4315499853f4f7b6cba18ffefbed05c1b0e",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850890880021835328"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[09:48:34.216] Scheduled build:
{
"id": "8685460818115405745",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29988,
"createdBy": "project:chromium",
"createTime": "2026-04-04T09:48:33.818286121Z",
"updateTime": "2026-04-04T09:48:33.818286121Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "6fb1f4315499853f4f7b6cba18ffefbed05c1b0e",
"ref": "refs/heads/main"
}
},
"canary": true
}
[09:48:34.216] Task URL: https://cr-buildbucket.appspot.com/build/8685460818115405745
[09:48:34.216] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:2:0) after 6m44s
[09:48:35.184] Received PubSub notification, asking Buildbucket for the build status
[09:48:35.338] Build status: SCHEDULED
[09:49:39.247] Received PubSub notification, asking Buildbucket for the build status
[09:49:39.311] Build status: STARTED
[09:55:18.270] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:2:0)
[09:55:18.293] Build status: STARTED
[09:55:18.293] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:5:0) after 8m10s
[10:03:28.411] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:5:0)
[10:03:28.491] Build status: STARTED
[10:03:28.491] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:6:0) after 8m53s
[10:12:21.504] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:6:0)
[10:12:21.555] Build status: STARTED
[10:12:21.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:7:0) after 6m53s
[10:19:14.683] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:7:0)
[10:19:14.703] Build status: STARTED
[10:19:14.703] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:8:0) after 6m12s
[10:25:26.753] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:8:0)
[10:25:26.768] Build status: STARTED
[10:25:26.768] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:9:0) after 8m34s
[10:34:00.784] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:9:0)
[10:34:00.871] Build status: STARTED
[10:34:00.871] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:10:0) after 8m26s
[10:42:26.887] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:10:0)
[10:42:26.906] Build status: STARTED
[10:42:26.906] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:11:0) after 2m25s
[10:44:52.075] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:11:0)
[10:44:52.097] Build status: STARTED
[10:44:52.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:12:0) after 6m53s
[10:51:45.108] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:12:0)
[10:51:45.125] Build status: STARTED
[10:51:45.125] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:13:0) after 4m24s
[10:56:09.260] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:13:0)
[10:56:09.277] Build status: STARTED
[10:56:09.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:14:0) after 2m54s
[10:59:03.324] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:14:0)
[10:59:03.505] Build status: STARTED
[10:59:03.505] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:15:0) after 1m16s
[11:00:19.617] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:15:0)
[11:00:19.635] Build status: STARTED
[11:00:19.635] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:16:0) after 6m38s
[11:06:58.091] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:16:0)
[11:06:58.144] Build status: STARTED
[11:06:58.144] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:17:0) after 7m26s
[11:14:24.158] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:17:0)
[11:14:24.247] Build status: STARTED
[11:14:24.247] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:18:0) after 9m21s
[11:23:45.318] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:18:0)
[11:23:45.371] Build status: STARTED
[11:23:45.371] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:19:0) after 3m46s
[11:27:31.388] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:19:0)
[11:27:31.402] Build status: STARTED
[11:27:31.403] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:20:0) after 4m35s
[11:32:06.417] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:20:0)
[11:32:06.441] Build status: STARTED
[11:32:06.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:21:0) after 5m34s
[11:37:40.438] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:21:0)
[11:37:40.461] Build status: STARTED
[11:37:40.461] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:22:0) after 9m41s
[11:47:21.501] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:22:0)
[11:47:21.535] Build status: STARTED
[11:47:21.535] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:23:0) after 8m13s
[11:55:34.552] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:23:0)
[11:55:34.599] Build status: STARTED
[11:55:34.599] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:24:0) after 3m53s
[11:59:27.698] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:24:0)
[11:59:27.719] Build status: STARTED
[11:59:27.719] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:25:0) after 3m24s
[12:02:51.709] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:25:0)
[12:02:51.760] Build status: STARTED
[12:02:51.760] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:26:0) after 3m57s
[12:06:48.931] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:26:0)
[12:06:48.948] Build status: STARTED
[12:06:48.948] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:27:0) after 5m29s
[12:12:18.000] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:27:0)
[12:12:18.023] Build status: STARTED
[12:12:18.023] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:28:0) after 1m52s
[12:14:10.228] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:28:0)
[12:14:10.246] Build status: STARTED
[12:14:10.246] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:29:0) after 2m9s
[12:16:19.640] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:29:0)
[12:16:19.660] Build status: STARTED
[12:16:19.660] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:30:0) after 5m35s
[12:21:54.676] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:30:0)
[12:21:54.695] Build status: STARTED
[12:21:54.695] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:31:0) after 1m58s
[12:23:52.710] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:31:0)
[12:23:52.725] Build status: STARTED
[12:23:52.725] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:32:0) after 1m28s
[12:25:20.731] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:32:0)
[12:25:20.756] Build status: STARTED
[12:25:20.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:33:0) after 1m49s
[12:27:09.875] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:33:0)
[12:27:09.901] Build status: STARTED
[12:27:09.901] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850890880021835328:34:0) after 7m38s
[12:33:53.141] Received PubSub notification, asking Buildbucket for the build status
[12:33:53.167] Build:
{
"id": "8685460818115405745",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29988,
"createdBy": "project:chromium",
"createTime": "2026-04-04T09:48:33.818286121Z",
"startTime": "2026-04-04T09:49:38.893530578Z",
"endTime": "2026-04-04T12:33:34.342580152Z",
"updateTime": "2026-04-04T12:33:34.342580152Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "6fb1f4315499853f4f7b6cba18ffefbed05c1b0e",
"ref": "refs/heads/main"
}
},
"canary": true
}
[12:33:53.167] Invocation finished in 2h45m20.55131563s with status FAILED