[04:07:42.208] New invocation is queued and will start shortly
[04:07:43.332] Starting the invocation (attempt 1)
[04:07:43.355] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/88fddf6e884c3353c0bf038ff9ff0167a2bebef9"
[04:07:43.356] Popped gitiles commit info from properties and tags
[04:07:43.356] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:07:43.356] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:07:43.356] Buildbucket request:
{
"requestId": "8843664566520554064",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "88fddf6e884c3353c0bf038ff9ff0167a2bebef9",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8843664566520554064"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:07:43.960] Scheduled build:
{
"id": "8678234504586153393",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31139,
"createdBy": "project:chromium",
"createTime": "2026-06-23T04:07:43.433133644Z",
"updateTime": "2026-06-23T04:07:43.433133644Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "88fddf6e884c3353c0bf038ff9ff0167a2bebef9",
"ref": "refs/heads/main"
}
}
}
[04:07:43.960] Task URL: https://cr-buildbucket.appspot.com/build/8678234504586153393
[04:07:43.960] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:2:0) after 5m10s
[04:07:45.308] Received PubSub notification, asking Buildbucket for the build status
[04:07:45.332] Build status: SCHEDULED
[04:09:00.140] Received PubSub notification, asking Buildbucket for the build status
[04:09:00.180] Build status: STARTED
[04:12:53.994] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:2:0)
[04:12:54.012] Build status: STARTED
[04:12:54.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:5:0) after 4m45s
[04:17:39.031] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:5:0)
[04:17:39.055] Build status: STARTED
[04:17:39.056] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:6:0) after 7m23s
[04:25:02.076] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:6:0)
[04:25:02.102] Build status: STARTED
[04:25:02.102] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:7:0) after 4m37s
[04:29:39.089] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:7:0)
[04:29:39.108] Build status: STARTED
[04:29:39.108] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:8:0) after 6m22s
[04:36:01.168] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:8:0)
[04:36:01.191] Build status: STARTED
[04:36:01.191] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:9:0) after 1m54s
[04:37:55.232] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:9:0)
[04:37:55.267] Build status: STARTED
[04:37:55.267] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:10:0) after 5m3s
[04:42:58.283] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:10:0)
[04:42:58.305] Build status: STARTED
[04:42:58.305] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:11:0) after 1m17s
[04:44:15.287] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:11:0)
[04:44:15.383] Build status: STARTED
[04:44:15.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:12:0) after 9m55s
[04:54:10.402] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:12:0)
[04:54:10.429] Build status: STARTED
[04:54:10.429] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:13:0) after 7m11s
[05:01:21.458] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:13:0)
[05:01:21.491] Build status: STARTED
[05:01:21.491] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:14:0) after 7m10s
[05:08:31.558] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:14:0)
[05:08:31.580] Build status: STARTED
[05:08:31.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:15:0) after 6m57s
[05:15:28.602] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:15:0)
[05:15:28.646] Build status: STARTED
[05:15:28.646] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:16:0) after 4m2s
[05:19:30.728] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:16:0)
[05:19:30.752] Build status: STARTED
[05:19:30.752] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:17:0) after 5m51s
[05:25:21.820] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:17:0)
[05:25:21.839] Build status: STARTED
[05:25:21.839] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:18:0) after 3m59s
[05:29:20.900] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:18:0)
[05:29:20.920] Build status: STARTED
[05:29:20.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:19:0) after 8m21s
[05:37:41.938] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:19:0)
[05:37:41.960] Build status: STARTED
[05:37:41.960] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:20:0) after 2m56s
[05:40:38.026] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:20:0)
[05:40:38.044] Build status: STARTED
[05:40:38.044] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:21:0) after 9m12s
[05:49:50.060] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:21:0)
[05:49:50.212] Build status: STARTED
[05:49:50.212] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:22:0) after 4m59s
[05:54:49.282] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:22:0)
[05:54:49.513] Build status: STARTED
[05:54:49.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:23:0) after 3m45s
[05:58:34.521] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:23:0)
[05:58:34.548] Build status: STARTED
[05:58:34.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:24:0) after 3m3s
[06:01:37.573] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:24:0)
[06:01:37.593] Build status: STARTED
[06:01:37.593] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:25:0) after 2m40s
[06:04:17.611] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:25:0)
[06:04:17.630] Build status: STARTED
[06:04:17.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:26:0) after 3m51s
[06:08:08.617] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:26:0)
[06:08:08.632] Build status: STARTED
[06:08:08.632] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:27:0) after 2m35s
[06:10:43.750] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:27:0)
[06:10:43.767] Build status: STARTED
[06:10:43.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:28:0) after 6m31s
[06:17:14.820] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:28:0)
[06:17:14.834] Build status: STARTED
[06:17:14.834] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:29:0) after 8m7s
[06:25:21.853] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:29:0)
[06:25:21.881] Build status: STARTED
[06:25:21.881] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:30:0) after 9m51s
[06:35:12.901] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:30:0)
[06:35:12.977] Build status: STARTED
[06:35:12.977] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:31:0) after 1m48s
[06:37:01.049] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:31:0)
[06:37:01.064] Build status: STARTED
[06:37:01.064] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:32:0) after 4m40s
[06:41:41.114] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:32:0)
[06:41:41.150] Build status: STARTED
[06:41:41.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:33:0) after 6m46s
[06:48:27.143] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:33:0)
[06:48:27.167] Build status: STARTED
[06:48:27.167] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:34:0) after 7m54s
[06:56:21.187] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:34:0)
[06:56:21.208] Build status: STARTED
[06:56:21.208] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:35:0) after 6m37s
[07:02:58.253] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:35:0)
[07:02:58.286] Build status: STARTED
[07:02:58.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:36:0) after 4m34s
[07:07:32.439] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:36:0)
[07:07:32.477] Build status: STARTED
[07:07:32.477] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:37:0) after 8m56s
[07:16:28.506] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:37:0)
[07:16:28.532] Build status: STARTED
[07:16:28.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:38:0) after 3m43s
[07:20:11.710] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:38:0)
[07:20:11.729] Build status: STARTED
[07:20:11.729] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:39:0) after 1m2s
[07:21:13.709] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:39:0)
[07:21:13.779] Build status: STARTED
[07:21:13.779] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:40:0) after 2m30s
[07:23:43.797] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:40:0)
[07:23:43.813] Build status: STARTED
[07:23:43.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:41:0) after 2m22s
[07:26:05.837] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:41:0)
[07:26:05.853] Build status: STARTED
[07:26:05.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843664566520554064:42:0) after 7m46s
[07:32:13.130] Received PubSub notification, asking Buildbucket for the build status
[07:32:13.147] Build:
{
"id": "8678234504586153393",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31139,
"createdBy": "project:chromium",
"createTime": "2026-06-23T04:07:43.433133644Z",
"startTime": "2026-06-23T04:08:59.135177460Z",
"endTime": "2026-06-23T07:32:12.808494130Z",
"updateTime": "2026-06-23T07:32:12.808494130Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "88fddf6e884c3353c0bf038ff9ff0167a2bebef9",
"ref": "refs/heads/main"
}
}
}
[07:32:13.148] Invocation finished in 3h24m30.94287273s with status FAILED