[21:12:47.657] New invocation is queued and will start shortly
[21:12:49.259] Starting the invocation (attempt 1)
[21:12:49.286] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0cf4791f29f4497bea266946cd8429ccc153c77a"
[21:12:49.286] Popped gitiles commit info from properties and tags
[21:12:49.286] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:12:49.286] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:12:49.286] Buildbucket request:
{
"requestId": "8835265152478334800",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0cf4791f29f4497bea266946cd8429ccc153c77a",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8835265152478334800"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:12:50.122] Scheduled build:
{
"id": "8669835089999410225",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 32761,
"createdBy": "project:chromium",
"createTime": "2026-09-23T21:12:49.397226051Z",
"updateTime": "2026-09-23T21:12:49.397226051Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0cf4791f29f4497bea266946cd8429ccc153c77a",
"ref": "refs/heads/main"
}
},
"canary": true
}
[21:12:50.122] Task URL: https://cr-buildbucket.appspot.com/build/8669835089999410225
[21:12:50.122] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:2:0) after 1m57s
[21:12:51.048] Received PubSub notification, asking Buildbucket for the build status
[21:12:51.072] Build status: SCHEDULED
[21:14:47.393] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:2:0)
[21:14:47.410] Build status: SCHEDULED
[21:14:47.410] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:4:0) after 1m53s
[21:16:40.393] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:4:0)
[21:16:40.429] Build status: SCHEDULED
[21:16:40.429] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:5:0) after 3m17s
[21:19:57.483] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:5:0)
[21:19:57.503] Build status: SCHEDULED
[21:19:57.503] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:6:0) after 1m52s
[21:21:49.607] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:6:0)
[21:21:49.636] Build status: SCHEDULED
[21:21:49.636] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:7:0) after 2m31s
[21:24:20.691] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:7:0)
[21:24:20.711] Build status: SCHEDULED
[21:24:20.711] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:8:0) after 1m29s
[21:25:49.767] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:8:0)
[21:25:49.817] Build status: SCHEDULED
[21:25:49.817] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:9:0) after 5m25s
[21:31:14.895] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:9:0)
[21:31:14.915] Build status: SCHEDULED
[21:31:14.915] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:10:0) after 3m38s
[21:34:52.933] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:10:0)
[21:34:52.958] Build status: SCHEDULED
[21:34:52.958] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:11:0) after 6m34s
[21:41:27.080] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:11:0)
[21:41:27.109] Build status: SCHEDULED
[21:41:27.109] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:12:0) after 3m4s
[21:44:31.137] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:12:0)
[21:44:31.156] Build status: SCHEDULED
[21:44:31.156] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:13:0) after 2m39s
[21:47:10.315] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:13:0)
[21:47:10.357] Build status: SCHEDULED
[21:47:10.357] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:14:0) after 3m18s
[21:50:28.363] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:14:0)
[21:50:28.382] Build status: SCHEDULED
[21:50:28.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:15:0) after 7m59s
[21:53:51.130] Received PubSub notification, asking Buildbucket for the build status
[21:53:51.164] Build status: STARTED
[21:58:27.438] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:15:0)
[21:58:27.458] Build status: STARTED
[21:58:27.458] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:17:0) after 5m14s
[22:03:41.579] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:17:0)
[22:03:41.595] Build status: STARTED
[22:03:41.595] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:18:0) after 9m9s
[22:12:50.616] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:18:0)
[22:12:50.652] Build status: STARTED
[22:12:50.652] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:19:0) after 1m3s
[22:13:53.744] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:19:0)
[22:13:53.769] Build status: STARTED
[22:13:53.769] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:20:0) after 7m8s
[22:21:01.835] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:20:0)
[22:21:01.859] Build status: STARTED
[22:21:01.859] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:21:0) after 7m44s
[22:28:45.927] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:21:0)
[22:28:46.019] Build status: STARTED
[22:28:46.019] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:22:0) after 8m14s
[22:37:00.110] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:22:0)
[22:37:00.133] Build status: STARTED
[22:37:00.133] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:23:0) after 1m19s
[22:38:19.278] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:23:0)
[22:38:19.303] Build status: STARTED
[22:38:19.303] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:24:0) after 9m41s
[22:48:00.594] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:24:0)
[22:48:00.625] Build status: STARTED
[22:48:00.625] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:25:0) after 6m5s
[22:54:05.731] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:25:0)
[22:54:05.825] Build status: STARTED
[22:54:05.825] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:26:0) after 2m0s
[22:56:05.883] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:26:0)
[22:56:05.941] Build status: STARTED
[22:56:05.941] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:27:0) after 9m31s
[23:05:37.010] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:27:0)
[23:05:37.029] Build status: STARTED
[23:05:37.029] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:28:0) after 6m35s
[23:12:12.049] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:28:0)
[23:12:12.065] Build status: STARTED
[23:12:12.065] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:29:0) after 4m0s
[23:16:12.112] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:29:0)
[23:16:12.130] Build status: STARTED
[23:16:12.130] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:30:0) after 7m9s
[23:23:21.204] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:30:0)
[23:23:21.219] Build status: STARTED
[23:23:21.219] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:31:0) after 6m43s
[23:30:04.261] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:31:0)
[23:30:04.346] Build status: STARTED
[23:30:04.346] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:32:0) after 5m28s
[23:35:32.388] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:32:0)
[23:35:32.404] Build status: STARTED
[23:35:32.404] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:33:0) after 4m18s
[23:39:50.424] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:33:0)
[23:39:50.447] Build status: STARTED
[23:39:50.447] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:34:0) after 1m17s
[23:41:07.640] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:34:0)
[23:41:07.659] Build status: STARTED
[23:41:07.659] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:35:0) after 1m18s
[23:42:25.745] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:35:0)
[23:42:25.832] Build status: STARTED
[23:42:25.832] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:36:0) after 6m33s
[23:48:58.866] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:36:0)
[23:48:58.893] Build status: STARTED
[23:48:58.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:37:0) after 6m57s
[23:55:56.049] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:37:0)
[23:55:56.074] Build status: STARTED
[23:55:56.074] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8835265152478334800:38:0) after 2m42s
[23:57:38.400] Received PubSub notification, asking Buildbucket for the build status
[23:57:38.419] Build:
{
"id": "8669835089999410225",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 32761,
"createdBy": "project:chromium",
"createTime": "2026-09-23T21:12:49.397226051Z",
"startTime": "2026-09-23T21:53:50.950228368Z",
"endTime": "2026-09-23T23:57:35.869064151Z",
"updateTime": "2026-09-23T23:57:35.869064151Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0cf4791f29f4497bea266946cd8429ccc153c77a",
"ref": "refs/heads/main"
}
},
"canary": true
}
[23:57:38.419] Invocation finished in 2h44m50.768543164s with status FAILED