[21:23:54.968] New invocation is queued and will start shortly
[21:23:56.103] Starting the invocation (attempt 1)
[21:23:56.130] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0ce2c45ad39eaab020ad0c2a204beb274128de82"
[21:23:56.130] Popped gitiles commit info from properties and tags
[21:23:56.130] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:23:56.130] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:23:56.130] Buildbucket request:
{
"requestId": "8850937727935641712",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0ce2c45ad39eaab020ad0c2a204beb274128de82",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850937727935641712"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:23:56.820] Scheduled build:
{
"id": "8685507666000059729",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29984,
"createdBy": "project:chromium",
"createTime": "2026-04-03T21:23:56.194342566Z",
"updateTime": "2026-04-03T21:23:56.194342566Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0ce2c45ad39eaab020ad0c2a204beb274128de82",
"ref": "refs/heads/main"
}
}
}
[21:23:56.820] Task URL: https://cr-buildbucket.appspot.com/build/8685507666000059729
[21:23:56.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:2:0) after 9m27s
[21:23:57.632] Received PubSub notification, asking Buildbucket for the build status
[21:23:57.650] Build status: SCHEDULED
[21:28:06.542] Received PubSub notification, asking Buildbucket for the build status
[21:28:06.632] Build status: STARTED
[21:33:23.839] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:2:0)
[21:33:23.859] Build status: STARTED
[21:33:23.859] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:5:0) after 3m46s
[21:37:09.880] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:5:0)
[21:37:09.932] Build status: STARTED
[21:37:09.932] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:6:0) after 7m41s
[21:44:51.049] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:6:0)
[21:44:51.082] Build status: STARTED
[21:44:51.082] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:7:0) after 4m43s
[21:49:34.186] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:7:0)
[21:49:34.239] Build status: STARTED
[21:49:34.239] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:8:0) after 2m43s
[21:52:17.405] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:8:0)
[21:52:17.422] Build status: STARTED
[21:52:17.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:9:0) after 5m35s
[21:57:52.417] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:9:0)
[21:57:52.434] Build status: STARTED
[21:57:52.434] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:10:0) after 2m8s
[22:00:00.478] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:10:0)
[22:00:00.494] Build status: STARTED
[22:00:00.494] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:11:0) after 6m35s
[22:06:35.624] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:11:0)
[22:06:35.645] Build status: STARTED
[22:06:35.645] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:12:0) after 4m7s
[22:10:42.713] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:12:0)
[22:10:42.732] Build status: STARTED
[22:10:42.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:13:0) after 1m18s
[22:12:00.872] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:13:0)
[22:12:00.890] Build status: STARTED
[22:12:00.890] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:14:0) after 7m54s
[22:19:54.965] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:14:0)
[22:19:54.982] Build status: STARTED
[22:19:54.982] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:15:0) after 5m38s
[22:25:32.999] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:15:0)
[22:25:33.019] Build status: STARTED
[22:25:33.020] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:16:0) after 7m44s
[22:33:17.014] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:16:0)
[22:33:17.164] Build status: STARTED
[22:33:17.164] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:17:0) after 7m53s
[22:41:10.189] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:17:0)
[22:41:10.210] Build status: STARTED
[22:41:10.210] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:18:0) after 3m57s
[22:45:07.236] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:18:0)
[22:45:07.266] Build status: STARTED
[22:45:07.266] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:19:0) after 1m8s
[22:46:15.284] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:19:0)
[22:46:15.301] Build status: STARTED
[22:46:15.301] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:20:0) after 2m21s
[22:48:36.328] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:20:0)
[22:48:36.347] Build status: STARTED
[22:48:36.347] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:21:0) after 4m40s
[22:53:16.503] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:21:0)
[22:53:16.519] Build status: STARTED
[22:53:16.519] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:22:0) after 8m17s
[23:01:33.572] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:22:0)
[23:01:33.591] Build status: STARTED
[23:01:33.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:23:0) after 8m29s
[23:10:02.571] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:23:0)
[23:10:02.635] Build status: STARTED
[23:10:02.635] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:24:0) after 6m19s
[23:16:21.697] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:24:0)
[23:16:21.714] Build status: STARTED
[23:16:21.714] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:25:0) after 7m48s
[23:24:09.731] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:25:0)
[23:24:09.748] Build status: STARTED
[23:24:09.748] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:26:0) after 5m49s
[23:29:58.777] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:26:0)
[23:29:58.794] Build status: STARTED
[23:29:58.794] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:27:0) after 6m46s
[23:36:44.878] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:27:0)
[23:36:44.897] Build status: STARTED
[23:36:44.897] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:28:0) after 4m2s
[23:40:46.914] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:28:0)
[23:40:46.954] Build status: STARTED
[23:40:46.954] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:29:0) after 8m44s
[23:49:30.971] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:29:0)
[23:49:31.000] Build status: STARTED
[23:49:31.000] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:30:0) after 9m40s
[23:59:11.164] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:30:0)
[23:59:11.184] Build status: STARTED
[23:59:11.184] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:31:0) after 4m53s
[00:04:04.306] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:31:0)
[00:04:04.323] Build status: STARTED
[00:04:04.323] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:32:0) after 9m9s
[00:13:13.341] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:32:0)
[00:13:13.359] Build status: STARTED
[00:13:13.359] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:33:0) after 5m34s
[00:18:47.406] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:33:0)
[00:18:47.422] Build status: STARTED
[00:18:47.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:34:0) after 5m6s
[00:23:53.452] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:34:0)
[00:23:53.469] Build status: STARTED
[00:23:53.469] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:35:0) after 1m50s
[00:25:43.489] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:35:0)
[00:25:43.512] Build status: STARTED
[00:25:43.512] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:36:0) after 1m25s
[00:27:08.551] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:36:0)
[00:27:08.639] Build status: STARTED
[00:27:08.639] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:37:0) after 6m18s
[00:33:26.654] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:37:0)
[00:33:26.738] Build status: STARTED
[00:33:26.738] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:38:0) after 3m12s
[00:36:38.829] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:38:0)
[00:36:38.866] Build status: STARTED
[00:36:38.866] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:39:0) after 4m45s
[00:41:23.868] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:39:0)
[00:41:23.889] Build status: STARTED
[00:41:23.889] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:40:0) after 4m12s
[00:45:35.941] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:40:0)
[00:45:35.968] Build status: STARTED
[00:45:35.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:41:0) after 8m58s
[00:54:34.028] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:41:0)
[00:54:34.055] Build status: STARTED
[00:54:34.055] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850937727935641712:42:0) after 7m19s
[01:01:52.074] Received PubSub notification, asking Buildbucket for the build status
[01:01:52.242] Build:
{
"id": "8685507666000059729",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29984,
"createdBy": "project:chromium",
"createTime": "2026-04-03T21:23:56.194342566Z",
"startTime": "2026-04-03T21:28:04.556891219Z",
"endTime": "2026-04-04T01:01:51.704577654Z",
"updateTime": "2026-04-04T01:01:51.704577654Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0ce2c45ad39eaab020ad0c2a204beb274128de82",
"ref": "refs/heads/main"
}
}
}
[01:01:52.242] Invocation finished in 3h37m57.278593523s with status FAILED