[01:34:51.131] New invocation is queued and will start shortly
[01:34:52.205] Starting the invocation (attempt 1)
[01:34:52.233] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/0b4371bdac738a1d3694383d097bfb3f1962c8ab"
[01:34:52.233] Popped gitiles commit info from properties and tags
[01:34:52.233] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[01:34:52.233] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[01:34:52.233] Buildbucket request:
{
"requestId": "8853096267600856912",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0b4371bdac738a1d3694383d097bfb3f1962c8ab",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8853096267600856912"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[01:34:52.756] Scheduled build:
{
"id": "8687666205730873873",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29729,
"createdBy": "project:chromium",
"createTime": "2026-03-11T01:34:52.291568210Z",
"updateTime": "2026-03-11T01:34:52.291568210Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0b4371bdac738a1d3694383d097bfb3f1962c8ab",
"ref": "refs/heads/main"
}
}
}
[01:34:52.756] Task URL: https://cr-buildbucket.appspot.com/build/8687666205730873873
[01:34:52.756] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:2:0) after 9m11s
[01:35:19.471] Received PubSub notification, asking Buildbucket for the build status
[01:35:19.494] Build status: SCHEDULED
[01:35:54.768] Received PubSub notification, asking Buildbucket for the build status
[01:35:54.783] Build status: STARTED
[01:44:04.064] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:2:0)
[01:44:04.086] Build status: STARTED
[01:44:04.086] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:5:0) after 6m56s
[01:51:00.122] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:5:0)
[01:51:00.144] Build status: STARTED
[01:51:00.144] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:6:0) after 4m54s
[01:55:54.190] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:6:0)
[01:55:54.223] Build status: STARTED
[01:55:54.223] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:7:0) after 8m1s
[02:03:55.290] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:7:0)
[02:03:55.307] Build status: STARTED
[02:03:55.307] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:8:0) after 4m37s
[02:08:32.384] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:8:0)
[02:08:32.400] Build status: STARTED
[02:08:32.401] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:9:0) after 7m40s
[02:16:12.468] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:9:0)
[02:16:12.489] Build status: STARTED
[02:16:12.489] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:10:0) after 1m42s
[02:17:54.547] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:10:0)
[02:17:54.589] Build status: STARTED
[02:17:54.589] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:11:0) after 7m10s
[02:25:04.613] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:11:0)
[02:25:04.638] Build status: STARTED
[02:25:04.638] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:12:0) after 7m34s
[02:32:38.658] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:12:0)
[02:32:38.753] Build status: STARTED
[02:32:38.753] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:13:0) after 4m12s
[02:36:50.797] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:13:0)
[02:36:50.823] Build status: STARTED
[02:36:50.824] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:14:0) after 5m43s
[02:42:33.846] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:14:0)
[02:42:33.863] Build status: STARTED
[02:42:33.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:15:0) after 8m38s
[02:51:11.945] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:15:0)
[02:51:11.987] Build status: STARTED
[02:51:11.987] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:16:0) after 5m25s
[02:56:37.052] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:16:0)
[02:56:37.118] Build status: STARTED
[02:56:37.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:17:0) after 8m1s
[03:04:38.186] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:17:0)
[03:04:38.222] Build status: STARTED
[03:04:38.222] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:18:0) after 8m45s
[03:13:23.314] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:18:0)
[03:13:23.336] Build status: STARTED
[03:13:23.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:19:0) after 4m36s
[03:17:59.358] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:19:0)
[03:17:59.379] Build status: STARTED
[03:17:59.379] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:20:0) after 6m30s
[03:24:29.491] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:20:0)
[03:24:29.507] Build status: STARTED
[03:24:29.507] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:21:0) after 5m46s
[03:30:15.522] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:21:0)
[03:30:15.539] Build status: STARTED
[03:30:15.539] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:22:0) after 4m21s
[03:34:36.563] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:22:0)
[03:34:36.580] Build status: STARTED
[03:34:36.581] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:23:0) after 1m9s
[03:35:45.652] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:23:0)
[03:35:45.669] Build status: STARTED
[03:35:45.669] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:24:0) after 7m12s
[03:42:57.763] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:24:0)
[03:42:57.779] Build status: STARTED
[03:42:57.779] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:25:0) after 8m44s
[03:51:41.843] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:25:0)
[03:51:41.863] Build status: STARTED
[03:51:41.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:26:0) after 7m25s
[03:59:06.934] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:26:0)
[03:59:07.010] Build status: STARTED
[03:59:07.010] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:27:0) after 4m15s
[04:03:22.090] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:27:0)
[04:03:22.110] Build status: STARTED
[04:03:22.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:28:0) after 2m38s
[04:06:00.141] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:28:0)
[04:06:00.214] Build status: STARTED
[04:06:00.214] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:29:0) after 4m44s
[04:10:44.239] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:29:0)
[04:10:44.267] Build status: STARTED
[04:10:44.267] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:30:0) after 3m16s
[04:14:00.286] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:30:0)
[04:14:00.318] Build status: STARTED
[04:14:00.318] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:31:0) after 2m58s
[04:16:58.387] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:31:0)
[04:16:58.405] Build status: STARTED
[04:16:58.405] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:32:0) after 6m0s
[04:22:58.521] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:32:0)
[04:22:58.597] Build status: STARTED
[04:22:58.597] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:33:0) after 3m19s
[04:26:17.689] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:33:0)
[04:26:17.717] Build status: STARTED
[04:26:17.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:34:0) after 4m11s
[04:30:28.792] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:34:0)
[04:30:28.843] Build status: STARTED
[04:30:28.843] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:35:0) after 2m11s
[04:32:39.925] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:35:0)
[04:32:39.999] Build status: STARTED
[04:32:39.999] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853096267600856912:36:0) after 5m58s
[04:37:15.773] Received PubSub notification, asking Buildbucket for the build status
[04:37:16.157] Build:
{
"id": "8687666205730873873",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29729,
"createdBy": "project:chromium",
"createTime": "2026-03-11T01:34:52.291568210Z",
"startTime": "2026-03-11T01:35:37.689107103Z",
"endTime": "2026-03-11T04:37:14.637540364Z",
"updateTime": "2026-03-11T04:37:14.637540364Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "0b4371bdac738a1d3694383d097bfb3f1962c8ab",
"ref": "refs/heads/main"
}
}
}
[04:37:16.157] Invocation finished in 3h2m25.032459647s with status FAILED