[15:24:53.033] New invocation is queued and will start shortly
[15:24:54.062] Starting the invocation (attempt 1)
[15:24:54.096] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/4a3bc7f99e00edca84ff57d813b5f5542391095f"
[15:24:54.096] Popped gitiles commit info from properties and tags
[15:24:54.096] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:24:54.097] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:24:54.097] Buildbucket request:
{
"requestId": "8858298670577571504",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4a3bc7f99e00edca84ff57d813b5f5542391095f",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8858298670577571504"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:24:54.635] Scheduled build:
{
"id": "8692868608723887233",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 28898,
"createdBy": "project:chromium",
"createTime": "2026-01-12T15:24:54.173416704Z",
"updateTime": "2026-01-12T15:24:54.173416704Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4a3bc7f99e00edca84ff57d813b5f5542391095f",
"ref": "refs/heads/main"
}
}
}
[15:24:54.635] Task URL: https://cr-buildbucket.appspot.com/build/8692868608723887233
[15:24:54.635] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:2:0) after 4m15s
[15:24:55.571] Received PubSub notification, asking Buildbucket for the build status
[15:24:55.746] Build status: SCHEDULED
[15:26:03.011] Received PubSub notification, asking Buildbucket for the build status
[15:26:03.030] Build status: STARTED
[15:29:09.700] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:2:0)
[15:29:09.721] Build status: STARTED
[15:29:09.721] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:5:0) after 3m55s
[15:33:04.742] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:5:0)
[15:33:04.757] Build status: STARTED
[15:33:04.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:6:0) after 3m12s
[15:36:16.772] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:6:0)
[15:36:16.785] Build status: STARTED
[15:36:16.785] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:7:0) after 2m25s
[15:38:41.810] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:7:0)
[15:38:41.858] Build status: STARTED
[15:38:41.858] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:8:0) after 5m37s
[15:44:18.836] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:8:0)
[15:44:18.849] Build status: STARTED
[15:44:18.849] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:9:0) after 4m45s
[15:49:03.870] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:9:0)
[15:49:03.897] Build status: STARTED
[15:49:03.897] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:10:0) after 8m53s
[15:57:56.960] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:10:0)
[15:57:56.974] Build status: STARTED
[15:57:56.974] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:11:0) after 8m47s
[16:06:43.989] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:11:0)
[16:06:44.007] Build status: STARTED
[16:06:44.007] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:12:0) after 8m33s
[16:15:17.021] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:12:0)
[16:15:17.037] Build status: STARTED
[16:15:17.038] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:13:0) after 6m9s
[16:21:26.055] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:13:0)
[16:21:26.074] Build status: STARTED
[16:21:26.075] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:14:0) after 6m39s
[16:28:05.140] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:14:0)
[16:28:05.157] Build status: STARTED
[16:28:05.157] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:15:0) after 1m45s
[16:29:50.176] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:15:0)
[16:29:50.225] Build status: STARTED
[16:29:50.225] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:16:0) after 5m18s
[16:35:08.246] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:16:0)
[16:35:08.289] Build status: STARTED
[16:35:08.289] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:17:0) after 2m46s
[16:37:54.404] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:17:0)
[16:37:54.417] Build status: STARTED
[16:37:54.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:18:0) after 6m26s
[16:44:20.431] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:18:0)
[16:44:20.446] Build status: STARTED
[16:44:20.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:19:0) after 3m59s
[16:48:19.513] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:19:0)
[16:48:19.528] Build status: STARTED
[16:48:19.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:20:0) after 7m57s
[16:56:16.543] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:20:0)
[16:56:16.558] Build status: STARTED
[16:56:16.558] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:21:0) after 6m49s
[17:03:05.579] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:21:0)
[17:03:05.598] Build status: STARTED
[17:03:05.598] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:22:0) after 3m24s
[17:06:29.737] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:22:0)
[17:06:29.758] Build status: STARTED
[17:06:29.758] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:23:0) after 3m28s
[17:09:57.962] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:23:0)
[17:09:57.981] Build status: STARTED
[17:09:57.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:24:0) after 7m2s
[17:17:00.006] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:24:0)
[17:17:00.045] Build status: STARTED
[17:17:00.045] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:25:0) after 2m14s
[17:19:14.068] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:25:0)
[17:19:14.085] Build status: STARTED
[17:19:14.085] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:26:0) after 5m18s
[17:24:32.180] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:26:0)
[17:24:32.200] Build status: STARTED
[17:24:32.200] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:27:0) after 7m9s
[17:31:41.270] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:27:0)
[17:31:41.288] Build status: STARTED
[17:31:41.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:28:0) after 6m18s
[17:37:59.481] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:28:0)
[17:37:59.497] Build status: STARTED
[17:37:59.497] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:29:0) after 3m5s
[17:41:04.516] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:29:0)
[17:41:04.536] Build status: STARTED
[17:41:04.536] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:30:0) after 4m39s
[17:45:43.632] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:30:0)
[17:45:43.703] Build status: STARTED
[17:45:43.703] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:31:0) after 8m47s
[17:54:30.750] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:31:0)
[17:54:30.766] Build status: STARTED
[17:54:30.766] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:32:0) after 5m25s
[17:59:55.933] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:32:0)
[17:59:55.950] Build status: STARTED
[17:59:55.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:33:0) after 6m22s
[18:06:18.017] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:33:0)
[18:06:18.033] Build status: STARTED
[18:06:18.033] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:34:0) after 1m53s
[18:08:11.024] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:34:0)
[18:08:11.040] Build status: STARTED
[18:08:11.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:35:0) after 1m36s
[18:09:47.055] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:35:0)
[18:09:47.071] Build status: STARTED
[18:09:47.071] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:36:0) after 2m13s
[18:12:00.094] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:36:0)
[18:12:00.115] Build status: STARTED
[18:12:00.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:37:0) after 9m47s
[18:21:47.199] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:37:0)
[18:21:47.216] Build status: STARTED
[18:21:47.216] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:38:0) after 7m3s
[18:28:50.232] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:38:0)
[18:28:50.302] Build status: STARTED
[18:28:50.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:39:0) after 5m28s
[18:34:18.317] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:39:0)
[18:34:18.341] Build status: STARTED
[18:34:18.341] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:40:0) after 3m4s
[18:37:22.330] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:40:0)
[18:37:22.349] Build status: STARTED
[18:37:22.349] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:41:0) after 9m2s
[18:46:24.363] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:41:0)
[18:46:24.383] Build status: STARTED
[18:46:24.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8858298670577571504:42:0) after 7m55s
[18:52:00.984] Received PubSub notification, asking Buildbucket for the build status
[18:52:01.000] Build:
{
"id": "8692868608723887233",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 28898,
"createdBy": "project:chromium",
"createTime": "2026-01-12T15:24:54.173416704Z",
"startTime": "2026-01-12T15:26:02.321308944Z",
"endTime": "2026-01-12T18:52:00.714573628Z",
"updateTime": "2026-01-12T18:52:00.714573628Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4a3bc7f99e00edca84ff57d813b5f5542391095f",
"ref": "refs/heads/main"
}
}
}
[18:52:01.000] Invocation finished in 3h27m7.978214907s with status FAILED