[00:19:50.892] New invocation is queued and will start shortly
[00:19:52.145] Starting the invocation (attempt 1)
[00:19:52.171] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/a8d67ad9516e6e53de07de42c423eff131a2dc92"
[00:19:52.171] Popped gitiles commit info from properties and tags
[00:19:52.171] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[00:19:52.171] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[00:19:52.172] Buildbucket request:
{
"requestId": "8843678901937521952",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a8d67ad9516e6e53de07de42c423eff131a2dc92",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8843678901937521952"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[00:19:52.739] Scheduled build:
{
"id": "8678248839876792817",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31138,
"createdBy": "project:chromium",
"createTime": "2026-06-23T00:19:52.235502153Z",
"updateTime": "2026-06-23T00:19:52.235502153Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a8d67ad9516e6e53de07de42c423eff131a2dc92",
"ref": "refs/heads/main"
}
}
}
[00:19:52.739] Task URL: https://cr-buildbucket.appspot.com/build/8678248839876792817
[00:19:52.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:2:0) after 4m35s
[00:19:53.674] Received PubSub notification, asking Buildbucket for the build status
[00:19:53.692] Build status: SCHEDULED
[00:21:10.971] Received PubSub notification, asking Buildbucket for the build status
[00:21:10.995] Build status: STARTED
[00:24:27.806] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:2:0)
[00:24:27.831] Build status: STARTED
[00:24:27.831] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:5:0) after 4m26s
[00:28:53.958] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:5:0)
[00:28:53.978] Build status: STARTED
[00:28:53.978] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:6:0) after 1m40s
[00:30:33.994] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:6:0)
[00:30:34.046] Build status: STARTED
[00:30:34.046] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:7:0) after 6m39s
[00:37:13.064] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:7:0)
[00:37:13.079] Build status: STARTED
[00:37:13.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:8:0) after 9m5s
[00:46:18.143] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:8:0)
[00:46:18.443] Build status: STARTED
[00:46:18.443] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:9:0) after 4m5s
[00:50:23.466] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:9:0)
[00:50:23.486] Build status: STARTED
[00:50:23.486] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:10:0) after 4m10s
[00:54:33.504] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:10:0)
[00:54:33.521] Build status: STARTED
[00:54:33.521] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:11:0) after 8m43s
[01:03:16.675] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:11:0)
[01:03:16.711] Build status: STARTED
[01:03:16.711] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:12:0) after 8m32s
[01:11:48.848] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:12:0)
[01:11:49.076] Build status: STARTED
[01:11:49.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:13:0) after 9m46s
[01:21:35.097] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:13:0)
[01:21:35.131] Build status: STARTED
[01:21:35.131] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:14:0) after 6m13s
[01:27:48.175] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:14:0)
[01:27:48.216] Build status: STARTED
[01:27:48.216] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:15:0) after 8m41s
[01:36:29.330] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:15:0)
[01:36:29.344] Build status: STARTED
[01:36:29.345] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:16:0) after 7m32s
[01:44:01.506] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:16:0)
[01:44:01.525] Build status: STARTED
[01:44:01.525] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:17:0) after 8m41s
[01:52:42.543] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:17:0)
[01:52:42.560] Build status: STARTED
[01:52:42.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:18:0) after 1m35s
[01:54:17.611] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:18:0)
[01:54:17.630] Build status: STARTED
[01:54:17.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:19:0) after 2m54s
[01:57:11.658] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:19:0)
[01:57:11.692] Build status: STARTED
[01:57:11.692] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:20:0) after 2m32s
[01:59:43.702] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:20:0)
[01:59:43.723] Build status: STARTED
[01:59:43.723] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:21:0) after 4m48s
[02:04:31.745] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:21:0)
[02:04:31.767] Build status: STARTED
[02:04:31.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:22:0) after 7m32s
[02:12:03.787] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:22:0)
[02:12:03.831] Build status: STARTED
[02:12:03.831] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:23:0) after 1m38s
[02:13:41.831] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:23:0)
[02:13:41.847] Build status: STARTED
[02:13:41.847] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:24:0) after 8m53s
[02:22:34.865] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:24:0)
[02:22:34.901] Build status: STARTED
[02:22:34.901] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:25:0) after 6m4s
[02:28:38.921] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:25:0)
[02:28:38.943] Build status: STARTED
[02:28:38.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:26:0) after 6m26s
[02:35:05.065] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:26:0)
[02:35:05.083] Build status: STARTED
[02:35:05.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:27:0) after 8m53s
[02:43:58.378] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:27:0)
[02:43:58.398] Build status: STARTED
[02:43:58.398] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:28:0) after 5m16s
[02:49:14.424] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:28:0)
[02:49:14.449] Build status: STARTED
[02:49:14.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:29:0) after 3m13s
[02:52:27.474] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:29:0)
[02:52:27.499] Build status: STARTED
[02:52:27.499] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:30:0) after 8m51s
[03:01:18.517] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:30:0)
[03:01:18.533] Build status: STARTED
[03:01:18.533] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:31:0) after 3m23s
[03:04:41.553] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:31:0)
[03:04:41.570] Build status: STARTED
[03:04:41.570] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:32:0) after 7m8s
[03:11:49.607] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:32:0)
[03:11:49.624] Build status: STARTED
[03:11:49.624] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:33:0) after 2m1s
[03:13:50.739] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:33:0)
[03:13:50.759] Build status: STARTED
[03:13:50.759] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:34:0) after 9m38s
[03:23:28.782] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:34:0)
[03:23:28.809] Build status: STARTED
[03:23:28.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:35:0) after 4m21s
[03:27:49.880] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:35:0)
[03:27:49.898] Build status: STARTED
[03:27:49.898] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:36:0) after 1m59s
[03:29:48.917] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:36:0)
[03:29:48.937] Build status: STARTED
[03:29:48.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:37:0) after 1m37s
[03:31:25.956] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:37:0)
[03:31:25.973] Build status: STARTED
[03:31:25.973] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:38:0) after 4m33s
[03:35:59.112] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:38:0)
[03:35:59.168] Build status: STARTED
[03:35:59.168] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:39:0) after 7m24s
[03:43:23.242] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:39:0)
[03:43:23.270] Build status: STARTED
[03:43:23.271] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:40:0) after 4m11s
[03:47:34.288] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:40:0)
[03:47:34.307] Build status: STARTED
[03:47:34.307] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:41:0) after 5m25s
[03:52:59.341] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:41:0)
[03:52:59.355] Build status: STARTED
[03:52:59.355] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:42:0) after 2m42s
[03:55:41.373] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:42:0)
[03:55:41.391] Build status: STARTED
[03:55:41.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:43:0) after 1m58s
[03:57:39.431] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:43:0)
[03:57:39.450] Build status: STARTED
[03:57:39.450] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:44:0) after 8m9s
[04:05:48.472] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:44:0)
[04:05:48.489] Build status: STARTED
[04:05:48.489] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8843678901937521952:45:0) after 6m12s
[04:07:40.310] Received PubSub notification, asking Buildbucket for the build status
[04:07:40.332] Build:
{
"id": "8678248839876792817",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31138,
"createdBy": "project:chromium",
"createTime": "2026-06-23T00:19:52.235502153Z",
"startTime": "2026-06-23T00:21:10.131517748Z",
"endTime": "2026-06-23T04:07:38.052558987Z",
"updateTime": "2026-06-23T04:07:38.052558987Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "a8d67ad9516e6e53de07de42c423eff131a2dc92",
"ref": "refs/heads/main"
}
}
}
[04:07:40.332] Invocation finished in 3h47m49.445772089s with status FAILED