[22:05:44.196] New invocation is queued and will start shortly
[22:05:46.102] Starting the invocation (attempt 1)
[22:05:46.127] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/964c357d73fab1256b9c0d87035d5642a4f1f1eb"
[22:05:46.127] Popped gitiles commit info from properties and tags
[22:05:46.127] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[22:05:46.127] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[22:05:46.127] Buildbucket request:
{
"requestId": "8853109424017865600",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "964c357d73fab1256b9c0d87035d5642a4f1f1eb",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8853109424017865600"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[22:05:46.832] Scheduled build:
{
"id": "8687679361102338065",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29728,
"createdBy": "project:chromium",
"createTime": "2026-03-10T22:05:46.352363545Z",
"updateTime": "2026-03-10T22:05:46.352363545Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "964c357d73fab1256b9c0d87035d5642a4f1f1eb",
"ref": "refs/heads/main"
}
}
}
[22:05:46.832] Task URL: https://cr-buildbucket.appspot.com/build/8687679361102338065
[22:05:46.832] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:2:0) after 3m49s
[22:05:47.797] Received PubSub notification, asking Buildbucket for the build status
[22:05:47.818] Build status: SCHEDULED
[22:05:51.494] Received PubSub notification, asking Buildbucket for the build status
[22:05:51.519] Build status: STARTED
[22:09:35.831] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:2:0)
[22:09:35.850] Build status: STARTED
[22:09:35.850] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:5:0) after 1m31s
[22:11:06.880] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:5:0)
[22:11:06.973] Build status: STARTED
[22:11:06.973] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:6:0) after 9m42s
[22:20:48.990] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:6:0)
[22:20:49.022] Build status: STARTED
[22:20:49.022] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:7:0) after 3m26s
[22:24:15.092] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:7:0)
[22:24:15.118] Build status: STARTED
[22:24:15.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:8:0) after 9m25s
[22:33:40.433] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:8:0)
[22:33:40.455] Build status: STARTED
[22:33:40.455] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:9:0) after 8m4s
[22:41:44.470] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:9:0)
[22:41:44.514] Build status: STARTED
[22:41:44.514] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:10:0) after 8m51s
[22:50:35.533] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:10:0)
[22:50:35.553] Build status: STARTED
[22:50:35.553] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:11:0) after 9m6s
[22:59:41.551] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:11:0)
[22:59:41.567] Build status: STARTED
[22:59:41.567] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:12:0) after 2m5s
[23:01:46.587] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:12:0)
[23:01:46.679] Build status: STARTED
[23:01:46.679] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:13:0) after 8m20s
[23:10:06.696] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:13:0)
[23:10:06.717] Build status: STARTED
[23:10:06.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:14:0) after 9m57s
[23:20:03.735] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:14:0)
[23:20:03.756] Build status: STARTED
[23:20:03.756] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:15:0) after 3m30s
[23:23:34.241] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:15:0)
[23:23:34.259] Build status: STARTED
[23:23:34.259] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:16:0) after 1m51s
[23:25:25.376] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:16:0)
[23:25:25.395] Build status: STARTED
[23:25:25.395] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:17:0) after 7m0s
[23:32:25.452] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:17:0)
[23:32:25.468] Build status: STARTED
[23:32:25.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:18:0) after 5m9s
[23:37:34.485] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:18:0)
[23:37:34.521] Build status: STARTED
[23:37:34.521] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:19:0) after 9m16s
[23:46:50.608] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:19:0)
[23:46:50.626] Build status: STARTED
[23:46:50.626] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:20:0) after 9m7s
[23:55:57.638] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:20:0)
[23:55:57.658] Build status: STARTED
[23:55:57.658] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:21:0) after 5m16s
[00:01:14.013] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:21:0)
[00:01:14.062] Build status: STARTED
[00:01:14.062] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:22:0) after 7m45s
[00:08:59.084] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:22:0)
[00:08:59.100] Build status: STARTED
[00:08:59.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:23:0) after 5m32s
[00:14:31.132] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:23:0)
[00:14:31.151] Build status: STARTED
[00:14:31.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:24:0) after 2m2s
[00:16:33.168] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:24:0)
[00:16:33.188] Build status: STARTED
[00:16:33.188] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:25:0) after 3m22s
[00:19:55.159] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:25:0)
[00:19:55.175] Build status: STARTED
[00:19:55.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:26:0) after 1m27s
[00:21:22.188] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:26:0)
[00:21:22.209] Build status: STARTED
[00:21:22.209] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:27:0) after 5m9s
[00:26:31.333] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:27:0)
[00:26:31.349] Build status: STARTED
[00:26:31.349] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:28:0) after 8m46s
[00:35:17.352] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:28:0)
[00:35:17.377] Build status: STARTED
[00:35:17.377] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:29:0) after 1m18s
[00:36:35.410] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:29:0)
[00:36:35.497] Build status: STARTED
[00:36:35.497] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:30:0) after 3m13s
[00:39:49.087] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:30:0)
[00:39:49.111] Build status: STARTED
[00:39:49.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:31:0) after 3m49s
[00:43:38.234] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:31:0)
[00:43:38.258] Build status: STARTED
[00:43:38.258] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:32:0) after 2m3s
[00:45:41.299] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:32:0)
[00:45:41.318] Build status: STARTED
[00:45:41.318] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:33:0) after 4m27s
[00:50:08.343] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:33:0)
[00:50:08.361] Build status: STARTED
[00:50:08.361] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:34:0) after 3m46s
[00:53:54.421] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:34:0)
[00:53:54.437] Build status: STARTED
[00:53:54.437] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:35:0) after 5m27s
[00:59:21.439] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:35:0)
[00:59:21.468] Build status: STARTED
[00:59:21.468] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:36:0) after 9m26s
[01:08:47.483] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:36:0)
[01:08:47.500] Build status: STARTED
[01:08:47.500] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:37:0) after 4m22s
[01:13:09.518] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:37:0)
[01:13:09.542] Build status: STARTED
[01:13:09.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:38:0) after 9m59s
[01:23:08.562] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:38:0)
[01:23:08.596] Build status: STARTED
[01:23:08.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:39:0) after 8m8s
[01:31:16.663] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:39:0)
[01:31:16.685] Build status: STARTED
[01:31:16.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:40:0) after 3m32s
[01:34:48.806] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8853109424017865600:40:0)
[01:34:48.823] Build:
{
"id": "8687679361102338065",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29728,
"createdBy": "project:chromium",
"createTime": "2026-03-10T22:05:46.352363545Z",
"startTime": "2026-03-10T22:05:51.202773202Z",
"endTime": "2026-03-11T01:34:34.594245569Z",
"updateTime": "2026-03-11T01:34:34.594245569Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "964c357d73fab1256b9c0d87035d5642a4f1f1eb",
"ref": "refs/heads/main"
}
}
}
[01:34:48.823] Invocation finished in 3h29m4.635317479s with status FAILED