[16:30:27.524] New invocation is queued and will start shortly
[16:30:28.782] Starting the invocation (attempt 1)
[16:30:28.807] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/78ed3b7f37599d8a7dba1aa8375a22a23740a31d"
[16:30:28.808] Popped gitiles commit info from properties and tags
[16:30:28.808] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[16:30:28.808] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[16:30:28.808] Buildbucket request:
{
"requestId": "8833470976173805136",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "78ed3b7f37599d8a7dba1aa8375a22a23740a31d",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8833470976173805136"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[16:30:29.374] Scheduled build:
{
"id": "8668040914092797057",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 33329,
"createdBy": "project:chromium",
"createTime": "2026-10-13T16:30:28.879559743Z",
"updateTime": "2026-10-13T16:30:28.879559743Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "78ed3b7f37599d8a7dba1aa8375a22a23740a31d",
"ref": "refs/heads/main"
}
}
}
[16:30:29.374] Task URL: https://cr-buildbucket.appspot.com/build/8668040914092797057
[16:30:29.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:2:0) after 3m16s
[16:30:30.343] Received PubSub notification, asking Buildbucket for the build status
[16:30:30.520] Build status: SCHEDULED
[16:33:45.388] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:2:0)
[16:33:45.404] Build status: SCHEDULED
[16:33:45.404] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:4:0) after 2m16s
[16:34:41.010] Received PubSub notification, asking Buildbucket for the build status
[16:34:41.031] Build status: STARTED
[16:36:01.469] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:4:0)
[16:36:01.503] Build status: STARTED
[16:36:01.503] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:6:0) after 7m31s
[16:43:32.521] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:6:0)
[16:43:32.576] Build status: STARTED
[16:43:32.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:7:0) after 9m54s
[16:53:26.671] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:7:0)
[16:53:26.691] Build status: STARTED
[16:53:26.691] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:8:0) after 7m47s
[17:01:13.689] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:8:0)
[17:01:13.708] Build status: STARTED
[17:01:13.708] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:9:0) after 4m42s
[17:05:55.724] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:9:0)
[17:05:55.744] Build status: STARTED
[17:05:55.744] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:10:0) after 5m35s
[17:11:30.849] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:10:0)
[17:11:30.870] Build status: STARTED
[17:11:30.870] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:11:0) after 3m42s
[17:15:12.947] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:11:0)
[17:15:12.967] Build status: STARTED
[17:15:12.967] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:12:0) after 1m37s
[17:16:49.940] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:12:0)
[17:16:49.962] Build status: STARTED
[17:16:49.962] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:13:0) after 3m45s
[17:20:35.083] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:13:0)
[17:20:35.160] Build status: STARTED
[17:20:35.160] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:14:0) after 8m54s
[17:29:29.492] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:14:0)
[17:29:29.515] Build status: STARTED
[17:29:29.515] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:15:0) after 6m42s
[17:36:11.594] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:15:0)
[17:36:11.668] Build status: STARTED
[17:36:11.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:16:0) after 6m50s
[17:43:01.730] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:16:0)
[17:43:01.763] Build status: STARTED
[17:43:01.763] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:17:0) after 7m12s
[17:50:13.837] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:17:0)
[17:50:13.858] Build status: STARTED
[17:50:13.858] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:18:0) after 4m38s
[17:54:51.893] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:18:0)
[17:54:51.917] Build status: STARTED
[17:54:51.917] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:19:0) after 3m9s
[17:58:00.944] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:19:0)
[17:58:00.965] Build status: STARTED
[17:58:00.965] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:20:0) after 4m6s
[18:02:07.065] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:20:0)
[18:02:07.083] Build status: STARTED
[18:02:07.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:21:0) after 6m53s
[18:09:00.063] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:21:0)
[18:09:00.084] Build status: STARTED
[18:09:00.084] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:22:0) after 3m11s
[18:12:11.145] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:22:0)
[18:12:11.175] Build status: STARTED
[18:12:11.175] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:23:0) after 6m51s
[18:19:02.191] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:23:0)
[18:19:02.219] Build status: STARTED
[18:19:02.219] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:24:0) after 3m14s
[18:22:16.428] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:24:0)
[18:22:16.454] Build status: STARTED
[18:22:16.454] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:25:0) after 4m20s
[18:26:36.470] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:25:0)
[18:26:36.556] Build status: STARTED
[18:26:36.556] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:26:0) after 1m8s
[18:27:44.696] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:26:0)
[18:27:44.747] Build status: STARTED
[18:27:44.747] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:27:0) after 7m35s
[18:35:19.803] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:27:0)
[18:35:19.877] Build status: STARTED
[18:35:19.878] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:28:0) after 2m35s
[18:37:54.904] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:28:0)
[18:37:54.923] Build status: STARTED
[18:37:54.924] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:29:0) after 4m6s
[18:42:00.940] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:29:0)
[18:42:00.963] Build status: STARTED
[18:42:00.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:30:0) after 8m0s
[18:50:01.107] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:30:0)
[18:50:01.162] Build status: STARTED
[18:50:01.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:31:0) after 5m51s
[18:55:52.313] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:31:0)
[18:55:52.331] Build status: STARTED
[18:55:52.331] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:32:0) after 5m33s
[19:01:25.425] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:32:0)
[19:01:25.463] Build status: STARTED
[19:01:25.463] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:33:0) after 9m11s
[19:10:36.540] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:33:0)
[19:10:36.561] Build status: STARTED
[19:10:36.562] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:34:0) after 1m36s
[19:12:12.599] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:34:0)
[19:12:12.617] Build status: STARTED
[19:12:12.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:35:0) after 7m14s
[19:19:26.635] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:35:0)
[19:19:26.656] Build status: STARTED
[19:19:26.656] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:36:0) after 3m22s
[19:22:48.734] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:36:0)
[19:22:48.755] Build status: STARTED
[19:22:48.756] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:37:0) after 9m59s
[19:32:47.773] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8833470976173805136:37:0)
[19:32:47.791] Build:
{
"id": "8668040914092797057",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 33329,
"createdBy": "project:chromium",
"createTime": "2026-10-13T16:30:28.879559743Z",
"startTime": "2026-10-13T16:34:40.286215523Z",
"endTime": "2026-10-13T19:32:45.418092907Z",
"updateTime": "2026-10-13T19:32:45.418092907Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "78ed3b7f37599d8a7dba1aa8375a22a23740a31d",
"ref": "refs/heads/main"
}
}
}
[19:32:47.791] Invocation finished in 3h2m20.281378463s with status FAILED