[08:18:44.242] New invocation is queued and will start shortly
[08:18:45.377] Starting the invocation (attempt 1)
[08:18:45.403] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/602363c5057c010bc6714e0124782cb8c22ee721"
[08:18:45.403] Popped gitiles commit info from properties and tags
[08:18:45.403] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[08:18:45.403] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[08:18:45.404] Buildbucket request:
{
"requestId": "8850987127127399392",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "602363c5057c010bc6714e0124782cb8c22ee721",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850987127127399392"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[08:18:45.882] Scheduled build:
{
"id": "8685557065181438753",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29980,
"createdBy": "project:chromium",
"createTime": "2026-04-03T08:18:45.463294142Z",
"updateTime": "2026-04-03T08:18:45.463294142Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "602363c5057c010bc6714e0124782cb8c22ee721",
"ref": "refs/heads/main"
}
}
}
[08:18:45.882] Task URL: https://cr-buildbucket.appspot.com/build/8685557065181438753
[08:18:45.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:2:0) after 8m19s
[08:18:46.256] Received PubSub notification, asking Buildbucket for the build status
[08:18:46.276] Build status: SCHEDULED
[08:19:46.937] Received PubSub notification, asking Buildbucket for the build status
[08:19:46.959] Build status: STARTED
[08:27:04.901] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:2:0)
[08:27:04.917] Build status: STARTED
[08:27:04.917] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:5:0) after 8m31s
[08:35:36.046] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:5:0)
[08:35:36.080] Build status: STARTED
[08:35:36.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:6:0) after 1m18s
[08:36:54.160] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:6:0)
[08:36:54.178] Build status: STARTED
[08:36:54.178] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:7:0) after 5m57s
[08:42:51.455] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:7:0)
[08:42:51.473] Build status: STARTED
[08:42:51.473] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:8:0) after 8m6s
[08:50:57.513] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:8:0)
[08:50:57.531] Build status: STARTED
[08:50:57.531] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:9:0) after 9m43s
[09:00:40.603] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:9:0)
[09:00:40.646] Build status: STARTED
[09:00:40.646] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:10:0) after 7m34s
[09:08:14.690] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:10:0)
[09:08:14.713] Build status: STARTED
[09:08:14.713] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:11:0) after 7m43s
[09:15:57.737] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:11:0)
[09:15:57.755] Build status: STARTED
[09:15:57.755] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:12:0) after 7m55s
[09:23:52.824] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:12:0)
[09:23:52.840] Build status: STARTED
[09:23:52.840] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:13:0) after 1m19s
[09:25:11.916] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:13:0)
[09:25:11.936] Build status: STARTED
[09:25:11.936] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:14:0) after 7m14s
[09:32:26.000] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:14:0)
[09:32:26.025] Build status: STARTED
[09:32:26.025] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:15:0) after 3m36s
[09:36:02.095] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:15:0)
[09:36:02.171] Build status: STARTED
[09:36:02.171] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:16:0) after 3m6s
[09:39:08.374] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:16:0)
[09:39:08.424] Build status: STARTED
[09:39:08.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:17:0) after 4m38s
[09:43:46.530] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:17:0)
[09:43:46.555] Build status: STARTED
[09:43:46.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:18:0) after 7m0s
[09:50:46.681] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:18:0)
[09:50:46.700] Build status: STARTED
[09:50:46.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:19:0) after 4m11s
[09:54:57.821] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:19:0)
[09:54:57.837] Build status: STARTED
[09:54:57.837] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:20:0) after 4m25s
[09:59:22.950] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:20:0)
[09:59:22.968] Build status: STARTED
[09:59:22.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:21:0) after 8m38s
[10:08:01.029] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:21:0)
[10:08:01.049] Build status: STARTED
[10:08:01.049] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:22:0) after 3m42s
[10:11:43.264] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:22:0)
[10:11:43.312] Build status: STARTED
[10:11:43.312] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:23:0) after 1m2s
[10:12:45.402] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:23:0)
[10:12:45.427] Build status: STARTED
[10:12:45.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:24:0) after 2m40s
[10:15:25.438] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:24:0)
[10:15:25.462] Build status: STARTED
[10:15:25.462] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:25:0) after 6m47s
[10:22:12.480] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:25:0)
[10:22:12.503] Build status: STARTED
[10:22:12.503] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:26:0) after 6m44s
[10:28:56.487] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:26:0)
[10:28:56.502] Build status: STARTED
[10:28:56.502] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:27:0) after 1m45s
[10:30:41.680] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:27:0)
[10:30:41.702] Build status: STARTED
[10:30:41.702] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:28:0) after 1m44s
[10:32:25.720] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:28:0)
[10:32:25.776] Build status: STARTED
[10:32:25.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:29:0) after 8m56s
[10:41:21.801] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:29:0)
[10:41:21.820] Build status: STARTED
[10:41:21.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:30:0) after 6m29s
[10:47:50.932] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:30:0)
[10:47:51.054] Build status: STARTED
[10:47:51.054] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:31:0) after 1m18s
[10:49:09.075] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:31:0)
[10:49:09.090] Build status: STARTED
[10:49:09.091] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:32:0) after 7m43s
[10:56:52.168] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:32:0)
[10:56:52.189] Build status: STARTED
[10:56:52.189] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:33:0) after 2m57s
[10:59:49.160] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:33:0)
[10:59:49.185] Build status: STARTED
[10:59:49.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:34:0) after 9m9s
[11:08:58.203] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:34:0)
[11:08:58.264] Build status: STARTED
[11:08:58.264] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:35:0) after 2m57s
[11:11:55.281] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:35:0)
[11:11:55.296] Build status: STARTED
[11:11:55.296] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:36:0) after 2m3s
[11:13:58.320] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:36:0)
[11:13:58.343] Build status: STARTED
[11:13:58.343] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850987127127399392:37:0) after 8m25s
[11:21:06.366] Received PubSub notification, asking Buildbucket for the build status
[11:21:06.386] Build:
{
"id": "8685557065181438753",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29980,
"createdBy": "project:chromium",
"createTime": "2026-04-03T08:18:45.463294142Z",
"startTime": "2026-04-03T08:19:46.668502714Z",
"endTime": "2026-04-03T11:21:06.117049318Z",
"updateTime": "2026-04-03T11:21:06.117049318Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "602363c5057c010bc6714e0124782cb8c22ee721",
"ref": "refs/heads/main"
}
}
}
[11:21:06.386] Invocation finished in 3h2m22.162217776s with status FAILED