[07:38:47.307] New invocation is queued and will start shortly
[07:38:48.617] Starting the invocation (attempt 1)
[07:38:48.649] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/b77917c309c94e0cb41739df74638e5275552708"
[07:38:48.649] Popped gitiles commit info from properties and tags
[07:38:48.649] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[07:38:48.649] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[07:38:48.649] Buildbucket request:
{
"requestId": "8846912776993004608",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b77917c309c94e0cb41739df74638e5275552708",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8846912776993004608"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[07:38:49.630] Scheduled build:
{
"id": "8681482714614221681",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 30570,
"createdBy": "project:chromium",
"createTime": "2026-05-18T07:38:48.955267417Z",
"updateTime": "2026-05-18T07:38:48.955267417Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b77917c309c94e0cb41739df74638e5275552708",
"ref": "refs/heads/main"
}
},
"canary": true
}
[07:38:49.630] Task URL: https://cr-buildbucket.appspot.com/build/8681482714614221681
[07:38:49.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:2:0) after 1m14s
[07:38:50.577] Received PubSub notification, asking Buildbucket for the build status
[07:38:50.596] Build status: SCHEDULED
[07:39:51.679] Received PubSub notification, asking Buildbucket for the build status
[07:39:51.713] Build status: STARTED
[07:40:03.752] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:2:0)
[07:40:03.802] Build status: STARTED
[07:40:03.802] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:5:0) after 9m44s
[07:49:48.238] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:5:0)
[07:49:48.264] Build status: STARTED
[07:49:48.265] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:6:0) after 6m0s
[07:55:48.285] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:6:0)
[07:55:48.315] Build status: STARTED
[07:55:48.315] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:7:0) after 6m51s
[08:02:39.799] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:7:0)
[08:02:39.823] Build status: STARTED
[08:02:39.823] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:8:0) after 6m14s
[08:08:53.934] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:8:0)
[08:08:53.965] Build status: STARTED
[08:08:53.965] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:9:0) after 7m10s
[08:16:04.179] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:9:0)
[08:16:04.196] Build status: STARTED
[08:16:04.196] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:10:0) after 1m28s
[08:17:32.588] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:10:0)
[08:17:32.607] Build status: STARTED
[08:17:32.607] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:11:0) after 8m47s
[08:26:19.813] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:11:0)
[08:26:19.829] Build status: STARTED
[08:26:19.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:12:0) after 7m51s
[08:34:10.844] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:12:0)
[08:34:10.879] Build status: STARTED
[08:34:10.880] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:13:0) after 8m47s
[08:42:57.964] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:13:0)
[08:42:58.026] Build status: STARTED
[08:42:58.026] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:14:0) after 2m51s
[08:45:49.048] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:14:0)
[08:45:49.094] Build status: STARTED
[08:45:49.094] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:15:0) after 7m16s
[08:53:05.116] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:15:0)
[08:53:05.147] Build status: STARTED
[08:53:05.147] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:16:0) after 1m7s
[08:54:12.228] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:16:0)
[08:54:12.247] Build status: STARTED
[08:54:12.247] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:17:0) after 4m6s
[08:58:18.321] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:17:0)
[08:58:18.340] Build status: STARTED
[08:58:18.340] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:18:0) after 3m18s
[09:01:36.352] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:18:0)
[09:01:36.416] Build status: STARTED
[09:01:36.416] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:19:0) after 7m42s
[09:09:18.425] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:19:0)
[09:09:18.441] Build status: STARTED
[09:09:18.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:20:0) after 1m42s
[09:11:00.457] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:20:0)
[09:11:00.476] Build status: STARTED
[09:11:00.476] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:21:0) after 8m5s
[09:19:05.583] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:21:0)
[09:19:05.603] Build status: STARTED
[09:19:05.603] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:22:0) after 4m56s
[09:24:01.674] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:22:0)
[09:24:01.688] Build status: STARTED
[09:24:01.688] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:23:0) after 6m32s
[09:30:33.705] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:23:0)
[09:30:33.725] Build status: STARTED
[09:30:33.725] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:24:0) after 1m26s
[09:31:59.740] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:24:0)
[09:31:59.761] Build status: STARTED
[09:31:59.761] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:25:0) after 1m41s
[09:33:40.798] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:25:0)
[09:33:40.816] Build status: STARTED
[09:33:40.816] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:26:0) after 5m0s
[09:38:40.952] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:26:0)
[09:38:40.968] Build status: STARTED
[09:38:40.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:27:0) after 5m23s
[09:44:03.987] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:27:0)
[09:44:04.005] Build status: STARTED
[09:44:04.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:28:0) after 5m38s
[09:49:42.072] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:28:0)
[09:49:42.135] Build status: STARTED
[09:49:42.135] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:29:0) after 9m18s
[09:59:00.201] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:29:0)
[09:59:00.218] Build status: STARTED
[09:59:00.218] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:30:0) after 6m15s
[10:05:15.257] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:30:0)
[10:05:15.359] Build status: STARTED
[10:05:15.359] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:31:0) after 4m25s
[10:09:40.426] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:31:0)
[10:09:40.444] Build status: STARTED
[10:09:40.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:32:0) after 4m54s
[10:14:34.449] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:32:0)
[10:14:34.470] Build status: STARTED
[10:14:34.470] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:33:0) after 1m27s
[10:16:01.529] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:33:0)
[10:16:01.547] Build status: STARTED
[10:16:01.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:34:0) after 4m48s
[10:20:49.562] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:34:0)
[10:20:49.579] Build status: STARTED
[10:20:49.579] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:35:0) after 9m49s
[10:30:38.590] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:35:0)
[10:30:38.607] Build status: STARTED
[10:30:38.607] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:36:0) after 1m36s
[10:32:14.623] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:36:0)
[10:32:14.643] Build status: STARTED
[10:32:14.643] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:37:0) after 3m20s
[10:35:34.699] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:37:0)
[10:35:34.715] Build status: STARTED
[10:35:34.715] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:38:0) after 5m38s
[10:41:12.775] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:38:0)
[10:41:12.835] Build status: STARTED
[10:41:12.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:39:0) after 7m13s
[10:48:26.180] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:39:0)
[10:48:26.199] Build status: STARTED
[10:48:26.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:40:0) after 5m37s
[10:54:03.513] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:40:0)
[10:54:03.530] Build status: STARTED
[10:54:03.530] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:41:0) after 2m56s
[10:56:59.648] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:41:0)
[10:56:59.687] Build status: STARTED
[10:56:59.687] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:42:0) after 1m29s
[10:58:28.873] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:42:0)
[10:58:28.896] Build status: STARTED
[10:58:28.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:43:0) after 6m11s
[11:04:40.056] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:43:0)
[11:04:40.095] Build status: STARTED
[11:04:40.095] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8846912776993004608:44:0) after 4m3s
[11:07:14.254] Received PubSub notification, asking Buildbucket for the build status
[11:07:14.306] Build:
{
"id": "8681482714614221681",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 30570,
"createdBy": "project:chromium",
"createTime": "2026-05-18T07:38:48.955267417Z",
"startTime": "2026-05-18T07:39:51.422626660Z",
"endTime": "2026-05-18T11:07:13.706784565Z",
"updateTime": "2026-05-18T11:07:13.706784565Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b77917c309c94e0cb41739df74638e5275552708",
"ref": "refs/heads/main"
}
},
"canary": true
}
[11:07:14.306] Invocation finished in 3h28m27.002987332s with status FAILED