[10:49:26.073] New invocation is queued and will start shortly
[10:49:27.135] Starting the invocation (attempt 1)
[10:49:27.165] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/e83e87d8e511082e066a26e0aeba698af4a843b3"
[10:49:27.165] Popped gitiles commit info from properties and tags
[10:49:27.165] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[10:49:27.166] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[10:49:27.166] Buildbucket request:
{
"requestId": "8842280337290920768",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e83e87d8e511082e066a26e0aeba698af4a843b3",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8842280337290920768"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[10:49:27.669] Scheduled build:
{
"id": "8676850275418348449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31337,
"createdBy": "project:chromium",
"createTime": "2026-07-08T10:49:27.239521216Z",
"updateTime": "2026-07-08T10:49:27.239521216Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e83e87d8e511082e066a26e0aeba698af4a843b3",
"ref": "refs/heads/main"
}
}
}
[10:49:27.669] Task URL: https://cr-buildbucket.appspot.com/build/8676850275418348449
[10:49:27.669] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:2:0) after 1m32s
[10:49:28.636] Received PubSub notification, asking Buildbucket for the build status
[10:49:28.657] Build status: SCHEDULED
[10:49:32.896] Received PubSub notification, asking Buildbucket for the build status
[10:49:32.913] Build status: STARTED
[10:50:59.696] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:2:0)
[10:50:59.716] Build status: STARTED
[10:50:59.716] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:5:0) after 8m59s
[10:59:58.770] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:5:0)
[10:59:58.790] Build status: STARTED
[10:59:58.790] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:6:0) after 4m38s
[11:04:36.945] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:6:0)
[11:04:36.967] Build status: STARTED
[11:04:36.967] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:7:0) after 9m43s
[11:14:20.027] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:7:0)
[11:14:20.049] Build status: STARTED
[11:14:20.049] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:8:0) after 4m16s
[11:18:36.078] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:8:0)
[11:18:36.097] Build status: STARTED
[11:18:36.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:9:0) after 1m8s
[11:19:44.154] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:9:0)
[11:19:44.169] Build status: STARTED
[11:19:44.169] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:10:0) after 8m44s
[11:28:28.224] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:10:0)
[11:28:28.254] Build status: STARTED
[11:28:28.254] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:11:0) after 9m34s
[11:38:02.411] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:11:0)
[11:38:02.436] Build status: STARTED
[11:38:02.436] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:12:0) after 3m56s
[11:41:58.454] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:12:0)
[11:41:58.482] Build status: STARTED
[11:41:58.482] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:13:0) after 6m44s
[11:48:42.505] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:13:0)
[11:48:42.527] Build status: STARTED
[11:48:42.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:14:0) after 1m51s
[11:50:33.501] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:14:0)
[11:50:33.547] Build status: STARTED
[11:50:33.547] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:15:0) after 9m24s
[11:59:57.563] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:15:0)
[11:59:57.584] Build status: STARTED
[11:59:57.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:16:0) after 2m33s
[12:02:30.600] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:16:0)
[12:02:30.618] Build status: STARTED
[12:02:30.618] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:17:0) after 6m5s
[12:08:35.665] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:17:0)
[12:08:35.689] Build status: STARTED
[12:08:35.689] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:18:0) after 5m10s
[12:13:45.837] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:18:0)
[12:13:45.852] Build status: STARTED
[12:13:45.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:19:0) after 1m45s
[12:15:30.942] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:19:0)
[12:15:30.965] Build status: STARTED
[12:15:30.965] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:20:0) after 2m49s
[12:18:19.979] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:20:0)
[12:18:19.999] Build status: STARTED
[12:18:19.999] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:21:0) after 5m54s
[12:24:14.015] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:21:0)
[12:24:14.119] Build status: STARTED
[12:24:14.119] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:22:0) after 4m36s
[12:28:50.185] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:22:0)
[12:28:50.199] Build status: STARTED
[12:28:50.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:23:0) after 8m25s
[12:37:15.212] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:23:0)
[12:37:15.244] Build status: STARTED
[12:37:15.244] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:24:0) after 2m30s
[12:39:45.264] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:24:0)
[12:39:45.293] Build status: STARTED
[12:39:45.293] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:25:0) after 8m12s
[12:47:57.341] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:25:0)
[12:47:57.360] Build status: STARTED
[12:47:57.360] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:26:0) after 7m14s
[12:55:11.378] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:26:0)
[12:55:11.406] Build status: STARTED
[12:55:11.406] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:27:0) after 8m19s
[13:03:30.421] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:27:0)
[13:03:30.533] Build status: STARTED
[13:03:30.533] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:28:0) after 1m27s
[13:04:57.574] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:28:0)
[13:04:57.590] Build status: STARTED
[13:04:57.590] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:29:0) after 2m38s
[13:07:35.679] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:29:0)
[13:07:35.709] Build status: STARTED
[13:07:35.709] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:30:0) after 2m9s
[13:09:44.723] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:30:0)
[13:09:44.752] Build status: STARTED
[13:09:44.752] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:31:0) after 9m52s
[13:19:36.770] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:31:0)
[13:19:36.796] Build status: STARTED
[13:19:36.796] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:32:0) after 1m9s
[13:20:45.840] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:32:0)
[13:20:45.861] Build status: STARTED
[13:20:45.861] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:33:0) after 9m8s
[13:29:53.876] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:33:0)
[13:29:53.893] Build status: STARTED
[13:29:53.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:34:0) after 4m26s
[13:34:19.899] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:34:0)
[13:34:19.922] Build status: STARTED
[13:34:19.922] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:35:0) after 1m0s
[13:35:19.945] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:35:0)
[13:35:19.981] Build status: STARTED
[13:35:19.982] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:36:0) after 4m26s
[13:39:46.008] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:36:0)
[13:39:46.031] Build status: STARTED
[13:39:46.031] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:37:0) after 3m5s
[13:42:51.193] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:37:0)
[13:42:51.220] Build status: STARTED
[13:42:51.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:38:0) after 7m21s
[13:50:12.265] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:38:0)
[13:50:12.285] Build status: STARTED
[13:50:12.285] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8842280337290920768:39:0) after 5m51s
[13:53:44.912] Received PubSub notification, asking Buildbucket for the build status
[13:53:45.091] Build:
{
"id": "8676850275418348449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 31337,
"createdBy": "project:chromium",
"createTime": "2026-07-08T10:49:27.239521216Z",
"startTime": "2026-07-08T10:49:32.726615856Z",
"endTime": "2026-07-08T13:53:42.642246030Z",
"updateTime": "2026-07-08T13:53:42.642246030Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e83e87d8e511082e066a26e0aeba698af4a843b3",
"ref": "refs/heads/main"
}
}
}
[13:53:45.091] Invocation finished in 3h4m19.022350629s with status FAILED