[04:49:47.100] New invocation is queued and will start shortly
[04:49:48.329] Starting the invocation (attempt 1)
[04:49:48.357] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/bb93a9290b6e3472f209b5ea47b2892c92d7531e"
[04:49:48.357] Popped gitiles commit info from properties and tags
[04:49:48.357] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:49:48.357] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:49:48.357] Buildbucket request:
{
"requestId": "8924021428178134032",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bb93a9290b6e3472f209b5ea47b2892c92d7531e",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8924021428178134032"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:49:48.917] Scheduled build:
{
"id": "8758591366155205297",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 19111,
"createdBy": "project:chromium",
"createTime": "2024-01-18T04:49:48.408875462Z",
"updateTime": "2024-01-18T04:49:48.408875462Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bb93a9290b6e3472f209b5ea47b2892c92d7531e",
"ref": "refs/heads/main"
}
}
}
[04:49:48.918] Task URL: https://cr-buildbucket.appspot.com/build/8758591366155205297
[04:49:48.918] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:2:0) after 6m37s
[04:49:49.756] Received PubSub notification, asking Buildbucket for the build status
[04:49:49.785] Build status: STARTED
[04:49:49.893] Received PubSub notification, asking Buildbucket for the build status
[04:49:49.911] Build status: STARTED
[04:56:26.223] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:2:0)
[04:56:26.243] Build status: STARTED
[04:56:26.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:5:0) after 1m35s
[04:58:01.268] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:5:0)
[04:58:01.515] Build status: STARTED
[04:58:01.515] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:6:0) after 4m7s
[05:02:08.546] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:6:0)
[05:02:08.591] Build status: STARTED
[05:02:08.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:7:0) after 1m10s
[05:03:18.723] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:7:0)
[05:03:18.740] Build status: STARTED
[05:03:18.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:8:0) after 7m46s
[05:11:04.771] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:8:0)
[05:11:04.789] Build status: STARTED
[05:11:04.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:9:0) after 6m55s
[05:17:59.770] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:9:0)
[05:17:59.795] Build status: STARTED
[05:17:59.795] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:10:0) after 9m20s
[05:27:19.920] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:10:0)
[05:27:19.945] Build status: STARTED
[05:27:19.945] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:11:0) after 6m45s
[05:34:05.110] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:11:0)
[05:34:05.191] Build status: STARTED
[05:34:05.191] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:12:0) after 3m38s
[05:37:43.441] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:12:0)
[05:37:43.463] Build status: STARTED
[05:37:43.463] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:13:0) after 5m29s
[05:43:12.531] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:13:0)
[05:43:12.548] Build status: STARTED
[05:43:12.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:14:0) after 9m6s
[05:52:18.574] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:14:0)
[05:52:18.590] Build status: STARTED
[05:52:18.590] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:15:0) after 6m36s
[05:58:54.651] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:15:0)
[05:58:54.679] Build status: STARTED
[05:58:54.679] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:16:0) after 3m41s
[06:02:35.733] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:16:0)
[06:02:35.751] Build status: STARTED
[06:02:35.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:17:0) after 9m20s
[06:11:55.820] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:17:0)
[06:11:55.838] Build status: STARTED
[06:11:55.838] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:18:0) after 9m35s
[06:21:30.863] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:18:0)
[06:21:30.880] Build status: STARTED
[06:21:30.880] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:19:0) after 7m25s
[06:28:55.897] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:19:0)
[06:28:55.934] Build status: STARTED
[06:28:55.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:20:0) after 9m33s
[06:38:29.049] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:20:0)
[06:38:29.076] Build status: STARTED
[06:38:29.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:21:0) after 7m31s
[06:46:00.176] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:21:0)
[06:46:00.197] Build status: STARTED
[06:46:00.197] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:22:0) after 6m6s
[06:52:06.256] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:22:0)
[06:52:06.273] Build status: STARTED
[06:52:06.273] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:23:0) after 4m49s
[06:56:55.294] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:23:0)
[06:56:55.349] Build status: STARTED
[06:56:55.349] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:24:0) after 2m19s
[06:59:14.366] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:24:0)
[06:59:14.393] Build status: STARTED
[06:59:14.393] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:25:0) after 7m52s
[07:07:06.423] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:25:0)
[07:07:06.442] Build status: STARTED
[07:07:06.442] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:26:0) after 1m48s
[07:08:54.462] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:26:0)
[07:08:54.479] Build status: STARTED
[07:08:54.479] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:27:0) after 8m50s
[07:17:45.037] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:27:0)
[07:17:45.055] Build status: STARTED
[07:17:45.055] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:28:0) after 7m18s
[07:25:03.073] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:28:0)
[07:25:03.162] Build status: STARTED
[07:25:03.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:29:0) after 2m49s
[07:27:52.234] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:29:0)
[07:27:52.270] Build status: STARTED
[07:27:52.270] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:30:0) after 5m14s
[07:33:06.388] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:30:0)
[07:33:06.417] Build status: STARTED
[07:33:06.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:31:0) after 6m51s
[07:39:57.695] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:31:0)
[07:39:57.723] Build status: STARTED
[07:39:57.723] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:32:0) after 9m29s
[07:49:26.778] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:32:0)
[07:49:26.797] Build status: STARTED
[07:49:26.797] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:33:0) after 3m25s
[07:52:51.826] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:33:0)
[07:52:51.984] Build status: STARTED
[07:52:51.984] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:34:0) after 3m31s
[07:56:23.280] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:34:0)
[07:56:23.374] Build status: STARTED
[07:56:23.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:35:0) after 5m49s
[08:02:12.483] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:35:0)
[08:02:12.536] Build status: STARTED
[08:02:12.536] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:36:0) after 5m55s
[08:08:07.557] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:36:0)
[08:08:07.714] Build status: STARTED
[08:08:07.714] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:37:0) after 9m57s
[08:18:04.863] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:37:0)
[08:18:04.909] Build status: STARTED
[08:18:04.909] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:38:0) after 8m12s
[08:26:17.028] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:38:0)
[08:26:17.043] Build status: STARTED
[08:26:17.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:39:0) after 3m36s
[08:29:53.102] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:39:0)
[08:29:53.121] Build status: STARTED
[08:29:53.121] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:40:0) after 1m24s
[08:31:17.340] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:40:0)
[08:31:17.391] Build status: STARTED
[08:31:17.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:41:0) after 8m13s
[08:39:30.505] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:41:0)
[08:39:30.528] Build status: STARTED
[08:39:30.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:42:0) after 6m0s
[08:45:30.864] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:42:0)
[08:45:30.885] Build status: STARTED
[08:45:30.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:43:0) after 7m48s
[08:53:18.908] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:43:0)
[08:53:18.976] Build status: STARTED
[08:53:18.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:44:0) after 9m50s
[09:03:08.991] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:44:0)
[09:03:09.017] Build status: STARTED
[09:03:09.017] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:45:0) after 5m55s
[09:09:04.110] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:45:0)
[09:09:04.127] Build status: STARTED
[09:09:04.128] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:46:0) after 4m19s
[09:13:23.162] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:46:0)
[09:13:23.184] Build status: STARTED
[09:13:23.184] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:47:0) after 1m4s
[09:14:27.204] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:47:0)
[09:14:27.244] Build status: STARTED
[09:14:27.245] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:48:0) after 1m59s
[09:16:26.272] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:48:0)
[09:16:26.287] Build status: STARTED
[09:16:26.287] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:49:0) after 9m56s
[09:26:22.304] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:49:0)
[09:26:22.323] Build status: STARTED
[09:26:22.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:50:0) after 6m57s
[09:33:19.785] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:50:0)
[09:33:19.860] Build status: STARTED
[09:33:19.860] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:51:0) after 2m9s
[09:35:28.897] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:51:0)
[09:35:28.914] Build status: STARTED
[09:35:28.914] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:52:0) after 3m44s
[09:39:12.938] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:52:0)
[09:39:12.956] Build status: STARTED
[09:39:12.956] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:53:0) after 1m32s
[09:40:44.978] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:53:0)
[09:40:44.997] Build status: STARTED
[09:40:44.997] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:54:0) after 3m36s
[09:44:21.026] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:54:0)
[09:44:21.043] Build status: STARTED
[09:44:21.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8924021428178134032:55:0) after 4m8s
[09:46:40.595] Received PubSub notification, asking Buildbucket for the build status
[09:46:40.611] Build:
{
"id": "8758591366155205297",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 19111,
"createdBy": "project:chromium",
"createTime": "2024-01-18T04:49:48.408875462Z",
"startTime": "2024-01-18T04:49:49.228750Z",
"endTime": "2024-01-18T09:46:39.983097Z",
"updateTime": "2024-01-18T09:46:39.983097Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bb93a9290b6e3472f209b5ea47b2892c92d7531e",
"ref": "refs/heads/main"
}
}
}
[09:46:40.611] Invocation finished in 4h56m53.515950924s with status FAILED