[00:32:08.363] New invocation is queued and will start shortly
[00:32:09.593] Starting the invocation (attempt 1)
[00:32:09.642] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/933a0f0ae5bd1e3a0cb192350e97806cd7d57590"
[00:32:09.642] Popped gitiles commit info from properties and tags
[00:32:09.642] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[00:32:09.642] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[00:32:09.642] Buildbucket request:
{
"requestId": "8976855669261382512",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "933a0f0ae5bd1e3a0cb192350e97806cd7d57590",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8976855669261382512"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[00:32:10.274] Scheduled build:
{
"id": "8811425607126029201",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 12830,
"createdBy": "project:chromium",
"createTime": "2022-06-14T00:32:09.767579714Z",
"updateTime": "2022-06-14T00:32:09.767579714Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "933a0f0ae5bd1e3a0cb192350e97806cd7d57590",
"ref": "refs/heads/main"
}
}
}
[00:32:10.274] Task URL: https://cr-buildbucket.appspot.com/build/8811425607126029201
[00:32:10.274] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:2:0) after 9m55s
[00:32:47.904] Received PubSub notification, asking Buildbucket for the build status
[00:32:47.940] Build status: STARTED
[00:42:05.682] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:2:0)
[00:42:05.706] Build status: STARTED
[00:42:05.706] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:4:0) after 7m20s
[00:49:25.730] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:4:0)
[00:49:25.755] Build status: STARTED
[00:49:25.755] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:5:0) after 7m57s
[00:57:22.773] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:5:0)
[00:57:22.842] Build status: STARTED
[00:57:22.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:6:0) after 4m54s
[01:02:16.874] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:6:0)
[01:02:16.897] Build status: STARTED
[01:02:16.897] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:7:0) after 2m55s
[01:05:11.933] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:7:0)
[01:05:11.962] Build status: STARTED
[01:05:11.962] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:8:0) after 5m46s
[01:10:58.089] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:8:0)
[01:10:58.110] Build status: STARTED
[01:10:58.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:9:0) after 7m56s
[01:18:54.600] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:9:0)
[01:18:54.625] Build status: STARTED
[01:18:54.625] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:10:0) after 5m59s
[01:24:53.651] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:10:0)
[01:24:53.675] Build status: STARTED
[01:24:53.675] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:11:0) after 3m13s
[01:28:06.765] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:11:0)
[01:28:06.797] Build status: STARTED
[01:28:06.797] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:12:0) after 4m46s
[01:32:52.887] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:12:0)
[01:32:52.943] Build status: STARTED
[01:32:52.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:13:0) after 1m41s
[01:34:34.266] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:13:0)
[01:34:34.289] Build status: STARTED
[01:34:34.289] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:14:0) after 6m32s
[01:41:06.328] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:14:0)
[01:41:06.353] Build status: STARTED
[01:41:06.353] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:15:0) after 6m59s
[01:48:05.548] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:15:0)
[01:48:05.567] Build status: STARTED
[01:48:05.567] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:16:0) after 1m53s
[01:49:58.594] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:16:0)
[01:49:58.626] Build status: STARTED
[01:49:58.626] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:17:0) after 3m21s
[01:53:20.269] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:17:0)
[01:53:20.290] Build status: STARTED
[01:53:20.290] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:18:0) after 7m11s
[02:00:31.394] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:18:0)
[02:00:31.419] Build status: STARTED
[02:00:31.419] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:19:0) after 8m13s
[02:08:44.443] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:19:0)
[02:08:44.467] Build status: STARTED
[02:08:44.467] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:20:0) after 6m28s
[02:15:12.491] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:20:0)
[02:15:12.523] Build status: STARTED
[02:15:12.523] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:21:0) after 2m2s
[02:17:14.705] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:21:0)
[02:17:14.779] Build status: STARTED
[02:17:14.779] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:22:0) after 5m17s
[02:22:31.893] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:22:0)
[02:22:31.917] Build status: STARTED
[02:22:31.917] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:23:0) after 7m47s
[02:30:19.049] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:23:0)
[02:30:19.074] Build status: STARTED
[02:30:19.074] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:24:0) after 6m53s
[02:37:12.099] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:24:0)
[02:37:12.154] Build status: STARTED
[02:37:12.154] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:25:0) after 8m29s
[02:45:41.211] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:25:0)
[02:45:41.232] Build status: STARTED
[02:45:41.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:26:0) after 6m33s
[02:52:14.242] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:26:0)
[02:52:14.264] Build status: STARTED
[02:52:14.264] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:27:0) after 8m36s
[03:00:50.286] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:27:0)
[03:00:50.307] Build status: STARTED
[03:00:50.307] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:28:0) after 7m11s
[03:08:02.038] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:28:0)
[03:08:02.064] Build status: STARTED
[03:08:02.064] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:29:0) after 4m14s
[03:12:16.089] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:29:0)
[03:12:16.207] Build status: STARTED
[03:12:16.207] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:30:0) after 1m55s
[03:14:11.227] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:30:0)
[03:14:11.252] Build status: STARTED
[03:14:11.252] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:31:0) after 6m44s
[03:20:55.279] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:31:0)
[03:20:55.372] Build status: STARTED
[03:20:55.372] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:32:0) after 9m48s
[03:30:44.153] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:32:0)
[03:30:44.182] Build status: STARTED
[03:30:44.182] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:33:0) after 3m29s
[03:34:13.350] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:33:0)
[03:34:13.374] Build status: STARTED
[03:34:13.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:34:0) after 4m57s
[03:39:10.690] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:34:0)
[03:39:10.714] Build status: STARTED
[03:39:10.714] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:35:0) after 9m27s
[03:48:37.726] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:35:0)
[03:48:37.764] Build status: STARTED
[03:48:37.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:36:0) after 7m22s
[03:55:59.887] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:36:0)
[03:55:59.911] Build status: STARTED
[03:55:59.911] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:37:0) after 4m5s
[04:00:05.052] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:37:0)
[04:00:05.111] Build status: STARTED
[04:00:05.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:38:0) after 3m46s
[04:03:51.596] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:38:0)
[04:03:51.670] Build status: STARTED
[04:03:51.670] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:39:0) after 4m51s
[04:08:42.759] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:39:0)
[04:08:42.790] Build status: STARTED
[04:08:42.790] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:40:0) after 8m9s
[04:16:51.814] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:40:0)
[04:16:51.840] Build status: STARTED
[04:16:51.840] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:41:0) after 2m5s
[04:18:56.930] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:41:0)
[04:18:56.951] Build status: STARTED
[04:18:56.951] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:42:0) after 9m12s
[04:28:08.977] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:42:0)
[04:28:09.010] Build status: STARTED
[04:28:09.010] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:43:0) after 7m26s
[04:35:35.036] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:43:0)
[04:35:35.060] Build status: STARTED
[04:35:35.060] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:44:0) after 5m38s
[04:41:13.120] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:44:0)
[04:41:13.142] Build status: STARTED
[04:41:13.142] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:45:0) after 7m4s
[04:48:17.416] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:45:0)
[04:48:17.444] Build status: STARTED
[04:48:17.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:46:0) after 9m35s
[04:57:52.471] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:46:0)
[04:57:52.505] Build status: STARTED
[04:57:52.505] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:47:0) after 2m35s
[05:00:27.514] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:47:0)
[05:00:27.568] Build status: STARTED
[05:00:27.568] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:48:0) after 5m57s
[05:06:24.672] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:48:0)
[05:06:24.744] Build status: STARTED
[05:06:24.744] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:49:0) after 4m3s
[05:10:27.862] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:49:0)
[05:10:27.885] Build status: STARTED
[05:10:27.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:50:0) after 3m37s
[05:14:04.911] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:50:0)
[05:14:04.943] Build status: STARTED
[05:14:04.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:51:0) after 3m26s
[05:17:31.037] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:51:0)
[05:17:31.061] Build status: STARTED
[05:17:31.061] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:52:0) after 8m59s
[05:26:30.087] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:52:0)
[05:26:30.110] Build status: STARTED
[05:26:30.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:53:0) after 7m9s
[05:33:39.477] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:53:0)
[05:33:39.510] Build status: STARTED
[05:33:39.510] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:54:0) after 2m19s
[05:35:58.534] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:54:0)
[05:35:58.556] Build status: STARTED
[05:35:58.556] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:55:0) after 8m9s
[05:44:07.605] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:55:0)
[05:44:07.644] Build status: STARTED
[05:44:07.644] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:56:0) after 6m51s
[05:50:58.790] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:56:0)
[05:50:58.815] Build status: STARTED
[05:50:58.815] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:57:0) after 6m20s
[05:57:19.170] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:57:0)
[05:57:19.194] Build status: STARTED
[05:57:19.195] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:58:0) after 6m0s
[06:03:19.238] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:58:0)
[06:03:19.262] Build status: STARTED
[06:03:19.262] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:59:0) after 4m10s
[06:07:29.386] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:59:0)
[06:07:29.423] Build status: STARTED
[06:07:29.423] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:60:0) after 1m50s
[06:09:19.455] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:60:0)
[06:09:19.477] Build status: STARTED
[06:09:19.477] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:61:0) after 8m51s
[06:18:10.500] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:61:0)
[06:18:10.541] Build status: STARTED
[06:18:10.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:62:0) after 5m29s
[06:23:39.824] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:62:0)
[06:23:39.849] Build status: STARTED
[06:23:39.849] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:63:0) after 8m7s
[06:31:46.979] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:63:0)
[06:31:47.002] Build status: STARTED
[06:31:47.002] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:64:0) after 1m1s
[06:32:48.297] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:64:0)
[06:32:48.341] Build status: STARTED
[06:32:48.341] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8976855669261382512:65:0) after 7m24s
[06:33:48.031] Received PubSub notification, asking Buildbucket for the build status
[06:33:48.074] Build:
{
"id": "8811425607126029201",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 12830,
"createdBy": "project:chromium",
"createTime": "2022-06-14T00:32:09.767579714Z",
"startTime": "2022-06-14T00:32:46.906934Z",
"endTime": "2022-06-14T06:33:47.701153086Z",
"updateTime": "2022-06-14T06:33:47.701153086Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "933a0f0ae5bd1e3a0cb192350e97806cd7d57590",
"ref": "refs/heads/main"
}
}
}
[06:33:48.074] Invocation finished in 6h1m39.727812862s with status FAILED