[23:08:06.889] New invocation is queued and will start shortly
[23:08:08.050] Starting the invocation (attempt 1)
[23:08:08.084] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/ed485cafd9bde824f6ca6b98158a80dc6bc4e7f9"
[23:08:08.084] Popped gitiles commit info from properties and tags
[23:08:08.084] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[23:08:08.084] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[23:08:08.084] Buildbucket request:
{
"requestId": "8923227551514369056",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ed485cafd9bde824f6ca6b98158a80dc6bc4e7f9",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8923227551514369056"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[23:08:09.210] Scheduled build:
{
"id": "8757797489465664065",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 19322,
"createdBy": "project:chromium",
"createTime": "2024-01-26T23:08:08.217725844Z",
"updateTime": "2024-01-26T23:08:08.217725844Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ed485cafd9bde824f6ca6b98158a80dc6bc4e7f9",
"ref": "refs/heads/main"
}
}
}
[23:08:09.210] Task URL: https://cr-buildbucket.appspot.com/build/8757797489465664065
[23:08:09.210] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:2:0) after 4m0s
[23:08:09.930] Received PubSub notification, asking Buildbucket for the build status
[23:08:09.987] Build status: SCHEDULED
[23:09:57.056] Received PubSub notification, asking Buildbucket for the build status
[23:09:57.094] Build status: STARTED
[23:12:09.278] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:2:0)
[23:12:09.299] Build status: STARTED
[23:12:09.299] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:5:0) after 3m25s
[23:15:34.322] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:5:0)
[23:15:34.348] Build status: STARTED
[23:15:34.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:6:0) after 5m33s
[23:21:07.507] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:6:0)
[23:21:07.532] Build status: STARTED
[23:21:07.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:7:0) after 5m39s
[23:26:46.658] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:7:0)
[23:26:46.717] Build status: STARTED
[23:26:46.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:8:0) after 3m0s
[23:29:46.796] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:8:0)
[23:29:46.820] Build status: STARTED
[23:29:46.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:9:0) after 1m32s
[23:31:19.308] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:9:0)
[23:31:19.325] Build status: STARTED
[23:31:19.325] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:10:0) after 3m23s
[23:34:42.359] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:10:0)
[23:34:42.392] Build status: STARTED
[23:34:42.392] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:11:0) after 6m17s
[23:40:59.594] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:11:0)
[23:40:59.623] Build status: STARTED
[23:40:59.623] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:12:0) after 2m26s
[23:43:25.915] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:12:0)
[23:43:25.943] Build status: STARTED
[23:43:25.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:13:0) after 5m3s
[23:48:29.225] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:13:0)
[23:48:29.249] Build status: STARTED
[23:48:29.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:14:0) after 8m7s
[23:56:36.282] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:14:0)
[23:56:36.307] Build status: STARTED
[23:56:36.307] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:15:0) after 8m58s
[00:05:34.325] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:15:0)
[00:05:34.344] Build status: STARTED
[00:05:34.344] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:16:0) after 9m21s
[00:14:55.342] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:16:0)
[00:14:55.392] Build status: STARTED
[00:14:55.392] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:17:0) after 3m9s
[00:18:04.424] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:17:0)
[00:18:04.452] Build status: STARTED
[00:18:04.452] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:18:0) after 7m14s
[00:25:18.473] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:18:0)
[00:25:18.718] Build status: STARTED
[00:25:18.718] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:19:0) after 2m25s
[00:27:43.737] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:19:0)
[00:27:43.761] Build status: STARTED
[00:27:43.761] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:20:0) after 6m26s
[00:34:09.778] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:20:0)
[00:34:09.800] Build status: STARTED
[00:34:09.800] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:21:0) after 3m22s
[00:37:31.841] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:21:0)
[00:37:31.858] Build status: STARTED
[00:37:31.858] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:22:0) after 9m50s
[00:47:21.941] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:22:0)
[00:47:21.967] Build status: STARTED
[00:47:21.967] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:23:0) after 5m46s
[00:53:08.460] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:23:0)
[00:53:08.490] Build status: STARTED
[00:53:08.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:24:0) after 8m28s
[01:01:36.811] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:24:0)
[01:01:36.833] Build status: STARTED
[01:01:36.833] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:25:0) after 2m14s
[01:03:51.008] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:25:0)
[01:03:51.031] Build status: STARTED
[01:03:51.031] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:26:0) after 2m11s
[01:06:02.431] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:26:0)
[01:06:02.493] Build status: STARTED
[01:06:02.493] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:27:0) after 6m57s
[01:12:59.522] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:27:0)
[01:12:59.541] Build status: STARTED
[01:12:59.541] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:28:0) after 2m21s
[01:15:20.542] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:28:0)
[01:15:20.563] Build status: STARTED
[01:15:20.563] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:29:0) after 9m7s
[01:24:28.004] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:29:0)
[01:24:28.118] Build status: STARTED
[01:24:28.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:30:0) after 3m57s
[01:28:25.633] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:30:0)
[01:28:25.721] Build status: STARTED
[01:28:25.721] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:31:0) after 3m4s
[01:31:29.740] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:31:0)
[01:31:29.760] Build status: STARTED
[01:31:29.760] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:32:0) after 6m46s
[01:38:15.802] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:32:0)
[01:38:15.821] Build status: STARTED
[01:38:15.821] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:33:0) after 4m10s
[01:42:26.092] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:33:0)
[01:42:26.168] Build status: STARTED
[01:42:26.168] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:34:0) after 5m51s
[01:48:17.425] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:34:0)
[01:48:17.526] Build status: STARTED
[01:48:17.526] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:35:0) after 7m48s
[01:56:05.545] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:35:0)
[01:56:05.566] Build status: STARTED
[01:56:05.566] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:36:0) after 2m51s
[01:58:56.611] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:36:0)
[01:58:56.629] Build status: STARTED
[01:58:56.629] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:37:0) after 3m3s
[02:01:59.666] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:37:0)
[02:01:59.685] Build status: STARTED
[02:01:59.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:38:0) after 8m50s
[02:10:49.928] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:38:0)
[02:10:49.950] Build status: STARTED
[02:10:49.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:39:0) after 7m47s
[02:18:37.163] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:39:0)
[02:18:37.181] Build status: STARTED
[02:18:37.181] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:40:0) after 4m56s
[02:23:33.303] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:40:0)
[02:23:33.364] Build status: STARTED
[02:23:33.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:41:0) after 8m10s
[02:31:43.452] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:41:0)
[02:31:43.494] Build status: STARTED
[02:31:43.494] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:42:0) after 8m54s
[02:40:37.470] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:42:0)
[02:40:37.533] Build status: STARTED
[02:40:37.533] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:43:0) after 6m6s
[02:46:43.528] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:43:0)
[02:46:43.557] Build status: STARTED
[02:46:43.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:44:0) after 2m21s
[02:49:05.021] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:44:0)
[02:49:05.047] Build status: STARTED
[02:49:05.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:45:0) after 6m34s
[02:55:39.363] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:45:0)
[02:55:39.383] Build status: STARTED
[02:55:39.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:46:0) after 3m47s
[02:59:26.767] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:46:0)
[02:59:26.824] Build status: STARTED
[02:59:26.824] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:47:0) after 9m15s
[03:08:42.373] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:47:0)
[03:08:42.532] Build status: STARTED
[03:08:42.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:48:0) after 2m35s
[03:11:17.558] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:48:0)
[03:11:17.633] Build status: STARTED
[03:11:17.633] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:49:0) after 4m14s
[03:15:31.669] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:49:0)
[03:15:31.690] Build status: STARTED
[03:15:31.690] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:50:0) after 6m9s
[03:21:40.847] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:50:0)
[03:21:40.865] Build status: STARTED
[03:21:40.866] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:51:0) after 6m50s
[03:28:31.131] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:51:0)
[03:28:31.158] Build status: STARTED
[03:28:31.158] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:52:0) after 1m2s
[03:29:33.349] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:52:0)
[03:29:33.374] Build status: STARTED
[03:29:33.374] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923227551514369056:53:0) after 8m15s
[03:37:16.095] Received PubSub notification, asking Buildbucket for the build status
[03:37:16.121] Build:
{
"id": "8757797489465664065",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 19322,
"createdBy": "project:chromium",
"createTime": "2024-01-26T23:08:08.217725844Z",
"startTime": "2024-01-26T23:09:56.097493Z",
"endTime": "2024-01-27T03:37:07.754370Z",
"updateTime": "2024-01-27T03:37:07.754370Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "ed485cafd9bde824f6ca6b98158a80dc6bc4e7f9",
"ref": "refs/heads/main"
}
}
}
[03:37:16.121] Invocation finished in 4h29m9.241743778s with status FAILED