[23:03:28.527] New invocation is queued and will start shortly
[23:03:30.320] Starting the invocation (attempt 1)
[23:03:30.350] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/bd885a69ebb7672139a02bc644ec6d70717c0dce"
[23:03:30.350] Popped gitiles commit info from properties and tags
[23:03:30.350] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[23:03:30.358] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[23:03:30.358] Buildbucket request:
{
"requestId": "8911359640804708320",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bd885a69ebb7672139a02bc644ec6d70717c0dce",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8911359640804708320"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[23:03:31.308] Scheduled build:
{
"id": "8745929578089505969",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 21614,
"createdBy": "project:chromium",
"createTime": "2024-06-05T23:03:30.486853837Z",
"updateTime": "2024-06-05T23:03:30.486853837Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bd885a69ebb7672139a02bc644ec6d70717c0dce",
"ref": "refs/heads/main"
}
}
}
[23:03:31.308] Task URL: https://cr-buildbucket.appspot.com/build/8745929578089505969
[23:03:31.308] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:2:0) after 5m55s
[23:03:32.260] Received PubSub notification, asking Buildbucket for the build status
[23:03:32.278] Build status: SCHEDULED
[23:08:05.044] Received PubSub notification, asking Buildbucket for the build status
[23:08:05.077] Build status: STARTED
[23:09:26.327] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:2:0)
[23:09:26.364] Build status: STARTED
[23:09:26.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:5:0) after 4m10s
[23:13:36.391] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:5:0)
[23:13:36.412] Build status: STARTED
[23:13:36.412] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:6:0) after 4m35s
[23:18:11.662] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:6:0)
[23:18:11.700] Build status: STARTED
[23:18:11.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:7:0) after 8m53s
[23:27:04.740] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:7:0)
[23:27:04.776] Build status: STARTED
[23:27:04.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:8:0) after 8m48s
[23:35:52.796] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:8:0)
[23:35:52.831] Build status: STARTED
[23:35:52.831] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:9:0) after 7m27s
[23:43:19.852] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:9:0)
[23:43:19.871] Build status: STARTED
[23:43:19.871] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:10:0) after 8m32s
[23:51:51.941] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:10:0)
[23:51:51.963] Build status: STARTED
[23:51:51.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:11:0) after 8m50s
[00:00:41.989] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:11:0)
[00:00:42.012] Build status: STARTED
[00:00:42.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:12:0) after 3m57s
[00:04:39.058] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:12:0)
[00:04:39.086] Build status: STARTED
[00:04:39.086] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:13:0) after 4m11s
[00:08:50.121] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:13:0)
[00:08:50.141] Build status: STARTED
[00:08:50.141] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:14:0) after 2m44s
[00:11:34.141] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:14:0)
[00:11:34.163] Build status: STARTED
[00:11:34.163] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:15:0) after 6m20s
[00:17:54.235] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:15:0)
[00:17:54.253] Build status: STARTED
[00:17:54.253] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:16:0) after 1m24s
[00:19:18.273] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:16:0)
[00:19:18.294] Build status: STARTED
[00:19:18.294] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:17:0) after 5m2s
[00:24:20.346] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:17:0)
[00:24:20.372] Build status: STARTED
[00:24:20.372] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:18:0) after 5m42s
[00:30:02.393] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:18:0)
[00:30:02.408] Build status: STARTED
[00:30:02.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:19:0) after 1m12s
[00:31:14.432] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:19:0)
[00:31:14.461] Build status: STARTED
[00:31:14.462] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:20:0) after 7m44s
[00:38:58.483] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:20:0)
[00:38:58.518] Build status: STARTED
[00:38:58.518] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:21:0) after 4m17s
[00:43:15.543] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:21:0)
[00:43:15.602] Build status: STARTED
[00:43:15.602] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:22:0) after 1m47s
[00:45:02.624] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:22:0)
[00:45:02.644] Build status: STARTED
[00:45:02.644] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:23:0) after 4m15s
[00:49:17.714] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:23:0)
[00:49:17.729] Build status: STARTED
[00:49:17.729] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:24:0) after 3m41s
[00:52:58.869] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:24:0)
[00:52:58.889] Build status: STARTED
[00:52:58.889] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:25:0) after 7m43s
[01:00:41.875] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:25:0)
[01:00:41.900] Build status: STARTED
[01:00:41.901] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:26:0) after 2m43s
[01:03:24.926] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:26:0)
[01:03:24.947] Build status: STARTED
[01:03:24.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:27:0) after 6m45s
[01:10:10.094] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:27:0)
[01:10:10.115] Build status: STARTED
[01:10:10.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:28:0) after 5m32s
[01:15:42.145] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:28:0)
[01:15:42.162] Build status: STARTED
[01:15:42.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:29:0) after 3m55s
[01:19:37.185] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:29:0)
[01:19:37.235] Build status: STARTED
[01:19:37.235] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:30:0) after 8m44s
[01:28:21.385] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:30:0)
[01:28:21.404] Build status: STARTED
[01:28:21.404] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:31:0) after 2m13s
[01:30:34.431] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:31:0)
[01:30:34.451] Build status: STARTED
[01:30:34.451] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:32:0) after 6m0s
[01:36:34.541] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:32:0)
[01:36:34.557] Build status: STARTED
[01:36:34.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:33:0) after 9m50s
[01:46:24.587] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:33:0)
[01:46:24.604] Build status: STARTED
[01:46:24.604] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:34:0) after 1m23s
[01:47:47.724] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:34:0)
[01:47:47.740] Build status: STARTED
[01:47:47.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:35:0) after 5m13s
[01:53:00.782] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:35:0)
[01:53:00.834] Build status: STARTED
[01:53:00.834] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:36:0) after 3m30s
[01:56:30.867] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:36:0)
[01:56:30.886] Build status: STARTED
[01:56:30.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:37:0) after 1m22s
[01:57:52.909] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:37:0)
[01:57:52.934] Build status: STARTED
[01:57:52.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:38:0) after 4m8s
[02:02:00.973] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:38:0)
[02:02:01.001] Build status: STARTED
[02:02:01.001] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:39:0) after 9m10s
[02:11:11.393] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:39:0)
[02:11:11.412] Build status: STARTED
[02:11:11.412] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:40:0) after 2m36s
[02:13:47.576] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:40:0)
[02:13:47.599] Build status: STARTED
[02:13:47.599] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:41:0) after 8m55s
[02:22:42.762] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:41:0)
[02:22:42.789] Build status: STARTED
[02:22:42.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:42:0) after 6m39s
[02:29:21.818] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:42:0)
[02:29:21.981] Build status: STARTED
[02:29:21.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:43:0) after 3m36s
[02:32:58.268] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:43:0)
[02:32:58.286] Build status: STARTED
[02:32:58.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:44:0) after 5m37s
[02:38:35.310] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:44:0)
[02:38:35.328] Build status: STARTED
[02:38:35.328] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:45:0) after 3m25s
[02:42:00.752] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:45:0)
[02:42:00.777] Build status: STARTED
[02:42:00.777] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:46:0) after 3m52s
[02:45:52.799] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:46:0)
[02:45:52.815] Build status: STARTED
[02:45:52.815] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:47:0) after 7m23s
[02:53:15.874] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:47:0)
[02:53:15.920] Build status: STARTED
[02:53:15.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:48:0) after 1m30s
[02:54:45.911] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:48:0)
[02:54:45.934] Build status: STARTED
[02:54:45.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:49:0) after 5m14s
[03:00:00.055] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:49:0)
[03:00:00.072] Build status: STARTED
[03:00:00.072] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:50:0) after 8m58s
[03:08:58.191] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:50:0)
[03:08:58.243] Build status: STARTED
[03:08:58.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:51:0) after 5m45s
[03:14:43.370] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:51:0)
[03:14:43.387] Build status: STARTED
[03:14:43.387] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:52:0) after 9m9s
[03:23:52.792] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:52:0)
[03:23:52.851] Build status: STARTED
[03:23:52.851] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:53:0) after 9m36s
[03:33:29.134] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:53:0)
[03:33:29.199] Build status: STARTED
[03:33:29.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:54:0) after 4m22s
[03:37:51.287] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:54:0)
[03:37:51.304] Build status: STARTED
[03:37:51.304] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8911359640804708320:55:0) after 8m55s
[03:43:21.582] Received PubSub notification, asking Buildbucket for the build status
[03:43:21.601] Build:
{
"id": "8745929578089505969",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 21614,
"createdBy": "project:chromium",
"createTime": "2024-06-05T23:03:30.486853837Z",
"startTime": "2024-06-05T23:07:40.802582564Z",
"endTime": "2024-06-06T03:43:21.213742492Z",
"updateTime": "2024-06-06T03:43:21.213742492Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "bd885a69ebb7672139a02bc644ec6d70717c0dce",
"ref": "refs/heads/main"
}
}
}
[03:43:21.601] Invocation finished in 4h39m53.088338749s with status FAILED