[17:35:50.515] New invocation is queued and will start shortly
[17:35:52.293] Starting the invocation (attempt 1)
[17:35:52.317] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/da9ee881bd44c9c58368c7bf4b108cbf5dae73b7"
[17:35:52.317] Popped gitiles commit info from properties and tags
[17:35:52.317] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[17:35:52.318] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[17:35:52.318] Buildbucket request:
{
"requestId": "8910927268888790880",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "da9ee881bd44c9c58368c7bf4b108cbf5dae73b7",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8910927268888790880"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[17:35:52.931] Scheduled build:
{
"id": "8745497206272278785",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 21654,
"createdBy": "project:chromium",
"createTime": "2024-06-10T17:35:52.383822259Z",
"updateTime": "2024-06-10T17:35:52.383822259Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "da9ee881bd44c9c58368c7bf4b108cbf5dae73b7",
"ref": "refs/heads/main"
}
},
"canary": true
}
[17:35:52.931] Task URL: https://cr-buildbucket.appspot.com/build/8745497206272278785
[17:35:52.931] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:2:0) after 5m9s
[17:35:54.493] Received PubSub notification, asking Buildbucket for the build status
[17:35:54.511] Build status: SCHEDULED
[17:37:02.405] Received PubSub notification, asking Buildbucket for the build status
[17:37:02.530] Build status: STARTED
[17:41:01.986] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:2:0)
[17:41:02.010] Build status: STARTED
[17:41:02.010] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:5:0) after 9m57s
[17:50:59.054] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:5:0)
[17:50:59.107] Build status: STARTED
[17:50:59.107] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:6:0) after 7m6s
[17:58:05.276] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:6:0)
[17:58:05.301] Build status: STARTED
[17:58:05.301] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:7:0) after 4m55s
[18:03:00.332] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:7:0)
[18:03:00.351] Build status: STARTED
[18:03:00.351] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:8:0) after 4m52s
[18:07:52.451] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:8:0)
[18:07:52.472] Build status: STARTED
[18:07:52.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:9:0) after 7m13s
[18:15:05.489] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:9:0)
[18:15:05.507] Build status: STARTED
[18:15:05.507] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:10:0) after 5m27s
[18:20:32.529] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:10:0)
[18:20:32.561] Build status: STARTED
[18:20:32.561] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:11:0) after 9m58s
[18:30:30.578] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:11:0)
[18:30:30.596] Build status: STARTED
[18:30:30.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:12:0) after 1m30s
[18:32:00.620] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:12:0)
[18:32:00.639] Build status: STARTED
[18:32:00.639] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:13:0) after 4m4s
[18:36:04.722] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:13:0)
[18:36:04.743] Build status: STARTED
[18:36:04.743] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:14:0) after 6m46s
[18:42:50.807] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:14:0)
[18:42:50.825] Build status: STARTED
[18:42:50.825] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:15:0) after 7m40s
[18:50:30.799] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:15:0)
[18:50:30.818] Build status: STARTED
[18:50:30.819] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:16:0) after 4m26s
[18:54:56.855] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:16:0)
[18:54:56.875] Build status: STARTED
[18:54:56.875] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:17:0) after 5m16s
[19:00:12.905] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:17:0)
[19:00:12.951] Build status: STARTED
[19:00:12.951] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:18:0) after 1m17s
[19:01:29.969] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:18:0)
[19:01:30.099] Build status: STARTED
[19:01:30.099] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:19:0) after 8m44s
[19:10:14.123] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:19:0)
[19:10:14.159] Build status: STARTED
[19:10:14.159] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:20:0) after 1m22s
[19:11:36.154] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:20:0)
[19:11:36.220] Build status: STARTED
[19:11:36.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:21:0) after 9m35s
[19:21:11.291] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:21:0)
[19:21:11.311] Build status: STARTED
[19:21:11.311] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:22:0) after 9m44s
[19:30:55.329] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:22:0)
[19:30:55.362] Build status: STARTED
[19:30:55.362] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:23:0) after 7m45s
[19:38:40.413] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:23:0)
[19:38:40.439] Build status: STARTED
[19:38:40.439] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:24:0) after 5m8s
[19:43:48.447] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:24:0)
[19:43:48.476] Build status: STARTED
[19:43:48.476] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:25:0) after 6m55s
[19:50:43.586] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:25:0)
[19:50:43.608] Build status: STARTED
[19:50:43.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:26:0) after 6m2s
[19:56:45.706] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:26:0)
[19:56:45.727] Build status: STARTED
[19:56:45.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:27:0) after 8m32s
[20:05:17.769] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:27:0)
[20:05:17.790] Build status: STARTED
[20:05:17.790] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:28:0) after 5m11s
[20:10:28.872] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:28:0)
[20:10:28.897] Build status: STARTED
[20:10:28.897] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:29:0) after 1m7s
[20:11:35.921] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:29:0)
[20:11:35.940] Build status: STARTED
[20:11:35.940] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:30:0) after 9m22s
[20:20:57.999] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:30:0)
[20:20:58.017] Build status: STARTED
[20:20:58.017] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:31:0) after 1m11s
[20:22:09.048] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:31:0)
[20:22:09.078] Build status: STARTED
[20:22:09.078] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:32:0) after 8m25s
[20:30:34.119] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:32:0)
[20:30:34.141] Build status: STARTED
[20:30:34.141] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:33:0) after 4m56s
[20:35:30.169] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:33:0)
[20:35:30.196] Build status: STARTED
[20:35:30.196] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:34:0) after 7m59s
[20:43:29.219] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:34:0)
[20:43:29.254] Build status: STARTED
[20:43:29.254] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:35:0) after 6m57s
[20:50:26.277] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:35:0)
[20:50:26.339] Build status: STARTED
[20:50:26.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:36:0) after 3m34s
[20:54:00.388] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:36:0)
[20:54:00.412] Build status: STARTED
[20:54:00.412] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:37:0) after 1m7s
[20:55:07.469] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:37:0)
[20:55:07.486] Build status: STARTED
[20:55:07.486] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:38:0) after 5m12s
[21:00:19.529] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:38:0)
[21:00:19.579] Build status: STARTED
[21:00:19.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:39:0) after 2m44s
[21:03:03.597] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:39:0)
[21:03:03.615] Build status: STARTED
[21:03:03.615] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:40:0) after 9m7s
[21:12:10.636] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:40:0)
[21:12:10.655] Build status: STARTED
[21:12:10.655] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:41:0) after 9m2s
[21:21:12.678] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:41:0)
[21:21:12.705] Build status: STARTED
[21:21:12.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:42:0) after 1m57s
[21:23:09.734] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:42:0)
[21:23:09.784] Build status: STARTED
[21:23:09.784] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:43:0) after 4m54s
[21:28:03.815] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:43:0)
[21:28:03.856] Build status: STARTED
[21:28:03.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:44:0) after 5m13s
[21:33:16.879] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:44:0)
[21:33:16.910] Build status: STARTED
[21:33:16.910] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:45:0) after 4m58s
[21:38:14.978] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:45:0)
[21:38:15.032] Build status: STARTED
[21:38:15.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:46:0) after 2m53s
[21:41:08.061] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:46:0)
[21:41:08.084] Build status: STARTED
[21:41:08.084] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:47:0) after 6m21s
[21:47:29.119] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:47:0)
[21:47:29.197] Build status: STARTED
[21:47:29.197] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:48:0) after 9m52s
[21:57:21.265] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:48:0)
[21:57:21.279] Build status: STARTED
[21:57:21.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8910927268888790880:49:0) after 8m6s
[22:03:17.246] Received PubSub notification, asking Buildbucket for the build status
[22:03:17.284] Build:
{
"id": "8745497206272278785",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 21654,
"createdBy": "project:chromium",
"createTime": "2024-06-10T17:35:52.383822259Z",
"startTime": "2024-06-10T17:37:02.188245890Z",
"endTime": "2024-06-10T22:03:16.955034320Z",
"updateTime": "2024-06-10T22:03:16.955034320Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "da9ee881bd44c9c58368c7bf4b108cbf5dae73b7",
"ref": "refs/heads/main"
}
},
"canary": true
}
[22:03:17.284] Invocation finished in 4h27m26.779538797s with status FAILED