[23:55:19.312] New invocation is queued and will start shortly
[23:55:20.448] Starting the invocation (attempt 1)
[23:55:20.496] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/af58c2d77ae9da0e31e700898f707c8a8bd94c65"
[23:55:20.496] Popped gitiles commit info from properties and tags
[23:55:20.496] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[23:55:20.496] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[23:55:20.496] Buildbucket request:
{
"requestId": "8955567698504832432",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "af58c2d77ae9da0e31e700898f707c8a8bd94c65",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8955567698504832432"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[23:55:21.235] Scheduled build:
{
"id": "8790137636464065361",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 16053,
"createdBy": "project:chromium",
"createTime": "2023-02-03T23:55:20.636649417Z",
"updateTime": "2023-02-03T23:55:20.636649417Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "af58c2d77ae9da0e31e700898f707c8a8bd94c65",
"ref": "refs/heads/main"
}
},
"canary": true
}
[23:55:21.235] Task URL: https://cr-buildbucket.appspot.com/build/8790137636464065361
[23:55:21.235] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:2:0) after 9m0s
[23:56:52.043] Received PubSub notification, asking Buildbucket for the build status
[23:56:52.071] Build status: STARTED
[00:04:21.668] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:2:0)
[00:04:21.704] Build status: STARTED
[00:04:21.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:4:0) after 6m6s
[00:10:27.727] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:4:0)
[00:10:27.754] Build status: STARTED
[00:10:27.754] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:5:0) after 7m56s
[00:18:24.003] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:5:0)
[00:18:24.032] Build status: STARTED
[00:18:24.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:6:0) after 3m58s
[00:22:22.123] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:6:0)
[00:22:22.145] Build status: STARTED
[00:22:22.145] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:7:0) after 4m38s
[00:27:00.191] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:7:0)
[00:27:00.265] Build status: STARTED
[00:27:00.265] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:8:0) after 6m25s
[00:33:25.328] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:8:0)
[00:33:25.369] Build status: STARTED
[00:33:25.369] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:9:0) after 9m52s
[00:43:17.446] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:9:0)
[00:43:17.467] Build status: STARTED
[00:43:17.467] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:10:0) after 3m7s
[00:46:24.565] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:10:0)
[00:46:24.586] Build status: STARTED
[00:46:24.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:11:0) after 4m55s
[00:51:19.749] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:11:0)
[00:51:19.769] Build status: STARTED
[00:51:19.769] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:12:0) after 4m7s
[00:55:26.878] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:12:0)
[00:55:26.903] Build status: STARTED
[00:55:26.903] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:13:0) after 1m10s
[00:56:37.016] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:13:0)
[00:56:37.063] Build status: STARTED
[00:56:37.063] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:14:0) after 7m21s
[01:03:58.559] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:14:0)
[01:03:58.581] Build status: STARTED
[01:03:58.581] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:15:0) after 1m18s
[01:05:16.758] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:15:0)
[01:05:16.779] Build status: STARTED
[01:05:16.779] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:16:0) after 2m50s
[01:08:07.374] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:16:0)
[01:08:07.397] Build status: STARTED
[01:08:07.397] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:17:0) after 4m11s
[01:12:18.840] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:17:0)
[01:12:18.868] Build status: STARTED
[01:12:18.869] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:18:0) after 7m55s
[01:20:14.306] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:18:0)
[01:20:14.329] Build status: STARTED
[01:20:14.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:19:0) after 7m19s
[01:27:33.523] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:19:0)
[01:27:33.553] Build status: STARTED
[01:27:33.553] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:20:0) after 6m34s
[01:34:07.794] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:20:0)
[01:34:08.003] Build status: STARTED
[01:34:08.003] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:21:0) after 8m26s
[01:42:34.052] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:21:0)
[01:42:34.074] Build status: STARTED
[01:42:34.074] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:22:0) after 8m13s
[01:50:47.133] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:22:0)
[01:50:47.192] Build status: STARTED
[01:50:47.192] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:23:0) after 4m18s
[01:55:05.564] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:23:0)
[01:55:05.585] Build status: STARTED
[01:55:05.585] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:24:0) after 1m12s
[01:56:17.704] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:24:0)
[01:56:17.729] Build status: STARTED
[01:56:17.729] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:25:0) after 6m17s
[02:02:34.754] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:25:0)
[02:02:34.778] Build status: STARTED
[02:02:34.778] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:26:0) after 2m27s
[02:05:01.800] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:26:0)
[02:05:01.820] Build status: STARTED
[02:05:01.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:27:0) after 1m37s
[02:06:39.039] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:27:0)
[02:06:39.063] Build status: STARTED
[02:06:39.063] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:28:0) after 9m7s
[02:15:46.080] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:28:0)
[02:15:46.108] Build status: STARTED
[02:15:46.108] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:29:0) after 7m19s
[02:23:05.156] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:29:0)
[02:23:05.201] Build status: STARTED
[02:23:05.202] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:30:0) after 3m11s
[02:26:16.278] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:30:0)
[02:26:16.300] Build status: STARTED
[02:26:16.300] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:31:0) after 3m18s
[02:29:34.299] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:31:0)
[02:29:34.319] Build status: STARTED
[02:29:34.319] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:32:0) after 7m53s
[02:37:27.364] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:32:0)
[02:37:27.383] Build status: STARTED
[02:37:27.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:33:0) after 7m2s
[02:44:29.402] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:33:0)
[02:44:29.455] Build status: STARTED
[02:44:29.455] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:34:0) after 3m48s
[02:48:17.551] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:34:0)
[02:48:17.571] Build status: STARTED
[02:48:17.571] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:35:0) after 8m43s
[02:57:00.607] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:35:0)
[02:57:00.628] Build status: STARTED
[02:57:00.628] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:36:0) after 1m52s
[02:58:52.825] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:36:0)
[02:58:52.856] Build status: STARTED
[02:58:52.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:37:0) after 4m5s
[03:02:57.929] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:37:0)
[03:02:57.954] Build status: STARTED
[03:02:57.954] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:38:0) after 1m31s
[03:04:28.982] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:38:0)
[03:04:29.010] Build status: STARTED
[03:04:29.010] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:39:0) after 2m43s
[03:07:12.063] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:39:0)
[03:07:12.085] Build status: STARTED
[03:07:12.085] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:40:0) after 4m27s
[03:11:39.823] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:40:0)
[03:11:39.877] Build status: STARTED
[03:11:39.877] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:41:0) after 3m30s
[03:15:10.001] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:41:0)
[03:15:10.021] Build status: STARTED
[03:15:10.021] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8955567698504832432:42:0) after 6m29s
[03:17:36.866] Received PubSub notification, asking Buildbucket for the build status
[03:17:36.916] Build:
{
"id": "8790137636464065361",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 16053,
"createdBy": "project:chromium",
"createTime": "2023-02-03T23:55:20.636649417Z",
"startTime": "2023-02-03T23:56:51.488956Z",
"endTime": "2023-02-04T03:17:36.601990076Z",
"updateTime": "2023-02-04T03:17:36.601990076Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "af58c2d77ae9da0e31e700898f707c8a8bd94c65",
"ref": "refs/heads/main"
}
},
"canary": true
}
[03:17:36.916] Invocation finished in 3h22m17.610372527s with status FAILED