[23:46:49.410] New invocation is queued and will start shortly
[23:46:50.651] Starting the invocation (attempt 1)
[23:46:50.686] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d35f57148dde64a4bbc6a12c6c31c3d87e2b7f2a"
[23:46:50.687] Popped gitiles commit info from properties and tags
[23:46:50.687] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[23:46:50.687] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[23:46:50.687] Buildbucket request:
{
"requestId": "8923315713144733120",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d35f57148dde64a4bbc6a12c6c31c3d87e2b7f2a",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8923315713144733120"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[23:46:51.206] Scheduled build:
{
"id": "8757885651080633617",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 21402,
"createdBy": "project:chromium",
"createTime": "2024-01-25T23:46:50.749691976Z",
"updateTime": "2024-01-25T23:46:50.749691976Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d35f57148dde64a4bbc6a12c6c31c3d87e2b7f2a",
"ref": "refs/heads/main"
}
}
}
[23:46:51.206] Task URL: https://cr-buildbucket.appspot.com/build/8757885651080633617
[23:46:51.206] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:2:0) after 9m34s
[23:46:52.280] Received PubSub notification, asking Buildbucket for the build status
[23:46:52.330] Build status: SCHEDULED
[23:48:44.020] Received PubSub notification, asking Buildbucket for the build status
[23:48:44.040] Build status: STARTED
[23:56:25.226] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:2:0)
[23:56:25.250] Build status: STARTED
[23:56:25.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:5:0) after 1m10s
[23:57:35.341] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:5:0)
[23:57:35.364] Build status: STARTED
[23:57:35.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:6:0) after 8m18s
[00:05:53.488] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:6:0)
[00:05:53.542] Build status: STARTED
[00:05:53.542] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:7:0) after 4m18s
[00:10:11.565] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:7:0)
[00:10:11.589] Build status: STARTED
[00:10:11.589] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:8:0) after 8m28s
[00:18:39.716] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:8:0)
[00:18:39.733] Build status: STARTED
[00:18:39.733] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:9:0) after 1m26s
[00:20:05.805] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:9:0)
[00:20:05.829] Build status: STARTED
[00:20:05.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:10:0) after 2m31s
[00:22:36.955] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:10:0)
[00:22:36.973] Build status: STARTED
[00:22:36.973] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:11:0) after 1m47s
[00:24:23.993] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:11:0)
[00:24:24.014] Build status: STARTED
[00:24:24.014] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:12:0) after 7m44s
[00:32:08.040] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:12:0)
[00:32:08.110] Build status: STARTED
[00:32:08.110] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:13:0) after 1m26s
[00:33:34.154] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:13:0)
[00:33:34.177] Build status: STARTED
[00:33:34.177] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:14:0) after 5m31s
[00:39:05.204] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:14:0)
[00:39:05.270] Build status: STARTED
[00:39:05.270] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:15:0) after 3m19s
[00:42:24.360] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:15:0)
[00:42:24.383] Build status: STARTED
[00:42:24.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:16:0) after 8m43s
[00:51:07.507] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:16:0)
[00:51:07.537] Build status: STARTED
[00:51:07.537] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:17:0) after 8m14s
[00:59:21.598] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:17:0)
[00:59:21.627] Build status: STARTED
[00:59:21.627] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:18:0) after 9m46s
[01:09:07.663] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:18:0)
[01:09:07.688] Build status: STARTED
[01:09:07.688] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:19:0) after 8m41s
[01:17:48.825] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:19:0)
[01:17:48.853] Build status: STARTED
[01:17:48.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:20:0) after 4m11s
[01:21:59.875] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:20:0)
[01:21:59.894] Build status: STARTED
[01:21:59.894] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:21:0) after 9m45s
[01:31:45.013] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:21:0)
[01:31:45.077] Build status: STARTED
[01:31:45.077] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:22:0) after 7m42s
[01:39:27.145] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:22:0)
[01:39:35.204] Build status: STARTED
[01:39:35.204] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:23:0) after 2m10s
[01:41:45.231] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:23:0)
[01:41:45.250] Build status: STARTED
[01:41:45.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:24:0) after 3m52s
[01:45:37.612] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:24:0)
[01:45:37.631] Build status: STARTED
[01:45:37.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:25:0) after 9m38s
[01:55:15.766] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:25:0)
[01:55:15.813] Build status: STARTED
[01:55:15.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:26:0) after 1m47s
[01:57:02.830] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:26:0)
[01:57:02.883] Build status: STARTED
[01:57:02.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:27:0) after 8m0s
[02:05:02.906] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:27:0)
[02:05:02.940] Build status: STARTED
[02:05:02.940] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:28:0) after 9m52s
[02:14:54.961] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:28:0)
[02:14:54.983] Build status: STARTED
[02:14:54.983] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:29:0) after 8m35s
[02:23:30.122] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:29:0)
[02:23:30.234] Build status: STARTED
[02:23:30.234] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:30:0) after 9m29s
[02:32:59.292] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:30:0)
[02:32:59.407] Build status: STARTED
[02:32:59.407] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:31:0) after 3m47s
[02:36:46.591] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:31:0)
[02:36:46.608] Build status: STARTED
[02:36:46.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:32:0) after 5m31s
[02:42:18.091] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:32:0)
[02:42:18.111] Build status: STARTED
[02:42:18.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:33:0) after 6m52s
[02:49:10.136] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:33:0)
[02:49:10.173] Build status: STARTED
[02:49:10.173] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:34:0) after 8m9s
[02:57:19.194] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:34:0)
[02:57:19.213] Build status: STARTED
[02:57:19.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:35:0) after 2m20s
[02:59:39.432] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:35:0)
[02:59:39.453] Build status: STARTED
[02:59:39.453] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:36:0) after 1m27s
[03:01:06.526] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:36:0)
[03:01:06.554] Build status: STARTED
[03:01:06.554] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:37:0) after 9m5s
[03:10:11.822] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:37:0)
[03:10:11.851] Build status: STARTED
[03:10:11.851] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:38:0) after 9m56s
[03:20:07.971] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:38:0)
[03:20:08.007] Build status: STARTED
[03:20:08.007] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:39:0) after 6m18s
[03:26:26.055] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:39:0)
[03:26:26.080] Build status: STARTED
[03:26:26.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:40:0) after 2m58s
[03:29:24.209] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:40:0)
[03:29:24.266] Build status: STARTED
[03:29:24.267] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:41:0) after 6m53s
[03:36:17.296] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:41:0)
[03:36:17.333] Build status: STARTED
[03:36:17.333] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:42:0) after 8m55s
[03:45:12.365] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:42:0)
[03:45:12.425] Build status: STARTED
[03:45:12.425] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:43:0) after 1m35s
[03:46:47.447] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:43:0)
[03:46:47.468] Build status: STARTED
[03:46:47.469] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:44:0) after 5m18s
[03:52:05.574] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:44:0)
[03:52:05.605] Build status: STARTED
[03:52:05.609] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:45:0) after 3m47s
[03:55:52.655] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:45:0)
[03:55:52.670] Build status: STARTED
[03:55:52.670] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:46:0) after 5m30s
[04:01:22.700] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:46:0)
[04:01:22.720] Build status: STARTED
[04:01:22.720] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:47:0) after 5m34s
[04:06:56.830] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:47:0)
[04:06:56.881] Build status: STARTED
[04:06:56.881] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:48:0) after 5m35s
[04:12:31.903] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:48:0)
[04:12:31.922] Build status: STARTED
[04:12:31.922] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:49:0) after 5m45s
[04:18:16.954] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:49:0)
[04:18:16.972] Build status: STARTED
[04:18:16.972] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:50:0) after 5m54s
[04:24:11.096] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:50:0)
[04:24:11.140] Build status: STARTED
[04:24:11.140] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:51:0) after 1m33s
[04:25:44.536] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:51:0)
[04:25:44.569] Build status: STARTED
[04:25:44.569] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:52:0) after 2m12s
[04:27:56.594] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:52:0)
[04:27:56.612] Build status: STARTED
[04:27:56.612] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:53:0) after 2m40s
[04:30:36.634] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:53:0)
[04:30:36.654] Build status: STARTED
[04:30:36.654] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:54:0) after 1m57s
[04:32:33.768] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:54:0)
[04:32:33.800] Build status: STARTED
[04:32:33.800] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:55:0) after 5m20s
[04:37:54.270] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:55:0)
[04:37:54.295] Build status: STARTED
[04:37:54.295] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923315713144733120:56:0) after 9m8s
[04:44:31.926] Received PubSub notification, asking Buildbucket for the build status
[04:44:31.951] Build:
{
"id": "8757885651080633617",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 21402,
"createdBy": "project:chromium",
"createTime": "2024-01-25T23:46:50.749691976Z",
"startTime": "2024-01-25T23:48:43.539354Z",
"endTime": "2024-01-26T04:44:31.036466Z",
"updateTime": "2024-01-26T04:44:31.036466Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d35f57148dde64a4bbc6a12c6c31c3d87e2b7f2a",
"ref": "refs/heads/main"
}
}
}
[04:44:31.951] Invocation finished in 4h57m42.553935873s with status FAILED