[18:52:56.066] New invocation is queued and will start shortly
[18:52:57.132] Starting the invocation (attempt 1)
[18:52:57.158] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/52568efb1654c7f2a196877d7ad3ec31fe6cc97f"
[18:52:57.158] Popped gitiles commit info from properties and tags
[18:52:57.158] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[18:52:57.159] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[18:52:57.159] Buildbucket request:
{
"requestId": "8892531234462501600",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "52568efb1654c7f2a196877d7ad3ec31fe6cc97f",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8892531234462501600"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[18:52:57.627] Scheduled build:
{
"id": "8727101172596401409",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 24009,
"createdBy": "project:chromium",
"createTime": "2024-12-30T18:52:57.223124573Z",
"updateTime": "2024-12-30T18:52:57.223124573Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "52568efb1654c7f2a196877d7ad3ec31fe6cc97f",
"ref": "refs/heads/main"
}
},
"canary": true
}
[18:52:57.627] Task URL: https://cr-buildbucket.appspot.com/build/8727101172596401409
[18:52:57.627] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:2:0) after 3m19s
[18:52:58.645] Received PubSub notification, asking Buildbucket for the build status
[18:52:58.672] Build status: SCHEDULED
[18:54:17.728] Received PubSub notification, asking Buildbucket for the build status
[18:54:17.750] Build status: STARTED
[18:56:16.655] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:2:0)
[18:56:16.712] Build status: STARTED
[18:56:16.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:5:0) after 1m19s
[18:57:35.770] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:5:0)
[18:57:35.790] Build status: STARTED
[18:57:35.790] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:6:0) after 9m20s
[19:06:56.226] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:6:0)
[19:06:56.253] Build status: STARTED
[19:06:56.253] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:7:0) after 8m19s
[19:15:16.146] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:7:0)
[19:15:16.171] Build status: STARTED
[19:15:16.171] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:8:0) after 2m2s
[19:17:18.401] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:8:0)
[19:17:18.464] Build status: STARTED
[19:17:18.464] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:9:0) after 6m15s
[19:23:34.828] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:9:0)
[19:23:34.854] Build status: STARTED
[19:23:34.854] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:10:0) after 1m42s
[19:25:16.945] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:10:0)
[19:25:16.967] Build status: STARTED
[19:25:16.967] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:11:0) after 2m38s
[19:27:54.998] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:11:0)
[19:27:55.021] Build status: STARTED
[19:27:55.021] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:12:0) after 5m42s
[19:33:37.684] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:12:0)
[19:33:37.702] Build status: STARTED
[19:33:37.702] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:13:0) after 2m4s
[19:35:41.771] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:13:0)
[19:35:41.834] Build status: STARTED
[19:35:41.834] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:14:0) after 4m7s
[19:39:49.052] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:14:0)
[19:39:49.073] Build status: STARTED
[19:39:49.073] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:15:0) after 7m38s
[19:47:27.586] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:15:0)
[19:47:27.608] Build status: STARTED
[19:47:27.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:16:0) after 2m5s
[19:49:32.613] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:16:0)
[19:49:32.629] Build status: STARTED
[19:49:32.629] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:17:0) after 5m9s
[19:54:41.650] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:17:0)
[19:54:41.666] Build status: STARTED
[19:54:41.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:18:0) after 1m52s
[19:56:33.828] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:18:0)
[19:56:33.855] Build status: STARTED
[19:56:33.855] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:19:0) after 7m4s
[20:03:38.043] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:19:0)
[20:03:38.060] Build status: STARTED
[20:03:38.060] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:20:0) after 2m43s
[20:06:21.328] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:20:0)
[20:06:21.349] Build status: STARTED
[20:06:21.349] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:21:0) after 1m35s
[20:07:56.462] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:21:0)
[20:07:56.481] Build status: STARTED
[20:07:56.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:22:0) after 2m41s
[20:10:37.509] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:22:0)
[20:10:37.524] Build status: STARTED
[20:10:37.524] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:23:0) after 2m6s
[20:12:43.549] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:23:0)
[20:12:43.565] Build status: STARTED
[20:12:43.565] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:24:0) after 1m39s
[20:14:22.654] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:24:0)
[20:14:22.681] Build status: STARTED
[20:14:22.681] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:25:0) after 3m48s
[20:18:10.708] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:25:0)
[20:18:10.887] Build status: STARTED
[20:18:10.887] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:26:0) after 7m51s
[20:26:01.906] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:26:0)
[20:26:02.091] Build status: STARTED
[20:26:02.091] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:27:0) after 5m39s
[20:31:41.096] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:27:0)
[20:31:41.138] Build status: STARTED
[20:31:41.138] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:28:0) after 7m22s
[20:39:03.239] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:28:0)
[20:39:03.262] Build status: STARTED
[20:39:03.262] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:29:0) after 7m24s
[20:46:27.285] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:29:0)
[20:46:27.358] Build status: STARTED
[20:46:27.358] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:30:0) after 7m9s
[20:53:36.447] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:30:0)
[20:53:36.535] Build status: STARTED
[20:53:36.535] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:31:0) after 8m36s
[21:02:12.617] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:31:0)
[21:02:12.636] Build status: STARTED
[21:02:12.637] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:32:0) after 7m41s
[21:09:53.782] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:32:0)
[21:09:53.804] Build status: STARTED
[21:09:53.804] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:33:0) after 5m15s
[21:15:08.832] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:33:0)
[21:15:08.978] Build status: STARTED
[21:15:08.978] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:34:0) after 9m9s
[21:24:18.004] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:34:0)
[21:24:18.022] Build status: STARTED
[21:24:18.022] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:35:0) after 2m47s
[21:27:05.065] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:35:0)
[21:27:05.080] Build status: STARTED
[21:27:05.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:36:0) after 4m13s
[21:31:18.106] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:36:0)
[21:31:18.194] Build status: STARTED
[21:31:18.194] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:37:0) after 5m33s
[21:36:51.272] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:37:0)
[21:36:51.290] Build status: STARTED
[21:36:51.290] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:38:0) after 9m13s
[21:46:04.312] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:38:0)
[21:46:04.345] Build status: STARTED
[21:46:04.345] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:39:0) after 9m53s
[21:55:57.366] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:39:0)
[21:55:57.380] Build status: STARTED
[21:55:57.380] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:40:0) after 9m32s
[22:05:29.403] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:40:0)
[22:05:29.420] Build status: STARTED
[22:05:29.420] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:41:0) after 9m57s
[22:15:26.457] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:41:0)
[22:15:26.472] Build status: STARTED
[22:15:26.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:42:0) after 6m43s
[22:22:09.503] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:42:0)
[22:22:09.531] Build status: STARTED
[22:22:09.531] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:43:0) after 9m16s
[22:31:25.560] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:43:0)
[22:31:25.585] Build status: STARTED
[22:31:25.585] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:44:0) after 5m25s
[22:36:50.737] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:44:0)
[22:36:50.761] Build status: STARTED
[22:36:50.761] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:45:0) after 4m51s
[22:41:41.801] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:45:0)
[22:41:41.816] Build status: STARTED
[22:41:41.816] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:46:0) after 6m21s
[22:48:02.845] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:46:0)
[22:48:02.860] Build status: STARTED
[22:48:02.860] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:47:0) after 1m54s
[22:49:56.937] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:47:0)
[22:49:56.959] Build status: STARTED
[22:49:56.959] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:48:0) after 3m39s
[22:53:35.978] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:48:0)
[22:53:35.992] Build status: STARTED
[22:53:35.992] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:49:0) after 9m28s
[23:03:04.015] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:49:0)
[23:03:04.047] Build status: STARTED
[23:03:04.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:50:0) after 1m19s
[23:04:23.074] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:50:0)
[23:04:23.100] Build status: STARTED
[23:04:23.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:51:0) after 9m8s
[23:13:31.120] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:51:0)
[23:13:31.139] Build status: STARTED
[23:13:31.139] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:52:0) after 7m52s
[23:21:23.157] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:52:0)
[23:21:23.173] Build status: STARTED
[23:21:23.174] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:53:0) after 9m2s
[23:30:25.199] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:53:0)
[23:30:25.256] Build status: STARTED
[23:30:25.256] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:54:0) after 2m11s
[23:32:36.281] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:54:0)
[23:32:36.296] Build status: STARTED
[23:32:36.296] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:55:0) after 7m44s
[23:40:20.566] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:55:0)
[23:40:20.587] Build status: STARTED
[23:40:20.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:56:0) after 7m40s
[23:48:00.715] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:56:0)
[23:48:00.731] Build status: STARTED
[23:48:00.731] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:57:0) after 6m7s
[23:54:07.779] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:57:0)
[23:54:07.811] Build status: STARTED
[23:54:07.811] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:58:0) after 6m59s
[00:01:06.852] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:58:0)
[00:01:06.876] Build status: STARTED
[00:01:06.876] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:59:0) after 1m20s
[00:02:26.927] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:59:0)
[00:02:26.943] Build status: STARTED
[00:02:26.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:60:0) after 3m25s
[00:05:51.969] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:60:0)
[00:05:52.145] Build status: STARTED
[00:05:52.145] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:61:0) after 9m24s
[00:15:16.217] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:61:0)
[00:15:16.235] Build status: STARTED
[00:15:16.235] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:62:0) after 2m48s
[00:18:04.301] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:62:0)
[00:18:04.332] Build status: STARTED
[00:18:04.332] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:63:0) after 2m31s
[00:20:35.427] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:63:0)
[00:20:35.446] Build status: STARTED
[00:20:35.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:64:0) after 2m23s
[00:22:58.478] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:64:0)
[00:22:58.528] Build status: STARTED
[00:22:58.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:65:0) after 8m30s
[00:31:28.589] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:65:0)
[00:31:28.613] Build status: STARTED
[00:31:28.614] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:66:0) after 6m48s
[00:38:16.979] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:66:0)
[00:38:16.995] Build status: STARTED
[00:38:16.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:67:0) after 3m39s
[00:41:56.022] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:67:0)
[00:41:56.040] Build status: STARTED
[00:41:56.040] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:68:0) after 6m0s
[00:47:56.211] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:68:0)
[00:47:56.232] Build status: STARTED
[00:47:56.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:69:0) after 6m15s
[00:54:11.462] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:69:0)
[00:54:11.488] Build status: STARTED
[00:54:11.488] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:70:0) after 6m1s
[01:00:12.917] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:70:0)
[01:00:12.951] Build status: STARTED
[01:00:12.951] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:71:0) after 5m8s
[01:05:21.295] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:71:0)
[01:05:21.315] Build status: STARTED
[01:05:21.315] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:72:0) after 2m18s
[01:07:39.341] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:72:0)
[01:07:39.367] Build status: STARTED
[01:07:39.367] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:73:0) after 8m28s
[01:16:07.409] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:73:0)
[01:16:07.429] Build status: STARTED
[01:16:07.429] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:74:0) after 4m21s
[01:20:28.774] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:74:0)
[01:20:28.789] Build status: STARTED
[01:20:28.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:75:0) after 7m42s
[01:28:10.839] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:75:0)
[01:28:10.859] Build status: STARTED
[01:28:10.859] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:76:0) after 1m14s
[01:29:24.865] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:76:0)
[01:29:24.881] Build status: STARTED
[01:29:24.881] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:77:0) after 3m36s
[01:33:00.904] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:77:0)
[01:33:00.932] Build status: STARTED
[01:33:00.932] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:78:0) after 8m13s
[01:41:14.092] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:78:0)
[01:41:14.116] Build status: STARTED
[01:41:14.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:79:0) after 7m36s
[01:48:50.216] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:79:0)
[01:48:50.288] Build status: STARTED
[01:48:50.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:80:0) after 9m23s
[01:58:13.360] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:80:0)
[01:58:13.378] Build status: STARTED
[01:58:13.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:81:0) after 3m35s
[02:01:48.414] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:81:0)
[02:01:48.437] Build status: STARTED
[02:01:48.437] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:82:0) after 5m21s
[02:07:09.516] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:82:0)
[02:07:09.549] Build status: STARTED
[02:07:09.549] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:83:0) after 8m39s
[02:15:48.575] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:83:0)
[02:15:48.712] Build status: STARTED
[02:15:48.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:84:0) after 2m26s
[02:18:14.744] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:84:0)
[02:18:14.761] Build status: STARTED
[02:18:14.761] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:85:0) after 9m47s
[02:28:01.801] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:85:0)
[02:28:01.817] Build status: STARTED
[02:28:01.817] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:86:0) after 9m17s
[02:37:18.879] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:86:0)
[02:37:18.950] Build status: STARTED
[02:37:18.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:87:0) after 3m51s
[02:41:10.098] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:87:0)
[02:41:10.116] Build status: STARTED
[02:41:10.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:88:0) after 9m40s
[02:50:50.144] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:88:0)
[02:50:50.179] Build status: STARTED
[02:50:50.179] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:89:0) after 1m9s
[02:51:59.302] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:89:0)
[02:51:59.319] Build status: STARTED
[02:51:59.319] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:90:0) after 3m9s
[02:55:08.445] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:90:0)
[02:55:08.459] Build status: STARTED
[02:55:08.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:91:0) after 1m46s
[02:56:54.998] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:91:0)
[02:56:55.018] Build status: STARTED
[02:56:55.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:92:0) after 3m17s
[03:00:12.045] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:92:0)
[03:00:12.068] Build status: STARTED
[03:00:12.068] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:93:0) after 7m25s
[03:07:37.067] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:93:0)
[03:07:37.157] Build status: STARTED
[03:07:37.157] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:94:0) after 7m44s
[03:15:21.269] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:94:0)
[03:15:21.334] Build status: STARTED
[03:15:21.334] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:95:0) after 6m39s
[03:22:00.707] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:95:0)
[03:22:00.722] Build status: STARTED
[03:22:00.722] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:96:0) after 3m17s
[03:25:17.804] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:96:0)
[03:25:17.824] Build status: STARTED
[03:25:17.824] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:97:0) after 4m0s
[03:29:17.852] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:97:0)
[03:29:17.869] Build status: STARTED
[03:29:17.869] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:98:0) after 3m3s
[03:32:20.899] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:98:0)
[03:32:20.927] Build status: STARTED
[03:32:20.927] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:99:0) after 5m18s
[03:37:38.966] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:99:0)
[03:37:39.012] Build status: STARTED
[03:37:39.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:100:0) after 1m21s
[03:39:00.030] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:100:0)
[03:39:00.054] Build status: STARTED
[03:39:00.054] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:101:0) after 6m41s
[03:45:41.219] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:101:0)
[03:45:41.263] Build status: STARTED
[03:45:41.263] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:102:0) after 8m14s
[03:53:55.304] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:102:0)
[03:53:55.321] Build status: STARTED
[03:53:55.321] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:103:0) after 4m55s
[03:58:50.331] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:103:0)
[03:58:50.379] Build status: STARTED
[03:58:50.379] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:104:0) after 6m43s
[04:05:33.541] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:104:0)
[04:05:33.559] Build status: STARTED
[04:05:33.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:105:0) after 4m5s
[04:09:38.683] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:105:0)
[04:09:38.705] Build status: STARTED
[04:09:38.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:106:0) after 2m25s
[04:12:03.746] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:106:0)
[04:12:03.765] Build status: STARTED
[04:12:03.765] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:107:0) after 6m20s
[04:18:23.792] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:107:0)
[04:18:23.807] Build status: STARTED
[04:18:23.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:108:0) after 8m34s
[04:26:57.812] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:108:0)
[04:26:57.837] Build status: STARTED
[04:26:57.837] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:109:0) after 1m19s
[04:28:16.874] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:109:0)
[04:28:16.903] Build status: STARTED
[04:28:16.903] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:110:0) after 4m14s
[04:32:30.926] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:110:0)
[04:32:30.943] Build status: STARTED
[04:32:30.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:111:0) after 4m30s
[04:37:00.972] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:111:0)
[04:37:01.001] Build status: STARTED
[04:37:01.001] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:112:0) after 3m5s
[04:40:06.001] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:112:0)
[04:40:06.018] Build status: STARTED
[04:40:06.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:113:0) after 8m43s
[04:48:49.138] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:113:0)
[04:48:49.163] Build status: STARTED
[04:48:49.163] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:114:0) after 4m37s
[04:53:26.185] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:114:0)
[04:53:26.203] Build status: STARTED
[04:53:26.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:115:0) after 3m39s
[04:57:05.223] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:115:0)
[04:57:05.237] Build status: STARTED
[04:57:05.237] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:116:0) after 2m47s
[04:59:52.265] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:116:0)
[04:59:52.333] Build status: STARTED
[04:59:52.333] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:117:0) after 8m26s
[05:08:18.526] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:117:0)
[05:08:18.552] Build status: STARTED
[05:08:18.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:118:0) after 5m59s
[05:14:17.585] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:118:0)
[05:14:17.600] Build status: STARTED
[05:14:17.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:119:0) after 6m33s
[05:20:50.727] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:119:0)
[05:20:50.764] Build status: STARTED
[05:20:50.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:120:0) after 3m29s
[05:24:19.784] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:120:0)
[05:24:19.806] Build status: STARTED
[05:24:19.806] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:121:0) after 3m34s
[05:27:54.069] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:121:0)
[05:27:54.089] Build status: STARTED
[05:27:54.089] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:122:0) after 7m52s
[05:35:46.118] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:122:0)
[05:35:46.204] Build status: STARTED
[05:35:46.204] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:123:0) after 4m38s
[05:40:24.236] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:123:0)
[05:40:24.263] Build status: STARTED
[05:40:24.263] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:124:0) after 9m39s
[05:50:03.345] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:124:0)
[05:50:03.361] Build status: STARTED
[05:50:03.361] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:125:0) after 7m0s
[05:57:03.387] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:125:0)
[05:57:03.413] Build status: STARTED
[05:57:03.413] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:126:0) after 7m12s
[06:04:15.506] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:126:0)
[06:04:15.528] Build status: STARTED
[06:04:15.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:127:0) after 5m37s
[06:09:52.555] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:127:0)
[06:09:52.610] Build status: STARTED
[06:09:52.610] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:128:0) after 9m14s
[06:19:06.638] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:128:0)
[06:19:06.681] Build status: STARTED
[06:19:06.681] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:129:0) after 3m59s
[06:23:05.702] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:129:0)
[06:23:05.718] Build status: STARTED
[06:23:05.718] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:130:0) after 5m54s
[06:28:59.747] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:130:0)
[06:28:59.763] Build status: STARTED
[06:28:59.763] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:131:0) after 9m12s
[06:38:11.908] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:131:0)
[06:38:11.924] Build status: STARTED
[06:38:11.924] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:132:0) after 8m2s
[06:46:13.958] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:132:0)
[06:46:14.041] Build status: STARTED
[06:46:14.041] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:133:0) after 8m26s
[06:54:40.068] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:133:0)
[06:54:40.083] Build status: STARTED
[06:54:40.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:134:0) after 8m31s
[07:03:11.155] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:134:0)
[07:03:11.177] Build status: STARTED
[07:03:11.177] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:135:0) after 2m37s
[07:05:48.298] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:135:0)
[07:05:48.314] Build status: STARTED
[07:05:48.315] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:136:0) after 7m51s
[07:13:39.336] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:136:0)
[07:13:39.353] Build status: STARTED
[07:13:39.353] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:137:0) after 2m1s
[07:15:40.453] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:137:0)
[07:15:40.480] Build status: STARTED
[07:15:40.480] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:138:0) after 7m59s
[07:23:39.597] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:138:0)
[07:23:39.614] Build status: STARTED
[07:23:39.614] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:139:0) after 1m37s
[07:25:16.645] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:139:0)
[07:25:16.671] Build status: STARTED
[07:25:16.671] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:140:0) after 3m3s
[07:28:19.697] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:140:0)
[07:28:19.718] Build status: STARTED
[07:28:19.718] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:141:0) after 6m29s
[07:34:48.742] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:141:0)
[07:34:48.843] Build status: STARTED
[07:34:48.843] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:142:0) after 1m33s
[07:36:21.873] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:142:0)
[07:36:21.932] Build status: STARTED
[07:36:21.932] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:143:0) after 3m45s
[07:40:07.014] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:143:0)
[07:40:07.029] Build status: STARTED
[07:40:07.029] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:144:0) after 8m56s
[07:49:03.069] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:144:0)
[07:49:03.089] Build status: STARTED
[07:49:03.089] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:145:0) after 9m48s
[07:58:51.119] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:145:0)
[07:58:51.137] Build status: STARTED
[07:58:51.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:146:0) after 5m23s
[08:04:14.219] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:146:0)
[08:04:14.237] Build status: STARTED
[08:04:14.237] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:147:0) after 3m32s
[08:07:46.315] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:147:0)
[08:07:46.390] Build status: STARTED
[08:07:46.390] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:148:0) after 4m23s
[08:12:09.418] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:148:0)
[08:12:09.436] Build status: STARTED
[08:12:09.436] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:149:0) after 9m39s
[08:21:48.460] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:149:0)
[08:21:48.477] Build status: STARTED
[08:21:48.477] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:150:0) after 4m8s
[08:25:56.502] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:150:0)
[08:25:56.598] Build status: STARTED
[08:25:56.598] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:151:0) after 1m10s
[08:27:06.721] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:151:0)
[08:27:06.741] Build status: STARTED
[08:27:06.741] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:152:0) after 5m25s
[08:32:31.769] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:152:0)
[08:32:31.787] Build status: STARTED
[08:32:31.787] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:153:0) after 9m3s
[08:41:34.825] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:153:0)
[08:41:34.842] Build status: STARTED
[08:41:34.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:154:0) after 7m29s
[08:49:03.984] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:154:0)
[08:49:04.001] Build status: STARTED
[08:49:04.001] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:155:0) after 6m5s
[08:55:09.028] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:155:0)
[08:55:09.047] Build status: STARTED
[08:55:09.047] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8892531234462501600:156:0) after 4m27s
[08:55:15.473] Received PubSub notification, asking Buildbucket for the build status
[08:55:15.488] Build:
{
"id": "8727101172596401409",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 24009,
"createdBy": "project:chromium",
"createTime": "2024-12-30T18:52:57.223124573Z",
"startTime": "2024-12-30T18:54:17.534661357Z",
"endTime": "2024-12-31T08:55:15.076023012Z",
"updateTime": "2024-12-31T08:55:15.076023012Z",
"status": "INFRA_FAILURE",
"statusDetails": {
"timeout": {
}
},
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "52568efb1654c7f2a196877d7ad3ec31fe6cc97f",
"ref": "refs/heads/main"
}
},
"canary": true
}
[08:55:15.488] Invocation finished in 14h2m19.428204207s with status FAILED