[19:35:23.269] New invocation is queued and will start shortly
[19:35:24.339] Starting the invocation (attempt 1)
[19:35:24.376] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/3c0c078a6c7069e505639caec332f41d6e3917b8"
[19:35:24.376] Popped gitiles commit info from properties and tags
[19:35:24.376] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[19:35:24.377] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[19:35:24.377] Buildbucket request:
{
"requestId": "8944350028312880944",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3c0c078a6c7069e505639caec332f41d6e3917b8",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8944350028312880944"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[19:35:25.385] Scheduled build:
{
"id": "8778919966233218081",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 19155,
"createdBy": "project:chromium",
"createTime": "2023-06-07T19:35:24.625246176Z",
"updateTime": "2023-06-07T19:35:24.625246176Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3c0c078a6c7069e505639caec332f41d6e3917b8",
"ref": "refs/heads/main"
}
}
}
[19:35:25.385] Task URL: https://cr-buildbucket.appspot.com/build/8778919966233218081
[19:35:25.385] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:2:0) after 5m50s
[19:40:08.507] Received PubSub notification, asking Buildbucket for the build status
[19:40:08.544] Build status: STARTED
[19:41:15.439] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:2:0)
[19:41:15.460] Build status: STARTED
[19:41:15.460] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:4:0) after 4m40s
[19:45:55.513] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:4:0)
[19:45:55.532] Build status: STARTED
[19:45:55.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:5:0) after 4m41s
[19:50:36.595] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:5:0)
[19:50:36.677] Build status: STARTED
[19:50:36.677] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:6:0) after 6m22s
[19:56:58.811] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:6:0)
[19:56:58.891] Build status: STARTED
[19:56:58.891] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:7:0) after 4m56s
[20:01:54.962] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:7:0)
[20:01:54.997] Build status: STARTED
[20:01:54.997] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:8:0) after 7m28s
[20:09:23.021] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:8:0)
[20:09:23.051] Build status: STARTED
[20:09:23.051] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:9:0) after 3m0s
[20:12:23.637] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:9:0)
[20:12:23.739] Build status: STARTED
[20:12:23.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:10:0) after 5m22s
[20:17:46.105] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:10:0)
[20:17:46.124] Build status: STARTED
[20:17:46.124] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:11:0) after 4m30s
[20:22:16.103] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:11:0)
[20:22:16.177] Build status: STARTED
[20:22:16.177] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:12:0) after 2m45s
[20:25:01.263] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:12:0)
[20:25:01.290] Build status: STARTED
[20:25:01.290] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:13:0) after 2m2s
[20:27:03.568] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:13:0)
[20:27:03.587] Build status: STARTED
[20:27:03.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:14:0) after 5m54s
[20:32:57.637] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:14:0)
[20:32:57.676] Build status: STARTED
[20:32:57.676] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:15:0) after 5m26s
[20:38:23.719] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:15:0)
[20:38:23.739] Build status: STARTED
[20:38:23.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:16:0) after 8m45s
[20:47:08.807] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:16:0)
[20:47:08.840] Build status: STARTED
[20:47:08.840] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:17:0) after 8m47s
[20:55:55.881] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:17:0)
[20:55:55.900] Build status: STARTED
[20:55:55.900] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:18:0) after 5m11s
[21:01:06.923] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:18:0)
[21:01:07.101] Build status: STARTED
[21:01:07.101] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:19:0) after 3m10s
[21:04:17.129] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:19:0)
[21:04:17.182] Build status: STARTED
[21:04:17.182] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:20:0) after 8m50s
[21:13:07.233] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:20:0)
[21:13:07.299] Build status: STARTED
[21:13:07.299] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:21:0) after 9m59s
[21:23:06.341] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:21:0)
[21:23:06.360] Build status: STARTED
[21:23:06.360] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:22:0) after 6m16s
[21:29:22.381] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:22:0)
[21:29:22.400] Build status: STARTED
[21:29:22.400] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:23:0) after 7m51s
[21:37:13.487] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:23:0)
[21:37:13.509] Build status: STARTED
[21:37:13.509] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:24:0) after 2m3s
[21:39:16.697] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:24:0)
[21:39:16.713] Build status: STARTED
[21:39:16.713] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:25:0) after 5m17s
[21:44:33.974] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:25:0)
[21:44:33.990] Build status: STARTED
[21:44:33.990] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:26:0) after 3m30s
[21:48:03.997] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:26:0)
[21:48:04.019] Build status: STARTED
[21:48:04.019] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:27:0) after 2m4s
[21:50:08.048] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:27:0)
[21:50:08.082] Build status: STARTED
[21:50:08.082] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:28:0) after 6m51s
[21:56:59.166] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:28:0)
[21:56:59.188] Build status: STARTED
[21:56:59.188] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:29:0) after 4m2s
[22:01:01.224] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:29:0)
[22:01:01.249] Build status: STARTED
[22:01:01.249] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:30:0) after 5m34s
[22:06:35.473] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:30:0)
[22:06:35.558] Build status: STARTED
[22:06:35.558] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:31:0) after 9m24s
[22:15:59.757] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:31:0)
[22:15:59.836] Build status: STARTED
[22:15:59.836] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:32:0) after 5m24s
[22:21:23.938] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:32:0)
[22:21:24.017] Build status: STARTED
[22:21:24.017] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:33:0) after 2m30s
[22:23:54.062] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:33:0)
[22:23:54.083] Build status: STARTED
[22:23:54.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:34:0) after 3m59s
[22:27:53.228] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:34:0)
[22:27:53.243] Build status: STARTED
[22:27:53.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:35:0) after 6m22s
[22:34:15.274] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:35:0)
[22:34:15.332] Build status: STARTED
[22:34:15.332] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:36:0) after 2m37s
[22:36:52.342] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:36:0)
[22:36:52.363] Build status: STARTED
[22:36:52.363] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:37:0) after 2m15s
[22:39:07.736] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:37:0)
[22:39:07.763] Build status: STARTED
[22:39:07.763] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:38:0) after 6m53s
[22:46:00.902] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:38:0)
[22:46:00.926] Build status: STARTED
[22:46:00.926] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:39:0) after 9m9s
[22:55:10.024] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:39:0)
[22:55:10.044] Build status: STARTED
[22:55:10.044] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:40:0) after 2m13s
[22:57:23.068] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:40:0)
[22:57:23.087] Build status: STARTED
[22:57:23.087] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:41:0) after 2m26s
[22:59:49.301] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:41:0)
[22:59:49.321] Build status: STARTED
[22:59:49.321] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:42:0) after 2m28s
[23:02:17.505] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:42:0)
[23:02:17.525] Build status: STARTED
[23:02:17.525] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:43:0) after 2m48s
[23:05:05.562] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:43:0)
[23:05:05.587] Build status: STARTED
[23:05:05.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:44:0) after 6m52s
[23:11:57.806] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:44:0)
[23:11:57.825] Build status: STARTED
[23:11:57.825] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:45:0) after 2m13s
[23:14:11.446] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:45:0)
[23:14:11.516] Build status: STARTED
[23:14:11.516] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:46:0) after 6m55s
[23:21:07.958] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:46:0)
[23:21:07.978] Build status: STARTED
[23:21:07.978] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:47:0) after 1m29s
[23:22:37.208] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:47:0)
[23:22:37.273] Build status: STARTED
[23:22:37.273] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:48:0) after 8m30s
[23:31:07.285] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:48:0)
[23:31:07.320] Build status: STARTED
[23:31:07.320] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:49:0) after 5m10s
[23:36:17.392] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:49:0)
[23:36:17.438] Build status: STARTED
[23:36:17.438] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:50:0) after 5m56s
[23:42:13.459] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:50:0)
[23:42:13.484] Build status: STARTED
[23:42:13.484] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:51:0) after 7m44s
[23:49:57.507] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:51:0)
[23:49:57.523] Build status: STARTED
[23:49:57.523] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:52:0) after 6m34s
[23:56:31.604] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:52:0)
[23:56:31.621] Build status: STARTED
[23:56:31.621] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:53:0) after 5m6s
[00:01:37.743] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:53:0)
[00:01:37.765] Build status: STARTED
[00:01:37.765] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:54:0) after 6m13s
[00:07:50.800] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:54:0)
[00:07:50.879] Build status: STARTED
[00:07:50.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:55:0) after 9m35s
[00:17:25.895] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:55:0)
[00:17:25.914] Build status: STARTED
[00:17:25.914] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:56:0) after 5m13s
[00:22:39.193] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:56:0)
[00:22:39.209] Build status: STARTED
[00:22:39.209] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:57:0) after 3m22s
[00:26:01.435] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:57:0)
[00:26:01.460] Build status: STARTED
[00:26:01.460] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:58:0) after 1m3s
[00:27:04.647] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:58:0)
[00:27:04.670] Build status: STARTED
[00:27:04.670] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:59:0) after 8m22s
[00:35:26.694] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:59:0)
[00:35:26.713] Build status: STARTED
[00:35:26.714] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:60:0) after 4m1s
[00:39:27.745] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:60:0)
[00:39:27.803] Build status: STARTED
[00:39:27.803] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:61:0) after 5m55s
[00:45:22.856] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:61:0)
[00:45:22.885] Build status: STARTED
[00:45:22.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:62:0) after 8m26s
[00:53:49.000] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:62:0)
[00:53:49.025] Build status: STARTED
[00:53:49.025] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:63:0) after 6m1s
[00:59:50.052] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:63:0)
[00:59:50.071] Build status: STARTED
[00:59:50.071] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:64:0) after 5m23s
[01:05:13.164] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:64:0)
[01:05:13.184] Build status: STARTED
[01:05:13.184] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:65:0) after 5m29s
[01:10:42.226] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:65:0)
[01:10:42.245] Build status: STARTED
[01:10:42.245] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:66:0) after 4m25s
[01:15:07.282] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:66:0)
[01:15:07.302] Build status: STARTED
[01:15:07.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:67:0) after 7m30s
[01:22:37.727] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:67:0)
[01:22:37.757] Build status: STARTED
[01:22:37.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:68:0) after 4m29s
[01:27:06.912] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:68:0)
[01:27:06.934] Build status: STARTED
[01:27:06.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:69:0) after 2m36s
[01:29:43.118] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:69:0)
[01:29:43.171] Build status: STARTED
[01:29:43.171] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:70:0) after 8m30s
[01:38:13.228] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:70:0)
[01:38:13.251] Build status: STARTED
[01:38:13.251] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:71:0) after 7m45s
[01:45:58.298] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:71:0)
[01:45:58.320] Build status: STARTED
[01:45:58.320] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:72:0) after 2m14s
[01:48:12.391] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:72:0)
[01:48:12.469] Build status: STARTED
[01:48:12.469] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:73:0) after 2m2s
[01:50:14.667] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:73:0)
[01:50:14.688] Build status: STARTED
[01:50:14.688] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:74:0) after 8m38s
[01:58:52.726] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:74:0)
[01:58:52.766] Build status: STARTED
[01:58:52.766] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:75:0) after 1m45s
[02:00:38.012] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:75:0)
[02:00:38.030] Build status: STARTED
[02:00:38.030] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:76:0) after 3m8s
[02:03:46.159] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:76:0)
[02:03:46.185] Build status: STARTED
[02:03:46.185] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:77:0) after 7m3s
[02:10:49.253] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:77:0)
[02:10:49.281] Build status: STARTED
[02:10:49.281] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:78:0) after 5m30s
[02:16:19.389] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:78:0)
[02:16:19.411] Build status: STARTED
[02:16:19.411] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:79:0) after 5m7s
[02:21:26.732] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:79:0)
[02:21:26.758] Build status: STARTED
[02:21:26.758] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:80:0) after 2m47s
[02:24:13.875] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:80:0)
[02:24:13.896] Build status: STARTED
[02:24:13.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:81:0) after 3m24s
[02:27:38.024] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:81:0)
[02:27:38.072] Build status: STARTED
[02:27:38.072] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:82:0) after 8m11s
[02:35:49.182] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:82:0)
[02:35:49.200] Build status: STARTED
[02:35:49.200] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:83:0) after 2m31s
[02:38:20.231] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:83:0)
[02:38:20.250] Build status: STARTED
[02:38:20.250] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:84:0) after 2m5s
[02:40:25.289] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:84:0)
[02:40:25.309] Build status: STARTED
[02:40:25.309] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:85:0) after 1m10s
[02:41:35.342] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:85:0)
[02:41:35.363] Build status: STARTED
[02:41:35.363] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:86:0) after 3m59s
[02:45:34.502] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:86:0)
[02:45:34.583] Build status: STARTED
[02:45:34.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:87:0) after 9m32s
[02:55:06.815] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:87:0)
[02:55:07.027] Build status: STARTED
[02:55:07.027] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:88:0) after 4m32s
[02:59:39.048] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:88:0)
[02:59:39.073] Build status: STARTED
[02:59:39.073] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:89:0) after 5m27s
[03:05:06.124] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:89:0)
[03:05:06.151] Build status: STARTED
[03:05:06.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:90:0) after 5m51s
[03:10:57.276] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:90:0)
[03:10:57.358] Build status: STARTED
[03:10:57.358] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:91:0) after 1m49s
[03:12:46.439] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:91:0)
[03:12:46.457] Build status: STARTED
[03:12:46.457] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:92:0) after 3m42s
[03:16:28.564] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:92:0)
[03:16:28.606] Build status: STARTED
[03:16:28.606] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:93:0) after 6m49s
[03:23:17.641] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:93:0)
[03:23:17.690] Build status: STARTED
[03:23:17.690] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:94:0) after 3m22s
[03:26:40.027] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:94:0)
[03:26:40.061] Build status: STARTED
[03:26:40.061] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:95:0) after 3m9s
[03:29:49.085] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:95:0)
[03:29:49.104] Build status: STARTED
[03:29:49.104] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:96:0) after 5m40s
[03:35:29.204] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:96:0)
[03:35:29.227] Build status: STARTED
[03:35:29.227] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:97:0) after 5m2s
[03:40:31.424] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:97:0)
[03:40:31.451] Build status: STARTED
[03:40:31.451] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:98:0) after 1m6s
[03:41:37.573] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:98:0)
[03:41:37.593] Build status: STARTED
[03:41:37.593] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:99:0) after 9m18s
[03:50:55.878] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:99:0)
[03:50:55.912] Build status: STARTED
[03:50:55.912] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:100:0) after 6m42s
[03:57:38.030] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:100:0)
[03:57:38.063] Build status: STARTED
[03:57:38.063] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:101:0) after 5m46s
[04:03:24.095] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:101:0)
[04:03:24.158] Build status: STARTED
[04:03:24.158] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:102:0) after 8m23s
[04:11:47.522] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:102:0)
[04:11:47.546] Build status: STARTED
[04:11:47.546] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:103:0) after 5m58s
[04:17:45.690] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:103:0)
[04:17:45.709] Build status: STARTED
[04:17:45.709] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:104:0) after 4m45s
[04:22:30.888] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:104:0)
[04:22:30.908] Build status: STARTED
[04:22:30.908] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:105:0) after 8m4s
[04:30:35.043] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:105:0)
[04:30:35.069] Build status: STARTED
[04:30:35.069] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:106:0) after 2m58s
[04:33:33.500] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:106:0)
[04:33:33.548] Build status: STARTED
[04:33:33.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:107:0) after 1m44s
[04:35:17.605] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:107:0)
[04:35:17.622] Build status: STARTED
[04:35:17.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:108:0) after 2m6s
[04:37:23.753] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:108:0)
[04:37:23.815] Build status: STARTED
[04:37:23.815] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:109:0) after 2m11s
[04:39:35.024] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:109:0)
[04:39:35.045] Build status: STARTED
[04:39:35.045] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:110:0) after 1m22s
[04:40:57.093] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:110:0)
[04:40:57.117] Build status: STARTED
[04:40:57.117] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:111:0) after 3m0s
[04:43:57.243] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:111:0)
[04:43:57.285] Build status: STARTED
[04:43:57.285] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:112:0) after 1m12s
[04:45:09.529] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:112:0)
[04:45:09.550] Build status: STARTED
[04:45:09.551] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:113:0) after 2m43s
[04:47:52.635] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:113:0)
[04:47:52.689] Build status: STARTED
[04:47:52.689] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:114:0) after 7m51s
[04:55:43.818] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:114:0)
[04:55:43.879] Build status: STARTED
[04:55:43.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:115:0) after 4m57s
[05:00:40.905] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:115:0)
[05:00:40.942] Build status: STARTED
[05:00:40.942] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:116:0) after 2m12s
[05:02:53.268] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:116:0)
[05:02:53.288] Build status: STARTED
[05:02:53.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:117:0) after 8m51s
[05:11:44.476] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:117:0)
[05:11:44.518] Build status: STARTED
[05:11:44.518] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:118:0) after 3m56s
[05:15:40.554] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:118:0)
[05:15:40.578] Build status: STARTED
[05:15:40.579] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:119:0) after 6m0s
[05:21:40.661] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:119:0)
[05:21:40.693] Build status: STARTED
[05:21:40.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:120:0) after 5m6s
[05:26:46.822] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:120:0)
[05:26:46.840] Build status: STARTED
[05:26:46.840] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:121:0) after 3m1s
[05:29:48.046] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:121:0)
[05:29:48.071] Build status: STARTED
[05:29:48.071] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:122:0) after 8m11s
[05:37:59.106] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:122:0)
[05:37:59.126] Build status: STARTED
[05:37:59.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:123:0) after 6m57s
[05:44:56.359] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:123:0)
[05:44:56.402] Build status: STARTED
[05:44:56.402] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:124:0) after 6m25s
[05:51:21.425] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:124:0)
[05:51:21.612] Build status: STARTED
[05:51:21.612] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:125:0) after 7m18s
[05:58:39.642] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:125:0)
[05:58:39.666] Build status: STARTED
[05:58:39.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:126:0) after 3m55s
[06:02:34.692] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:126:0)
[06:02:34.716] Build status: STARTED
[06:02:34.716] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:127:0) after 4m18s
[06:06:52.786] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:127:0)
[06:06:52.805] Build status: STARTED
[06:06:52.805] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:128:0) after 5m8s
[06:12:00.933] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:128:0)
[06:12:00.951] Build status: STARTED
[06:12:00.951] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:129:0) after 2m54s
[06:14:55.031] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:129:0)
[06:14:55.054] Build status: STARTED
[06:14:55.054] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:130:0) after 1m49s
[06:16:44.133] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:130:0)
[06:16:44.164] Build status: STARTED
[06:16:44.164] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:131:0) after 7m21s
[06:24:05.259] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:131:0)
[06:24:05.286] Build status: STARTED
[06:24:05.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:132:0) after 5m0s
[06:29:05.314] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:132:0)
[06:29:05.339] Build status: STARTED
[06:29:05.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:133:0) after 8m43s
[06:37:48.789] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:133:0)
[06:37:48.810] Build status: STARTED
[06:37:48.810] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:134:0) after 3m18s
[06:41:06.835] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:134:0)
[06:41:06.855] Build status: STARTED
[06:41:06.855] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:135:0) after 5m48s
[06:46:55.041] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:135:0)
[06:46:55.062] Build status: STARTED
[06:46:55.062] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:136:0) after 5m40s
[06:52:35.085] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:136:0)
[06:52:35.104] Build status: STARTED
[06:52:35.104] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8944350028312880944:137:0) after 7m32s
[06:54:39.768] Received PubSub notification, asking Buildbucket for the build status
[06:54:39.856] Build:
{
"id": "8778919966233218081",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 19155,
"createdBy": "project:chromium",
"createTime": "2023-06-07T19:35:24.625246176Z",
"startTime": "2023-06-07T19:40:07.742212Z",
"endTime": "2023-06-08T06:54:39.145230899Z",
"updateTime": "2023-06-08T06:54:39.145230899Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3c0c078a6c7069e505639caec332f41d6e3917b8",
"ref": "refs/heads/main"
}
}
}
[06:54:39.856] Invocation finished in 11h19m16.598608602s with status FAILED