[09:02:25.510] New invocation is queued and will start shortly
[09:02:26.585] Starting the invocation (attempt 1)
[09:02:26.628] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/423951cfe5a27c283b49eb27bab4db80652b3ac4"
[09:02:26.628] Popped gitiles commit info from properties and tags
[09:02:26.628] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[09:02:26.628] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[09:02:26.629] Buildbucket request:
{
"requestId": "8946111193242785072",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "423951cfe5a27c283b49eb27bab4db80652b3ac4",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8946111193242785072"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[09:02:27.232] Scheduled build:
{
"id": "8780681131323678673",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 18999,
"createdBy": "project:chromium",
"createTime": "2023-05-19T09:02:26.711981167Z",
"updateTime": "2023-05-19T09:02:26.711981167Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "423951cfe5a27c283b49eb27bab4db80652b3ac4",
"ref": "refs/heads/main"
}
}
}
[09:02:27.232] Task URL: https://cr-buildbucket.appspot.com/build/8780681131323678673
[09:02:27.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:2:0) after 9m30s
[09:06:21.788] Received PubSub notification, asking Buildbucket for the build status
[09:06:21.812] Build status: STARTED
[09:11:57.340] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:2:0)
[09:11:57.361] Build status: STARTED
[09:11:57.361] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:4:0) after 6m43s
[09:18:40.495] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:4:0)
[09:18:40.528] Build status: STARTED
[09:18:40.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:5:0) after 1m27s
[09:20:07.601] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:5:0)
[09:20:07.627] Build status: STARTED
[09:20:07.627] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:6:0) after 8m14s
[09:28:21.656] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:6:0)
[09:28:21.673] Build status: STARTED
[09:28:21.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:7:0) after 9m14s
[09:37:35.740] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:7:0)
[09:37:35.757] Build status: STARTED
[09:37:35.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:8:0) after 6m19s
[09:43:54.792] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:8:0)
[09:43:54.812] Build status: STARTED
[09:43:54.812] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:9:0) after 7m36s
[09:51:30.835] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:9:0)
[09:51:30.920] Build status: STARTED
[09:51:30.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:10:0) after 8m24s
[09:59:55.037] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:10:0)
[09:59:55.054] Build status: STARTED
[09:59:55.054] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:11:0) after 5m3s
[10:04:58.083] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:11:0)
[10:04:58.100] Build status: STARTED
[10:04:58.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:12:0) after 6m45s
[10:11:43.128] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:12:0)
[10:11:43.148] Build status: STARTED
[10:11:43.148] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:13:0) after 3m37s
[10:15:20.208] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:13:0)
[10:15:20.231] Build status: STARTED
[10:15:20.231] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:14:0) after 9m24s
[10:24:44.226] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:14:0)
[10:24:44.246] Build status: STARTED
[10:24:44.246] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:15:0) after 2m16s
[10:27:00.275] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:15:0)
[10:27:00.311] Build status: STARTED
[10:27:00.311] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:16:0) after 3m21s
[10:30:21.555] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:16:0)
[10:30:21.587] Build status: STARTED
[10:30:21.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:17:0) after 1m4s
[10:31:25.606] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:17:0)
[10:31:25.630] Build status: STARTED
[10:31:25.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:18:0) after 6m32s
[10:37:57.648] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:18:0)
[10:37:57.668] Build status: STARTED
[10:37:57.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:19:0) after 7m34s
[10:45:31.787] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:19:0)
[10:45:31.804] Build status: STARTED
[10:45:31.804] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:20:0) after 4m41s
[10:50:12.892] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:20:0)
[10:50:12.916] Build status: STARTED
[10:50:12.916] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:21:0) after 2m38s
[10:52:51.082] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:21:0)
[10:52:51.100] Build status: STARTED
[10:52:51.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:22:0) after 2m44s
[10:55:35.242] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:22:0)
[10:55:35.260] Build status: STARTED
[10:55:35.260] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:23:0) after 5m46s
[11:01:21.379] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:23:0)
[11:01:21.399] Build status: STARTED
[11:01:21.399] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:24:0) after 2m27s
[11:03:48.473] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:24:0)
[11:03:48.499] Build status: STARTED
[11:03:48.499] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:25:0) after 9m55s
[11:13:43.738] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:25:0)
[11:13:43.757] Build status: STARTED
[11:13:43.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:26:0) after 9m58s
[11:23:41.944] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:26:0)
[11:23:41.968] Build status: STARTED
[11:23:41.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:27:0) after 8m58s
[11:32:39.987] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:27:0)
[11:32:40.005] Build status: STARTED
[11:32:40.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:28:0) after 5m12s
[11:37:52.026] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:28:0)
[11:37:52.052] Build status: STARTED
[11:37:52.052] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:29:0) after 3m4s
[11:40:56.126] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:29:0)
[11:40:56.147] Build status: STARTED
[11:40:56.147] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:30:0) after 5m18s
[11:46:14.175] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:30:0)
[11:46:14.197] Build status: STARTED
[11:46:14.197] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:31:0) after 6m21s
[11:52:35.224] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:31:0)
[11:52:35.263] Build status: STARTED
[11:52:35.263] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:32:0) after 3m47s
[11:56:22.279] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:32:0)
[11:56:22.302] Build status: STARTED
[11:56:22.302] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:33:0) after 5m31s
[12:01:53.423] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:33:0)
[12:01:53.442] Build status: STARTED
[12:01:53.442] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:34:0) after 9m45s
[12:11:38.992] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:34:0)
[12:11:39.012] Build status: STARTED
[12:11:39.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:35:0) after 6m22s
[12:18:01.041] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:35:0)
[12:18:01.058] Build status: STARTED
[12:18:01.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:36:0) after 2m9s
[12:20:10.090] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:36:0)
[12:20:10.108] Build status: STARTED
[12:20:10.108] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:37:0) after 8m20s
[12:28:30.132] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:37:0)
[12:28:30.159] Build status: STARTED
[12:28:30.159] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:38:0) after 9m55s
[12:38:25.193] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:38:0)
[12:38:25.230] Build status: STARTED
[12:38:25.231] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:39:0) after 3m30s
[12:41:55.251] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:39:0)
[12:41:55.271] Build status: STARTED
[12:41:55.271] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:40:0) after 8m17s
[12:50:12.340] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:40:0)
[12:50:12.372] Build status: STARTED
[12:50:12.372] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:41:0) after 8m55s
[12:59:07.468] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:41:0)
[12:59:07.487] Build status: STARTED
[12:59:07.487] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:42:0) after 3m7s
[13:02:14.521] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:42:0)
[13:02:14.548] Build status: STARTED
[13:02:14.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:43:0) after 3m9s
[13:05:23.588] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:43:0)
[13:05:23.618] Build status: STARTED
[13:05:23.618] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:44:0) after 8m49s
[13:14:12.690] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:44:0)
[13:14:12.709] Build status: STARTED
[13:14:12.710] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:45:0) after 1m25s
[13:15:37.736] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:45:0)
[13:15:37.781] Build status: STARTED
[13:15:37.781] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:46:0) after 8m45s
[13:24:22.915] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:46:0)
[13:24:22.937] Build status: STARTED
[13:24:22.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:47:0) after 3m35s
[13:27:58.136] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:47:0)
[13:27:58.155] Build status: STARTED
[13:27:58.155] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:48:0) after 5m30s
[13:33:28.292] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:48:0)
[13:33:28.314] Build status: STARTED
[13:33:28.314] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:49:0) after 1m57s
[13:35:25.345] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:49:0)
[13:35:25.378] Build status: STARTED
[13:35:25.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:50:0) after 7m16s
[13:42:41.432] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:50:0)
[13:42:41.460] Build status: STARTED
[13:42:41.460] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:51:0) after 8m18s
[13:50:59.586] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:51:0)
[13:50:59.610] Build status: STARTED
[13:50:59.610] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:52:0) after 6m51s
[13:57:50.733] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:52:0)
[13:57:50.752] Build status: STARTED
[13:57:50.752] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:53:0) after 1m42s
[13:59:32.870] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:53:0)
[13:59:32.911] Build status: STARTED
[13:59:32.912] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:54:0) after 9m12s
[14:08:45.225] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:54:0)
[14:08:45.265] Build status: STARTED
[14:08:45.265] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:55:0) after 5m7s
[14:13:52.255] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:55:0)
[14:13:52.404] Build status: STARTED
[14:13:52.404] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:56:0) after 3m51s
[14:17:44.285] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:56:0)
[14:17:44.303] Build status: STARTED
[14:17:44.303] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:57:0) after 5m16s
[14:23:00.416] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:57:0)
[14:23:00.483] Build status: STARTED
[14:23:00.484] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:58:0) after 1m48s
[14:24:48.704] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:58:0)
[14:24:48.727] Build status: STARTED
[14:24:48.727] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:59:0) after 9m40s
[14:34:28.876] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:59:0)
[14:34:28.899] Build status: STARTED
[14:34:28.899] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:60:0) after 2m28s
[14:36:56.939] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:60:0)
[14:36:56.968] Build status: STARTED
[14:36:56.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:61:0) after 3m1s
[14:39:58.119] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:61:0)
[14:39:58.140] Build status: STARTED
[14:39:58.140] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:62:0) after 5m27s
[14:45:25.162] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:62:0)
[14:45:25.187] Build status: STARTED
[14:45:25.188] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:63:0) after 7m16s
[14:52:41.504] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:63:0)
[14:52:41.552] Build status: STARTED
[14:52:41.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:64:0) after 6m27s
[14:59:08.880] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:64:0)
[14:59:08.899] Build status: STARTED
[14:59:08.899] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:65:0) after 6m48s
[15:05:56.987] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:65:0)
[15:05:57.009] Build status: STARTED
[15:05:57.009] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:66:0) after 7m56s
[15:13:53.264] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:66:0)
[15:13:53.287] Build status: STARTED
[15:13:53.287] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:67:0) after 5m30s
[15:19:23.394] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:67:0)
[15:19:23.415] Build status: STARTED
[15:19:23.415] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:68:0) after 1m21s
[15:20:44.729] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:68:0)
[15:20:44.758] Build status: STARTED
[15:20:44.758] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:69:0) after 4m6s
[15:24:50.815] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:69:0)
[15:24:50.837] Build status: STARTED
[15:24:50.837] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:70:0) after 5m51s
[15:30:41.924] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:70:0)
[15:30:41.950] Build status: STARTED
[15:30:41.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:71:0) after 4m58s
[15:35:40.292] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:71:0)
[15:35:40.364] Build status: STARTED
[15:35:40.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:72:0) after 4m18s
[15:39:58.725] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:72:0)
[15:39:58.746] Build status: STARTED
[15:39:58.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:73:0) after 7m48s
[15:47:46.768] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:73:0)
[15:47:46.785] Build status: STARTED
[15:47:46.785] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:74:0) after 6m49s
[15:54:36.085] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:74:0)
[15:54:36.105] Build status: STARTED
[15:54:36.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:75:0) after 1m2s
[15:55:38.375] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:75:0)
[15:55:38.397] Build status: STARTED
[15:55:38.397] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:76:0) after 6m34s
[16:02:12.556] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:76:0)
[16:02:12.575] Build status: STARTED
[16:02:12.575] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:77:0) after 5m35s
[16:07:47.609] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:77:0)
[16:07:47.632] Build status: STARTED
[16:07:47.632] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:78:0) after 4m48s
[16:12:35.858] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:78:0)
[16:12:35.892] Build status: STARTED
[16:12:35.892] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:79:0) after 6m58s
[16:19:34.250] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:79:0)
[16:19:34.286] Build status: STARTED
[16:19:34.286] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:80:0) after 1m4s
[16:20:38.309] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:80:0)
[16:20:38.330] Build status: STARTED
[16:20:38.330] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:81:0) after 6m14s
[16:26:52.597] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:81:0)
[16:26:52.617] Build status: STARTED
[16:26:52.617] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:82:0) after 5m35s
[16:32:27.659] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:82:0)
[16:32:27.700] Build status: STARTED
[16:32:27.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:83:0) after 8m42s
[16:41:09.843] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:83:0)
[16:41:09.868] Build status: STARTED
[16:41:09.868] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:84:0) after 7m20s
[16:48:29.894] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:84:0)
[16:48:29.914] Build status: STARTED
[16:48:29.915] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:85:0) after 8m19s
[16:56:49.775] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:85:0)
[16:56:49.807] Build status: STARTED
[16:56:49.807] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:86:0) after 5m28s
[17:02:18.528] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:86:0)
[17:02:18.556] Build status: STARTED
[17:02:18.556] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:87:0) after 8m54s
[17:11:12.826] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:87:0)
[17:11:12.846] Build status: STARTED
[17:11:12.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:88:0) after 8m40s
[17:19:53.154] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:88:0)
[17:19:53.173] Build status: STARTED
[17:19:53.173] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:89:0) after 5m22s
[17:25:15.332] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:89:0)
[17:25:15.353] Build status: STARTED
[17:25:15.353] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:90:0) after 4m14s
[17:29:29.671] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:90:0)
[17:29:29.693] Build status: STARTED
[17:29:29.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:91:0) after 5m59s
[17:35:28.876] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:91:0)
[17:35:28.899] Build status: STARTED
[17:35:28.899] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:92:0) after 1m48s
[17:37:17.038] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:92:0)
[17:37:17.057] Build status: STARTED
[17:37:17.057] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:93:0) after 8m36s
[17:45:53.185] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:93:0)
[17:45:53.217] Build status: STARTED
[17:45:53.217] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:94:0) after 5m1s
[17:50:54.245] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:94:0)
[17:50:54.282] Build status: STARTED
[17:50:54.282] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:95:0) after 2m44s
[17:53:38.308] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:95:0)
[17:53:38.330] Build status: STARTED
[17:53:38.330] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:96:0) after 1m16s
[17:54:54.480] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:96:0)
[17:54:54.500] Build status: STARTED
[17:54:54.500] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:97:0) after 3m24s
[17:58:18.615] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:97:0)
[17:58:18.639] Build status: STARTED
[17:58:18.639] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:98:0) after 7m43s
[18:06:01.789] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:98:0)
[18:06:01.811] Build status: STARTED
[18:06:01.811] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:99:0) after 3m59s
[18:10:00.988] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:99:0)
[18:10:01.005] Build status: STARTED
[18:10:01.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:100:0) after 5m14s
[18:15:15.813] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:100:0)
[18:15:15.841] Build status: STARTED
[18:15:15.841] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:101:0) after 3m5s
[18:18:20.865] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:101:0)
[18:18:20.917] Build status: STARTED
[18:18:20.917] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:102:0) after 8m27s
[18:26:47.942] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:102:0)
[18:26:47.964] Build status: STARTED
[18:26:47.964] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:103:0) after 7m32s
[18:34:19.997] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:103:0)
[18:34:20.018] Build status: STARTED
[18:34:20.018] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:104:0) after 2m14s
[18:36:33.994] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:104:0)
[18:36:34.024] Build status: STARTED
[18:36:34.024] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:105:0) after 3m23s
[18:39:57.203] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:105:0)
[18:39:57.224] Build status: STARTED
[18:39:57.224] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:106:0) after 4m12s
[18:44:09.364] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:106:0)
[18:44:09.382] Build status: STARTED
[18:44:09.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:107:0) after 4m31s
[18:48:40.436] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:107:0)
[18:48:40.462] Build status: STARTED
[18:48:40.462] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:108:0) after 3m29s
[18:52:09.802] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:108:0)
[18:52:09.823] Build status: STARTED
[18:52:09.823] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:109:0) after 3m50s
[18:55:59.859] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:109:0)
[18:55:59.879] Build status: STARTED
[18:55:59.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:110:0) after 9m41s
[19:05:41.007] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:110:0)
[19:05:41.028] Build status: STARTED
[19:05:41.028] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:111:0) after 1m15s
[19:06:56.339] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:111:0)
[19:06:56.362] Build status: STARTED
[19:06:56.362] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:112:0) after 8m2s
[19:14:58.456] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:112:0)
[19:14:58.476] Build status: STARTED
[19:14:58.476] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:113:0) after 6m57s
[19:21:55.545] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:113:0)
[19:21:55.564] Build status: STARTED
[19:21:55.564] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:114:0) after 6m33s
[19:28:28.552] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:114:0)
[19:28:28.572] Build status: STARTED
[19:28:28.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:115:0) after 7m47s
[19:36:15.598] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:115:0)
[19:36:15.622] Build status: STARTED
[19:36:15.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:116:0) after 7m53s
[19:44:08.642] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:116:0)
[19:44:08.663] Build status: STARTED
[19:44:08.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:117:0) after 6m41s
[19:50:49.823] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:117:0)
[19:50:49.849] Build status: STARTED
[19:50:49.849] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:118:0) after 5m16s
[19:56:05.967] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:118:0)
[19:56:05.989] Build status: STARTED
[19:56:05.989] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8946111193242785072:119:0) after 9m5s
[20:01:16.583] Received PubSub notification, asking Buildbucket for the build status
[20:01:16.607] Build:
{
"id": "8780681131323678673",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 18999,
"createdBy": "project:chromium",
"createTime": "2023-05-19T09:02:26.711981167Z",
"startTime": "2023-05-19T09:06:21.023322Z",
"endTime": "2023-05-19T20:01:15.952151703Z",
"updateTime": "2023-05-19T20:01:15.952151703Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "423951cfe5a27c283b49eb27bab4db80652b3ac4",
"ref": "refs/heads/main"
}
}
}
[20:01:16.607] Invocation finished in 10h58m51.109713067s with status FAILED