[04:04:02.515] New invocation is queued and will start shortly
[04:04:03.654] Starting the invocation (attempt 1)
[04:04:03.713] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/67f8e5867f0e5ae2530fa26221d6a0e1cfebc402"
[04:04:03.713] Popped gitiles commit info from properties and tags
[04:04:03.713] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:04:03.713] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:04:03.713] Buildbucket request:
{
"requestId": "8973490249769227888",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "67f8e5867f0e5ae2530fa26221d6a0e1cfebc402",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8973490249769227888"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:04:04.477] Scheduled build:
{
"id": "8808060187743810417",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 16235,
"createdBy": "project:chromium",
"createTime": "2022-07-21T04:04:03.796076201Z",
"updateTime": "2022-07-21T04:04:03.796076201Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "67f8e5867f0e5ae2530fa26221d6a0e1cfebc402",
"ref": "refs/heads/main"
}
}
}
[04:04:04.477] Task URL: https://cr-buildbucket.appspot.com/build/8808060187743810417
[04:04:04.477] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:2:0) after 2m27s
[04:05:13.759] Received PubSub notification, asking Buildbucket for the build status
[04:05:13.779] Build status: STARTED
[04:06:31.595] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:2:0)
[04:06:31.627] Build status: STARTED
[04:06:31.627] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:4:0) after 3m47s
[04:10:18.816] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:4:0)
[04:10:18.835] Build status: STARTED
[04:10:18.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:5:0) after 1m24s
[04:11:42.861] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:5:0)
[04:11:42.877] Build status: STARTED
[04:11:42.877] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:6:0) after 8m29s
[04:20:11.924] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:6:0)
[04:20:11.983] Build status: STARTED
[04:20:11.983] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:7:0) after 9m22s
[04:29:34.353] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:7:0)
[04:29:34.382] Build status: STARTED
[04:29:34.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:8:0) after 6m26s
[04:36:00.546] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:8:0)
[04:36:00.580] Build status: STARTED
[04:36:00.580] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:9:0) after 5m24s
[04:41:24.740] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:9:0)
[04:41:24.759] Build status: STARTED
[04:41:24.759] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:10:0) after 4m48s
[04:46:12.841] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:10:0)
[04:46:12.919] Build status: STARTED
[04:46:12.919] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:11:0) after 5m29s
[04:51:41.940] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:11:0)
[04:51:41.957] Build status: STARTED
[04:51:41.957] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:12:0) after 9m26s
[05:01:08.077] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:12:0)
[05:01:08.215] Build status: STARTED
[05:01:08.215] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:13:0) after 6m9s
[05:07:17.202] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:13:0)
[05:07:17.222] Build status: STARTED
[05:07:17.222] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:14:0) after 1m24s
[05:08:41.301] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:14:0)
[05:08:41.322] Build status: STARTED
[05:08:41.322] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:15:0) after 2m24s
[05:11:05.345] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:15:0)
[05:11:05.426] Build status: STARTED
[05:11:05.426] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:16:0) after 1m30s
[05:12:35.544] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:16:0)
[05:12:35.571] Build status: STARTED
[05:12:35.571] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:17:0) after 2m12s
[05:14:47.593] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:17:0)
[05:14:47.613] Build status: STARTED
[05:14:47.613] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:18:0) after 4m2s
[05:18:49.721] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:18:0)
[05:18:49.742] Build status: STARTED
[05:18:49.742] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:19:0) after 7m2s
[05:25:51.770] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:19:0)
[05:25:51.790] Build status: STARTED
[05:25:51.790] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:20:0) after 1m20s
[05:27:11.821] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:20:0)
[05:27:11.843] Build status: STARTED
[05:27:11.843] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:21:0) after 9m51s
[05:37:02.876] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:21:0)
[05:37:02.898] Build status: STARTED
[05:37:02.898] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:22:0) after 7m46s
[05:44:49.166] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:22:0)
[05:44:49.385] Build status: STARTED
[05:44:49.385] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:23:0) after 7m40s
[05:52:29.407] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:23:0)
[05:52:29.432] Build status: STARTED
[05:52:29.432] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:24:0) after 1m23s
[05:53:52.453] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:24:0)
[05:53:52.473] Build status: STARTED
[05:53:52.473] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:25:0) after 4m33s
[05:58:25.522] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:25:0)
[05:58:25.546] Build status: STARTED
[05:58:25.546] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:26:0) after 4m7s
[06:02:32.615] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:26:0)
[06:02:32.648] Build status: STARTED
[06:02:32.648] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:27:0) after 5m8s
[06:07:40.677] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:27:0)
[06:07:40.697] Build status: STARTED
[06:07:40.697] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:28:0) after 6m52s
[06:14:32.965] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:28:0)
[06:14:33.037] Build status: STARTED
[06:14:33.037] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:29:0) after 5m24s
[06:19:57.059] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:29:0)
[06:19:57.128] Build status: STARTED
[06:19:57.128] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:30:0) after 6m46s
[06:26:43.110] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:30:0)
[06:26:43.152] Build status: STARTED
[06:26:43.152] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:31:0) after 6m11s
[06:32:54.176] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:31:0)
[06:32:54.213] Build status: STARTED
[06:32:54.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:32:0) after 6m7s
[06:39:01.266] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:32:0)
[06:39:01.298] Build status: STARTED
[06:39:01.298] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:33:0) after 9m42s
[06:48:43.699] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:33:0)
[06:48:43.727] Build status: STARTED
[06:48:43.728] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:34:0) after 6m30s
[06:55:13.881] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:34:0)
[06:55:13.900] Build status: STARTED
[06:55:13.900] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:35:0) after 9m25s
[07:04:39.242] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:35:0)
[07:04:39.275] Build status: STARTED
[07:04:39.275] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:36:0) after 3m22s
[07:08:01.320] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:36:0)
[07:08:01.342] Build status: STARTED
[07:08:01.342] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:37:0) after 9m37s
[07:17:38.473] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:37:0)
[07:17:38.505] Build status: STARTED
[07:17:38.505] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:38:0) after 2m51s
[07:20:29.628] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:38:0)
[07:20:29.649] Build status: STARTED
[07:20:29.649] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:39:0) after 1m37s
[07:22:06.675] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:39:0)
[07:22:06.695] Build status: STARTED
[07:22:06.695] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:40:0) after 3m18s
[07:25:24.854] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:40:0)
[07:25:24.883] Build status: STARTED
[07:25:24.883] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:41:0) after 5m28s
[07:30:52.906] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:41:0)
[07:30:52.939] Build status: STARTED
[07:30:52.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:42:0) after 9m23s
[07:40:15.976] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:42:0)
[07:40:15.999] Build status: STARTED
[07:40:15.999] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:43:0) after 9m47s
[07:50:03.047] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:43:0)
[07:50:03.083] Build status: STARTED
[07:50:03.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:44:0) after 1m54s
[07:51:57.222] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:44:0)
[07:51:57.249] Build status: STARTED
[07:51:57.249] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:45:0) after 7m19s
[07:59:16.313] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:45:0)
[07:59:16.346] Build status: STARTED
[07:59:16.346] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:46:0) after 7m25s
[08:06:41.400] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:46:0)
[08:06:41.472] Build status: STARTED
[08:06:41.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:47:0) after 1m42s
[08:08:23.540] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:47:0)
[08:08:23.809] Build status: STARTED
[08:08:23.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:48:0) after 3m36s
[08:11:59.945] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:48:0)
[08:11:59.970] Build status: STARTED
[08:11:59.970] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:49:0) after 7m7s
[08:19:06.995] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:49:0)
[08:19:07.022] Build status: STARTED
[08:19:07.022] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:50:0) after 6m0s
[08:25:07.089] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:50:0)
[08:25:07.130] Build status: STARTED
[08:25:07.130] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:51:0) after 7m21s
[08:32:28.107] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:51:0)
[08:32:28.138] Build status: STARTED
[08:32:28.138] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:52:0) after 3m55s
[08:36:23.217] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:52:0)
[08:36:23.241] Build status: STARTED
[08:36:23.241] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:53:0) after 4m2s
[08:40:25.432] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:53:0)
[08:40:25.462] Build status: STARTED
[08:40:25.462] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:54:0) after 1m50s
[08:42:15.493] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:54:0)
[08:42:15.525] Build status: STARTED
[08:42:15.526] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:55:0) after 5m49s
[08:48:04.554] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:55:0)
[08:48:04.575] Build status: STARTED
[08:48:04.575] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:56:0) after 3m51s
[08:51:55.825] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:56:0)
[08:51:55.965] Build status: STARTED
[08:51:55.965] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:57:0) after 9m31s
[09:01:26.996] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:57:0)
[09:01:36.604] Build status: STARTED
[09:01:36.604] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:58:0) after 7m34s
[09:09:10.751] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:58:0)
[09:09:10.782] Build status: STARTED
[09:09:10.782] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:59:0) after 7m21s
[09:16:31.827] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:59:0)
[09:16:31.865] Build status: STARTED
[09:16:31.865] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:60:0) after 8m49s
[09:25:20.934] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:60:0)
[09:25:20.979] Build status: STARTED
[09:25:20.979] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:61:0) after 5m31s
[09:30:52.042] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:61:0)
[09:30:52.076] Build status: STARTED
[09:30:52.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:62:0) after 3m58s
[09:34:50.175] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:62:0)
[09:34:50.200] Build status: STARTED
[09:34:50.200] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8973490249769227888:63:0) after 4m32s
[09:37:25.845] Received PubSub notification, asking Buildbucket for the build status
[09:37:25.864] Build:
{
"id": "8808060187743810417",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 16235,
"createdBy": "project:chromium",
"createTime": "2022-07-21T04:04:03.796076201Z",
"startTime": "2022-07-21T04:05:09.788314Z",
"endTime": "2022-07-21T09:37:25.294985566Z",
"updateTime": "2022-07-21T09:37:25.294985566Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "67f8e5867f0e5ae2530fa26221d6a0e1cfebc402",
"ref": "refs/heads/main"
}
}
}
[09:37:25.864] Invocation finished in 5h33m23.382773474s with status FAILED