[12:08:27.408] New invocation is queued and will start shortly
[12:08:28.613] Starting the invocation (attempt 1)
[12:08:28.643] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/6169b57be3fcb656274f96c3ce825984cebd0eb7"
[12:08:28.643] Popped gitiles commit info from properties and tags
[12:08:28.643] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[12:08:28.643] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[12:08:28.643] Buildbucket request:
{
"requestId": "8845989841267059968",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "6169b57be3fcb656274f96c3ce825984cebd0eb7",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8845989841267059968"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[12:08:29.193] Scheduled build:
{
"id": "8680559779251912913",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 32664,
"createdBy": "project:chromium",
"createTime": "2026-05-28T12:08:28.706189024Z",
"updateTime": "2026-05-28T12:08:28.706189024Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "6169b57be3fcb656274f96c3ce825984cebd0eb7",
"ref": "refs/heads/main"
}
}
}
[12:08:29.193] Task URL: https://cr-buildbucket.appspot.com/build/8680559779251912913
[12:08:29.193] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:2:0) after 6m1s
[12:08:30.105] Received PubSub notification, asking Buildbucket for the build status
[12:08:30.127] Build status: SCHEDULED
[12:09:13.128] Received PubSub notification, asking Buildbucket for the build status
[12:09:13.224] Build status: STARTED
[12:14:30.372] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:2:0)
[12:14:30.408] Build status: STARTED
[12:14:30.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:5:0) after 9m24s
[12:23:54.428] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:5:0)
[12:23:54.459] Build status: STARTED
[12:23:54.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:6:0) after 7m21s
[12:31:15.472] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:6:0)
[12:31:15.497] Build status: STARTED
[12:31:15.497] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:7:0) after 3m7s
[12:34:22.631] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:7:0)
[12:34:22.709] Build status: STARTED
[12:34:22.709] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:8:0) after 5m57s
[12:40:19.723] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:8:0)
[12:40:19.767] Build status: STARTED
[12:40:19.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:9:0) after 9m43s
[12:50:02.788] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:9:0)
[12:50:02.808] Build status: STARTED
[12:50:02.808] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:10:0) after 3m27s
[12:53:30.200] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:10:0)
[12:53:30.216] Build status: STARTED
[12:53:30.216] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:11:0) after 7m18s
[13:00:48.263] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:11:0)
[13:00:48.344] Build status: STARTED
[13:00:48.344] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:12:0) after 4m37s
[13:05:25.460] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:12:0)
[13:05:25.480] Build status: STARTED
[13:05:25.480] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:13:0) after 5m52s
[13:11:17.496] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:13:0)
[13:11:17.517] Build status: STARTED
[13:11:17.517] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:14:0) after 4m31s
[13:15:48.699] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:14:0)
[13:15:48.730] Build status: STARTED
[13:15:48.730] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:15:0) after 8m36s
[13:24:24.772] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:15:0)
[13:24:24.789] Build status: STARTED
[13:24:24.789] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:16:0) after 1m41s
[13:26:05.925] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:16:0)
[13:26:06.015] Build status: STARTED
[13:26:06.015] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:17:0) after 9m20s
[13:35:26.193] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:17:0)
[13:35:26.238] Build status: STARTED
[13:35:26.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:18:0) after 6m1s
[13:41:27.298] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:18:0)
[13:41:27.317] Build status: STARTED
[13:41:27.317] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:19:0) after 6m1s
[13:47:28.450] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:19:0)
[13:47:28.471] Build status: STARTED
[13:47:28.471] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:20:0) after 4m36s
[13:52:04.485] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:20:0)
[13:52:04.566] Build status: STARTED
[13:52:04.566] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:21:0) after 5m35s
[13:57:39.560] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:21:0)
[13:57:39.615] Build status: STARTED
[13:57:39.615] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:22:0) after 8m54s
[14:06:33.686] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:22:0)
[14:06:33.733] Build status: STARTED
[14:06:33.733] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:23:0) after 1m8s
[14:07:41.748] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:23:0)
[14:07:41.768] Build status: STARTED
[14:07:41.768] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:24:0) after 9m8s
[14:16:49.836] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:24:0)
[14:16:49.871] Build status: STARTED
[14:16:49.872] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:25:0) after 8m25s
[14:25:14.886] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:25:0)
[14:25:14.907] Build status: STARTED
[14:25:14.907] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:26:0) after 8m27s
[14:33:42.018] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:26:0)
[14:33:42.042] Build status: STARTED
[14:33:42.042] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:27:0) after 4m42s
[14:38:24.059] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:27:0)
[14:38:24.077] Build status: STARTED
[14:38:24.077] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:28:0) after 7m14s
[14:45:38.100] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:28:0)
[14:45:38.120] Build status: STARTED
[14:45:38.120] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:29:0) after 8m7s
[14:53:45.217] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:29:0)
[14:53:45.329] Build status: STARTED
[14:53:45.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:30:0) after 7m50s
[15:01:35.389] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:30:0)
[15:01:35.439] Build status: STARTED
[15:01:35.439] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:31:0) after 1m44s
[15:03:19.462] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:31:0)
[15:03:19.477] Build status: STARTED
[15:03:19.477] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:32:0) after 6m16s
[15:09:35.993] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:32:0)
[15:09:36.015] Build status: STARTED
[15:09:36.015] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:33:0) after 6m23s
[15:15:59.092] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:33:0)
[15:15:59.135] Build status: STARTED
[15:15:59.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:34:0) after 3m47s
[15:19:46.226] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:34:0)
[15:19:46.288] Build status: STARTED
[15:19:46.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:35:0) after 2m47s
[15:22:33.294] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:35:0)
[15:22:33.313] Build status: STARTED
[15:22:33.313] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:36:0) after 1m10s
[15:23:43.383] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:36:0)
[15:23:43.466] Build status: STARTED
[15:23:43.466] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:37:0) after 2m7s
[15:25:50.504] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:37:0)
[15:25:50.528] Build status: STARTED
[15:25:50.528] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:38:0) after 8m53s
[15:34:43.542] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:38:0)
[15:34:43.566] Build status: STARTED
[15:34:43.566] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:39:0) after 2m27s
[15:37:10.585] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:39:0)
[15:37:10.633] Build status: STARTED
[15:37:10.633] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:40:0) after 7m58s
[15:45:08.741] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:40:0)
[15:45:08.762] Build status: STARTED
[15:45:08.762] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:41:0) after 1m38s
[15:46:46.902] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:41:0)
[15:46:46.918] Build status: STARTED
[15:46:46.918] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:42:0) after 5m28s
[15:52:14.977] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:42:0)
[15:52:14.995] Build status: STARTED
[15:52:14.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:43:0) after 9m48s
[16:02:03.255] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:43:0)
[16:02:03.300] Build status: STARTED
[16:02:03.300] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:44:0) after 6m27s
[16:08:30.313] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:44:0)
[16:08:30.329] Build status: STARTED
[16:08:30.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:45:0) after 3m33s
[16:12:03.329] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:45:0)
[16:12:03.351] Build status: STARTED
[16:12:03.351] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:46:0) after 1m43s
[16:13:46.378] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:46:0)
[16:13:46.397] Build status: STARTED
[16:13:46.397] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:47:0) after 5m42s
[16:19:28.540] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:47:0)
[16:19:28.622] Build status: STARTED
[16:19:28.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8845989841267059968:48:0) after 5m36s
[16:24:03.351] Received PubSub notification, asking Buildbucket for the build status
[16:24:03.383] Build:
{
"id": "8680559779251912913",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 32664,
"createdBy": "project:chromium",
"createTime": "2026-05-28T12:08:28.706189024Z",
"startTime": "2026-05-28T12:09:12.922187953Z",
"endTime": "2026-05-28T16:24:02.506128873Z",
"updateTime": "2026-05-28T16:24:02.506128873Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "6169b57be3fcb656274f96c3ce825984cebd0eb7",
"ref": "refs/heads/main"
}
}
}
[16:24:03.383] Invocation finished in 4h15m35.981592897s with status FAILED