[15:40:56.208] New invocation is queued and will start shortly
[15:40:57.308] Starting the invocation (attempt 1)
[15:40:57.333] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/9eee1d89d91b62c4f3d7a727b80de2381c47d2bd"
[15:40:57.334] Popped gitiles commit info from properties and tags
[15:40:57.334] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:40:57.334] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:40:57.334] Buildbucket request:
{
"requestId": "8837913343168020480",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9eee1d89d91b62c4f3d7a727b80de2381c47d2bd",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8837913343168020480"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:40:57.900] Scheduled build:
{
"id": "8672483281260147281",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 35192,
"createdBy": "project:chromium",
"createTime": "2026-08-25T15:40:57.407864160Z",
"updateTime": "2026-08-25T15:40:57.407864160Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9eee1d89d91b62c4f3d7a727b80de2381c47d2bd",
"ref": "refs/heads/main"
}
},
"canary": true
}
[15:40:57.900] Task URL: https://cr-buildbucket.appspot.com/build/8672483281260147281
[15:40:57.900] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:2:0) after 3m18s
[15:40:58.871] Received PubSub notification, asking Buildbucket for the build status
[15:40:58.893] Build status: SCHEDULED
[15:42:53.106] Received PubSub notification, asking Buildbucket for the build status
[15:42:53.133] Build status: STARTED
[15:44:15.930] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:2:0)
[15:44:15.952] Build status: STARTED
[15:44:15.952] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:5:0) after 9m33s
[15:53:49.107] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:5:0)
[15:53:49.128] Build status: STARTED
[15:53:49.128] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:6:0) after 2m33s
[15:56:22.291] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:6:0)
[15:56:22.359] Build status: STARTED
[15:56:22.359] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:7:0) after 3m31s
[15:59:53.436] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:7:0)
[15:59:53.459] Build status: STARTED
[15:59:53.460] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:8:0) after 8m10s
[16:08:03.552] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:8:0)
[16:08:03.571] Build status: STARTED
[16:08:03.571] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:9:0) after 1m14s
[16:09:17.654] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:9:0)
[16:09:17.739] Build status: STARTED
[16:09:17.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:10:0) after 8m27s
[16:17:44.757] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:10:0)
[16:17:44.922] Build status: STARTED
[16:17:44.922] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:11:0) after 7m23s
[16:25:08.003] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:11:0)
[16:25:08.073] Build status: STARTED
[16:25:08.073] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:12:0) after 9m35s
[16:34:43.178] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:12:0)
[16:34:43.197] Build status: STARTED
[16:34:43.197] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:13:0) after 6m30s
[16:41:13.186] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:13:0)
[16:41:13.347] Build status: STARTED
[16:41:13.347] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:14:0) after 4m59s
[16:46:12.357] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:14:0)
[16:46:12.400] Build status: STARTED
[16:46:12.400] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:15:0) after 5m42s
[16:51:54.590] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:15:0)
[16:51:54.608] Build status: STARTED
[16:51:54.609] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:16:0) after 7m10s
[16:59:04.637] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:16:0)
[16:59:04.663] Build status: STARTED
[16:59:04.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:17:0) after 3m1s
[17:02:05.741] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:17:0)
[17:02:05.762] Build status: STARTED
[17:02:05.762] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:18:0) after 4m53s
[17:06:58.778] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:18:0)
[17:06:59.548] Build status: STARTED
[17:06:59.548] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:19:0) after 2m9s
[17:09:08.575] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:19:0)
[17:09:08.601] Build status: STARTED
[17:09:08.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:20:0) after 3m1s
[17:12:09.727] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:20:0)
[17:12:09.746] Build status: STARTED
[17:12:09.746] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:21:0) after 5m44s
[17:17:53.868] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:21:0)
[17:17:53.885] Build status: STARTED
[17:17:53.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:22:0) after 9m34s
[17:27:27.909] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:22:0)
[17:27:27.930] Build status: STARTED
[17:27:27.930] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:23:0) after 1m16s
[17:28:43.958] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:23:0)
[17:28:43.981] Build status: STARTED
[17:28:43.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:24:0) after 1m57s
[17:30:41.000] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:24:0)
[17:30:41.056] Build status: STARTED
[17:30:41.056] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:25:0) after 6m1s
[17:36:42.091] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:25:0)
[17:36:42.113] Build status: STARTED
[17:36:42.113] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:26:0) after 2m2s
[17:38:44.128] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:26:0)
[17:38:44.143] Build status: STARTED
[17:38:44.143] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:27:0) after 6m42s
[17:45:26.200] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:27:0)
[17:45:26.235] Build status: STARTED
[17:45:26.235] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:28:0) after 4m36s
[17:50:02.254] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:28:0)
[17:50:02.279] Build status: STARTED
[17:50:02.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:29:0) after 1m57s
[17:51:59.304] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:29:0)
[17:51:59.325] Build status: STARTED
[17:51:59.325] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:30:0) after 9m1s
[18:01:00.360] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:30:0)
[18:01:00.383] Build status: STARTED
[18:01:00.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:31:0) after 3m57s
[18:04:57.560] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:31:0)
[18:04:57.577] Build status: STARTED
[18:04:57.577] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:32:0) after 7m7s
[18:12:04.685] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:32:0)
[18:12:04.705] Build status: STARTED
[18:12:04.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:33:0) after 9m36s
[18:21:40.748] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:33:0)
[18:21:40.776] Build status: STARTED
[18:21:40.776] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:34:0) after 4m56s
[18:26:36.928] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:34:0)
[18:26:36.950] Build status: STARTED
[18:26:36.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:35:0) after 2m41s
[18:29:18.016] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:35:0)
[18:29:18.037] Build status: STARTED
[18:29:18.037] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:36:0) after 4m30s
[18:33:48.098] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:36:0)
[18:33:48.115] Build status: STARTED
[18:33:48.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:37:0) after 1m0s
[18:34:48.166] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:37:0)
[18:34:48.189] Build status: STARTED
[18:34:48.189] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:38:0) after 7m49s
[18:42:37.277] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:38:0)
[18:42:37.296] Build status: STARTED
[18:42:37.296] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:39:0) after 5m58s
[18:48:35.322] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:39:0)
[18:48:35.348] Build status: STARTED
[18:48:35.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:40:0) after 4m48s
[18:53:23.413] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:40:0)
[18:53:23.444] Build status: STARTED
[18:53:23.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:41:0) after 8m16s
[19:01:39.536] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:41:0)
[19:01:39.555] Build status: STARTED
[19:01:39.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:42:0) after 6m24s
[19:08:03.577] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:42:0)
[19:08:03.596] Build status: STARTED
[19:08:03.596] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:43:0) after 2m59s
[19:11:02.735] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:43:0)
[19:11:02.756] Build status: STARTED
[19:11:02.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:44:0) after 3m51s
[19:14:53.842] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:44:0)
[19:14:53.857] Build status: STARTED
[19:14:53.857] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:45:0) after 2m34s
[19:17:27.841] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:45:0)
[19:17:27.878] Build status: STARTED
[19:17:27.878] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:46:0) after 8m30s
[19:25:57.937] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:46:0)
[19:25:57.974] Build status: STARTED
[19:25:57.974] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:47:0) after 6m21s
[19:32:18.984] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:47:0)
[19:32:19.014] Build status: STARTED
[19:32:19.014] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:48:0) after 3m15s
[19:35:34.064] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:48:0)
[19:35:34.121] Build status: STARTED
[19:35:34.121] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:49:0) after 9m52s
[19:45:26.173] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:49:0)
[19:45:26.197] Build status: STARTED
[19:45:26.197] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:50:0) after 1m44s
[19:47:10.206] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:50:0)
[19:47:10.230] Build status: STARTED
[19:47:10.230] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:51:0) after 1m7s
[19:48:17.252] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:51:0)
[19:48:17.291] Build status: STARTED
[19:48:17.291] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:52:0) after 8m6s
[19:56:23.302] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:52:0)
[19:56:23.335] Build status: STARTED
[19:56:23.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:53:0) after 1m26s
[19:57:49.356] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:53:0)
[19:57:49.521] Build status: STARTED
[19:57:49.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:54:0) after 8m8s
[20:05:57.542] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:54:0)
[20:05:57.578] Build status: STARTED
[20:05:57.578] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:55:0) after 3m6s
[20:09:03.593] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:55:0)
[20:09:03.627] Build status: STARTED
[20:09:03.627] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:56:0) after 2m56s
[20:11:59.665] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:56:0)
[20:11:59.689] Build status: STARTED
[20:11:59.689] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8837913343168020480:57:0) after 4m26s
[20:15:34.303] Received PubSub notification, asking Buildbucket for the build status
[20:15:34.321] Build:
{
"id": "8672483281260147281",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 35192,
"createdBy": "project:chromium",
"createTime": "2026-08-25T15:40:57.407864160Z",
"startTime": "2026-08-25T15:42:52.824971516Z",
"endTime": "2026-08-25T20:15:33.902814094Z",
"updateTime": "2026-08-25T20:15:33.902814094Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "9eee1d89d91b62c4f3d7a727b80de2381c47d2bd",
"ref": "refs/heads/main"
}
},
"canary": true
}
[20:15:34.321] Invocation finished in 4h34m38.116627448s with status FAILED