[18:13:35.264] New invocation is queued and will start shortly
[18:13:36.421] Starting the invocation (attempt 1)
[18:13:36.456] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/3a30f9c0ab1d16def124926e04e494b6f4f2483f"
[18:13:36.456] Popped gitiles commit info from properties and tags
[18:13:36.456] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[18:13:36.456] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[18:13:36.456] Buildbucket request:
{
"requestId": "8850587314506946928",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3a30f9c0ab1d16def124926e04e494b6f4f2483f",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850587314506946928"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[18:13:37.603] Scheduled build:
{
"id": "8685157252514825665",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 31788,
"createdBy": "project:chromium",
"createTime": "2026-04-07T18:13:36.535027057Z",
"updateTime": "2026-04-07T18:13:36.535027057Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3a30f9c0ab1d16def124926e04e494b6f4f2483f",
"ref": "refs/heads/main"
}
}
}
[18:13:37.603] Task URL: https://cr-buildbucket.appspot.com/build/8685157252514825665
[18:13:37.603] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:2:0) after 8m43s
[18:13:38.447] Received PubSub notification, asking Buildbucket for the build status
[18:13:38.474] Build status: SCHEDULED
[18:14:35.329] Received PubSub notification, asking Buildbucket for the build status
[18:14:35.418] Build status: STARTED
[18:22:20.704] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:2:0)
[18:22:20.724] Build status: STARTED
[18:22:20.724] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:5:0) after 1m41s
[18:24:01.799] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:5:0)
[18:24:01.820] Build status: STARTED
[18:24:01.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:6:0) after 1m27s
[18:25:28.921] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:6:0)
[18:25:28.939] Build status: STARTED
[18:25:28.940] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:7:0) after 1m8s
[18:26:36.992] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:7:0)
[18:26:37.012] Build status: STARTED
[18:26:37.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:8:0) after 7m32s
[18:34:09.265] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:8:0)
[18:34:09.336] Build status: STARTED
[18:34:09.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:9:0) after 5m58s
[18:40:07.367] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:9:0)
[18:40:07.384] Build status: STARTED
[18:40:07.384] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:10:0) after 8m54s
[18:49:01.541] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:10:0)
[18:49:01.581] Build status: STARTED
[18:49:01.581] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:11:0) after 6m19s
[18:55:20.695] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:11:0)
[18:55:20.712] Build status: STARTED
[18:55:20.712] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:12:0) after 3m15s
[18:58:35.775] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:12:0)
[18:58:35.792] Build status: STARTED
[18:58:35.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:13:0) after 1m7s
[18:59:42.920] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:13:0)
[18:59:42.940] Build status: STARTED
[18:59:42.940] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:14:0) after 4m54s
[19:04:37.043] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:14:0)
[19:04:37.090] Build status: STARTED
[19:04:37.090] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:15:0) after 6m29s
[19:11:06.108] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:15:0)
[19:11:06.133] Build status: STARTED
[19:11:06.133] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:16:0) after 8m41s
[19:19:47.382] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:16:0)
[19:19:47.432] Build status: STARTED
[19:19:47.432] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:17:0) after 8m28s
[19:28:15.456] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:17:0)
[19:28:15.473] Build status: STARTED
[19:28:15.473] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:18:0) after 4m44s
[19:32:59.571] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:18:0)
[19:32:59.590] Build status: STARTED
[19:32:59.590] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:19:0) after 5m39s
[19:38:38.618] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:19:0)
[19:38:38.633] Build status: STARTED
[19:38:38.633] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:20:0) after 4m39s
[19:43:17.623] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:20:0)
[19:43:17.722] Build status: STARTED
[19:43:17.722] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:21:0) after 8m45s
[19:52:02.831] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:21:0)
[19:52:02.858] Build status: STARTED
[19:52:02.858] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:22:0) after 3m1s
[19:55:03.881] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:22:0)
[19:55:03.898] Build status: STARTED
[19:55:03.898] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:23:0) after 9m8s
[20:04:11.943] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:23:0)
[20:04:11.963] Build status: STARTED
[20:04:11.963] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:24:0) after 7m26s
[20:11:38.131] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:24:0)
[20:11:38.147] Build status: STARTED
[20:11:38.147] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:25:0) after 7m34s
[20:19:12.172] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:25:0)
[20:19:12.236] Build status: STARTED
[20:19:12.236] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:26:0) after 7m34s
[20:26:46.349] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:26:0)
[20:26:46.365] Build status: STARTED
[20:26:46.365] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:27:0) after 9m31s
[20:36:17.610] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:27:0)
[20:36:17.630] Build status: STARTED
[20:36:17.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:28:0) after 7m39s
[20:43:56.777] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:28:0)
[20:43:56.815] Build status: STARTED
[20:43:56.815] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:29:0) after 6m24s
[20:50:20.828] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:29:0)
[20:50:20.853] Build status: STARTED
[20:50:20.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:30:0) after 5m7s
[20:55:27.980] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:30:0)
[20:55:28.000] Build status: STARTED
[20:55:28.000] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:31:0) after 9m44s
[21:05:12.438] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:31:0)
[21:05:12.496] Build status: STARTED
[21:05:12.497] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:32:0) after 7m33s
[21:12:45.516] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:32:0)
[21:12:45.534] Build status: STARTED
[21:12:45.534] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:33:0) after 3m48s
[21:16:33.583] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:33:0)
[21:16:33.608] Build status: STARTED
[21:16:33.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:34:0) after 1m29s
[21:18:02.656] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:34:0)
[21:18:02.713] Build status: STARTED
[21:18:02.713] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:35:0) after 6m49s
[21:24:51.794] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:35:0)
[21:24:51.814] Build status: STARTED
[21:24:51.814] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:36:0) after 1m17s
[21:26:08.844] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:36:0)
[21:26:08.884] Build status: STARTED
[21:26:08.884] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:37:0) after 4m52s
[21:31:01.053] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:37:0)
[21:31:01.080] Build status: STARTED
[21:31:01.080] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:38:0) after 3m56s
[21:34:57.278] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:38:0)
[21:34:57.305] Build status: STARTED
[21:34:57.305] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:39:0) after 6m2s
[21:40:59.363] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:39:0)
[21:40:59.378] Build status: STARTED
[21:40:59.378] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:40:0) after 2m54s
[21:43:53.445] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:40:0)
[21:43:53.471] Build status: STARTED
[21:43:53.471] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:41:0) after 7m27s
[21:51:20.652] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:41:0)
[21:51:20.673] Build status: STARTED
[21:51:20.673] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:42:0) after 6m49s
[21:58:09.754] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:42:0)
[21:58:09.771] Build status: STARTED
[21:58:09.771] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:43:0) after 6m9s
[22:04:18.789] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:43:0)
[22:04:18.809] Build status: STARTED
[22:04:18.810] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:44:0) after 3m19s
[22:07:37.849] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:44:0)
[22:07:37.870] Build status: STARTED
[22:07:37.870] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:45:0) after 4m9s
[22:11:47.062] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:45:0)
[22:11:47.083] Build status: STARTED
[22:11:47.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:46:0) after 4m1s
[22:15:48.128] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:46:0)
[22:15:48.296] Build status: STARTED
[22:15:48.296] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:47:0) after 4m24s
[22:20:12.356] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:47:0)
[22:20:12.390] Build status: STARTED
[22:20:12.390] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:48:0) after 3m51s
[22:24:03.414] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:48:0)
[22:24:03.436] Build status: STARTED
[22:24:03.436] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:49:0) after 4m38s
[22:28:41.922] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:49:0)
[22:28:41.935] Build status: STARTED
[22:28:41.935] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:50:0) after 7m52s
[22:36:34.070] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:50:0)
[22:36:34.100] Build status: STARTED
[22:36:34.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:51:0) after 1m24s
[22:37:58.194] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:51:0)
[22:37:58.275] Build status: STARTED
[22:37:58.275] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:52:0) after 5m8s
[22:43:06.379] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:52:0)
[22:43:06.393] Build status: STARTED
[22:43:06.393] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:53:0) after 3m34s
[22:46:40.504] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:53:0)
[22:46:40.531] Build status: STARTED
[22:46:40.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:54:0) after 8m14s
[22:54:54.782] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:54:0)
[22:54:54.801] Build status: STARTED
[22:54:54.801] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:55:0) after 5m3s
[22:59:57.973] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8850587314506946928:55:0)
[22:59:57.994] Build:
{
"id": "8685157252514825665",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 31788,
"createdBy": "project:chromium",
"createTime": "2026-04-07T18:13:36.535027057Z",
"startTime": "2026-04-07T18:14:35.105853603Z",
"endTime": "2026-04-07T22:59:52.659877889Z",
"updateTime": "2026-04-07T22:59:52.659877889Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "3a30f9c0ab1d16def124926e04e494b6f4f2483f",
"ref": "refs/heads/main"
}
}
}
[22:59:57.994] Invocation finished in 4h46m22.743153991s with status FAILED