[21:06:56.424] New invocation is queued and will start shortly
[21:06:57.618] Starting the invocation (attempt 1)
[21:06:57.649] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/628168700b8ce53f755f480319cb3ded0e5eed19"
[21:06:57.649] Popped gitiles commit info from properties and tags
[21:06:57.650] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:06:57.650] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:06:57.650] Buildbucket request:
{
"requestId": "8851844765621547472",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "628168700b8ce53f755f480319cb3ded0e5eed19",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851844765621547472"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:06:58.216] Scheduled build:
{
"id": "8686414703620123969",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 31601,
"createdBy": "project:chromium",
"createTime": "2026-03-24T21:06:57.711676743Z",
"updateTime": "2026-03-24T21:06:57.711676743Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "628168700b8ce53f755f480319cb3ded0e5eed19",
"ref": "refs/heads/main"
}
}
}
[21:06:58.216] Task URL: https://cr-buildbucket.appspot.com/build/8686414703620123969
[21:06:58.216] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:2:0) after 3m51s
[21:06:59.214] Received PubSub notification, asking Buildbucket for the build status
[21:06:59.246] Build status: SCHEDULED
[21:07:50.774] Received PubSub notification, asking Buildbucket for the build status
[21:07:50.794] Build status: STARTED
[21:10:49.235] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:2:0)
[21:10:49.253] Build status: STARTED
[21:10:49.253] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:5:0) after 8m6s
[21:18:55.275] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:5:0)
[21:18:55.300] Build status: STARTED
[21:18:55.300] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:6:0) after 1m11s
[21:20:06.304] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:6:0)
[21:20:06.324] Build status: STARTED
[21:20:06.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:7:0) after 7m22s
[21:27:28.346] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:7:0)
[21:27:28.370] Build status: STARTED
[21:27:28.371] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:8:0) after 2m12s
[21:29:40.447] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:8:0)
[21:29:40.466] Build status: STARTED
[21:29:40.466] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:9:0) after 7m28s
[21:37:08.490] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:9:0)
[21:37:08.555] Build status: STARTED
[21:37:08.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:10:0) after 8m40s
[21:45:48.578] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:10:0)
[21:45:48.595] Build status: STARTED
[21:45:48.595] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:11:0) after 2m33s
[21:48:21.665] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:11:0)
[21:48:21.694] Build status: STARTED
[21:48:21.694] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:12:0) after 1m57s
[21:50:18.700] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:12:0)
[21:50:18.716] Build status: STARTED
[21:50:18.716] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:13:0) after 7m49s
[21:58:07.784] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:13:0)
[21:58:07.800] Build status: STARTED
[21:58:07.800] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:14:0) after 2m32s
[22:00:39.820] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:14:0)
[22:00:39.837] Build status: STARTED
[22:00:39.837] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:15:0) after 6m27s
[22:07:07.119] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:15:0)
[22:07:07.139] Build status: STARTED
[22:07:07.139] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:16:0) after 2m36s
[22:09:43.199] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:16:0)
[22:09:43.218] Build status: STARTED
[22:09:43.218] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:17:0) after 9m10s
[22:18:53.235] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:17:0)
[22:18:53.256] Build status: STARTED
[22:18:53.256] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:18:0) after 9m1s
[22:27:54.300] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:18:0)
[22:27:54.319] Build status: STARTED
[22:27:54.319] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:19:0) after 6m27s
[22:34:21.391] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:19:0)
[22:34:21.417] Build status: STARTED
[22:34:21.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:20:0) after 6m20s
[22:40:41.441] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:20:0)
[22:40:41.459] Build status: STARTED
[22:40:41.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:21:0) after 2m2s
[22:42:43.484] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:21:0)
[22:42:43.504] Build status: STARTED
[22:42:43.504] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:22:0) after 8m51s
[22:51:34.550] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:22:0)
[22:51:34.581] Build status: STARTED
[22:51:34.581] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:23:0) after 4m49s
[22:56:23.655] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:23:0)
[22:56:23.676] Build status: STARTED
[22:56:23.676] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:24:0) after 4m34s
[23:00:57.733] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:24:0)
[23:00:57.751] Build status: STARTED
[23:00:57.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:25:0) after 7m56s
[23:08:53.809] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:25:0)
[23:08:53.827] Build status: STARTED
[23:08:53.827] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:26:0) after 1m43s
[23:10:36.895] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:26:0)
[23:10:36.923] Build status: STARTED
[23:10:36.923] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:27:0) after 9m8s
[23:19:45.018] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:27:0)
[23:19:45.070] Build status: STARTED
[23:19:45.070] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:28:0) after 8m48s
[23:28:33.091] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:28:0)
[23:28:33.153] Build status: STARTED
[23:28:33.153] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:29:0) after 5m42s
[23:34:15.172] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:29:0)
[23:34:15.188] Build status: STARTED
[23:34:15.188] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:30:0) after 8m24s
[23:42:39.212] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:30:0)
[23:42:39.243] Build status: STARTED
[23:42:39.243] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:31:0) after 9m7s
[23:51:46.284] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:31:0)
[23:51:46.347] Build status: STARTED
[23:51:46.347] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:32:0) after 9m20s
[00:01:06.468] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:32:0)
[00:01:06.484] Build status: STARTED
[00:01:06.484] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:33:0) after 6m10s
[00:07:16.601] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:33:0)
[00:07:16.635] Build status: STARTED
[00:07:16.635] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:34:0) after 2m38s
[00:09:54.752] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:34:0)
[00:09:54.770] Build status: STARTED
[00:09:54.770] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:35:0) after 8m59s
[00:18:53.835] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:35:0)
[00:18:53.850] Build status: STARTED
[00:18:53.850] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:36:0) after 4m26s
[00:23:19.869] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:36:0)
[00:23:19.941] Build status: STARTED
[00:23:19.941] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:37:0) after 9m58s
[00:33:18.028] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:37:0)
[00:33:18.048] Build status: STARTED
[00:33:18.048] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:38:0) after 9m49s
[00:43:07.210] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:38:0)
[00:43:07.232] Build status: STARTED
[00:43:07.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:39:0) after 6m54s
[00:50:01.258] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:39:0)
[00:50:01.318] Build status: STARTED
[00:50:01.318] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:40:0) after 4m22s
[00:54:23.351] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:40:0)
[00:54:23.370] Build status: STARTED
[00:54:23.370] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:41:0) after 2m55s
[00:57:18.406] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:41:0)
[00:57:18.427] Build status: STARTED
[00:57:18.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:42:0) after 5m17s
[01:02:35.472] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:42:0)
[01:02:35.504] Build status: STARTED
[01:02:35.504] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:43:0) after 5m49s
[01:08:24.554] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:43:0)
[01:08:24.577] Build status: STARTED
[01:08:24.577] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8851844765621547472:44:0) after 2m9s
[01:09:16.188] Received PubSub notification, asking Buildbucket for the build status
[01:09:16.207] Build:
{
"id": "8686414703620123969",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 31601,
"createdBy": "project:chromium",
"createTime": "2026-03-24T21:06:57.711676743Z",
"startTime": "2026-03-24T21:07:50.443745062Z",
"endTime": "2026-03-25T01:09:15.919743375Z",
"updateTime": "2026-03-25T01:09:15.919743375Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "628168700b8ce53f755f480319cb3ded0e5eed19",
"ref": "refs/heads/main"
}
}
}
[01:09:16.207] Invocation finished in 4h2m19.788870555s with status FAILED