[02:58:12.342] New invocation is queued and will start shortly
[02:58:13.437] Starting the invocation (attempt 1)
[02:58:13.462] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/09cc86a61509659b768d02f70ab52c6643890d99"
[02:58:13.462] Popped gitiles commit info from properties and tags
[02:58:13.462] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[02:58:13.462] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[02:58:13.462] Buildbucket request:
{
"requestId": "8923031881520236800",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "09cc86a61509659b768d02f70ab52c6643890d99",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8923031881520236800"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[02:58:14.127] Scheduled build:
{
"id": "8757601819620052417",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 21417,
"createdBy": "project:chromium",
"createTime": "2024-01-29T02:58:13.523080005Z",
"updateTime": "2024-01-29T02:58:13.523080005Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "09cc86a61509659b768d02f70ab52c6643890d99",
"ref": "refs/heads/main"
}
}
}
[02:58:14.127] Task URL: https://cr-buildbucket.appspot.com/build/8757601819620052417
[02:58:14.127] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:2:0) after 3m3s
[02:58:14.244] Received PubSub notification, asking Buildbucket for the build status
[02:58:14.272] Build status: SCHEDULED
[02:59:49.868] Received PubSub notification, asking Buildbucket for the build status
[02:59:49.895] Build status: STARTED
[03:01:17.334] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:2:0)
[03:01:17.466] Build status: STARTED
[03:01:17.466] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:5:0) after 8m57s
[03:10:14.565] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:5:0)
[03:10:14.597] Build status: STARTED
[03:10:14.598] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:6:0) after 2m11s
[03:12:25.636] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:6:0)
[03:12:25.669] Build status: STARTED
[03:12:25.672] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:7:0) after 8m46s
[03:21:11.691] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:7:0)
[03:21:11.871] Build status: STARTED
[03:21:11.872] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:8:0) after 5m29s
[03:26:40.889] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:8:0)
[03:26:40.906] Build status: STARTED
[03:26:40.906] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:9:0) after 8m23s
[03:35:04.039] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:9:0)
[03:35:04.075] Build status: STARTED
[03:35:04.075] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:10:0) after 3m32s
[03:38:36.091] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:10:0)
[03:38:36.117] Build status: STARTED
[03:38:36.117] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:11:0) after 1m59s
[03:40:35.158] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:11:0)
[03:40:35.177] Build status: STARTED
[03:40:35.177] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:12:0) after 4m21s
[03:44:56.322] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:12:0)
[03:44:56.341] Build status: STARTED
[03:44:56.341] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:13:0) after 4m6s
[03:49:02.361] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:13:0)
[03:49:02.382] Build status: STARTED
[03:49:02.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:14:0) after 7m42s
[03:56:44.398] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:14:0)
[03:56:44.422] Build status: STARTED
[03:56:44.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:15:0) after 9m9s
[04:05:53.480] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:15:0)
[04:05:53.557] Build status: STARTED
[04:05:53.557] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:16:0) after 8m38s
[04:14:31.592] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:16:0)
[04:14:31.610] Build status: STARTED
[04:14:31.610] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:17:0) after 6m54s
[04:21:25.718] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:17:0)
[04:21:25.739] Build status: STARTED
[04:21:25.739] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:18:0) after 8m8s
[04:29:34.026] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:18:0)
[04:29:34.045] Build status: STARTED
[04:29:34.046] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:19:0) after 3m22s
[04:32:56.063] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:19:0)
[04:32:56.125] Build status: STARTED
[04:32:56.125] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:20:0) after 2m57s
[04:35:53.101] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:20:0)
[04:35:53.185] Build status: STARTED
[04:35:53.185] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:21:0) after 6m22s
[04:42:15.176] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:21:0)
[04:42:15.193] Build status: STARTED
[04:42:15.193] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:22:0) after 3m47s
[04:46:02.209] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:22:0)
[04:46:02.245] Build status: STARTED
[04:46:02.245] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:23:0) after 1m59s
[04:48:01.274] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:23:0)
[04:48:01.324] Build status: STARTED
[04:48:01.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:24:0) after 7m0s
[04:55:01.342] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:24:0)
[04:55:01.367] Build status: STARTED
[04:55:01.367] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:25:0) after 4m14s
[04:59:15.517] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:25:0)
[04:59:15.535] Build status: STARTED
[04:59:15.535] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:26:0) after 2m13s
[05:01:28.562] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:26:0)
[05:01:28.588] Build status: STARTED
[05:01:28.588] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:27:0) after 3m56s
[05:05:24.604] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:27:0)
[05:05:24.653] Build status: STARTED
[05:05:24.653] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:28:0) after 9m17s
[05:14:41.715] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:28:0)
[05:14:41.734] Build status: STARTED
[05:14:41.734] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:29:0) after 7m29s
[05:22:10.760] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:29:0)
[05:22:10.781] Build status: STARTED
[05:22:10.781] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:30:0) after 8m50s
[05:31:00.802] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:30:0)
[05:31:00.819] Build status: STARTED
[05:31:00.819] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:31:0) after 9m5s
[05:40:05.855] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:31:0)
[05:40:05.873] Build status: STARTED
[05:40:05.873] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:32:0) after 8m47s
[05:48:52.889] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:32:0)
[05:48:52.910] Build status: STARTED
[05:48:52.910] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:33:0) after 2m41s
[05:51:34.114] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:33:0)
[05:51:34.141] Build status: STARTED
[05:51:34.141] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:34:0) after 9m39s
[06:01:13.216] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:34:0)
[06:01:13.261] Build status: STARTED
[06:01:13.261] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:35:0) after 2m36s
[06:03:49.379] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:35:0)
[06:03:49.398] Build status: STARTED
[06:03:49.398] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:36:0) after 7m47s
[06:11:36.450] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:36:0)
[06:11:36.474] Build status: STARTED
[06:11:36.474] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:37:0) after 5m39s
[06:17:15.526] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:37:0)
[06:17:15.579] Build status: STARTED
[06:17:15.579] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:38:0) after 7m50s
[06:25:05.654] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:38:0)
[06:25:05.680] Build status: STARTED
[06:25:05.680] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:39:0) after 3m24s
[06:28:29.940] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:39:0)
[06:28:30.013] Build status: STARTED
[06:28:30.013] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:40:0) after 6m23s
[06:34:53.079] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:40:0)
[06:34:53.097] Build status: STARTED
[06:34:53.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:41:0) after 1m42s
[06:36:35.112] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:41:0)
[06:36:35.131] Build status: STARTED
[06:36:35.131] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:42:0) after 6m32s
[06:43:07.148] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:42:0)
[06:43:07.185] Build status: STARTED
[06:43:07.185] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:43:0) after 4m52s
[06:47:59.292] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:43:0)
[06:47:59.310] Build status: STARTED
[06:47:59.310] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:44:0) after 7m35s
[06:55:34.328] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:44:0)
[06:55:34.346] Build status: STARTED
[06:55:34.346] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:45:0) after 8m0s
[07:03:34.461] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:45:0)
[07:03:34.481] Build status: STARTED
[07:03:34.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:46:0) after 1m46s
[07:05:20.541] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:46:0)
[07:05:20.567] Build status: STARTED
[07:05:20.567] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:47:0) after 5m26s
[07:10:46.683] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:47:0)
[07:10:46.704] Build status: STARTED
[07:10:46.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:48:0) after 6m2s
[07:16:48.893] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:48:0)
[07:16:48.934] Build status: STARTED
[07:16:48.934] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:49:0) after 9m31s
[07:26:19.994] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:49:0)
[07:26:20.012] Build status: STARTED
[07:26:20.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:50:0) after 2m50s
[07:29:10.160] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:50:0)
[07:29:10.176] Build status: STARTED
[07:29:10.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:51:0) after 6m14s
[07:35:24.200] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:51:0)
[07:35:24.220] Build status: STARTED
[07:35:24.220] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:52:0) after 7m40s
[07:43:04.348] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:52:0)
[07:43:04.365] Build status: STARTED
[07:43:04.365] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:53:0) after 6m17s
[07:49:21.490] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:53:0)
[07:49:21.510] Build status: STARTED
[07:49:21.511] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:54:0) after 1m30s
[07:50:51.591] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:54:0)
[07:50:51.613] Build status: STARTED
[07:50:51.613] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:55:0) after 5m34s
[07:56:25.635] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:55:0)
[07:56:25.652] Build status: STARTED
[07:56:25.652] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:56:0) after 9m11s
[08:05:36.679] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:56:0)
[08:05:36.700] Build status: STARTED
[08:05:36.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923031881520236800:57:0) after 9m1s
[08:07:00.392] Received PubSub notification, asking Buildbucket for the build status
[08:07:00.458] Build:
{
"id": "8757601819620052417",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 21417,
"createdBy": "project:chromium",
"createTime": "2024-01-29T02:58:13.523080005Z",
"startTime": "2024-01-29T02:59:49.383980Z",
"endTime": "2024-01-29T08:06:59.713624Z",
"updateTime": "2024-01-29T08:06:59.713624Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "09cc86a61509659b768d02f70ab52c6643890d99",
"ref": "refs/heads/main"
}
}
}
[08:07:00.458] Invocation finished in 5h8m48.130291576s with status FAILED