[18:21:59.099] New invocation is queued and will start shortly
[18:22:00.213] Starting the invocation (attempt 1)
[18:22:00.249] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/2413f227ee53feca65029b75bb59c184ad09a228"
[18:22:00.249] Popped gitiles commit info from properties and tags
[18:22:00.249] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[18:22:00.249] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[18:22:00.249] Buildbucket request:
{
"requestId": "8922973762353835728",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2413f227ee53feca65029b75bb59c184ad09a228",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8922973762353835728"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[18:22:00.988] Scheduled build:
{
"id": "8757543700410588017",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 21420,
"createdBy": "project:chromium",
"createTime": "2024-01-29T18:22:00.321955483Z",
"updateTime": "2024-01-29T18:22:00.321955483Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2413f227ee53feca65029b75bb59c184ad09a228",
"ref": "refs/heads/main"
}
}
}
[18:22:00.989] Task URL: https://cr-buildbucket.appspot.com/build/8757543700410588017
[18:22:00.989] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:2:0) after 3m34s
[18:22:01.823] Received PubSub notification, asking Buildbucket for the build status
[18:22:01.844] Build status: SCHEDULED
[18:23:28.026] Received PubSub notification, asking Buildbucket for the build status
[18:23:28.302] Build status: STARTED
[18:25:35.021] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:2:0)
[18:25:35.055] Build status: STARTED
[18:25:35.055] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:5:0) after 6m59s
[18:32:34.088] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:5:0)
[18:32:34.174] Build status: STARTED
[18:32:34.174] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:6:0) after 6m2s
[18:38:36.252] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:6:0)
[18:38:36.279] Build status: STARTED
[18:38:36.280] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:7:0) after 9m22s
[18:47:58.312] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:7:0)
[18:47:58.336] Build status: STARTED
[18:47:58.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:8:0) after 9m12s
[18:57:10.403] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:8:0)
[18:57:10.423] Build status: STARTED
[18:57:10.423] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:9:0) after 8m10s
[19:05:20.447] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:9:0)
[19:05:20.517] Build status: STARTED
[19:05:20.517] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:10:0) after 2m27s
[19:07:47.569] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:10:0)
[19:07:47.593] Build status: STARTED
[19:07:47.593] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:11:0) after 3m49s
[19:11:36.672] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:11:0)
[19:11:36.692] Build status: STARTED
[19:11:36.692] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:12:0) after 2m47s
[19:14:23.724] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:12:0)
[19:14:23.747] Build status: STARTED
[19:14:23.747] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:13:0) after 5m23s
[19:19:46.773] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:13:0)
[19:19:46.835] Build status: STARTED
[19:19:46.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:14:0) after 2m1s
[19:21:47.931] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:14:0)
[19:21:47.962] Build status: STARTED
[19:21:47.962] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:15:0) after 2m25s
[19:24:12.985] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:15:0)
[19:24:13.019] Build status: STARTED
[19:24:13.019] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:16:0) after 5m41s
[19:29:54.042] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:16:0)
[19:29:54.063] Build status: STARTED
[19:29:54.063] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:17:0) after 7m23s
[19:37:17.071] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:17:0)
[19:37:17.103] Build status: STARTED
[19:37:17.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:18:0) after 7m17s
[19:44:34.210] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:18:0)
[19:44:34.229] Build status: STARTED
[19:44:34.229] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:19:0) after 1m10s
[19:45:44.251] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:19:0)
[19:45:44.271] Build status: STARTED
[19:45:44.271] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:20:0) after 6m6s
[19:51:50.362] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:20:0)
[19:51:50.459] Build status: STARTED
[19:51:50.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:21:0) after 1m15s
[19:53:05.535] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:21:0)
[19:53:05.560] Build status: STARTED
[19:53:05.560] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:22:0) after 9m21s
[20:02:26.582] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:22:0)
[20:02:26.608] Build status: STARTED
[20:02:26.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:23:0) after 5m18s
[20:07:44.817] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:23:0)
[20:07:44.837] Build status: STARTED
[20:07:44.837] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:24:0) after 2m8s
[20:09:52.864] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:24:0)
[20:09:52.888] Build status: STARTED
[20:09:52.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:25:0) after 6m24s
[20:16:16.931] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:25:0)
[20:16:16.947] Build status: STARTED
[20:16:16.947] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:26:0) after 5m19s
[20:21:36.089] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:26:0)
[20:21:36.116] Build status: STARTED
[20:21:36.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:27:0) after 8m4s
[20:29:40.138] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:27:0)
[20:29:40.203] Build status: STARTED
[20:29:40.203] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:28:0) after 2m11s
[20:31:51.293] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:28:0)
[20:31:51.311] Build status: STARTED
[20:31:51.311] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:29:0) after 9m8s
[20:40:59.329] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:29:0)
[20:40:59.348] Build status: STARTED
[20:40:59.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:30:0) after 1m4s
[20:42:03.371] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:30:0)
[20:42:03.391] Build status: STARTED
[20:42:03.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:31:0) after 7m44s
[20:49:47.434] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:31:0)
[20:49:47.464] Build status: STARTED
[20:49:47.464] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:32:0) after 3m25s
[20:53:12.499] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:32:0)
[20:53:12.515] Build status: STARTED
[20:53:12.515] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:33:0) after 1m10s
[20:54:22.683] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:33:0)
[20:54:22.703] Build status: STARTED
[20:54:22.703] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:34:0) after 2m53s
[20:57:15.790] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:34:0)
[20:57:15.809] Build status: STARTED
[20:57:15.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:35:0) after 9m2s
[21:06:17.873] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:35:0)
[21:06:17.891] Build status: STARTED
[21:06:17.891] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:36:0) after 5m28s
[21:11:45.936] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:36:0)
[21:11:45.972] Build status: STARTED
[21:11:45.972] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:37:0) after 4m41s
[21:16:27.001] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:37:0)
[21:16:27.025] Build status: STARTED
[21:16:27.025] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:38:0) after 8m15s
[21:24:42.046] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:38:0)
[21:24:42.065] Build status: STARTED
[21:24:42.065] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:39:0) after 7m9s
[21:31:51.432] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:39:0)
[21:31:51.497] Build status: STARTED
[21:31:51.497] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:40:0) after 3m3s
[21:34:54.800] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:40:0)
[21:34:54.821] Build status: STARTED
[21:34:54.821] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:41:0) after 3m40s
[21:38:34.855] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:41:0)
[21:38:34.872] Build status: STARTED
[21:38:34.872] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:42:0) after 4m19s
[21:42:53.900] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:42:0)
[21:42:53.920] Build status: STARTED
[21:42:53.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:43:0) after 3m35s
[21:46:29.044] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:43:0)
[21:46:29.064] Build status: STARTED
[21:46:29.064] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:44:0) after 8m59s
[21:55:28.084] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:44:0)
[21:55:28.129] Build status: STARTED
[21:55:28.129] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:45:0) after 1m1s
[21:56:29.149] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:45:0)
[21:56:29.169] Build status: STARTED
[21:56:29.169] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:46:0) after 5m37s
[22:02:06.158] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:46:0)
[22:02:06.182] Build status: STARTED
[22:02:06.182] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:47:0) after 9m2s
[22:11:08.253] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:47:0)
[22:11:08.269] Build status: STARTED
[22:11:08.269] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:48:0) after 1m5s
[22:12:13.297] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:48:0)
[22:12:13.313] Build status: STARTED
[22:12:13.314] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:49:0) after 5m36s
[22:17:49.515] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:49:0)
[22:17:49.663] Build status: STARTED
[22:17:49.663] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:50:0) after 6m40s
[22:24:29.710] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:50:0)
[22:24:29.758] Build status: STARTED
[22:24:29.758] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:51:0) after 2m10s
[22:26:39.943] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:51:0)
[22:26:39.958] Build status: STARTED
[22:26:39.958] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:52:0) after 1m29s
[22:28:08.992] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:52:0)
[22:28:09.059] Build status: STARTED
[22:28:09.059] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:53:0) after 5m41s
[22:33:50.137] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:53:0)
[22:33:50.159] Build status: STARTED
[22:33:50.159] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:54:0) after 6m14s
[22:40:04.159] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:54:0)
[22:40:04.183] Build status: STARTED
[22:40:04.183] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:55:0) after 2m35s
[22:42:39.254] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:55:0)
[22:42:39.278] Build status: STARTED
[22:42:39.278] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:56:0) after 3m14s
[22:45:53.300] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:56:0)
[22:45:53.327] Build status: STARTED
[22:45:53.327] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:57:0) after 1m0s
[22:46:53.369] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:57:0)
[22:46:53.388] Build status: STARTED
[22:46:53.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:58:0) after 1m15s
[22:48:08.414] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:58:0)
[22:48:08.444] Build status: STARTED
[22:48:08.444] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:59:0) after 5m35s
[22:53:43.563] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:59:0)
[22:53:43.586] Build status: STARTED
[22:53:43.586] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:60:0) after 8m58s
[23:02:41.609] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:60:0)
[23:02:41.704] Build status: STARTED
[23:02:41.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:61:0) after 2m32s
[23:05:13.963] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:61:0)
[23:05:13.985] Build status: STARTED
[23:05:13.985] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:62:0) after 8m17s
[23:13:31.013] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:62:0)
[23:13:31.033] Build status: STARTED
[23:13:31.033] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:63:0) after 2m58s
[23:16:29.040] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:63:0)
[23:16:29.056] Build status: STARTED
[23:16:29.056] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:64:0) after 8m54s
[23:25:23.144] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:64:0)
[23:25:23.165] Build status: STARTED
[23:25:23.165] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:65:0) after 1m37s
[23:27:00.271] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:65:0)
[23:27:00.294] Build status: STARTED
[23:27:00.294] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8922973762353835728:66:0) after 7m51s
[23:29:37.917] Received PubSub notification, asking Buildbucket for the build status
[23:29:37.943] Build:
{
"id": "8757543700410588017",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 21420,
"createdBy": "project:chromium",
"createTime": "2024-01-29T18:22:00.321955483Z",
"startTime": "2024-01-29T18:23:27.272141Z",
"endTime": "2024-01-29T23:29:37.254517Z",
"updateTime": "2024-01-29T23:29:37.254517Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2413f227ee53feca65029b75bb59c184ad09a228",
"ref": "refs/heads/main"
}
}
}
[23:29:37.943] Invocation finished in 5h7m38.860166759s with status FAILED