[19:44:58.119] New invocation is queued and will start shortly
[19:44:59.212] Starting the invocation (attempt 1)
[19:44:59.261] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/690ef5b209556085d473f47f8b4720cb491ff238"
[19:44:59.262] Popped gitiles commit info from properties and tags
[19:44:59.262] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[19:44:59.262] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[19:44:59.262] Buildbucket request:
{
"requestId": "8972343888161436240",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "690ef5b209556085d473f47f8b4720cb491ff238",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8972343888161436240"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[19:44:59.803] Scheduled build:
{
"id": "8806913826212327505",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 16467,
"createdBy": "project:chromium",
"createTime": "2022-08-02T19:44:59.346005901Z",
"updateTime": "2022-08-02T19:44:59.346005901Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "690ef5b209556085d473f47f8b4720cb491ff238",
"ref": "refs/heads/main"
}
}
}
[19:44:59.803] Task URL: https://cr-buildbucket.appspot.com/build/8806913826212327505
[19:44:59.804] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:2:0) after 9m13s
[19:48:13.806] Received PubSub notification, asking Buildbucket for the build status
[19:48:13.864] Build status: STARTED
[19:54:12.869] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:2:0)
[19:54:12.901] Build status: STARTED
[19:54:12.901] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:4:0) after 6m54s
[20:01:07.062] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:4:0)
[20:01:07.102] Build status: STARTED
[20:01:07.102] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:5:0) after 1m44s
[20:02:51.394] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:5:0)
[20:02:51.509] Build status: STARTED
[20:02:51.509] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:6:0) after 7m55s
[20:10:46.695] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:6:0)
[20:10:46.741] Build status: STARTED
[20:10:46.741] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:7:0) after 6m50s
[20:17:37.330] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:7:0)
[20:17:37.362] Build status: STARTED
[20:17:37.362] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:8:0) after 6m14s
[20:23:51.487] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:8:0)
[20:23:51.531] Build status: STARTED
[20:23:51.531] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:9:0) after 9m43s
[20:33:34.689] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:9:0)
[20:33:34.723] Build status: STARTED
[20:33:34.723] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:10:0) after 4m2s
[20:37:36.791] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:10:0)
[20:37:36.813] Build status: STARTED
[20:37:36.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:11:0) after 1m39s
[20:39:15.884] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:11:0)
[20:39:15.937] Build status: STARTED
[20:39:15.937] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:12:0) after 7m47s
[20:47:02.958] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:12:0)
[20:47:02.981] Build status: STARTED
[20:47:02.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:13:0) after 3m8s
[20:50:11.003] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:13:0)
[20:50:11.055] Build status: STARTED
[20:50:11.055] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:14:0) after 4m46s
[20:54:57.083] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:14:0)
[20:54:57.119] Build status: STARTED
[20:54:57.119] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:15:0) after 2m12s
[20:57:09.189] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:15:0)
[20:57:09.221] Build status: STARTED
[20:57:09.221] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:16:0) after 6m54s
[21:04:03.441] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:16:0)
[21:04:03.464] Build status: STARTED
[21:04:03.464] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:17:0) after 7m27s
[21:11:30.695] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:17:0)
[21:11:30.720] Build status: STARTED
[21:11:30.720] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:18:0) after 6m16s
[21:17:47.068] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:18:0)
[21:17:47.089] Build status: STARTED
[21:17:47.089] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:19:0) after 5m24s
[21:23:11.177] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:19:0)
[21:23:11.221] Build status: STARTED
[21:23:11.221] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:20:0) after 6m23s
[21:29:34.327] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:20:0)
[21:29:34.352] Build status: STARTED
[21:29:34.352] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:21:0) after 6m51s
[21:36:25.380] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:21:0)
[21:36:25.449] Build status: STARTED
[21:36:25.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:22:0) after 4m47s
[21:41:12.440] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:22:0)
[21:41:12.466] Build status: STARTED
[21:41:12.466] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:23:0) after 1m24s
[21:42:36.492] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:23:0)
[21:42:36.512] Build status: STARTED
[21:42:36.512] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:24:0) after 7m3s
[21:49:39.490] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:24:0)
[21:49:39.511] Build status: STARTED
[21:49:39.511] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:25:0) after 2m55s
[21:52:34.545] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:25:0)
[21:52:34.567] Build status: STARTED
[21:52:34.567] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:26:0) after 4m5s
[21:56:39.695] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:26:0)
[21:56:39.715] Build status: STARTED
[21:56:39.715] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:27:0) after 5m16s
[22:01:56.040] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:27:0)
[22:01:56.061] Build status: STARTED
[22:01:56.061] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:28:0) after 2m50s
[22:04:46.082] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:28:0)
[22:04:46.105] Build status: STARTED
[22:04:46.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:29:0) after 1m32s
[22:06:18.214] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:29:0)
[22:06:18.252] Build status: STARTED
[22:06:18.252] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:30:0) after 4m30s
[22:10:48.397] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:30:0)
[22:10:48.446] Build status: STARTED
[22:10:48.446] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:31:0) after 6m19s
[22:17:07.916] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:31:0)
[22:17:07.954] Build status: STARTED
[22:17:07.954] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:32:0) after 9m44s
[22:26:52.015] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:32:0)
[22:26:52.064] Build status: STARTED
[22:26:52.064] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:33:0) after 3m49s
[22:30:41.318] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:33:0)
[22:30:41.358] Build status: STARTED
[22:30:41.358] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:34:0) after 2m48s
[22:33:29.392] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:34:0)
[22:33:29.417] Build status: STARTED
[22:33:29.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:35:0) after 5m59s
[22:39:28.545] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:35:0)
[22:39:28.567] Build status: STARTED
[22:39:28.568] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:36:0) after 8m6s
[22:47:34.604] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:36:0)
[22:47:34.651] Build status: STARTED
[22:47:34.651] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:37:0) after 9m57s
[22:57:31.666] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:37:0)
[22:57:31.707] Build status: STARTED
[22:57:31.707] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:38:0) after 4m41s
[23:02:12.733] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:38:0)
[23:02:13.201] Build status: STARTED
[23:02:13.201] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:39:0) after 5m47s
[23:08:00.375] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:39:0)
[23:08:00.398] Build status: STARTED
[23:08:00.398] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:40:0) after 2m20s
[23:10:20.521] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:40:0)
[23:10:20.584] Build status: STARTED
[23:10:20.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:41:0) after 9m33s
[23:19:53.605] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:41:0)
[23:19:53.630] Build status: STARTED
[23:19:53.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:42:0) after 3m19s
[23:23:12.651] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:42:0)
[23:23:12.677] Build status: STARTED
[23:23:12.677] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:43:0) after 9m42s
[23:32:54.706] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:43:0)
[23:32:54.735] Build status: STARTED
[23:32:54.735] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:44:0) after 1m42s
[23:34:36.978] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:44:0)
[23:34:37.007] Build status: STARTED
[23:34:37.007] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:45:0) after 1m2s
[23:35:39.087] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:45:0)
[23:35:39.206] Build status: STARTED
[23:35:39.206] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:46:0) after 7m31s
[23:43:10.324] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:46:0)
[23:43:10.348] Build status: STARTED
[23:43:10.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:47:0) after 3m39s
[23:46:49.443] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:47:0)
[23:46:49.481] Build status: STARTED
[23:46:49.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:48:0) after 1m39s
[23:48:28.913] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:48:0)
[23:48:28.939] Build status: STARTED
[23:48:28.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:49:0) after 9m34s
[23:58:03.070] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:49:0)
[23:58:03.111] Build status: STARTED
[23:58:03.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:50:0) after 5m8s
[00:03:11.309] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:50:0)
[00:03:11.339] Build status: STARTED
[00:03:11.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:51:0) after 5m44s
[00:08:55.398] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:51:0)
[00:08:55.435] Build status: STARTED
[00:08:55.435] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:52:0) after 5m17s
[00:14:12.623] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:52:0)
[00:14:12.656] Build status: STARTED
[00:14:12.656] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:53:0) after 9m46s
[00:23:58.738] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:53:0)
[00:23:58.760] Build status: STARTED
[00:23:58.760] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:54:0) after 4m42s
[00:28:40.824] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:54:0)
[00:28:40.846] Build status: STARTED
[00:28:40.846] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:55:0) after 1m7s
[00:29:47.871] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:55:0)
[00:29:47.907] Build status: STARTED
[00:29:47.907] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:56:0) after 3m40s
[00:33:28.165] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:56:0)
[00:33:28.201] Build status: STARTED
[00:33:28.201] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:57:0) after 6m54s
[00:40:22.224] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:57:0)
[00:40:22.248] Build status: STARTED
[00:40:22.248] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:58:0) after 1m2s
[00:41:24.369] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:58:0)
[00:41:24.395] Build status: STARTED
[00:41:24.395] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:59:0) after 8m57s
[00:50:21.500] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:59:0)
[00:50:21.659] Build status: STARTED
[00:50:21.659] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:60:0) after 9m25s
[00:59:46.682] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:60:0)
[00:59:46.706] Build status: STARTED
[00:59:46.706] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:61:0) after 1m35s
[01:01:21.806] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:61:0)
[01:01:21.830] Build status: STARTED
[01:01:21.830] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:62:0) after 5m8s
[01:06:29.881] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:62:0)
[01:06:29.904] Build status: STARTED
[01:06:29.904] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:63:0) after 7m18s
[01:13:48.203] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:63:0)
[01:13:48.245] Build status: STARTED
[01:13:48.245] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:64:0) after 2m32s
[01:16:20.264] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:64:0)
[01:16:20.289] Build status: STARTED
[01:16:20.289] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:65:0) after 4m55s
[01:21:15.388] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:65:0)
[01:21:15.414] Build status: STARTED
[01:21:15.414] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8972343888161436240:66:0) after 5m30s
[01:23:13.571] Received PubSub notification, asking Buildbucket for the build status
[01:23:13.591] Build:
{
"id": "8806913826212327505",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 16467,
"createdBy": "project:chromium",
"createTime": "2022-08-02T19:44:59.346005901Z",
"startTime": "2022-08-02T19:48:12.765579Z",
"endTime": "2022-08-03T01:23:13.255734429Z",
"updateTime": "2022-08-03T01:23:13.255734429Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "690ef5b209556085d473f47f8b4720cb491ff238",
"ref": "refs/heads/main"
}
}
}
[01:23:13.591] Invocation finished in 5h38m15.487958313s with status FAILED