[11:12:16.897] New invocation is queued and will start shortly
[11:12:17.977] Starting the invocation (attempt 1)
[11:12:18.005] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/e7369b7a0b27454ee594c7a6f6cc58b0499d8a84"
[11:12:18.005] Popped gitiles commit info from properties and tags
[11:12:18.005] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[11:12:18.005] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[11:12:18.005] Buildbucket request:
{
"requestId": "8916387218404315136",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e7369b7a0b27454ee594c7a6f6cc58b0499d8a84",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8916387218404315136"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[11:12:18.597] Scheduled build:
{
"id": "8750957156514648321",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 22357,
"createdBy": "project:chromium",
"createTime": "2024-04-11T11:12:18.068510907Z",
"updateTime": "2024-04-11T11:12:18.068510907Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e7369b7a0b27454ee594c7a6f6cc58b0499d8a84",
"ref": "refs/heads/main"
}
}
}
[11:12:18.597] Task URL: https://cr-buildbucket.appspot.com/build/8750957156514648321
[11:12:18.597] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:2:0) after 6m35s
[11:12:19.552] Received PubSub notification, asking Buildbucket for the build status
[11:12:19.572] Build status: SCHEDULED
[11:14:07.546] Received PubSub notification, asking Buildbucket for the build status
[11:14:07.594] Build status: STARTED
[11:18:53.678] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:2:0)
[11:18:53.701] Build status: STARTED
[11:18:53.701] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:5:0) after 5m8s
[11:24:01.740] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:5:0)
[11:24:01.769] Build status: STARTED
[11:24:01.769] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:6:0) after 1m11s
[11:25:12.858] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:6:0)
[11:25:12.942] Build status: STARTED
[11:25:12.942] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:7:0) after 9m55s
[11:35:07.968] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:7:0)
[11:35:07.989] Build status: STARTED
[11:35:07.989] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:8:0) after 7m6s
[11:42:14.203] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:8:0)
[11:42:14.228] Build status: STARTED
[11:42:14.228] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:9:0) after 9m12s
[11:51:26.254] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:9:0)
[11:51:26.273] Build status: STARTED
[11:51:26.273] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:10:0) after 1m12s
[11:52:38.378] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:10:0)
[11:52:38.396] Build status: STARTED
[11:52:38.396] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:11:0) after 1m0s
[11:53:38.493] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:11:0)
[11:53:38.605] Build status: STARTED
[11:53:38.605] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:12:0) after 3m16s
[11:56:54.705] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:12:0)
[11:56:54.734] Build status: STARTED
[11:56:54.734] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:13:0) after 6m48s
[12:03:42.803] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:13:0)
[12:03:42.910] Build status: STARTED
[12:03:42.910] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:14:0) after 2m53s
[12:06:36.177] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:14:0)
[12:06:37.556] Build status: STARTED
[12:06:37.556] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:15:0) after 2m22s
[12:08:59.661] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:15:0)
[12:08:59.678] Build status: STARTED
[12:08:59.678] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:16:0) after 5m11s
[12:14:10.731] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:16:0)
[12:14:10.777] Build status: STARTED
[12:14:10.777] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:17:0) after 3m38s
[12:17:48.871] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:17:0)
[12:17:48.894] Build status: STARTED
[12:17:48.894] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:18:0) after 7m10s
[12:24:58.937] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:18:0)
[12:24:58.974] Build status: STARTED
[12:24:58.974] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:19:0) after 9m56s
[12:34:55.009] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:19:0)
[12:34:55.032] Build status: STARTED
[12:34:55.032] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:20:0) after 8m28s
[12:43:23.366] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:20:0)
[12:43:23.391] Build status: STARTED
[12:43:23.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:21:0) after 7m53s
[12:51:16.546] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:21:0)
[12:51:16.563] Build status: STARTED
[12:51:16.563] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:22:0) after 3m54s
[12:55:11.202] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:22:0)
[12:55:11.263] Build status: STARTED
[12:55:11.263] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:23:0) after 6m22s
[13:01:33.509] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:23:0)
[13:01:33.540] Build status: STARTED
[13:01:33.540] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:24:0) after 2m13s
[13:03:46.755] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:24:0)
[13:03:46.848] Build status: STARTED
[13:03:46.848] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:25:0) after 4m24s
[13:08:11.113] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:25:0)
[13:08:11.139] Build status: STARTED
[13:08:11.139] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:26:0) after 1m23s
[13:09:34.285] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:26:0)
[13:09:34.312] Build status: STARTED
[13:09:34.316] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:27:0) after 6m47s
[13:16:21.475] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:27:0)
[13:16:21.500] Build status: STARTED
[13:16:21.500] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:28:0) after 7m18s
[13:23:39.617] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:28:0)
[13:23:39.640] Build status: STARTED
[13:23:39.640] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:29:0) after 4m48s
[13:28:27.763] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:29:0)
[13:28:27.785] Build status: STARTED
[13:28:27.786] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:30:0) after 2m30s
[13:30:57.936] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:30:0)
[13:30:58.103] Build status: STARTED
[13:30:58.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:31:0) after 3m9s
[13:34:07.416] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:31:0)
[13:34:07.498] Build status: STARTED
[13:34:07.498] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:32:0) after 2m55s
[13:37:02.634] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:32:0)
[13:37:02.652] Build status: STARTED
[13:37:02.652] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:33:0) after 2m12s
[13:39:14.774] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:33:0)
[13:39:14.978] Build status: STARTED
[13:39:14.979] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:34:0) after 7m45s
[13:47:00.062] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:34:0)
[13:47:00.100] Build status: STARTED
[13:47:00.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:35:0) after 7m45s
[13:54:45.232] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:35:0)
[13:54:45.295] Build status: STARTED
[13:54:45.295] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:36:0) after 4m46s
[13:59:31.501] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:36:0)
[13:59:31.522] Build status: STARTED
[13:59:31.522] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:37:0) after 7m22s
[14:06:53.682] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:37:0)
[14:06:53.713] Build status: STARTED
[14:06:53.713] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:38:0) after 2m35s
[14:09:29.000] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:38:0)
[14:09:29.270] Build status: STARTED
[14:09:29.270] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:39:0) after 2m13s
[14:11:42.274] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:39:0)
[14:11:42.297] Build status: STARTED
[14:11:42.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:40:0) after 2m19s
[14:14:01.628] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:40:0)
[14:14:01.651] Build status: STARTED
[14:14:01.651] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:41:0) after 1m44s
[14:15:45.902] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:41:0)
[14:15:45.920] Build status: STARTED
[14:15:45.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:42:0) after 9m21s
[14:25:06.979] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:42:0)
[14:25:07.053] Build status: STARTED
[14:25:07.053] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:43:0) after 3m40s
[14:28:47.114] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:43:0)
[14:28:47.151] Build status: STARTED
[14:28:47.151] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:44:0) after 2m45s
[14:31:32.279] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:44:0)
[14:31:32.297] Build status: STARTED
[14:31:32.297] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:45:0) after 8m13s
[14:39:45.407] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:45:0)
[14:39:45.605] Build status: STARTED
[14:39:45.605] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:46:0) after 5m29s
[14:45:14.753] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:46:0)
[14:45:14.873] Build status: STARTED
[14:45:14.874] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:47:0) after 5m50s
[14:51:05.428] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:47:0)
[14:51:05.472] Build status: STARTED
[14:51:05.472] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:48:0) after 6m44s
[14:57:49.493] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:48:0)
[14:57:49.587] Build status: STARTED
[14:57:49.587] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:49:0) after 1m24s
[14:59:13.971] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:49:0)
[14:59:13.995] Build status: STARTED
[14:59:13.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:50:0) after 5m36s
[15:04:50.084] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:50:0)
[15:04:50.104] Build status: STARTED
[15:04:50.104] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:51:0) after 1m37s
[15:06:27.489] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:51:0)
[15:06:27.521] Build status: STARTED
[15:06:27.521] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:52:0) after 5m42s
[15:12:09.522] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:52:0)
[15:12:09.544] Build status: STARTED
[15:12:09.544] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:53:0) after 2m59s
[15:15:08.615] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:53:0)
[15:15:08.665] Build status: STARTED
[15:15:08.665] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:54:0) after 1m10s
[15:16:18.721] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:54:0)
[15:16:18.749] Build status: STARTED
[15:16:18.749] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:55:0) after 6m22s
[15:22:40.820] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:55:0)
[15:22:41.351] Build status: STARTED
[15:22:41.351] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:56:0) after 3m44s
[15:26:25.374] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:56:0)
[15:26:25.394] Build status: STARTED
[15:26:25.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:57:0) after 3m1s
[15:29:26.466] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:57:0)
[15:29:26.581] Build status: STARTED
[15:29:26.581] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:58:0) after 7m49s
[15:37:15.823] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:58:0)
[15:37:15.853] Build status: STARTED
[15:37:15.853] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:59:0) after 4m40s
[15:41:55.986] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:59:0)
[15:41:56.005] Build status: STARTED
[15:41:56.005] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:60:0) after 6m29s
[15:48:25.897] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:60:0)
[15:48:25.984] Build status: STARTED
[15:48:25.984] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:61:0) after 5m29s
[15:53:55.109] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:61:0)
[15:53:55.139] Build status: STARTED
[15:53:55.139] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8916387218404315136:62:0) after 3m18s
[15:55:25.292] Received PubSub notification, asking Buildbucket for the build status
[15:55:25.319] Build:
{
"id": "8750957156514648321",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 22357,
"createdBy": "project:chromium",
"createTime": "2024-04-11T11:12:18.068510907Z",
"startTime": "2024-04-11T11:14:07.259565514Z",
"endTime": "2024-04-11T15:55:24.147815577Z",
"updateTime": "2024-04-11T15:55:24.147815577Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e7369b7a0b27454ee594c7a6f6cc58b0499d8a84",
"ref": "refs/heads/main"
}
}
}
[15:55:25.319] Invocation finished in 4h43m8.436980153s with status FAILED