[21:39:09.346] New invocation is queued and will start shortly
[21:39:10.415] Starting the invocation (attempt 1)
[21:39:10.442] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d0a14bcc9b533dd4b1953053de9644065531b3fb"
[21:39:10.442] Popped gitiles commit info from properties and tags
[21:39:10.442] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:39:10.442] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:39:10.443] Buildbucket request:
{
"requestId": "8923051954397650000",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d0a14bcc9b533dd4b1953053de9644065531b3fb",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8923051954397650000"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxMSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:39:11.451] Scheduled build:
{
"id": "8757621892512351905",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 21416,
"createdBy": "project:chromium",
"createTime": "2024-01-28T21:39:10.521922890Z",
"updateTime": "2024-01-28T21:39:10.521922890Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d0a14bcc9b533dd4b1953053de9644065531b3fb",
"ref": "refs/heads/main"
}
}
}
[21:39:11.451] Task URL: https://cr-buildbucket.appspot.com/build/8757621892512351905
[21:39:11.452] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:2:0) after 6m23s
[21:39:17.705] Received PubSub notification, asking Buildbucket for the build status
[21:39:17.721] Build status: SCHEDULED
[21:45:34.473] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:2:0)
[21:45:34.506] Build status: SCHEDULED
[21:45:34.506] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:4:0) after 4m3s
[21:46:04.614] Received PubSub notification, asking Buildbucket for the build status
[21:46:04.652] Build status: STARTED
[21:49:37.523] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:4:0)
[21:49:37.552] Build status: STARTED
[21:49:37.552] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:6:0) after 8m12s
[21:57:49.708] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:6:0)
[21:57:49.729] Build status: STARTED
[21:57:49.729] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:7:0) after 4m45s
[22:02:34.745] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:7:0)
[22:02:34.785] Build status: STARTED
[22:02:34.785] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:8:0) after 8m21s
[22:10:55.803] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:8:0)
[22:10:55.851] Build status: STARTED
[22:10:55.851] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:9:0) after 3m1s
[22:13:57.003] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:9:0)
[22:13:57.105] Build status: STARTED
[22:13:57.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:10:0) after 1m38s
[22:15:35.123] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:10:0)
[22:15:35.287] Build status: STARTED
[22:15:35.287] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:11:0) after 5m11s
[22:20:46.304] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:11:0)
[22:20:46.322] Build status: STARTED
[22:20:46.322] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:12:0) after 3m18s
[22:24:04.349] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:12:0)
[22:24:04.398] Build status: STARTED
[22:24:04.398] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:13:0) after 7m13s
[22:31:17.437] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:13:0)
[22:31:17.455] Build status: STARTED
[22:31:17.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:14:0) after 3m5s
[22:34:22.470] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:14:0)
[22:34:22.490] Build status: STARTED
[22:34:22.491] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:15:0) after 2m16s
[22:36:38.594] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:15:0)
[22:36:38.622] Build status: STARTED
[22:36:38.622] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:16:0) after 6m53s
[22:43:31.643] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:16:0)
[22:43:31.660] Build status: STARTED
[22:43:31.660] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:17:0) after 2m17s
[22:45:48.708] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:17:0)
[22:45:48.734] Build status: STARTED
[22:45:48.734] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:18:0) after 9m27s
[22:55:15.971] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:18:0)
[22:55:15.995] Build status: STARTED
[22:55:15.996] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:19:0) after 8m12s
[23:03:28.129] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:19:0)
[23:03:28.266] Build status: STARTED
[23:03:28.266] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:20:0) after 2m0s
[23:05:28.486] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:20:0)
[23:05:28.598] Build status: STARTED
[23:05:28.598] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:21:0) after 3m45s
[23:09:13.624] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:21:0)
[23:09:13.772] Build status: STARTED
[23:09:13.772] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:22:0) after 8m28s
[23:17:41.963] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:22:0)
[23:17:41.980] Build status: STARTED
[23:17:41.980] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:23:0) after 3m12s
[23:20:54.124] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:23:0)
[23:20:54.143] Build status: STARTED
[23:20:54.143] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:24:0) after 2m52s
[23:23:46.175] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:24:0)
[23:23:46.200] Build status: STARTED
[23:23:46.200] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:25:0) after 7m57s
[23:31:43.321] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:25:0)
[23:31:43.344] Build status: STARTED
[23:31:43.344] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:26:0) after 7m26s
[23:39:09.372] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:26:0)
[23:39:09.394] Build status: STARTED
[23:39:09.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:27:0) after 2m19s
[23:41:28.420] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:27:0)
[23:41:28.449] Build status: STARTED
[23:41:28.449] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:28:0) after 1m0s
[23:42:28.520] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:28:0)
[23:42:28.576] Build status: STARTED
[23:42:28.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:29:0) after 6m30s
[23:48:58.593] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:29:0)
[23:48:58.618] Build status: STARTED
[23:48:58.618] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:30:0) after 5m52s
[23:54:50.637] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:30:0)
[23:54:50.656] Build status: STARTED
[23:54:50.656] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:31:0) after 7m2s
[00:01:52.692] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:31:0)
[00:01:52.766] Build status: STARTED
[00:01:52.766] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:32:0) after 4m21s
[00:06:13.801] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:32:0)
[00:06:13.825] Build status: STARTED
[00:06:13.825] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:33:0) after 5m23s
[00:11:36.843] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:33:0)
[00:11:36.864] Build status: STARTED
[00:11:36.864] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:34:0) after 1m38s
[00:13:14.893] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:34:0)
[00:13:14.917] Build status: STARTED
[00:13:14.917] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:35:0) after 8m16s
[00:21:30.946] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:35:0)
[00:21:30.988] Build status: STARTED
[00:21:30.989] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:36:0) after 2m51s
[00:24:22.013] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:36:0)
[00:24:22.039] Build status: STARTED
[00:24:22.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:37:0) after 4m25s
[00:28:47.302] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:37:0)
[00:28:47.320] Build status: STARTED
[00:28:47.320] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:38:0) after 6m23s
[00:35:10.609] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:38:0)
[00:35:10.695] Build status: STARTED
[00:35:10.695] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:39:0) after 6m1s
[00:41:11.726] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:39:0)
[00:41:11.749] Build status: STARTED
[00:41:11.749] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:40:0) after 1m32s
[00:42:43.935] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:40:0)
[00:42:43.952] Build status: STARTED
[00:42:43.952] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:41:0) after 6m28s
[00:49:12.156] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:41:0)
[00:49:12.189] Build status: STARTED
[00:49:12.189] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:42:0) after 7m6s
[00:56:18.329] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:42:0)
[00:56:18.348] Build status: STARTED
[00:56:18.348] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:43:0) after 6m25s
[01:02:43.378] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:43:0)
[01:02:43.399] Build status: STARTED
[01:02:43.399] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:44:0) after 6m34s
[01:09:17.413] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:44:0)
[01:09:17.442] Build status: STARTED
[01:09:17.442] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:45:0) after 3m10s
[01:12:27.535] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:45:0)
[01:12:27.565] Build status: STARTED
[01:12:27.565] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:46:0) after 8m9s
[01:20:36.580] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:46:0)
[01:20:36.607] Build status: STARTED
[01:20:36.607] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:47:0) after 9m1s
[01:29:37.701] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:47:0)
[01:29:37.717] Build status: STARTED
[01:29:37.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:48:0) after 7m7s
[01:36:44.839] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:48:0)
[01:36:44.860] Build status: STARTED
[01:36:44.860] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:49:0) after 8m1s
[01:44:45.965] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:49:0)
[01:44:45.995] Build status: STARTED
[01:44:45.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:50:0) after 9m43s
[01:54:29.013] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:50:0)
[01:54:29.037] Build status: STARTED
[01:54:29.037] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:51:0) after 7m12s
[02:01:41.056] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:51:0)
[02:01:41.199] Build status: STARTED
[02:01:41.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:52:0) after 2m43s
[02:04:24.344] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:52:0)
[02:04:24.406] Build status: STARTED
[02:04:24.406] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:53:0) after 9m21s
[02:13:45.578] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:53:0)
[02:13:45.634] Build status: STARTED
[02:13:45.635] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:54:0) after 4m4s
[02:17:49.650] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:54:0)
[02:17:49.671] Build status: STARTED
[02:17:49.671] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:55:0) after 9m55s
[02:27:44.688] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:55:0)
[02:27:44.707] Build status: STARTED
[02:27:44.707] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:56:0) after 7m55s
[02:35:39.830] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:56:0)
[02:35:39.861] Build status: STARTED
[02:35:39.861] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:57:0) after 2m46s
[02:38:25.879] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:57:0)
[02:38:25.894] Build status: STARTED
[02:38:25.894] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:58:0) after 3m49s
[02:42:14.917] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:58:0)
[02:42:15.150] Build status: STARTED
[02:42:15.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:59:0) after 8m56s
[02:51:11.196] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:59:0)
[02:51:11.384] Build status: STARTED
[02:51:11.384] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:60:0) after 2m33s
[02:53:44.402] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:60:0)
[02:53:44.419] Build status: STARTED
[02:53:44.420] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxMSan:8923051954397650000:61:0) after 7m46s
[02:58:10.438] Received PubSub notification, asking Buildbucket for the build status
[02:58:10.507] Build:
{
"id": "8757621892512351905",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxMSan"
},
"number": 21416,
"createdBy": "project:chromium",
"createTime": "2024-01-28T21:39:10.521922890Z",
"startTime": "2024-01-28T21:46:03.819217Z",
"endTime": "2024-01-29T02:58:09.983081Z",
"updateTime": "2024-01-29T02:58:09.983081Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d0a14bcc9b533dd4b1953053de9644065531b3fb",
"ref": "refs/heads/main"
}
}
}
[02:58:10.507] Invocation finished in 5h19m1.166404607s with status FAILED