[04:00:07.345] New invocation is queued and will start shortly
[04:00:08.825] Starting the invocation (attempt 1)
[04:00:08.857] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/e80537f37d2f23f63a8a1718a9b5780494ed5b99"
[04:00:08.857] Popped gitiles commit info from properties and tags
[04:00:08.857] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:00:08.857] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:00:08.857] Buildbucket request:
{
"requestId": "8869756969826622864",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-code-coverage"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e80537f37d2f23f63a8a1718a9b5780494ed5b99",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8869756969826622864"
},
{
"key": "scheduler_job_id",
"value": "chromium/mac-code-coverage"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:00:09.396] Scheduled build:
{
"id": "8704326907507100913",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-code-coverage"
},
"number": 10253,
"createdBy": "project:chromium",
"createTime": "2025-09-08T04:00:08.914452901Z",
"updateTime": "2025-09-08T04:00:08.914452901Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e80537f37d2f23f63a8a1718a9b5780494ed5b99",
"ref": "refs/heads/main"
}
}
}
[04:00:09.396] Task URL: https://cr-buildbucket.appspot.com/build/8704326907507100913
[04:00:09.396] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:2:0) after 9m4s
[04:00:10.462] Received PubSub notification, asking Buildbucket for the build status
[04:00:10.479] Build status: SCHEDULED
[04:00:16.573] Received PubSub notification, asking Buildbucket for the build status
[04:00:16.606] Build status: STARTED
[04:09:13.413] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:2:0)
[04:09:13.429] Build status: STARTED
[04:09:13.429] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:5:0) after 2m32s
[04:11:45.521] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:5:0)
[04:11:45.546] Build status: STARTED
[04:11:45.546] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:6:0) after 9m11s
[04:20:56.604] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:6:0)
[04:20:56.616] Build status: STARTED
[04:20:56.616] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:7:0) after 4m33s
[04:25:29.633] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:7:0)
[04:25:29.664] Build status: STARTED
[04:25:29.664] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:8:0) after 7m40s
[04:33:09.681] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:8:0)
[04:33:09.706] Build status: STARTED
[04:33:09.707] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:9:0) after 7m56s
[04:41:05.746] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:9:0)
[04:41:05.866] Build status: STARTED
[04:41:05.866] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:10:0) after 5m9s
[04:46:14.885] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:10:0)
[04:46:14.965] Build status: STARTED
[04:46:14.965] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:11:0) after 7m55s
[04:54:09.980] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:11:0)
[04:54:09.995] Build status: STARTED
[04:54:09.995] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:12:0) after 5m47s
[04:59:57.014] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:12:0)
[04:59:57.039] Build status: STARTED
[04:59:57.039] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:13:0) after 4m44s
[05:04:41.121] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:13:0)
[05:04:41.139] Build status: STARTED
[05:04:41.139] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:14:0) after 1m59s
[05:06:40.750] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:14:0)
[05:06:40.769] Build status: STARTED
[05:06:40.769] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:15:0) after 8m53s
[05:15:33.902] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:15:0)
[05:15:33.918] Build status: STARTED
[05:15:33.918] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:16:0) after 1m38s
[05:17:11.940] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:16:0)
[05:17:11.986] Build status: STARTED
[05:17:11.986] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:17:0) after 4m59s
[05:22:11.171] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:17:0)
[05:22:11.189] Build status: STARTED
[05:22:11.189] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:18:0) after 1m46s
[05:23:57.220] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:18:0)
[05:23:57.238] Build status: STARTED
[05:23:57.238] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:19:0) after 6m8s
[05:30:05.660] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:19:0)
[05:30:05.679] Build status: STARTED
[05:30:05.679] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:20:0) after 9m34s
[05:39:39.822] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:20:0)
[05:39:39.869] Build status: STARTED
[05:39:39.869] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:21:0) after 4m44s
[05:44:23.885] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:21:0)
[05:44:23.902] Build status: STARTED
[05:44:23.902] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:22:0) after 1m53s
[05:46:16.890] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:22:0)
[05:46:16.913] Build status: STARTED
[05:46:16.913] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:23:0) after 4m22s
[05:50:39.298] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:23:0)
[05:50:39.312] Build status: STARTED
[05:50:39.312] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:24:0) after 1m53s
[05:52:32.445] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:24:0)
[05:52:32.523] Build status: STARTED
[05:52:32.523] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:25:0) after 5m12s
[05:57:44.540] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:25:0)
[05:57:44.572] Build status: STARTED
[05:57:44.572] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:26:0) after 3m13s
[06:00:57.541] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:26:0)
[06:00:57.558] Build status: STARTED
[06:00:57.558] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:27:0) after 7m29s
[06:08:26.573] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:27:0)
[06:08:26.589] Build status: STARTED
[06:08:26.590] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:28:0) after 3m35s
[06:12:01.605] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:28:0)
[06:12:01.623] Build status: STARTED
[06:12:01.623] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:29:0) after 6m46s
[06:18:47.801] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:29:0)
[06:18:47.820] Build status: STARTED
[06:18:47.820] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:30:0) after 4m25s
[06:23:12.989] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:30:0)
[06:23:13.004] Build status: STARTED
[06:23:13.004] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:31:0) after 7m21s
[06:30:34.126] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:31:0)
[06:30:34.146] Build status: STARTED
[06:30:34.146] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:32:0) after 3m35s
[06:34:09.164] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:32:0)
[06:34:09.220] Build status: STARTED
[06:34:09.220] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:33:0) after 7m56s
[06:42:05.236] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:33:0)
[06:42:05.255] Build status: STARTED
[06:42:05.255] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:34:0) after 7m13s
[06:49:18.612] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:34:0)
[06:49:18.627] Build status: STARTED
[06:49:18.627] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:35:0) after 4m27s
[06:53:45.622] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:35:0)
[06:53:45.637] Build status: STARTED
[06:53:45.638] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:36:0) after 6m37s
[07:00:22.761] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:36:0)
[07:00:22.779] Build status: STARTED
[07:00:22.779] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:37:0) after 4m7s
[07:04:29.912] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:37:0)
[07:04:29.939] Build status: STARTED
[07:04:29.939] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:38:0) after 9m39s
[07:14:09.161] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:38:0)
[07:14:09.176] Build status: STARTED
[07:14:09.176] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:39:0) after 2m2s
[07:16:11.249] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:39:0)
[07:16:11.265] Build status: STARTED
[07:16:11.265] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:40:0) after 6m59s
[07:23:10.251] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:40:0)
[07:23:10.319] Build status: STARTED
[07:23:10.319] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:41:0) after 3m32s
[07:26:42.334] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:41:0)
[07:26:42.362] Build status: STARTED
[07:26:42.362] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:42:0) after 2m39s
[07:29:21.407] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:42:0)
[07:29:21.428] Build status: STARTED
[07:29:21.428] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:43:0) after 9m21s
[07:38:42.498] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:43:0)
[07:38:42.514] Build status: STARTED
[07:38:42.514] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:44:0) after 4m37s
[07:43:19.533] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:44:0)
[07:43:19.556] Build status: STARTED
[07:43:19.556] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:45:0) after 2m36s
[07:45:55.667] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:45:0)
[07:45:55.719] Build status: STARTED
[07:45:55.719] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:46:0) after 3m53s
[07:49:48.762] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:46:0)
[07:49:48.777] Build status: STARTED
[07:49:48.777] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:47:0) after 4m13s
[07:54:01.791] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:47:0)
[07:54:01.805] Build status: STARTED
[07:54:01.805] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:48:0) after 3m22s
[07:57:23.821] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:48:0)
[07:57:23.879] Build status: STARTED
[07:57:23.879] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:49:0) after 1m40s
[07:59:03.937] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:49:0)
[07:59:03.955] Build status: STARTED
[07:59:03.955] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:50:0) after 4m7s
[08:03:11.068] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:50:0)
[08:03:11.107] Build status: STARTED
[08:03:11.107] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:51:0) after 9m22s
[08:12:33.123] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:51:0)
[08:12:33.145] Build status: STARTED
[08:12:33.145] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:52:0) after 6m22s
[08:18:55.159] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:52:0)
[08:18:55.180] Build status: STARTED
[08:18:55.180] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:53:0) after 5m47s
[08:24:42.251] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:53:0)
[08:24:42.303] Build status: STARTED
[08:24:42.303] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:54:0) after 3m47s
[08:28:29.323] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:54:0)
[08:28:29.343] Build status: STARTED
[08:28:29.343] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:55:0) after 5m16s
[08:33:45.379] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:55:0)
[08:33:45.399] Build status: STARTED
[08:33:45.399] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:56:0) after 8m4s
[08:41:49.376] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:56:0)
[08:41:49.398] Build status: STARTED
[08:41:49.398] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:57:0) after 3m44s
[08:45:33.495] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:57:0)
[08:45:33.514] Build status: STARTED
[08:45:33.514] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:58:0) after 7m19s
[08:52:52.548] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:58:0)
[08:52:52.567] Build status: STARTED
[08:52:52.567] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:59:0) after 4m57s
[08:57:49.585] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:59:0)
[08:57:49.603] Build status: STARTED
[08:57:49.603] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:60:0) after 1m41s
[08:59:30.659] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:60:0)
[08:59:30.677] Build status: STARTED
[08:59:30.677] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:61:0) after 8m36s
[09:08:06.691] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:61:0)
[09:08:06.708] Build status: STARTED
[09:08:06.708] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:62:0) after 4m54s
[09:13:00.721] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:62:0)
[09:13:00.743] Build status: STARTED
[09:13:00.743] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:63:0) after 1m20s
[09:14:20.838] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:63:0)
[09:14:20.857] Build status: STARTED
[09:14:20.857] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:64:0) after 8m28s
[09:22:48.842] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:64:0)
[09:22:48.858] Build status: STARTED
[09:22:48.858] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:65:0) after 2m11s
[09:24:59.842] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:65:0)
[09:24:59.862] Build status: STARTED
[09:24:59.862] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:66:0) after 8m18s
[09:33:17.884] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:66:0)
[09:33:17.933] Build status: STARTED
[09:33:17.933] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:67:0) after 8m10s
[09:41:27.952] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:67:0)
[09:41:27.965] Build status: STARTED
[09:41:27.965] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:68:0) after 6m24s
[09:47:51.988] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:68:0)
[09:47:52.007] Build status: STARTED
[09:47:52.007] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8869756969826622864:69:0) after 4m38s
[09:50:26.123] Received PubSub notification, asking Buildbucket for the build status
[09:50:26.153] Build:
{
"id": "8704326907507100913",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-code-coverage"
},
"number": 10253,
"createdBy": "project:chromium",
"createTime": "2025-09-08T04:00:08.914452901Z",
"startTime": "2025-09-08T04:00:15.558280974Z",
"endTime": "2025-09-08T09:50:25.041257449Z",
"updateTime": "2025-09-08T09:50:25.041257449Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e80537f37d2f23f63a8a1718a9b5780494ed5b99",
"ref": "refs/heads/main"
}
}
}
[09:50:26.153] Invocation finished in 5h50m18.833711412s with status FAILED