[04:00:08.687] New invocation is queued and will start shortly
[04:00:09.722] Starting the invocation (attempt 1)
[04:00:09.749] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/85c4f95dee86d1ec66442b61933435af66dae483"
[04:00:09.749] Popped gitiles commit info from properties and tags
[04:00:09.750] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:00:09.750] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:00:09.750] Buildbucket request:
{
"requestId": "8853721305343047152",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-code-coverage"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "85c4f95dee86d1ec66442b61933435af66dae483",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8853721305343047152"
},
{
"key": "scheduler_job_id",
"value": "chromium/mac-code-coverage"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:00:10.269] Scheduled build:
{
"id": "8688291243527309985",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-code-coverage"
},
"number": 10430,
"createdBy": "project:chromium",
"createTime": "2026-03-04T04:00:09.798559375Z",
"updateTime": "2026-03-04T04:00:09.798559375Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "85c4f95dee86d1ec66442b61933435af66dae483",
"ref": "refs/heads/main"
}
}
}
[04:00:10.269] Task URL: https://cr-buildbucket.appspot.com/build/8688291243527309985
[04:00:10.269] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:2:0) after 5m0s
[04:00:13.146] Received PubSub notification, asking Buildbucket for the build status
[04:00:13.164] Build status: SCHEDULED
[04:00:54.383] Received PubSub notification, asking Buildbucket for the build status
[04:00:54.409] Build status: STARTED
[04:05:10.293] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:2:0)
[04:05:10.308] Build status: STARTED
[04:05:10.308] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:5:0) after 1m16s
[04:06:26.320] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:5:0)
[04:06:26.337] Build status: STARTED
[04:06:26.337] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:6:0) after 6m13s
[04:12:39.362] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:6:0)
[04:12:39.379] Build status: STARTED
[04:12:39.379] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:7:0) after 4m53s
[04:17:32.395] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:7:0)
[04:17:32.412] Build status: STARTED
[04:17:32.412] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:8:0) after 4m42s
[04:22:14.443] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:8:0)
[04:22:14.461] Build status: STARTED
[04:22:14.461] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:9:0) after 3m58s
[04:26:12.524] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:9:0)
[04:26:12.542] Build status: STARTED
[04:26:12.542] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:10:0) after 8m29s
[04:34:41.667] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:10:0)
[04:34:41.700] Build status: STARTED
[04:34:41.700] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:11:0) after 3m19s
[04:38:00.741] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:11:0)
[04:38:00.758] Build status: STARTED
[04:38:00.758] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:12:0) after 1m15s
[04:39:15.808] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:12:0)
[04:39:15.833] Build status: STARTED
[04:39:15.833] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:13:0) after 8m8s
[04:47:23.852] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:13:0)
[04:47:23.890] Build status: STARTED
[04:47:23.890] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:14:0) after 1m51s
[04:49:14.907] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:14:0)
[04:49:14.923] Build status: STARTED
[04:49:14.923] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:15:0) after 4m54s
[04:54:08.929] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:15:0)
[04:54:08.945] Build status: STARTED
[04:54:08.945] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:16:0) after 5m28s
[04:59:36.986] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:16:0)
[04:59:36.998] Build status: STARTED
[04:59:36.999] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:17:0) after 2m25s
[05:02:02.017] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:17:0)
[05:02:02.035] Build status: STARTED
[05:02:02.035] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:18:0) after 1m52s
[05:03:54.074] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:18:0)
[05:03:54.120] Build status: STARTED
[05:03:54.120] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:19:0) after 5m14s
[05:09:08.185] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:19:0)
[05:09:08.198] Build status: STARTED
[05:09:08.199] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:20:0) after 6m49s
[05:15:57.214] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:20:0)
[05:15:57.234] Build status: STARTED
[05:15:57.234] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:21:0) after 2m33s
[05:18:30.331] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:21:0)
[05:18:30.347] Build status: STARTED
[05:18:30.348] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:22:0) after 3m35s
[05:22:05.364] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:22:0)
[05:22:05.377] Build status: STARTED
[05:22:05.377] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:23:0) after 2m6s
[05:24:11.392] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:23:0)
[05:24:11.414] Build status: STARTED
[05:24:11.414] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:24:0) after 9m5s
[05:33:16.431] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:24:0)
[05:33:16.458] Build status: STARTED
[05:33:16.458] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:25:0) after 5m54s
[05:39:10.519] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:25:0)
[05:39:10.581] Build status: STARTED
[05:39:10.581] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:26:0) after 2m51s
[05:42:01.602] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:26:0)
[05:42:01.685] Build status: STARTED
[05:42:01.685] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:27:0) after 7m12s
[05:49:13.701] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:27:0)
[05:49:13.716] Build status: STARTED
[05:49:13.716] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:28:0) after 3m56s
[05:53:09.765] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:28:0)
[05:53:09.780] Build status: STARTED
[05:53:09.780] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:29:0) after 1m52s
[05:55:01.807] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:29:0)
[05:55:01.833] Build status: STARTED
[05:55:01.833] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:30:0) after 6m33s
[06:01:35.423] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:30:0)
[06:01:35.463] Build status: STARTED
[06:01:35.463] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:31:0) after 2m52s
[06:04:27.558] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:31:0)
[06:04:27.641] Build status: STARTED
[06:04:27.641] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:32:0) after 4m54s
[06:09:21.746] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:32:0)
[06:09:21.777] Build status: STARTED
[06:09:21.777] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:33:0) after 8m14s
[06:17:35.790] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:33:0)
[06:17:35.850] Build status: STARTED
[06:17:35.850] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:34:0) after 3m8s
[06:20:43.895] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:34:0)
[06:20:43.927] Build status: STARTED
[06:20:43.927] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:35:0) after 5m14s
[06:25:58.071] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:35:0)
[06:25:58.084] Build status: STARTED
[06:25:58.085] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:36:0) after 1m41s
[06:27:39.103] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:36:0)
[06:27:39.182] Build status: STARTED
[06:27:39.182] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:37:0) after 7m41s
[06:35:20.238] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:37:0)
[06:35:20.267] Build status: STARTED
[06:35:20.267] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:38:0) after 5m26s
[06:40:46.397] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:38:0)
[06:40:46.480] Build status: STARTED
[06:40:46.480] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:39:0) after 2m28s
[06:43:14.503] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:39:0)
[06:43:14.525] Build status: STARTED
[06:43:14.525] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:40:0) after 3m19s
[06:46:33.544] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:40:0)
[06:46:33.580] Build status: STARTED
[06:46:33.580] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:41:0) after 7m40s
[06:54:13.597] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:41:0)
[06:54:13.615] Build status: STARTED
[06:54:13.615] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:42:0) after 6m39s
[07:00:52.600] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:42:0)
[07:00:52.618] Build status: STARTED
[07:00:52.618] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:43:0) after 4m49s
[07:05:41.685] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:43:0)
[07:05:41.711] Build status: STARTED
[07:05:41.711] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:44:0) after 3m1s
[07:08:42.717] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:44:0)
[07:08:42.753] Build status: STARTED
[07:08:42.753] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:45:0) after 6m31s
[07:15:13.819] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:45:0)
[07:15:13.834] Build status: STARTED
[07:15:13.834] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:46:0) after 4m19s
[07:19:32.851] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:46:0)
[07:19:32.865] Build status: STARTED
[07:19:32.865] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:47:0) after 1m11s
[07:20:43.851] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:47:0)
[07:20:43.867] Build status: STARTED
[07:20:43.867] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:48:0) after 8m8s
[07:28:51.987] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:48:0)
[07:28:52.001] Build status: STARTED
[07:28:52.001] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:49:0) after 8m52s
[07:37:44.042] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:49:0)
[07:37:44.059] Build status: STARTED
[07:37:44.059] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:50:0) after 3m52s
[07:41:36.080] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:50:0)
[07:41:36.097] Build status: STARTED
[07:41:36.097] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:51:0) after 5m22s
[07:46:58.107] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:51:0)
[07:46:58.123] Build status: STARTED
[07:46:58.123] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:52:0) after 3m41s
[07:50:39.138] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:52:0)
[07:50:39.156] Build status: STARTED
[07:50:39.156] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:53:0) after 7m59s
[07:58:38.220] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:53:0)
[07:58:38.241] Build status: STARTED
[07:58:38.241] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:54:0) after 1m9s
[07:59:47.253] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:54:0)
[07:59:47.277] Build status: STARTED
[07:59:47.277] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:55:0) after 1m55s
[08:01:42.332] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:55:0)
[08:01:42.359] Build status: STARTED
[08:01:42.359] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:56:0) after 7m20s
[08:09:02.376] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:56:0)
[08:09:02.394] Build status: STARTED
[08:09:02.394] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:57:0) after 1m39s
[08:10:41.458] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:57:0)
[08:10:41.503] Build status: STARTED
[08:10:41.503] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:58:0) after 2m36s
[08:13:17.522] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:58:0)
[08:13:17.580] Build status: STARTED
[08:13:17.580] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:59:0) after 2m35s
[08:15:52.596] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:59:0)
[08:15:52.612] Build status: STARTED
[08:15:52.612] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:60:0) after 2m24s
[08:18:16.618] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:60:0)
[08:18:16.682] Build status: STARTED
[08:18:16.682] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:61:0) after 4m17s
[08:22:33.665] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:61:0)
[08:22:33.684] Build status: STARTED
[08:22:33.684] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:62:0) after 2m42s
[08:25:15.747] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:62:0)
[08:25:15.779] Build status: STARTED
[08:25:15.779] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:63:0) after 7m32s
[08:32:47.800] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:63:0)
[08:32:48.014] Build status: STARTED
[08:32:48.014] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:64:0) after 7m53s
[08:40:41.058] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:64:0)
[08:40:41.079] Build status: STARTED
[08:40:41.079] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:65:0) after 2m0s
[08:42:41.127] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:65:0)
[08:42:41.179] Build status: STARTED
[08:42:41.180] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:66:0) after 2m10s
[08:44:51.223] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:66:0)
[08:44:51.249] Build status: STARTED
[08:44:51.249] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8853721305343047152:67:0) after 6m59s
[08:51:03.052] Received PubSub notification, asking Buildbucket for the build status
[08:51:03.116] Build:
{
"id": "8688291243527309985",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-code-coverage"
},
"number": 10430,
"createdBy": "project:chromium",
"createTime": "2026-03-04T04:00:09.798559375Z",
"startTime": "2026-03-04T04:00:44.539449377Z",
"endTime": "2026-03-04T08:51:02.716769746Z",
"updateTime": "2026-03-04T08:51:02.716769746Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "85c4f95dee86d1ec66442b61933435af66dae483",
"ref": "refs/heads/main"
}
}
}
[08:51:03.116] Invocation finished in 4h50m54.433416968s with status FAILED