[04:00:06.617] New invocation is queued and will start shortly
[04:00:07.821] Starting the invocation (attempt 1)
[04:00:07.846] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/2a38f4ebd84c680a48d6ed67242822e2174ee0b1"
[04:00:07.846] Popped gitiles commit info from properties and tags
[04:00:07.846] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:00:07.846] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:00:07.846] Buildbucket request:
{
"requestId": "8852815337854841584",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-code-coverage"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2a38f4ebd84c680a48d6ed67242822e2174ee0b1",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8852815337854841584"
},
{
"key": "scheduler_job_id",
"value": "chromium/mac-code-coverage"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:00:08.487] Scheduled build:
{
"id": "8687385275781278145",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-code-coverage"
},
"number": 10440,
"createdBy": "project:chromium",
"createTime": "2026-03-14T04:00:07.969179901Z",
"updateTime": "2026-03-14T04:00:07.969179901Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2a38f4ebd84c680a48d6ed67242822e2174ee0b1",
"ref": "refs/heads/main"
}
}
}
[04:00:08.487] Task URL: https://cr-buildbucket.appspot.com/build/8687385275781278145
[04:00:08.488] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:2:0) after 8m8s
[04:00:09.542] Received PubSub notification, asking Buildbucket for the build status
[04:00:09.596] Build status: SCHEDULED
[04:00:11.689] Received PubSub notification, asking Buildbucket for the build status
[04:00:11.708] Build status: STARTED
[04:08:16.583] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:2:0)
[04:08:16.623] Build status: STARTED
[04:08:16.623] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:5:0) after 9m21s
[04:17:37.689] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:5:0)
[04:17:37.710] Build status: STARTED
[04:17:37.710] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:6:0) after 4m27s
[04:22:04.826] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:6:0)
[04:22:04.853] Build status: STARTED
[04:22:04.853] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:7:0) after 4m1s
[04:26:05.867] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:7:0)
[04:26:05.882] Build status: STARTED
[04:26:05.882] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:8:0) after 4m21s
[04:30:26.900] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:8:0)
[04:30:26.930] Build status: STARTED
[04:30:26.930] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:9:0) after 6m13s
[04:36:39.956] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:9:0)
[04:36:39.985] Build status: STARTED
[04:36:39.985] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:10:0) after 3m16s
[04:39:56.003] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:10:0)
[04:39:56.019] Build status: STARTED
[04:39:56.019] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:11:0) after 7m23s
[04:47:19.036] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:11:0)
[04:47:19.059] Build status: STARTED
[04:47:19.059] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:12:0) after 4m26s
[04:51:45.099] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:12:0)
[04:51:45.125] Build status: STARTED
[04:51:45.125] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:13:0) after 2m6s
[04:53:51.412] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:13:0)
[04:53:51.473] Build status: STARTED
[04:53:51.473] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:14:0) after 2m1s
[04:55:52.474] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:14:0)
[04:55:52.574] Build status: STARTED
[04:55:52.574] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:15:0) after 1m52s
[04:57:44.639] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:15:0)
[04:57:44.681] Build status: STARTED
[04:57:44.681] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:16:0) after 5m34s
[05:03:18.746] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:16:0)
[05:03:18.856] Build status: STARTED
[05:03:18.856] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:17:0) after 3m41s
[05:06:59.840] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:17:0)
[05:06:59.863] Build status: STARTED
[05:06:59.863] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:18:0) after 8m42s
[05:15:41.930] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:18:0)
[05:15:41.963] Build status: STARTED
[05:15:41.963] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:19:0) after 7m52s
[05:23:34.099] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:19:0)
[05:23:34.153] Build status: STARTED
[05:23:34.153] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:20:0) after 6m10s
[05:29:44.272] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:20:0)
[05:29:44.314] Build status: STARTED
[05:29:44.314] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:21:0) after 4m26s
[05:34:10.448] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:21:0)
[05:34:10.504] Build status: STARTED
[05:34:10.504] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:22:0) after 3m29s
[05:37:39.576] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:22:0)
[05:37:39.589] Build status: STARTED
[05:37:39.589] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:23:0) after 2m12s
[05:39:51.645] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:23:0)
[05:39:51.685] Build status: STARTED
[05:39:51.685] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:24:0) after 8m38s
[05:48:29.811] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:24:0)
[05:48:29.829] Build status: STARTED
[05:48:29.829] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:25:0) after 9m21s
[05:57:51.035] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:25:0)
[05:57:51.053] Build status: STARTED
[05:57:51.053] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:26:0) after 2m29s
[06:00:20.568] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:26:0)
[06:00:20.582] Build status: STARTED
[06:00:20.582] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:27:0) after 5m34s
[06:05:54.688] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:27:0)
[06:05:54.711] Build status: STARTED
[06:05:54.711] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:28:0) after 8m1s
[06:13:55.730] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:28:0)
[06:13:55.759] Build status: STARTED
[06:13:55.759] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:29:0) after 6m39s
[06:20:34.775] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:29:0)
[06:20:34.796] Build status: STARTED
[06:20:34.796] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:30:0) after 8m31s
[06:29:05.813] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:30:0)
[06:29:05.833] Build status: STARTED
[06:29:05.833] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:31:0) after 6m2s
[06:35:08.129] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:31:0)
[06:35:08.145] Build status: STARTED
[06:35:08.145] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:32:0) after 9m43s
[06:44:51.163] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:32:0)
[06:44:51.229] Build status: STARTED
[06:44:51.229] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:33:0) after 6m13s
[06:51:04.274] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:33:0)
[06:51:04.321] Build status: STARTED
[06:51:04.321] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:34:0) after 4m36s
[06:55:40.703] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:34:0)
[06:55:40.727] Build status: STARTED
[06:55:40.727] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:35:0) after 3m36s
[06:59:16.745] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:35:0)
[06:59:16.768] Build status: STARTED
[06:59:16.768] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:36:0) after 1m19s
[07:00:35.811] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:36:0)
[07:00:35.828] Build status: STARTED
[07:00:35.828] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:37:0) after 4m21s
[07:04:56.907] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:37:0)
[07:04:56.938] Build status: STARTED
[07:04:56.938] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:38:0) after 1m49s
[07:06:45.930] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:38:0)
[07:06:45.984] Build status: STARTED
[07:06:45.984] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:39:0) after 7m18s
[07:14:04.199] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:39:0)
[07:14:04.217] Build status: STARTED
[07:14:04.217] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:40:0) after 4m15s
[07:18:19.333] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:40:0)
[07:18:19.357] Build status: STARTED
[07:18:19.357] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:41:0) after 2m50s
[07:21:50.361] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:41:0)
[07:21:50.384] Build status: STARTED
[07:21:50.384] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:42:0) after 9m55s
[07:31:45.600] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:42:0)
[07:31:45.665] Build status: STARTED
[07:31:45.665] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:43:0) after 1m28s
[07:33:13.681] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:43:0)
[07:33:13.698] Build status: STARTED
[07:33:13.698] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:44:0) after 1m32s
[07:34:45.815] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:44:0)
[07:34:45.834] Build status: STARTED
[07:34:45.834] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:45:0) after 7m46s
[07:42:32.010] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:45:0)
[07:42:32.060] Build status: STARTED
[07:42:32.060] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:46:0) after 4m12s
[07:46:44.169] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:46:0)
[07:46:44.186] Build status: STARTED
[07:46:44.187] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:47:0) after 1m58s
[07:48:42.274] Handling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:47:0)
[07:48:42.292] Build status: STARTED
[07:48:42.292] Scheduling timer "check-buildbucket-build-status" (chromium/mac-code-coverage:8852815337854841584:48:0) after 9m16s
[07:51:51.759] Received PubSub notification, asking Buildbucket for the build status
[07:51:51.784] Build:
{
"id": "8687385275781278145",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-code-coverage"
},
"number": 10440,
"createdBy": "project:chromium",
"createTime": "2026-03-14T04:00:07.969179901Z",
"startTime": "2026-03-14T04:00:11.494159589Z",
"endTime": "2026-03-14T07:51:51.454123898Z",
"updateTime": "2026-03-14T07:51:51.454123898Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2a38f4ebd84c680a48d6ed67242822e2174ee0b1",
"ref": "refs/heads/main"
}
}
}
[07:51:51.784] Invocation finished in 3h51m45.176674189s with status FAILED