[02:50:21.062] New invocation is queued and will start shortly
[02:50:22.131] Starting the invocation (attempt 1)
[02:50:22.164] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/2cc9b35154f9cbc9465f1bb4faaad5dabbbf2665"
[02:50:22.164] Popped gitiles commit info from properties and tags
[02:50:22.164] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[02:50:22.164] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[02:50:22.164] Buildbucket request:
{
"requestId": "8835334512209495008",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-official"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2cc9b35154f9cbc9465f1bb4faaad5dabbbf2665",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8835334512209495008"
},
{
"key": "scheduler_job_id",
"value": "chromium/mac-official"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[02:50:22.743] Scheduled build:
{
"id": "8669904450313871825",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-official"
},
"number": 15163,
"createdBy": "project:chromium",
"createTime": "2026-09-23T02:50:22.247843294Z",
"updateTime": "2026-09-23T02:50:22.247843294Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2cc9b35154f9cbc9465f1bb4faaad5dabbbf2665",
"ref": "refs/heads/main"
}
},
"canary": true
}
[02:50:22.743] Task URL: https://cr-buildbucket.appspot.com/build/8669904450313871825
[02:50:22.743] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:2:0) after 6m0s
[02:50:23.754] Received PubSub notification, asking Buildbucket for the build status
[02:50:23.774] Build status: SCHEDULED
[02:56:22.730] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:2:0)
[02:56:22.751] Build status: SCHEDULED
[02:56:22.751] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:4:0) after 7m26s
[03:03:48.773] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:4:0)
[03:03:48.793] Build status: SCHEDULED
[03:03:48.793] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:5:0) after 6m12s
[03:10:00.927] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:5:0)
[03:10:00.965] Build status: SCHEDULED
[03:10:00.966] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:6:0) after 5m36s
[03:11:41.990] Received PubSub notification, asking Buildbucket for the build status
[03:11:42.039] Build status: STARTED
[03:15:36.989] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:6:0)
[03:15:37.111] Build status: STARTED
[03:15:37.111] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:8:0) after 1m42s
[03:17:19.226] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:8:0)
[03:17:19.266] Build status: STARTED
[03:17:19.266] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:9:0) after 9m10s
[03:26:29.287] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:9:0)
[03:26:29.306] Build status: STARTED
[03:26:29.307] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:10:0) after 3m21s
[03:29:50.284] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:10:0)
[03:29:50.305] Build status: STARTED
[03:29:50.305] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:11:0) after 2m42s
[03:32:32.373] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:11:0)
[03:32:32.459] Build status: STARTED
[03:32:32.459] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:12:0) after 9m59s
[03:42:31.478] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:12:0)
[03:42:31.514] Build status: STARTED
[03:42:31.514] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:13:0) after 8m41s
[03:51:12.536] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:13:0)
[03:51:12.554] Build status: STARTED
[03:51:12.554] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:14:0) after 4m29s
[03:55:41.535] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:14:0)
[03:55:41.648] Build status: STARTED
[03:55:41.648] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:15:0) after 3m40s
[03:59:21.673] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:15:0)
[03:59:21.703] Build status: STARTED
[03:59:21.703] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:16:0) after 1m11s
[04:00:32.786] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:16:0)
[04:00:32.804] Build status: STARTED
[04:00:32.804] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:17:0) after 5m2s
[04:05:34.827] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:17:0)
[04:05:34.846] Build status: STARTED
[04:05:34.846] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:18:0) after 2m29s
[04:08:03.965] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:18:0)
[04:08:03.987] Build status: STARTED
[04:08:03.987] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:19:0) after 8m49s
[04:16:53.011] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:19:0)
[04:16:53.037] Build status: STARTED
[04:16:53.037] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:20:0) after 8m5s
[04:24:58.185] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:20:0)
[04:24:58.206] Build status: STARTED
[04:24:58.206] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:21:0) after 8m13s
[04:33:11.269] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:21:0)
[04:33:11.285] Build status: STARTED
[04:33:11.285] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:22:0) after 1m6s
[04:34:17.502] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:22:0)
[04:34:17.519] Build status: STARTED
[04:34:17.519] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:23:0) after 1m11s
[04:35:28.535] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:23:0)
[04:35:28.569] Build status: STARTED
[04:35:28.569] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:24:0) after 4m35s
[04:40:03.591] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:24:0)
[04:40:03.611] Build status: STARTED
[04:40:03.611] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:25:0) after 5m3s
[04:45:06.675] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:25:0)
[04:45:06.689] Build status: STARTED
[04:45:06.689] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:26:0) after 3m33s
[04:48:39.882] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:26:0)
[04:48:39.899] Build status: STARTED
[04:48:39.899] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:27:0) after 3m42s
[04:52:21.919] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:27:0)
[04:52:21.999] Build status: STARTED
[04:52:21.999] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:28:0) after 3m43s
[04:56:05.040] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:28:0)
[04:56:05.060] Build status: STARTED
[04:56:05.060] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:29:0) after 8m51s
[05:04:56.114] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:29:0)
[05:04:56.136] Build status: STARTED
[05:04:56.136] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:30:0) after 1m19s
[05:06:15.226] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:30:0)
[05:06:15.255] Build status: STARTED
[05:06:15.255] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:31:0) after 5m33s
[05:11:48.304] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:31:0)
[05:11:48.402] Build status: STARTED
[05:11:48.402] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:32:0) after 1m1s
[05:12:49.502] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:32:0)
[05:12:49.521] Build status: STARTED
[05:12:49.521] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:33:0) after 4m18s
[05:17:07.545] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:33:0)
[05:17:07.567] Build status: STARTED
[05:17:07.567] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:34:0) after 8m9s
[05:25:16.623] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:34:0)
[05:25:16.694] Build status: STARTED
[05:25:16.694] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:35:0) after 5m48s
[05:31:04.772] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:35:0)
[05:31:04.789] Build status: STARTED
[05:31:04.789] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:36:0) after 5m2s
[05:36:06.852] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:36:0)
[05:36:06.869] Build status: STARTED
[05:36:06.869] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:37:0) after 9m52s
[05:45:58.899] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:37:0)
[05:45:58.970] Build status: STARTED
[05:45:58.971] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:38:0) after 8m31s
[05:54:30.140] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:38:0)
[05:54:30.159] Build status: STARTED
[05:54:30.159] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:39:0) after 7m54s
[06:02:24.214] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:39:0)
[06:02:24.233] Build status: STARTED
[06:02:24.233] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:40:0) after 2m2s
[06:04:26.303] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:40:0)
[06:04:26.336] Build status: STARTED
[06:04:26.336] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:41:0) after 5m9s
[06:09:35.417] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:41:0)
[06:09:35.489] Build status: STARTED
[06:09:35.489] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:42:0) after 4m6s
[06:13:41.582] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:42:0)
[06:13:41.634] Build status: STARTED
[06:13:41.634] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:43:0) after 1m44s
[06:15:25.658] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:43:0)
[06:15:25.683] Build status: STARTED
[06:15:25.683] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:44:0) after 2m54s
[06:18:19.705] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:44:0)
[06:18:19.726] Build status: STARTED
[06:18:19.726] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:45:0) after 8m2s
[06:26:21.715] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:45:0)
[06:26:21.732] Build status: STARTED
[06:26:21.732] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:46:0) after 7m57s
[06:34:18.803] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:46:0)
[06:34:18.848] Build status: STARTED
[06:34:18.848] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:47:0) after 4m20s
[06:38:38.958] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:47:0)
[06:38:38.979] Build status: STARTED
[06:38:38.979] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:48:0) after 9m17s
[06:47:56.069] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:48:0)
[06:47:56.087] Build status: STARTED
[06:47:56.087] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:49:0) after 7m17s
[06:55:13.182] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:49:0)
[06:55:13.211] Build status: STARTED
[06:55:13.211] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:50:0) after 5m58s
[07:01:11.233] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:50:0)
[07:01:11.277] Build status: STARTED
[07:01:11.277] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:51:0) after 8m56s
[07:10:07.299] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:51:0)
[07:10:07.337] Build status: STARTED
[07:10:07.337] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:52:0) after 6m14s
[07:16:21.376] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:52:0)
[07:16:21.394] Build status: STARTED
[07:16:21.394] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:53:0) after 8m7s
[07:24:28.413] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:53:0)
[07:24:28.430] Build status: STARTED
[07:24:28.431] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:54:0) after 8m54s
[07:33:22.448] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:54:0)
[07:33:22.465] Build status: STARTED
[07:33:22.465] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:55:0) after 1m44s
[07:35:06.486] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:55:0)
[07:35:06.512] Build status: STARTED
[07:35:06.512] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:56:0) after 9m58s
[07:45:04.533] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:56:0)
[07:45:04.552] Build status: STARTED
[07:45:04.552] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:57:0) after 5m1s
[07:50:05.576] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:57:0)
[07:50:05.598] Build status: STARTED
[07:50:05.598] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:58:0) after 1m42s
[07:51:47.626] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:58:0)
[07:51:47.659] Build status: STARTED
[07:51:47.659] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:59:0) after 5m21s
[07:57:08.654] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:59:0)
[07:57:08.709] Build status: STARTED
[07:57:08.709] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:60:0) after 9m7s
[08:06:15.851] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:60:0)
[08:06:15.911] Build status: STARTED
[08:06:15.911] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:61:0) after 3m24s
[08:09:39.938] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:61:0)
[08:09:39.964] Build status: STARTED
[08:09:39.964] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:62:0) after 3m49s
[08:13:29.036] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:62:0)
[08:13:29.055] Build status: STARTED
[08:13:29.055] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:63:0) after 3m58s
[08:17:27.077] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:63:0)
[08:17:27.097] Build status: STARTED
[08:17:27.097] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:64:0) after 2m10s
[08:19:37.105] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:64:0)
[08:19:37.133] Build status: STARTED
[08:19:37.134] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:65:0) after 5m51s
[08:25:28.158] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:65:0)
[08:25:28.178] Build status: STARTED
[08:25:28.178] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:66:0) after 2m20s
[08:27:48.251] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:66:0)
[08:27:48.272] Build status: STARTED
[08:27:48.272] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:67:0) after 2m57s
[08:30:45.344] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:67:0)
[08:30:45.364] Build status: STARTED
[08:30:45.364] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:68:0) after 4m57s
[08:35:42.477] Handling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:68:0)
[08:35:42.497] Build status: STARTED
[08:35:42.497] Scheduling timer "check-buildbucket-build-status" (chromium/mac-official:8835334512209495008:69:0) after 5m28s
[08:39:27.391] Received PubSub notification, asking Buildbucket for the build status
[08:39:27.477] Build:
{
"id": "8669904450313871825",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "mac-official"
},
"number": 15163,
"createdBy": "project:chromium",
"createTime": "2026-09-23T02:50:22.247843294Z",
"startTime": "2026-09-23T03:11:41.784421740Z",
"endTime": "2026-09-23T08:39:26.833341059Z",
"updateTime": "2026-09-23T08:39:26.833341059Z",
"status": "SUCCESS",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "2cc9b35154f9cbc9465f1bb4faaad5dabbbf2665",
"ref": "refs/heads/main"
}
},
"canary": true
}
[08:39:27.477] Invocation finished in 5h49m6.420711368s with status SUCCEEDED