[21:25:21.787] New invocation is queued and will start shortly
[21:25:23.159] Starting the invocation (attempt 1)
[21:25:23.190] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/99d088645e59a91da0038df8c59976e92a746405"
[21:25:23.190] Popped gitiles commit info from properties and tags
[21:25:23.190] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:25:23.190] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:25:23.190] Buildbucket request:
{
"requestId": "8856645245790103984",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "99d088645e59a91da0038df8c59976e92a746405",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8856645245790103984"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMac"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:25:24.007] Scheduled build:
{
"id": "8691215183576212817",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 39326,
"createdBy": "project:chromium",
"createTime": "2026-01-30T21:25:23.271346172Z",
"updateTime": "2026-01-30T21:25:23.271346172Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "99d088645e59a91da0038df8c59976e92a746405",
"ref": "refs/heads/main"
}
},
"canary": true
}
[21:25:24.007] Task URL: https://cr-buildbucket.appspot.com/build/8691215183576212817
[21:25:24.007] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:2:0) after 4m17s
[21:25:24.979] Received PubSub notification, asking Buildbucket for the build status
[21:25:25.011] Build status: SCHEDULED
[21:26:35.847] Received PubSub notification, asking Buildbucket for the build status
[21:26:35.875] Build status: STARTED
[21:29:41.013] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:2:0)
[21:29:41.054] Build status: STARTED
[21:29:41.054] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:5:0) after 1m44s
[21:31:25.079] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:5:0)
[21:31:25.128] Build status: STARTED
[21:31:25.129] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:6:0) after 5m2s
[21:36:27.186] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:6:0)
[21:36:27.230] Build status: STARTED
[21:36:27.231] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:7:0) after 9m11s
[21:45:38.242] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:7:0)
[21:45:38.261] Build status: STARTED
[21:45:38.261] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:8:0) after 3m40s
[21:49:18.292] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:8:0)
[21:49:18.364] Build status: STARTED
[21:49:18.364] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:9:0) after 3m47s
[21:53:05.415] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:9:0)
[21:53:05.431] Build status: STARTED
[21:53:05.431] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:10:0) after 7m48s
[22:00:53.684] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:10:0)
[22:00:53.700] Build status: STARTED
[22:00:53.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:11:0) after 8m50s
[22:09:43.723] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:11:0)
[22:09:43.740] Build status: STARTED
[22:09:43.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:12:0) after 3m32s
[22:13:15.750] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:12:0)
[22:13:15.764] Build status: STARTED
[22:13:15.764] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:13:0) after 8m26s
[22:21:42.083] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:13:0)
[22:21:42.097] Build status: STARTED
[22:21:42.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:14:0) after 6m21s
[22:28:03.143] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:14:0)
[22:28:03.229] Build status: STARTED
[22:28:03.229] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:15:0) after 8m33s
[22:36:36.231] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:15:0)
[22:36:36.251] Build status: STARTED
[22:36:36.251] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:16:0) after 8m5s
[22:44:41.311] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:16:0)
[22:44:41.331] Build status: STARTED
[22:44:41.331] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:17:0) after 9m22s
[22:54:03.382] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:17:0)
[22:54:03.397] Build status: STARTED
[22:54:03.397] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:18:0) after 9m38s
[23:03:41.608] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:18:0)
[23:03:41.626] Build status: STARTED
[23:03:41.626] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:19:0) after 1m55s
[23:05:36.652] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:19:0)
[23:05:36.687] Build status: STARTED
[23:05:36.687] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:20:0) after 8m38s
[23:14:14.755] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:20:0)
[23:14:14.775] Build status: STARTED
[23:14:14.775] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:21:0) after 9m53s
[23:24:07.797] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:21:0)
[23:24:07.818] Build status: STARTED
[23:24:07.818] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:22:0) after 1m51s
[23:25:58.845] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:22:0)
[23:25:58.863] Build status: STARTED
[23:25:58.863] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:23:0) after 5m16s
[23:31:14.970] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:23:0)
[23:31:15.025] Build status: STARTED
[23:31:15.025] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:24:0) after 5m52s
[23:37:07.074] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:24:0)
[23:37:07.135] Build status: STARTED
[23:37:07.135] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:25:0) after 2m39s
[23:39:46.312] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:25:0)
[23:39:46.335] Build status: STARTED
[23:39:46.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:26:0) after 6m59s
[23:46:45.411] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:26:0)
[23:46:45.428] Build status: STARTED
[23:46:45.428] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:27:0) after 2m56s
[23:49:41.455] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:27:0)
[23:49:41.471] Build status: STARTED
[23:49:41.471] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:28:0) after 6m50s
[23:56:31.502] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:28:0)
[23:56:31.516] Build status: STARTED
[23:56:31.516] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:29:0) after 5m48s
[00:02:19.550] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:29:0)
[00:02:19.585] Build status: STARTED
[00:02:19.585] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:30:0) after 5m1s
[00:07:20.607] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:30:0)
[00:07:20.643] Build status: STARTED
[00:07:20.643] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:31:0) after 9m57s
[00:17:17.655] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:31:0)
[00:17:17.704] Build status: STARTED
[00:17:17.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:32:0) after 3m29s
[00:20:46.777] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:32:0)
[00:20:46.805] Build status: STARTED
[00:20:46.805] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:33:0) after 4m8s
[00:24:54.829] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:33:0)
[00:24:54.842] Build status: STARTED
[00:24:54.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:34:0) after 7m21s
[00:32:15.929] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:34:0)
[00:32:15.948] Build status: STARTED
[00:32:15.948] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:35:0) after 1m59s
[00:34:15.119] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:35:0)
[00:34:15.136] Build status: STARTED
[00:34:15.136] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:36:0) after 3m45s
[00:38:00.265] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:36:0)
[00:38:00.288] Build status: STARTED
[00:38:00.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:37:0) after 9m24s
[00:47:24.324] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:37:0)
[00:47:24.422] Build status: STARTED
[00:47:24.422] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:38:0) after 4m55s
[00:52:19.537] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:38:0)
[00:52:19.560] Build status: STARTED
[00:52:19.561] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:39:0) after 7m38s
[00:59:57.762] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:39:0)
[00:59:57.792] Build status: STARTED
[00:59:57.792] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:40:0) after 5m37s
[01:05:34.900] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:40:0)
[01:05:34.919] Build status: STARTED
[01:05:34.919] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:41:0) after 6m20s
[01:11:55.018] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:41:0)
[01:11:55.068] Build status: STARTED
[01:11:55.068] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:42:0) after 6m12s
[01:18:07.095] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:42:0)
[01:18:07.115] Build status: STARTED
[01:18:07.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:43:0) after 3m41s
[01:21:48.139] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:43:0)
[01:21:48.154] Build status: STARTED
[01:21:48.154] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:44:0) after 6m51s
[01:28:39.237] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:44:0)
[01:28:39.251] Build status: STARTED
[01:28:39.251] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:45:0) after 4m33s
[01:33:12.274] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:45:0)
[01:33:12.291] Build status: STARTED
[01:33:12.291] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:46:0) after 1m6s
[01:34:18.336] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:46:0)
[01:34:18.359] Build status: STARTED
[01:34:18.359] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:47:0) after 9m11s
[01:43:29.395] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:47:0)
[01:43:29.409] Build status: STARTED
[01:43:29.409] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:48:0) after 5m3s
[01:48:32.428] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:48:0)
[01:48:32.440] Build status: STARTED
[01:48:32.440] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:49:0) after 4m58s
[01:53:30.513] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:49:0)
[01:53:30.577] Build status: STARTED
[01:53:30.577] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:50:0) after 4m31s
[01:58:01.669] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:50:0)
[01:58:01.692] Build status: STARTED
[01:58:01.692] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:51:0) after 3m53s
[02:01:54.796] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:51:0)
[02:01:54.834] Build status: STARTED
[02:01:54.834] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:52:0) after 5m55s
[02:07:49.864] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:52:0)
[02:07:49.904] Build status: STARTED
[02:07:49.904] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:53:0) after 8m13s
[02:16:03.028] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:53:0)
[02:16:03.045] Build status: STARTED
[02:16:03.045] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:54:0) after 2m41s
[02:18:44.076] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:54:0)
[02:18:44.134] Build status: STARTED
[02:18:44.134] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:55:0) after 8m17s
[02:27:01.214] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:55:0)
[02:27:01.326] Build status: STARTED
[02:27:01.326] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:56:0) after 8m32s
[02:35:33.364] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:56:0)
[02:35:33.537] Build status: STARTED
[02:35:33.538] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:57:0) after 5m9s
[02:40:42.563] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:57:0)
[02:40:42.581] Build status: STARTED
[02:40:42.581] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:58:0) after 8m15s
[02:48:57.668] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:58:0)
[02:48:57.682] Build status: STARTED
[02:48:57.682] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:59:0) after 2m28s
[02:51:25.765] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:59:0)
[02:51:25.793] Build status: STARTED
[02:51:25.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:60:0) after 9m0s
[03:00:25.924] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:60:0)
[03:00:25.938] Build status: STARTED
[03:00:25.938] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:61:0) after 9m31s
[03:09:57.068] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:61:0)
[03:09:57.085] Build status: STARTED
[03:09:57.085] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:62:0) after 1m3s
[03:11:00.245] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:62:0)
[03:11:00.271] Build status: STARTED
[03:11:00.271] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:63:0) after 8m55s
[03:19:55.356] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:63:0)
[03:19:55.386] Build status: STARTED
[03:19:55.386] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:64:0) after 2m35s
[03:22:30.412] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:64:0)
[03:22:30.432] Build status: STARTED
[03:22:30.433] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:65:0) after 8m28s
[03:30:58.508] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:65:0)
[03:30:58.572] Build status: STARTED
[03:30:58.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:66:0) after 8m2s
[03:39:00.599] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:66:0)
[03:39:00.621] Build status: STARTED
[03:39:00.621] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:67:0) after 9m15s
[03:48:15.649] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:67:0)
[03:48:15.668] Build status: STARTED
[03:48:15.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:68:0) after 7m10s
[03:55:25.690] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:68:0)
[03:55:25.716] Build status: STARTED
[03:55:25.716] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:69:0) after 9m41s
[04:05:06.753] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:69:0)
[04:05:06.848] Build status: STARTED
[04:05:06.848] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:70:0) after 5m34s
[04:10:40.870] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:70:0)
[04:10:40.900] Build status: STARTED
[04:10:40.900] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:71:0) after 6m18s
[04:16:58.995] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:71:0)
[04:16:59.013] Build status: STARTED
[04:16:59.013] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:72:0) after 8m53s
[04:25:52.039] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:72:0)
[04:25:52.058] Build status: STARTED
[04:25:52.058] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8856645245790103984:73:0) after 8m40s
[04:32:48.638] Received PubSub notification, asking Buildbucket for the build status
[04:32:48.653] Build:
{
"id": "8691215183576212817",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 39326,
"createdBy": "project:chromium",
"createTime": "2026-01-30T21:25:23.271346172Z",
"startTime": "2026-01-30T21:26:35.690178973Z",
"endTime": "2026-01-31T04:32:47.733795708Z",
"updateTime": "2026-01-31T04:32:47.733795708Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "99d088645e59a91da0038df8c59976e92a746405",
"ref": "refs/heads/main"
}
},
"canary": true
}
[04:32:48.653] Invocation finished in 7h7m26.876684294s with status FAILED