[09:25:12.585] New invocation is queued and will start shortly
[09:25:13.651] Starting the invocation (attempt 1)
[09:25:13.679] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/eb8fecbf0579291e5b741d07f24e1206c326c29a"
[09:25:13.679] Popped gitiles commit info from properties and tags
[09:25:13.679] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[09:25:13.679] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[09:25:13.679] Buildbucket request:
{
"requestId": "8851345332896255008",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "eb8fecbf0579291e5b741d07f24e1206c326c29a",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8851345332896255008"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMac"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[09:25:14.158] Scheduled build:
{
"id": "8685915271051279185",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 40106,
"createdBy": "project:chromium",
"createTime": "2026-03-30T09:25:13.725525682Z",
"updateTime": "2026-03-30T09:25:13.725525682Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "eb8fecbf0579291e5b741d07f24e1206c326c29a",
"ref": "refs/heads/main"
}
}
}
[09:25:14.158] Task URL: https://cr-buildbucket.appspot.com/build/8685915271051279185
[09:25:14.158] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:2:0) after 3m18s
[09:25:15.144] Received PubSub notification, asking Buildbucket for the build status
[09:25:15.168] Build status: SCHEDULED
[09:25:54.988] Received PubSub notification, asking Buildbucket for the build status
[09:25:55.011] Build status: STARTED
[09:28:32.177] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:2:0)
[09:28:32.190] Build status: STARTED
[09:28:32.190] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:5:0) after 9m50s
[09:38:22.210] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:5:0)
[09:38:22.232] Build status: STARTED
[09:38:22.232] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:6:0) after 4m31s
[09:42:53.249] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:6:0)
[09:42:53.269] Build status: STARTED
[09:42:53.269] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:7:0) after 6m23s
[09:49:16.288] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:7:0)
[09:49:16.304] Build status: STARTED
[09:49:16.304] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:8:0) after 6m4s
[09:55:20.288] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:8:0)
[09:55:20.336] Build status: STARTED
[09:55:20.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:9:0) after 6m14s
[10:01:34.368] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:9:0)
[10:01:34.394] Build status: STARTED
[10:01:34.394] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:10:0) after 5m26s
[10:07:00.414] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:10:0)
[10:07:00.492] Build status: STARTED
[10:07:00.492] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:11:0) after 6m57s
[10:13:57.630] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:11:0)
[10:13:57.646] Build status: STARTED
[10:13:57.646] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:12:0) after 1m42s
[10:15:39.661] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:12:0)
[10:15:39.722] Build status: STARTED
[10:15:39.722] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:13:0) after 5m3s
[10:20:42.740] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:13:0)
[10:20:42.758] Build status: STARTED
[10:20:42.758] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:14:0) after 5m0s
[10:25:42.787] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:14:0)
[10:25:42.880] Build status: STARTED
[10:25:42.881] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:15:0) after 6m36s
[10:32:18.981] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:15:0)
[10:32:19.002] Build status: STARTED
[10:32:19.002] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:16:0) after 9m34s
[10:41:53.024] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:16:0)
[10:41:53.049] Build status: STARTED
[10:41:53.049] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:17:0) after 7m48s
[10:49:41.165] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:17:0)
[10:49:41.181] Build status: STARTED
[10:49:41.181] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:18:0) after 9m38s
[10:59:19.199] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:18:0)
[10:59:19.217] Build status: STARTED
[10:59:19.217] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:19:0) after 8m34s
[11:07:53.221] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:19:0)
[11:07:53.242] Build status: STARTED
[11:07:53.242] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:20:0) after 2m26s
[11:10:19.309] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:20:0)
[11:10:19.330] Build status: STARTED
[11:10:19.330] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:21:0) after 1m1s
[11:11:20.350] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:21:0)
[11:11:20.370] Build status: STARTED
[11:11:20.370] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:22:0) after 9m6s
[11:20:26.497] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:22:0)
[11:20:26.516] Build status: STARTED
[11:20:26.516] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:23:0) after 2m13s
[11:22:39.532] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:23:0)
[11:22:39.576] Build status: STARTED
[11:22:39.576] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:24:0) after 5m21s
[11:28:00.650] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:24:0)
[11:28:00.667] Build status: STARTED
[11:28:00.667] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:25:0) after 2m14s
[11:30:14.700] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:25:0)
[11:30:14.717] Build status: STARTED
[11:30:14.717] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:26:0) after 1m33s
[11:31:47.823] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:26:0)
[11:31:47.842] Build status: STARTED
[11:31:47.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:27:0) after 5m21s
[11:37:08.904] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:27:0)
[11:37:08.921] Build status: STARTED
[11:37:08.921] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:28:0) after 5m47s
[11:42:55.910] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:28:0)
[11:42:55.926] Build status: STARTED
[11:42:55.926] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:29:0) after 6m5s
[11:49:00.942] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:29:0)
[11:49:00.958] Build status: STARTED
[11:49:00.958] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:30:0) after 1m39s
[11:50:39.976] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:30:0)
[11:50:39.993] Build status: STARTED
[11:50:39.993] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:31:0) after 7m43s
[11:58:23.043] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:31:0)
[11:58:23.060] Build status: STARTED
[11:58:23.060] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:32:0) after 1m37s
[12:00:00.077] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:32:0)
[12:00:00.095] Build status: STARTED
[12:00:00.096] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:33:0) after 5m7s
[12:05:07.115] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:33:0)
[12:05:07.134] Build status: STARTED
[12:05:07.134] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:34:0) after 8m3s
[12:13:10.156] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:34:0)
[12:13:10.179] Build status: STARTED
[12:13:10.179] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:35:0) after 2m58s
[12:16:08.273] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:35:0)
[12:16:08.293] Build status: STARTED
[12:16:08.293] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:36:0) after 9m22s
[12:25:30.354] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:36:0)
[12:25:30.371] Build status: STARTED
[12:25:30.371] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:37:0) after 3m29s
[12:28:59.394] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:37:0)
[12:28:59.408] Build status: STARTED
[12:28:59.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:38:0) after 2m40s
[12:31:39.564] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:38:0)
[12:31:39.583] Build status: STARTED
[12:31:39.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:39:0) after 7m32s
[12:39:11.629] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:39:0)
[12:39:11.656] Build status: STARTED
[12:39:11.656] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:40:0) after 7m35s
[12:46:46.724] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:40:0)
[12:46:46.859] Build status: STARTED
[12:46:46.859] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:41:0) after 6m51s
[12:53:37.880] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:41:0)
[12:53:37.896] Build status: STARTED
[12:53:37.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:42:0) after 5m17s
[12:58:54.960] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:42:0)
[12:58:54.979] Build status: STARTED
[12:58:54.980] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:43:0) after 7m18s
[13:06:12.966] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:43:0)
[13:06:12.999] Build status: STARTED
[13:06:12.999] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:44:0) after 2m6s
[13:08:19.019] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:44:0)
[13:08:19.042] Build status: STARTED
[13:08:19.042] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:45:0) after 5m40s
[13:13:59.110] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:45:0)
[13:13:59.137] Build status: STARTED
[13:13:59.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:46:0) after 1m41s
[13:15:40.205] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:46:0)
[13:15:40.223] Build status: STARTED
[13:15:40.223] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:47:0) after 1m11s
[13:16:51.309] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:47:0)
[13:16:51.330] Build status: STARTED
[13:16:51.330] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:48:0) after 1m25s
[13:18:16.379] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:48:0)
[13:18:16.398] Build status: STARTED
[13:18:16.398] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:49:0) after 8m1s
[13:26:17.468] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:49:0)
[13:26:17.486] Build status: STARTED
[13:26:17.486] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:50:0) after 1m28s
[13:27:45.476] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:50:0)
[13:27:45.529] Build status: STARTED
[13:27:45.529] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:51:0) after 4m55s
[13:32:40.578] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:51:0)
[13:32:40.594] Build status: STARTED
[13:32:40.594] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:52:0) after 4m25s
[13:37:05.610] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:52:0)
[13:37:05.625] Build status: STARTED
[13:37:05.625] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:53:0) after 5m13s
[13:42:18.664] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:53:0)
[13:42:18.680] Build status: STARTED
[13:42:18.680] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:54:0) after 8m40s
[13:50:58.774] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:54:0)
[13:50:58.790] Build status: STARTED
[13:50:58.790] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:55:0) after 4m21s
[13:55:19.814] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:55:0)
[13:55:19.841] Build status: STARTED
[13:55:19.841] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:56:0) after 7m23s
[14:02:42.873] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:56:0)
[14:02:42.896] Build status: STARTED
[14:02:42.897] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:57:0) after 6m55s
[14:09:37.951] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:57:0)
[14:09:37.974] Build status: STARTED
[14:09:37.974] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:58:0) after 6m15s
[14:15:52.993] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:58:0)
[14:15:53.012] Build status: STARTED
[14:15:53.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:59:0) after 6m19s
[14:22:12.029] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:59:0)
[14:22:12.045] Build status: STARTED
[14:22:12.045] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:60:0) after 6m58s
[14:29:10.061] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:60:0)
[14:29:10.079] Build status: STARTED
[14:29:10.079] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:61:0) after 3m44s
[14:32:54.100] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:61:0)
[14:32:54.121] Build status: STARTED
[14:32:54.121] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:62:0) after 8m39s
[14:41:33.427] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:62:0)
[14:41:33.553] Build status: STARTED
[14:41:33.553] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:63:0) after 2m16s
[14:43:49.686] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:63:0)
[14:43:49.704] Build status: STARTED
[14:43:49.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:64:0) after 6m40s
[14:50:29.684] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:64:0)
[14:50:29.702] Build status: STARTED
[14:50:29.702] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:65:0) after 4m41s
[14:55:10.728] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:65:0)
[14:55:10.763] Build status: STARTED
[14:55:10.763] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:66:0) after 6m42s
[15:01:52.783] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:66:0)
[15:01:52.804] Build status: STARTED
[15:01:52.804] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:67:0) after 8m26s
[15:10:18.873] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:67:0)
[15:10:18.888] Build status: STARTED
[15:10:18.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:68:0) after 3m5s
[15:13:23.871] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:68:0)
[15:13:23.891] Build status: STARTED
[15:13:23.891] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:69:0) after 4m4s
[15:17:27.910] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:69:0)
[15:17:27.928] Build status: STARTED
[15:17:27.928] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:70:0) after 3m54s
[15:21:22.050] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:70:0)
[15:21:22.067] Build status: STARTED
[15:21:22.067] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:71:0) after 2m29s
[15:23:51.087] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:71:0)
[15:23:51.101] Build status: STARTED
[15:23:51.101] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8851345332896255008:72:0) after 6m2s
[15:24:09.322] Received PubSub notification, asking Buildbucket for the build status
[15:24:09.339] Build:
{
"id": "8685915271051279185",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 40106,
"createdBy": "project:chromium",
"createTime": "2026-03-30T09:25:13.725525682Z",
"startTime": "2026-03-30T09:25:54.745641017Z",
"endTime": "2026-03-30T15:24:08.679584220Z",
"updateTime": "2026-03-30T15:24:08.679584220Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "eb8fecbf0579291e5b741d07f24e1206c326c29a",
"ref": "refs/heads/main"
}
}
}
[15:24:09.339] Invocation finished in 5h58m56.757381976s with status FAILED