[04:31:58.252] New invocation is queued and will start shortly
[04:31:59.398] Starting the invocation (attempt 1)
[04:31:59.434] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/e3c0b9ca8088e3b72dfce9a1ed2ad2de86ce9b4e"
[04:31:59.434] Popped gitiles commit info from properties and tags
[04:31:59.434] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[04:31:59.434] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[04:31:59.434] Buildbucket request:
{
"requestId": "8881623168683333104",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e3c0b9ca8088e3b72dfce9a1ed2ad2de86ce9b4e",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8881623168683333104"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMac"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[04:31:59.923] Scheduled build:
{
"id": "8716193106704692865",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 35978,
"createdBy": "project:chromium",
"createTime": "2025-04-30T04:31:59.506347767Z",
"updateTime": "2025-04-30T04:31:59.506347767Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e3c0b9ca8088e3b72dfce9a1ed2ad2de86ce9b4e",
"ref": "refs/heads/main"
}
},
"canary": true
}
[04:31:59.923] Task URL: https://cr-buildbucket.appspot.com/build/8716193106704692865
[04:31:59.923] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:2:0) after 7m11s
[04:32:00.907] Received PubSub notification, asking Buildbucket for the build status
[04:32:00.990] Build status: SCHEDULED
[04:32:03.207] Received PubSub notification, asking Buildbucket for the build status
[04:32:03.234] Build status: STARTED
[04:39:10.942] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:2:0)
[04:39:10.966] Build status: STARTED
[04:39:10.966] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:5:0) after 7m8s
[04:46:19.009] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:5:0)
[04:46:19.027] Build status: STARTED
[04:46:19.027] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:6:0) after 9m58s
[04:56:17.051] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:6:0)
[04:56:17.074] Build status: STARTED
[04:56:17.074] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:7:0) after 7m58s
[05:04:15.095] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:7:0)
[05:04:15.116] Build status: STARTED
[05:04:15.116] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:8:0) after 8m57s
[05:13:12.198] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:8:0)
[05:13:12.225] Build status: STARTED
[05:13:12.225] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:9:0) after 7m6s
[05:20:18.232] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:9:0)
[05:20:18.292] Build status: STARTED
[05:20:18.292] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:10:0) after 9m34s
[05:29:52.318] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:10:0)
[05:29:52.356] Build status: STARTED
[05:29:52.356] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:11:0) after 3m21s
[05:33:13.385] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:11:0)
[05:33:13.413] Build status: STARTED
[05:33:13.413] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:12:0) after 4m15s
[05:37:28.439] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:12:0)
[05:37:28.459] Build status: STARTED
[05:37:28.459] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:13:0) after 2m8s
[05:39:36.583] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:13:0)
[05:39:36.608] Build status: STARTED
[05:39:36.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:14:0) after 6m34s
[05:46:10.652] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:14:0)
[05:46:10.682] Build status: STARTED
[05:46:10.682] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:15:0) after 1m3s
[05:47:13.705] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:15:0)
[05:47:13.754] Build status: STARTED
[05:47:13.754] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:16:0) after 4m3s
[05:51:16.800] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:16:0)
[05:51:16.828] Build status: STARTED
[05:51:16.828] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:17:0) after 7m28s
[05:58:44.819] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:17:0)
[05:58:44.842] Build status: STARTED
[05:58:44.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:18:0) after 4m45s
[06:03:29.864] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:18:0)
[06:03:29.944] Build status: STARTED
[06:03:29.944] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:19:0) after 8m40s
[06:12:09.972] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:19:0)
[06:12:09.994] Build status: STARTED
[06:12:09.994] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:20:0) after 4m20s
[06:16:30.017] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:20:0)
[06:16:30.033] Build status: STARTED
[06:16:30.033] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:21:0) after 4m29s
[06:20:59.060] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:21:0)
[06:20:59.084] Build status: STARTED
[06:20:59.084] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:22:0) after 2m30s
[06:23:29.081] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:22:0)
[06:23:29.102] Build status: STARTED
[06:23:29.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:23:0) after 2m16s
[06:25:45.168] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:23:0)
[06:25:45.186] Build status: STARTED
[06:25:45.186] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:24:0) after 8m47s
[06:34:32.253] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:24:0)
[06:34:32.276] Build status: STARTED
[06:34:32.276] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:25:0) after 9m41s
[06:44:13.299] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:25:0)
[06:44:13.323] Build status: STARTED
[06:44:13.323] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:26:0) after 9m2s
[06:53:15.347] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:26:0)
[06:53:15.363] Build status: STARTED
[06:53:15.363] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:27:0) after 1m29s
[06:54:44.499] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:27:0)
[06:54:44.515] Build status: STARTED
[06:54:44.515] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:28:0) after 8m45s
[07:03:29.588] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:28:0)
[07:03:29.697] Build status: STARTED
[07:03:29.697] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:29:0) after 4m29s
[07:07:58.722] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:29:0)
[07:07:58.751] Build status: STARTED
[07:07:58.751] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:30:0) after 2m1s
[07:09:59.773] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:30:0)
[07:09:59.842] Build status: STARTED
[07:09:59.842] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:31:0) after 9m57s
[07:19:56.866] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:31:0)
[07:19:56.886] Build status: STARTED
[07:19:56.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:32:0) after 8m29s
[07:28:26.009] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:32:0)
[07:28:26.029] Build status: STARTED
[07:28:26.029] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:33:0) after 5m10s
[07:33:36.136] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:33:0)
[07:33:36.154] Build status: STARTED
[07:33:36.154] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:34:0) after 1m35s
[07:35:11.215] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:34:0)
[07:35:11.237] Build status: STARTED
[07:35:11.237] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:35:0) after 1m7s
[07:36:18.280] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:35:0)
[07:36:18.298] Build status: STARTED
[07:36:18.298] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:36:0) after 8m17s
[07:44:35.317] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:36:0)
[07:44:35.340] Build status: STARTED
[07:44:35.340] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:37:0) after 6m8s
[07:50:43.472] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:37:0)
[07:50:43.487] Build status: STARTED
[07:50:43.487] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:38:0) after 2m49s
[07:53:32.508] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:38:0)
[07:53:32.524] Build status: STARTED
[07:53:32.524] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:39:0) after 1m20s
[07:54:52.590] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:39:0)
[07:54:52.608] Build status: STARTED
[07:54:52.609] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:40:0) after 1m15s
[07:56:07.726] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:40:0)
[07:56:07.745] Build status: STARTED
[07:56:07.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:41:0) after 1m30s
[07:57:37.772] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:41:0)
[07:57:37.813] Build status: STARTED
[07:57:37.813] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:42:0) after 9m17s
[08:06:54.950] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:42:0)
[08:06:55.133] Build status: STARTED
[08:06:55.133] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:43:0) after 8m56s
[08:15:51.218] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:43:0)
[08:15:51.237] Build status: STARTED
[08:15:51.237] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:44:0) after 1m11s
[08:17:02.261] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:44:0)
[08:17:02.277] Build status: STARTED
[08:17:02.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:45:0) after 3m8s
[08:20:10.294] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:45:0)
[08:20:10.339] Build status: STARTED
[08:20:10.339] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:46:0) after 1m14s
[08:21:24.367] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:46:0)
[08:21:24.383] Build status: STARTED
[08:21:24.383] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:47:0) after 8m49s
[08:30:13.411] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:47:0)
[08:30:13.431] Build status: STARTED
[08:30:13.431] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:48:0) after 2m30s
[08:32:43.458] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:48:0)
[08:32:43.482] Build status: STARTED
[08:32:43.482] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:49:0) after 6m17s
[08:39:00.704] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:49:0)
[08:39:00.724] Build status: STARTED
[08:39:00.724] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:50:0) after 9m0s
[08:48:00.721] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:50:0)
[08:48:00.744] Build status: STARTED
[08:48:00.744] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:51:0) after 9m41s
[08:57:41.817] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:51:0)
[08:57:41.839] Build status: STARTED
[08:57:41.839] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:52:0) after 9m44s
[09:07:25.982] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:52:0)
[09:07:25.995] Build status: STARTED
[09:07:25.995] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:53:0) after 9m50s
[09:17:16.028] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:53:0)
[09:17:16.055] Build status: STARTED
[09:17:16.055] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:54:0) after 5m30s
[09:22:46.085] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:54:0)
[09:22:46.103] Build status: STARTED
[09:22:46.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:55:0) after 9m8s
[09:31:54.133] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:55:0)
[09:31:54.147] Build status: STARTED
[09:31:54.147] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:56:0) after 3m7s
[09:35:01.416] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:56:0)
[09:35:01.431] Build status: STARTED
[09:35:01.432] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:57:0) after 1m10s
[09:36:11.460] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:57:0)
[09:36:11.481] Build status: STARTED
[09:36:11.481] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:58:0) after 4m12s
[09:40:23.689] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:58:0)
[09:40:23.704] Build status: STARTED
[09:40:23.704] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:59:0) after 7m32s
[09:47:56.058] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:59:0)
[09:47:56.075] Build status: STARTED
[09:47:56.076] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:60:0) after 6m0s
[09:53:56.101] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:60:0)
[09:53:56.120] Build status: STARTED
[09:53:56.120] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:61:0) after 2m48s
[09:56:44.250] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:61:0)
[09:56:44.270] Build status: STARTED
[09:56:44.270] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:62:0) after 9m29s
[10:06:13.465] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:62:0)
[10:06:13.478] Build status: STARTED
[10:06:13.478] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:63:0) after 9m47s
[10:16:00.736] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:63:0)
[10:16:00.766] Build status: STARTED
[10:16:00.766] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:64:0) after 3m8s
[10:19:08.798] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:64:0)
[10:19:08.815] Build status: STARTED
[10:19:08.815] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8881623168683333104:65:0) after 9m28s
[10:22:35.317] Received PubSub notification, asking Buildbucket for the build status
[10:22:35.370] Build:
{
"id": "8716193106704692865",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 35978,
"createdBy": "project:chromium",
"createTime": "2025-04-30T04:31:59.506347767Z",
"startTime": "2025-04-30T04:32:02.970179091Z",
"endTime": "2025-04-30T10:22:35.005965302Z",
"updateTime": "2025-04-30T10:22:35.005965302Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "e3c0b9ca8088e3b72dfce9a1ed2ad2de86ce9b4e",
"ref": "refs/heads/main"
}
},
"canary": true
}
[10:22:35.370] Invocation finished in 5h50m37.134370995s with status FAILED