[03:49:25.530] New invocation is queued and will start shortly
[03:49:27.188] Starting the invocation (attempt 1)
[03:49:27.212] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/6fc114eecd6ab78d740565bb1326805670382bd2"
[03:49:27.212] Popped gitiles commit info from properties and tags
[03:49:27.212] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[03:49:27.214] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[03:49:27.214] Buildbucket request:
{
"requestId": "8841581986245853520",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "linux-arm64-rel-fyi"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "6fc114eecd6ab78d740565bb1326805670382bd2",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8841581986245853520"
},
{
"key": "scheduler_job_id",
"value": "chromium/linux-arm64-rel-fyi"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[03:49:27.637] Scheduled build:
{
"id": "8676151923767602305",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "linux-arm64-rel-fyi"
},
"number": 963,
"createdBy": "project:chromium",
"createTime": "2026-07-16T03:49:27.272845363Z",
"updateTime": "2026-07-16T03:49:27.272845363Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "6fc114eecd6ab78d740565bb1326805670382bd2",
"ref": "refs/heads/main"
}
},
"canary": true
}
[03:49:27.637] Task URL: https://cr-buildbucket.appspot.com/build/8676151923767602305
[03:49:27.637] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:2:0) after 5m48s
[03:49:28.646] Received PubSub notification, asking Buildbucket for the build status
[03:49:28.664] Build status: SCHEDULED
[03:50:16.714] Received PubSub notification, asking Buildbucket for the build status
[03:50:17.037] Build status: STARTED
[03:55:15.825] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:2:0)
[03:55:15.843] Build status: STARTED
[03:55:15.843] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:5:0) after 3m16s
[03:58:31.964] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:5:0)
[03:58:31.981] Build status: STARTED
[03:58:31.981] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:6:0) after 9m14s
[04:07:46.013] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:6:0)
[04:07:46.078] Build status: STARTED
[04:07:46.078] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:7:0) after 5m35s
[04:13:21.098] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:7:0)
[04:13:21.117] Build status: STARTED
[04:13:21.117] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:8:0) after 2m12s
[04:15:33.160] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:8:0)
[04:15:33.255] Build status: STARTED
[04:15:33.255] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:9:0) after 7m11s
[04:22:44.297] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:9:0)
[04:22:44.323] Build status: STARTED
[04:22:44.323] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:10:0) after 8m28s
[04:31:12.316] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:10:0)
[04:31:12.336] Build status: STARTED
[04:31:12.336] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:11:0) after 3m43s
[04:34:55.393] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:11:0)
[04:34:55.427] Build status: STARTED
[04:34:55.427] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:12:0) after 8m7s
[04:43:02.420] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:12:0)
[04:43:02.523] Build status: STARTED
[04:43:02.523] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:13:0) after 3m14s
[04:46:16.536] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:13:0)
[04:46:16.559] Build status: STARTED
[04:46:16.559] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:14:0) after 8m48s
[04:55:04.626] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:14:0)
[04:55:04.640] Build status: STARTED
[04:55:04.641] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:15:0) after 6m52s
[05:01:56.685] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:15:0)
[05:01:56.722] Build status: STARTED
[05:01:56.722] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:16:0) after 8m59s
[05:10:56.030] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:16:0)
[05:10:56.051] Build status: STARTED
[05:10:56.051] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:17:0) after 3m15s
[05:14:11.166] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:17:0)
[05:14:11.186] Build status: STARTED
[05:14:11.186] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:18:0) after 7m5s
[05:21:16.210] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:18:0)
[05:21:16.306] Build status: STARTED
[05:21:16.306] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:19:0) after 2m25s
[05:23:41.344] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:19:0)
[05:23:41.361] Build status: STARTED
[05:23:41.361] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:20:0) after 2m21s
[05:26:02.382] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:20:0)
[05:26:02.407] Build status: STARTED
[05:26:02.408] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:21:0) after 5m35s
[05:31:37.436] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:21:0)
[05:31:37.455] Build status: STARTED
[05:31:37.455] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:22:0) after 2m46s
[05:34:23.548] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:22:0)
[05:34:23.573] Build status: STARTED
[05:34:23.573] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:23:0) after 5m39s
[05:40:02.594] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:23:0)
[05:40:02.621] Build status: STARTED
[05:40:02.622] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:24:0) after 8m31s
[05:48:33.689] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:24:0)
[05:48:33.706] Build status: STARTED
[05:48:33.706] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:25:0) after 1m17s
[05:49:50.852] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:25:0)
[05:49:50.911] Build status: STARTED
[05:49:50.911] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:26:0) after 4m20s
[05:54:10.965] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:26:0)
[05:54:11.011] Build status: STARTED
[05:54:11.011] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:27:0) after 4m20s
[05:58:31.027] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:27:0)
[05:58:31.044] Build status: STARTED
[05:58:31.044] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:28:0) after 9m37s
[06:08:08.115] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:28:0)
[06:08:08.133] Build status: STARTED
[06:08:08.133] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:29:0) after 9m9s
[06:17:17.214] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:29:0)
[06:17:17.231] Build status: STARTED
[06:17:17.231] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:30:0) after 6m37s
[06:23:54.285] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:30:0)
[06:23:54.305] Build status: STARTED
[06:23:54.305] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:31:0) after 1m10s
[06:25:04.327] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:31:0)
[06:25:04.353] Build status: STARTED
[06:25:04.353] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:32:0) after 1m38s
[06:26:42.397] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:32:0)
[06:26:42.423] Build status: STARTED
[06:26:42.423] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:33:0) after 9m53s
[06:36:35.441] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:33:0)
[06:36:35.458] Build status: STARTED
[06:36:35.458] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:34:0) after 1m44s
[06:38:19.598] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:34:0)
[06:38:19.618] Build status: STARTED
[06:38:19.618] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:35:0) after 2m11s
[06:40:30.645] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:35:0)
[06:40:30.682] Build status: STARTED
[06:40:30.682] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:36:0) after 7m33s
[06:48:03.715] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:36:0)
[06:48:03.733] Build status: STARTED
[06:48:03.733] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:37:0) after 3m15s
[06:51:18.913] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:37:0)
[06:51:18.960] Build status: STARTED
[06:51:18.960] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:38:0) after 3m28s
[06:54:47.066] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:38:0)
[06:54:47.085] Build status: STARTED
[06:54:47.085] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:39:0) after 1m51s
[06:56:38.206] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:39:0)
[06:56:38.229] Build status: STARTED
[06:56:38.229] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:40:0) after 7m42s
[07:04:20.226] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:40:0)
[07:04:20.248] Build status: STARTED
[07:04:20.248] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:41:0) after 7m30s
[07:11:50.320] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:41:0)
[07:11:50.336] Build status: STARTED
[07:11:50.336] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:42:0) after 3m22s
[07:15:12.370] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:42:0)
[07:15:12.428] Build status: STARTED
[07:15:12.428] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:43:0) after 9m11s
[07:24:23.512] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:43:0)
[07:24:23.529] Build status: STARTED
[07:24:23.530] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:44:0) after 5m47s
[07:30:10.555] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:44:0)
[07:30:10.588] Build status: STARTED
[07:30:10.588] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:45:0) after 9m50s
[07:40:00.607] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:45:0)
[07:40:00.649] Build status: STARTED
[07:40:00.649] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:46:0) after 7m21s
[07:47:21.779] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:46:0)
[07:47:21.800] Build status: STARTED
[07:47:21.800] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:47:0) after 2m54s
[07:50:15.868] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:47:0)
[07:50:15.893] Build status: STARTED
[07:50:15.893] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:48:0) after 4m44s
[07:54:59.911] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:48:0)
[07:54:59.949] Build status: STARTED
[07:54:59.949] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:49:0) after 6m9s
[08:01:09.016] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:49:0)
[08:01:09.037] Build status: STARTED
[08:01:09.037] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:50:0) after 6m24s
[08:07:33.097] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:50:0)
[08:07:33.117] Build status: STARTED
[08:07:33.117] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:51:0) after 3m41s
[08:11:14.092] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:51:0)
[08:11:14.129] Build status: STARTED
[08:11:14.130] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:52:0) after 5m26s
[08:16:40.224] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:52:0)
[08:16:40.422] Build status: STARTED
[08:16:40.422] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:53:0) after 8m18s
[08:24:58.447] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:53:0)
[08:24:58.488] Build status: STARTED
[08:24:58.488] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:54:0) after 5m23s
[08:30:21.566] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:54:0)
[08:30:21.602] Build status: STARTED
[08:30:21.602] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:55:0) after 1m51s
[08:32:12.671] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:55:0)
[08:32:12.700] Build status: STARTED
[08:32:12.700] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:56:0) after 9m23s
[08:41:35.761] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:56:0)
[08:41:35.816] Build status: STARTED
[08:41:35.816] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:57:0) after 7m21s
[08:48:56.900] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:57:0)
[08:48:56.917] Build status: STARTED
[08:48:56.917] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:58:0) after 5m59s
[08:54:56.001] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:58:0)
[08:54:56.018] Build status: STARTED
[08:54:56.018] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:59:0) after 8m26s
[09:03:22.029] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:59:0)
[09:03:22.043] Build status: STARTED
[09:03:22.043] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:60:0) after 7m18s
[09:10:40.076] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:60:0)
[09:10:40.097] Build status: STARTED
[09:10:40.097] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:61:0) after 6m59s
[09:17:39.166] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:61:0)
[09:17:39.185] Build status: STARTED
[09:17:39.185] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:62:0) after 7m24s
[09:25:03.305] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:62:0)
[09:25:03.325] Build status: STARTED
[09:25:03.325] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:63:0) after 8m15s
[09:33:18.346] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:63:0)
[09:33:18.420] Build status: STARTED
[09:33:18.420] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:64:0) after 6m19s
[09:39:37.564] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:64:0)
[09:39:37.587] Build status: STARTED
[09:39:37.587] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:65:0) after 5m34s
[09:45:11.602] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:65:0)
[09:45:11.624] Build status: STARTED
[09:45:11.624] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:66:0) after 5m27s
[09:50:38.652] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:66:0)
[09:50:38.687] Build status: STARTED
[09:50:38.687] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:67:0) after 7m40s
[09:58:18.692] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:67:0)
[09:58:18.752] Build status: STARTED
[09:58:18.753] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:68:0) after 8m41s
[10:06:59.772] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:68:0)
[10:06:59.801] Build status: STARTED
[10:06:59.801] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:69:0) after 7m40s
[10:14:39.978] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:69:0)
[10:14:40.000] Build status: STARTED
[10:14:40.000] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:70:0) after 2m56s
[10:17:36.263] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:70:0)
[10:17:36.288] Build status: STARTED
[10:17:36.289] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:71:0) after 7m19s
[10:24:55.357] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:71:0)
[10:24:55.430] Build status: STARTED
[10:24:55.430] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:72:0) after 7m33s
[10:32:28.459] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:72:0)
[10:32:28.478] Build status: STARTED
[10:32:28.478] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:73:0) after 3m0s
[10:35:28.499] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:73:0)
[10:35:28.539] Build status: STARTED
[10:35:28.539] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:74:0) after 3m54s
[10:39:22.587] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:74:0)
[10:39:22.606] Build status: STARTED
[10:39:22.606] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:75:0) after 5m12s
[10:44:34.660] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:75:0)
[10:44:34.686] Build status: STARTED
[10:44:34.686] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:76:0) after 4m53s
[10:49:27.709] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:76:0)
[10:49:27.729] Build status: STARTED
[10:49:27.729] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:77:0) after 2m29s
[10:51:56.821] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:77:0)
[10:51:56.840] Build status: STARTED
[10:51:56.840] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:78:0) after 7m17s
[10:59:13.901] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:78:0)
[10:59:13.923] Build status: STARTED
[10:59:13.923] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:79:0) after 6m15s
[11:05:29.038] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:79:0)
[11:05:29.061] Build status: STARTED
[11:05:29.061] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:80:0) after 5m1s
[11:10:30.090] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:80:0)
[11:10:30.107] Build status: STARTED
[11:10:30.107] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:81:0) after 6m29s
[11:16:59.178] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:81:0)
[11:16:59.205] Build status: STARTED
[11:16:59.205] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:82:0) after 8m18s
[11:25:17.264] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:82:0)
[11:25:17.289] Build status: STARTED
[11:25:17.289] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:83:0) after 6m24s
[11:31:41.388] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:83:0)
[11:31:41.410] Build status: STARTED
[11:31:41.410] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:84:0) after 2m0s
[11:33:41.454] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:84:0)
[11:33:41.600] Build status: STARTED
[11:33:41.600] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:85:0) after 3m43s
[11:37:24.710] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:85:0)
[11:37:24.746] Build status: STARTED
[11:37:24.746] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:86:0) after 3m33s
[11:40:57.745] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:86:0)
[11:40:57.786] Build status: STARTED
[11:40:57.786] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:87:0) after 6m37s
[11:47:34.846] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:87:0)
[11:47:34.867] Build status: STARTED
[11:47:34.867] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:88:0) after 7m7s
[11:54:41.920] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:88:0)
[11:54:42.002] Build status: STARTED
[11:54:42.002] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:89:0) after 1m1s
[11:55:43.129] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:89:0)
[11:55:43.148] Build status: STARTED
[11:55:43.148] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:90:0) after 2m53s
[11:58:36.170] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:90:0)
[11:58:36.193] Build status: STARTED
[11:58:36.193] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:91:0) after 3m52s
[12:02:28.264] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:91:0)
[12:02:28.282] Build status: STARTED
[12:02:28.282] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:92:0) after 5m23s
[12:07:51.369] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:92:0)
[12:07:51.405] Build status: STARTED
[12:07:51.405] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:93:0) after 6m26s
[12:14:17.481] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:93:0)
[12:14:17.502] Build status: STARTED
[12:14:17.503] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:94:0) after 1m58s
[12:16:15.535] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:94:0)
[12:16:15.560] Build status: STARTED
[12:16:15.561] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:95:0) after 5m9s
[12:21:24.597] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:95:0)
[12:21:24.614] Build status: STARTED
[12:21:24.615] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:96:0) after 6m34s
[12:27:58.695] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:96:0)
[12:27:58.725] Build status: STARTED
[12:27:58.725] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:97:0) after 5m0s
[12:32:58.755] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:97:0)
[12:32:58.773] Build status: STARTED
[12:32:58.773] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:98:0) after 4m17s
[12:37:15.804] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:98:0)
[12:37:15.828] Build status: STARTED
[12:37:15.828] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:99:0) after 7m17s
[12:44:32.953] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:99:0)
[12:44:32.987] Build status: STARTED
[12:44:32.987] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:100:0) after 5m17s
[12:49:50.078] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:100:0)
[12:49:50.097] Build status: STARTED
[12:49:50.097] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:101:0) after 6m0s
[12:55:50.177] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:101:0)
[12:55:50.202] Build status: STARTED
[12:55:50.202] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:102:0) after 1m46s
[12:57:36.219] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:102:0)
[12:57:36.240] Build status: STARTED
[12:57:36.240] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:103:0) after 4m41s
[13:02:17.288] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:103:0)
[13:02:17.310] Build status: STARTED
[13:02:17.310] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:104:0) after 6m44s
[13:09:01.372] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:104:0)
[13:09:01.389] Build status: STARTED
[13:09:01.389] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:105:0) after 4m0s
[13:13:01.415] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:105:0)
[13:13:01.494] Build status: STARTED
[13:13:01.494] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:106:0) after 6m42s
[13:19:43.517] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:106:0)
[13:19:43.534] Build status: STARTED
[13:19:43.534] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:107:0) after 2m28s
[13:22:11.549] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:107:0)
[13:22:11.582] Build status: STARTED
[13:22:11.582] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:108:0) after 7m47s
[13:29:58.652] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:108:0)
[13:29:58.718] Build status: STARTED
[13:29:58.718] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:109:0) after 9m13s
[13:39:11.807] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:109:0)
[13:39:11.848] Build status: STARTED
[13:39:11.849] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:110:0) after 6m16s
[13:45:27.902] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:110:0)
[13:45:27.927] Build status: STARTED
[13:45:27.927] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:111:0) after 2m39s
[13:48:06.949] Handling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:111:0)
[13:48:06.967] Build status: STARTED
[13:48:06.967] Scheduling timer "check-buildbucket-build-status" (chromium/linux-arm64-rel-fyi:8841581986245853520:112:0) after 5m14s
[13:53:00.165] Received PubSub notification, asking Buildbucket for the build status
[13:53:00.183] Build:
{
"id": "8676151923767602305",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "linux-arm64-rel-fyi"
},
"number": 963,
"createdBy": "project:chromium",
"createTime": "2026-07-16T03:49:27.272845363Z",
"startTime": "2026-07-16T03:50:16.525700440Z",
"endTime": "2026-07-16T13:52:59.793269672Z",
"updateTime": "2026-07-16T13:52:59.793269672Z",
"status": "INFRA_FAILURE",
"statusDetails": {
"timeout": {
}
},
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "6fc114eecd6ab78d740565bb1326805670382bd2",
"ref": "refs/heads/main"
}
},
"canary": true
}
[13:53:00.183] Invocation finished in 10h3m34.6582016s with status FAILED