[01:04:19.317] New invocation is queued and will start shortly
[01:04:21.192] Starting the invocation (attempt 1)
[01:04:21.223] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/b4f8bf2e92718d6832e901f1fcd143a20423bdd1"
[01:04:21.223] Popped gitiles commit info from properties and tags
[01:04:21.223] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[01:04:21.223] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[01:04:21.223] Buildbucket request:
{
"requestId": "8874660266404326784",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b4f8bf2e92718d6832e901f1fcd143a20423bdd1",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8874660266404326784"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMac"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[01:04:21.792] Scheduled build:
{
"id": "8709230203633434801",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 37076,
"createdBy": "project:chromium",
"createTime": "2025-07-16T01:04:21.333338479Z",
"updateTime": "2025-07-16T01:04:21.333338479Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b4f8bf2e92718d6832e901f1fcd143a20423bdd1",
"ref": "refs/heads/main"
}
},
"canary": true
}
[01:04:21.792] Task URL: https://cr-buildbucket.appspot.com/build/8709230203633434801
[01:04:21.792] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:2:0) after 1m8s
[01:04:22.847] Received PubSub notification, asking Buildbucket for the build status
[01:04:22.866] Build status: SCHEDULED
[01:04:25.017] Received PubSub notification, asking Buildbucket for the build status
[01:04:25.037] Build status: STARTED
[01:05:29.858] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:2:0)
[01:05:29.885] Build status: STARTED
[01:05:29.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:5:0) after 2m49s
[01:08:18.904] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:5:0)
[01:08:18.932] Build status: STARTED
[01:08:18.932] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:6:0) after 8m53s
[01:17:11.953] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:6:0)
[01:17:11.982] Build status: STARTED
[01:17:11.982] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:7:0) after 8m58s
[01:26:10.106] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:7:0)
[01:26:10.134] Build status: STARTED
[01:26:10.134] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:8:0) after 4m29s
[01:30:39.204] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:8:0)
[01:30:39.254] Build status: STARTED
[01:30:39.254] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:9:0) after 7m0s
[01:37:39.335] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:9:0)
[01:37:39.368] Build status: STARTED
[01:37:39.368] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:10:0) after 6m2s
[01:43:41.393] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:10:0)
[01:43:41.421] Build status: STARTED
[01:43:41.421] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:11:0) after 3m22s
[01:47:03.440] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:11:0)
[01:47:03.495] Build status: STARTED
[01:47:03.495] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:12:0) after 3m28s
[01:50:31.563] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:12:0)
[01:50:31.584] Build status: STARTED
[01:50:31.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:13:0) after 3m1s
[01:53:32.560] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:13:0)
[01:53:32.584] Build status: STARTED
[01:53:32.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:14:0) after 7m36s
[02:01:08.638] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:14:0)
[02:01:08.676] Build status: STARTED
[02:01:08.676] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:15:0) after 1m36s
[02:02:44.689] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:15:0)
[02:02:44.707] Build status: STARTED
[02:02:44.707] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:16:0) after 3m0s
[02:05:44.798] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:16:0)
[02:05:44.827] Build status: STARTED
[02:05:44.827] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:17:0) after 6m11s
[02:11:55.852] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:17:0)
[02:11:55.903] Build status: STARTED
[02:11:55.903] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:18:0) after 8m14s
[02:20:09.977] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:18:0)
[02:20:09.998] Build status: STARTED
[02:20:09.998] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:19:0) after 5m26s
[02:25:36.022] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:19:0)
[02:25:36.042] Build status: STARTED
[02:25:36.042] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:20:0) after 6m31s
[02:32:07.073] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:20:0)
[02:32:07.088] Build status: STARTED
[02:32:07.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:21:0) after 8m23s
[02:40:30.156] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:21:0)
[02:40:30.177] Build status: STARTED
[02:40:30.177] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:22:0) after 1m26s
[02:41:56.199] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:22:0)
[02:41:56.226] Build status: STARTED
[02:41:56.226] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:23:0) after 3m57s
[02:45:53.245] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:23:0)
[02:45:53.406] Build status: STARTED
[02:45:53.406] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:24:0) after 8m50s
[02:54:43.490] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:24:0)
[02:54:43.563] Build status: STARTED
[02:54:43.563] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:25:0) after 1m34s
[02:56:17.629] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:25:0)
[02:56:17.650] Build status: STARTED
[02:56:17.651] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:26:0) after 5m34s
[03:01:51.699] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:26:0)
[03:01:51.753] Build status: STARTED
[03:01:51.754] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:27:0) after 3m50s
[03:05:41.874] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:27:0)
[03:05:41.941] Build status: STARTED
[03:05:41.941] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:28:0) after 5m11s
[03:10:52.961] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:28:0)
[03:10:53.017] Build status: STARTED
[03:10:53.017] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:29:0) after 6m0s
[03:16:53.036] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:29:0)
[03:16:53.183] Build status: STARTED
[03:16:53.183] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:30:0) after 2m40s
[03:19:33.207] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:30:0)
[03:19:33.236] Build status: STARTED
[03:19:33.236] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:31:0) after 8m36s
[03:28:09.260] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:31:0)
[03:28:09.279] Build status: STARTED
[03:28:09.279] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:32:0) after 4m35s
[03:32:44.302] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:32:0)
[03:32:44.333] Build status: STARTED
[03:32:44.333] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:33:0) after 3m22s
[03:36:06.406] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:33:0)
[03:36:06.427] Build status: STARTED
[03:36:06.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:34:0) after 9m46s
[03:45:52.496] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:34:0)
[03:45:52.511] Build status: STARTED
[03:45:52.511] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:35:0) after 8m41s
[03:54:33.580] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:35:0)
[03:54:33.600] Build status: STARTED
[03:54:33.600] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:36:0) after 2m50s
[03:57:23.591] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:36:0)
[03:57:24.039] Build status: STARTED
[03:57:24.039] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:37:0) after 5m34s
[04:02:58.105] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:37:0)
[04:02:58.135] Build status: STARTED
[04:02:58.135] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:38:0) after 8m0s
[04:10:58.165] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:38:0)
[04:10:58.194] Build status: STARTED
[04:10:58.194] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:39:0) after 9m0s
[04:19:58.319] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:39:0)
[04:19:58.336] Build status: STARTED
[04:19:58.336] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:40:0) after 9m48s
[04:29:46.420] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:40:0)
[04:29:46.461] Build status: STARTED
[04:29:46.461] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:41:0) after 5m32s
[04:35:18.496] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:41:0)
[04:35:18.516] Build status: STARTED
[04:35:18.516] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:42:0) after 1m24s
[04:36:42.538] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:42:0)
[04:36:42.551] Build status: STARTED
[04:36:42.551] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:43:0) after 3m1s
[04:39:43.614] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:43:0)
[04:39:43.630] Build status: STARTED
[04:39:43.630] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:44:0) after 9m30s
[04:49:13.655] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:44:0)
[04:49:13.740] Build status: STARTED
[04:49:13.740] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:45:0) after 2m57s
[04:52:10.762] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:45:0)
[04:52:10.781] Build status: STARTED
[04:52:10.781] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:46:0) after 6m46s
[04:58:56.804] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:46:0)
[04:58:56.829] Build status: STARTED
[04:58:56.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:47:0) after 5m51s
[05:04:47.986] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:47:0)
[05:04:48.003] Build status: STARTED
[05:04:48.003] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:48:0) after 7m23s
[05:12:11.103] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:48:0)
[05:12:11.126] Build status: STARTED
[05:12:11.126] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:49:0) after 3m15s
[05:15:26.287] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:49:0)
[05:15:26.304] Build status: STARTED
[05:15:26.304] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:50:0) after 4m32s
[05:19:58.470] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:50:0)
[05:19:58.490] Build status: STARTED
[05:19:58.490] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:51:0) after 8m12s
[05:28:10.513] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:51:0)
[05:28:10.637] Build status: STARTED
[05:28:10.637] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:52:0) after 5m49s
[05:33:59.668] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:52:0)
[05:33:59.693] Build status: STARTED
[05:33:59.693] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:53:0) after 4m2s
[05:38:01.881] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:53:0)
[05:38:01.900] Build status: STARTED
[05:38:01.900] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:54:0) after 4m25s
[05:42:26.933] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:54:0)
[05:42:26.953] Build status: STARTED
[05:42:26.953] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:55:0) after 3m8s
[05:45:34.978] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:55:0)
[05:45:35.004] Build status: STARTED
[05:45:35.004] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:56:0) after 1m5s
[05:46:40.078] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:56:0)
[05:46:40.102] Build status: STARTED
[05:46:40.102] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:57:0) after 1m30s
[05:48:10.128] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:57:0)
[05:48:10.145] Build status: STARTED
[05:48:10.145] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:58:0) after 9m52s
[05:58:02.169] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:58:0)
[05:58:02.189] Build status: STARTED
[05:58:02.189] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:59:0) after 7m30s
[06:05:32.203] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:59:0)
[06:05:32.341] Build status: STARTED
[06:05:32.341] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:60:0) after 5m32s
[06:11:04.488] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:60:0)
[06:11:04.511] Build status: STARTED
[06:11:04.511] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:61:0) after 4m20s
[06:15:24.563] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:61:0)
[06:15:24.582] Build status: STARTED
[06:15:24.582] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:62:0) after 6m44s
[06:22:08.602] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:62:0)
[06:22:08.634] Build status: STARTED
[06:22:08.634] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:63:0) after 4m33s
[06:26:41.666] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:63:0)
[06:26:41.697] Build status: STARTED
[06:26:41.697] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:64:0) after 3m13s
[06:29:54.845] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:64:0)
[06:29:54.865] Build status: STARTED
[06:29:54.865] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:65:0) after 3m45s
[06:33:40.289] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:65:0)
[06:33:40.360] Build status: STARTED
[06:33:40.360] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8874660266404326784:66:0) after 9m22s
[06:34:56.574] Received PubSub notification, asking Buildbucket for the build status
[06:34:56.597] Build:
{
"id": "8709230203633434801",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 37076,
"createdBy": "project:chromium",
"createTime": "2025-07-16T01:04:21.333338479Z",
"startTime": "2025-07-16T01:04:24.747185790Z",
"endTime": "2025-07-16T06:34:55.744959284Z",
"updateTime": "2025-07-16T06:34:55.744959284Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "b4f8bf2e92718d6832e901f1fcd143a20423bdd1",
"ref": "refs/heads/main"
}
},
"canary": true
}
[06:34:56.597] Invocation finished in 5h30m37.290375237s with status FAILED