[01:47:04.098] New invocation is queued and will start shortly
[01:47:05.185] Starting the invocation (attempt 1)
[01:47:05.214] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/f3222b2949ea6d1e8d7fe09a93aad1b83303141a"
[01:47:05.214] Popped gitiles commit info from properties and tags
[01:47:05.214] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[01:47:05.215] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[01:47:05.215] Buildbucket request:
{
"requestId": "8885982197833250928",
"builder": {
"project": "chromium",
"bucket": "reclient",
"builder": "Linux Builder (canonical wd) (reclient compare)"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "f3222b2949ea6d1e8d7fe09a93aad1b83303141a",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8885982197833250928"
},
{
"key": "scheduler_job_id",
"value": "chromium/Linux Builder (canonical wd) (reclient compare)"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[01:47:06.054] Scheduled build:
{
"id": "8720552135952430497",
"builder": {
"project": "chromium",
"bucket": "reclient",
"builder": "Linux Builder (canonical wd) (reclient compare)"
},
"number": 10046,
"createdBy": "project:chromium",
"createTime": "2025-03-13T01:47:05.268320887Z",
"updateTime": "2025-03-13T01:47:05.268320887Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "f3222b2949ea6d1e8d7fe09a93aad1b83303141a",
"ref": "refs/heads/main"
}
},
"canary": true
}
[01:47:06.054] Task URL: https://cr-buildbucket.appspot.com/build/8720552135952430497
[01:47:06.054] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:2:0) after 4m39s
[01:47:07.018] Received PubSub notification, asking Buildbucket for the build status
[01:47:07.058] Build status: SCHEDULED
[01:48:17.089] Received PubSub notification, asking Buildbucket for the build status
[01:48:17.115] Build status: STARTED
[01:51:45.112] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:2:0)
[01:51:45.141] Build status: STARTED
[01:51:45.141] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:5:0) after 7m10s
[01:58:55.168] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:5:0)
[01:58:55.190] Build status: STARTED
[01:58:55.190] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:6:0) after 6m16s
[02:05:11.208] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:6:0)
[02:05:11.244] Build status: STARTED
[02:05:11.245] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:7:0) after 4m30s
[02:09:43.181] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:7:0)
[02:09:43.222] Build status: STARTED
[02:09:43.222] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:8:0) after 1m39s
[02:11:22.306] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:8:0)
[02:11:22.329] Build status: STARTED
[02:11:22.329] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:9:0) after 4m39s
[02:16:01.356] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:9:0)
[02:16:01.393] Build status: STARTED
[02:16:01.393] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:10:0) after 4m1s
[02:20:02.424] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:10:0)
[02:20:02.472] Build status: STARTED
[02:20:02.472] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:11:0) after 4m32s
[02:24:34.494] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:11:0)
[02:24:34.519] Build status: STARTED
[02:24:34.519] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:12:0) after 9m56s
[02:34:30.576] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:12:0)
[02:34:30.605] Build status: STARTED
[02:34:30.605] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:13:0) after 3m13s
[02:37:43.595] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:13:0)
[02:37:43.628] Build status: STARTED
[02:37:43.628] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:14:0) after 2m38s
[02:40:21.729] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:14:0)
[02:40:21.756] Build status: STARTED
[02:40:21.756] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:15:0) after 1m54s
[02:42:15.791] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:15:0)
[02:42:15.818] Build status: STARTED
[02:42:15.818] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:16:0) after 5m34s
[02:47:49.920] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:16:0)
[02:47:49.970] Build status: STARTED
[02:47:49.970] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:17:0) after 7m49s
[02:55:38.993] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:17:0)
[02:55:39.046] Build status: STARTED
[02:55:39.047] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:18:0) after 5m39s
[03:01:18.111] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:18:0)
[03:01:18.161] Build status: STARTED
[03:01:18.161] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:19:0) after 2m36s
[03:03:54.650] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:19:0)
[03:03:54.680] Build status: STARTED
[03:03:54.680] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:20:0) after 9m34s
[03:13:28.745] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:20:0)
[03:13:28.776] Build status: STARTED
[03:13:28.776] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:21:0) after 1m19s
[03:14:47.800] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:21:0)
[03:14:47.823] Build status: STARTED
[03:14:47.823] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:22:0) after 2m44s
[03:17:31.849] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:22:0)
[03:17:31.880] Build status: STARTED
[03:17:31.880] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:23:0) after 3m19s
[03:20:51.004] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:23:0)
[03:20:51.031] Build status: STARTED
[03:20:51.031] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:24:0) after 6m54s
[03:27:45.053] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:24:0)
[03:27:45.084] Build status: STARTED
[03:27:45.084] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:25:0) after 9m42s
[03:37:27.113] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:25:0)
[03:37:27.140] Build status: STARTED
[03:37:27.140] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:26:0) after 8m43s
[03:46:10.208] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:26:0)
[03:46:10.226] Build status: STARTED
[03:46:10.226] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:27:0) after 1m25s
[03:47:35.253] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:27:0)
[03:47:35.279] Build status: STARTED
[03:47:35.279] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:28:0) after 4m9s
[03:51:44.306] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:28:0)
[03:51:44.329] Build status: STARTED
[03:51:44.329] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:29:0) after 6m17s
[03:58:01.351] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:29:0)
[03:58:01.369] Build status: STARTED
[03:58:01.369] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:30:0) after 8m33s
[04:06:34.425] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:30:0)
[04:06:34.446] Build status: STARTED
[04:06:34.446] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:31:0) after 4m12s
[04:10:46.547] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:31:0)
[04:10:46.568] Build status: STARTED
[04:10:46.568] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:32:0) after 2m28s
[04:13:14.598] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:32:0)
[04:13:14.615] Build status: STARTED
[04:13:14.615] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:33:0) after 1m12s
[04:14:26.729] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:33:0)
[04:14:26.751] Build status: STARTED
[04:14:26.751] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:34:0) after 1m43s
[04:16:10.387] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:34:0)
[04:16:10.418] Build status: STARTED
[04:16:10.418] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:35:0) after 8m32s
[04:24:42.545] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:35:0)
[04:24:42.600] Build status: STARTED
[04:24:42.600] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:36:0) after 2m53s
[04:27:35.971] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:36:0)
[04:27:36.004] Build status: STARTED
[04:27:36.004] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:37:0) after 5m40s
[04:33:16.114] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:37:0)
[04:33:16.131] Build status: STARTED
[04:33:16.131] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:38:0) after 1m41s
[04:34:57.260] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:38:0)
[04:34:57.297] Build status: STARTED
[04:34:57.297] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:39:0) after 6m36s
[04:41:33.419] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:39:0)
[04:41:33.440] Build status: STARTED
[04:41:33.440] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:40:0) after 3m43s
[04:45:16.537] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:40:0)
[04:45:16.552] Build status: STARTED
[04:45:16.553] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:41:0) after 7m26s
[04:52:42.936] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:41:0)
[04:52:42.961] Build status: STARTED
[04:52:42.961] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:42:0) after 7m0s
[04:59:43.008] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:42:0)
[04:59:43.190] Build status: STARTED
[04:59:43.190] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:43:0) after 1m12s
[05:00:55.219] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:43:0)
[05:00:55.239] Build status: STARTED
[05:00:55.239] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:44:0) after 4m51s
[05:05:46.450] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:44:0)
[05:05:46.470] Build status: STARTED
[05:05:46.470] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:45:0) after 3m43s
[05:09:29.762] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:45:0)
[05:09:29.783] Build status: STARTED
[05:09:29.783] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:46:0) after 1m2s
[05:10:31.954] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:46:0)
[05:10:31.988] Build status: STARTED
[05:10:31.989] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:47:0) after 1m25s
[05:11:57.353] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:47:0)
[05:11:57.375] Build status: STARTED
[05:11:57.375] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:48:0) after 2m42s
[05:14:39.399] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:48:0)
[05:14:39.416] Build status: STARTED
[05:14:39.416] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:49:0) after 1m37s
[05:16:16.432] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:49:0)
[05:16:16.449] Build status: STARTED
[05:16:16.449] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:50:0) after 1m26s
[05:17:42.529] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:50:0)
[05:17:42.553] Build status: STARTED
[05:17:42.553] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:51:0) after 4m48s
[05:22:30.613] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:51:0)
[05:22:30.630] Build status: STARTED
[05:22:30.630] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:52:0) after 9m45s
[05:32:15.642] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:52:0)
[05:32:15.673] Build status: STARTED
[05:32:15.673] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:53:0) after 3m12s
[05:35:27.698] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:53:0)
[05:35:27.739] Build status: STARTED
[05:35:27.739] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:54:0) after 9m55s
[05:45:22.741] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:54:0)
[05:45:22.780] Build status: STARTED
[05:45:22.780] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:55:0) after 1m36s
[05:46:58.800] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:55:0)
[05:46:58.826] Build status: STARTED
[05:46:58.826] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:56:0) after 5m50s
[05:52:48.956] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:56:0)
[05:52:48.971] Build status: STARTED
[05:52:48.971] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:57:0) after 6m38s
[05:59:27.038] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:57:0)
[05:59:27.055] Build status: STARTED
[05:59:27.055] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:58:0) after 3m29s
[06:02:56.301] Handling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:58:0)
[06:02:56.319] Build status: STARTED
[06:02:56.319] Scheduling timer "check-buildbucket-build-status" (chromium/Linux Builder (canonical wd) (reclient compare):8885982197833250928:59:0) after 3m53s
[06:03:34.688] Received PubSub notification, asking Buildbucket for the build status
[06:03:34.744] Build:
{
"id": "8720552135952430497",
"builder": {
"project": "chromium",
"bucket": "reclient",
"builder": "Linux Builder (canonical wd) (reclient compare)"
},
"number": 10046,
"createdBy": "project:chromium",
"createTime": "2025-03-13T01:47:05.268320887Z",
"startTime": "2025-03-13T01:48:16.856533658Z",
"endTime": "2025-03-13T06:03:34.141264343Z",
"updateTime": "2025-03-13T06:03:34.141264343Z",
"status": "INFRA_FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "f3222b2949ea6d1e8d7fe09a93aad1b83303141a",
"ref": "refs/heads/main"
}
},
"canary": true
}
[06:03:34.744] Invocation finished in 4h16m30.652429497s with status FAILED