[03:56:09.198] New invocation is queued and will start shortly
[03:56:10.390] Starting the invocation (attempt 1)
[03:56:10.421] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/158d090b76c878788f81a98cc1363b529e4c81ec"
[03:56:10.421] Popped gitiles commit info from properties and tags
[03:56:10.421] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[03:56:10.421] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[03:56:10.421] Buildbucket request:
{
"requestId": "8949210759056587168",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "Mac deterministic"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "158d090b76c878788f81a98cc1363b529e4c81ec",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8949210759056587168"
},
{
"key": "scheduler_job_id",
"value": "chromium/Mac deterministic"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[03:56:11.056] Scheduled build:
{
"id": "8783780697041838001",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "Mac deterministic"
},
"number": 60579,
"createdBy": "project:chromium",
"createTime": "2023-04-15T03:56:10.494552252Z",
"updateTime": "2023-04-15T03:56:10.494552252Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "158d090b76c878788f81a98cc1363b529e4c81ec",
"ref": "refs/heads/main"
}
}
}
[03:56:11.056] Task URL: https://cr-buildbucket.appspot.com/build/8783780697041838001
[03:56:11.056] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:2:0) after 3m59s
[04:00:10.088] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:2:0)
[04:00:10.108] Build status: SCHEDULED
[04:00:10.108] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:3:0) after 7m30s
[04:07:40.129] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:3:0)
[04:07:40.152] Build status: SCHEDULED
[04:07:40.152] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:4:0) after 9m44s
[04:17:24.173] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:4:0)
[04:17:24.190] Build status: SCHEDULED
[04:17:24.190] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:5:0) after 6m3s
[04:23:27.321] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:5:0)
[04:23:27.339] Build status: SCHEDULED
[04:23:27.339] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:6:0) after 7m17s
[04:30:44.362] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:6:0)
[04:30:44.391] Build status: SCHEDULED
[04:30:44.391] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:7:0) after 9m30s
[04:40:14.432] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:7:0)
[04:40:14.457] Build status: SCHEDULED
[04:40:14.457] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:8:0) after 3m35s
[04:43:49.579] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:8:0)
[04:43:49.596] Build status: SCHEDULED
[04:43:49.596] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:9:0) after 4m40s
[04:48:29.615] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:9:0)
[04:48:29.636] Build status: SCHEDULED
[04:48:29.636] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:10:0) after 7m38s
[04:56:07.688] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:10:0)
[04:56:07.711] Build status: SCHEDULED
[04:56:07.711] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:11:0) after 1m9s
[04:57:16.752] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:11:0)
[04:57:16.770] Build status: SCHEDULED
[04:57:16.770] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:12:0) after 3m14s
[05:00:30.822] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:12:0)
[05:00:30.860] Build status: SCHEDULED
[05:00:30.860] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:13:0) after 9m34s
[05:10:04.988] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:13:0)
[05:10:05.032] Build status: SCHEDULED
[05:10:05.032] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:14:0) after 4m41s
[05:14:46.057] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:14:0)
[05:14:46.078] Build status: SCHEDULED
[05:14:46.078] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:15:0) after 8m54s
[05:23:40.235] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:15:0)
[05:23:40.259] Build status: SCHEDULED
[05:23:40.259] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:16:0) after 4m16s
[05:27:56.370] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:16:0)
[05:27:56.393] Build status: SCHEDULED
[05:27:56.393] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:17:0) after 6m33s
[05:34:29.413] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:17:0)
[05:34:29.434] Build status: SCHEDULED
[05:34:29.434] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:18:0) after 4m22s
[05:38:51.564] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:18:0)
[05:38:51.593] Build status: SCHEDULED
[05:38:51.593] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:19:0) after 8m23s
[05:47:14.799] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:19:0)
[05:47:14.819] Build status: SCHEDULED
[05:47:14.819] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:20:0) after 2m50s
[05:50:04.888] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:20:0)
[05:50:04.910] Build status: SCHEDULED
[05:50:04.910] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:21:0) after 2m29s
[05:52:33.934] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:21:0)
[05:52:33.965] Build status: SCHEDULED
[05:52:33.965] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:22:0) after 1m37s
[05:54:10.956] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:22:0)
[05:54:10.975] Build status: SCHEDULED
[05:54:10.975] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:23:0) after 3m20s
[05:57:31.045] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:23:0)
[05:57:31.068] Build status: SCHEDULED
[05:57:31.068] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:24:0) after 1m51s
[05:59:22.157] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:24:0)
[05:59:22.185] Build status: SCHEDULED
[05:59:22.185] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:25:0) after 6m24s
[06:05:46.272] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:25:0)
[06:05:46.290] Build status: SCHEDULED
[06:05:46.290] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:26:0) after 7m26s
[06:13:12.415] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:26:0)
[06:13:12.437] Build status: SCHEDULED
[06:13:12.437] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:27:0) after 7m39s
[06:20:51.467] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:27:0)
[06:20:51.492] Build status: SCHEDULED
[06:20:51.492] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:28:0) after 3m46s
[06:24:37.643] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:28:0)
[06:24:37.699] Build status: SCHEDULED
[06:24:37.699] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:29:0) after 9m33s
[06:34:10.721] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:29:0)
[06:34:10.744] Build status: SCHEDULED
[06:34:10.744] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:30:0) after 4m20s
[06:38:30.763] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:30:0)
[06:38:30.780] Build status: SCHEDULED
[06:38:30.780] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:31:0) after 3m30s
[06:42:00.813] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:31:0)
[06:42:00.880] Build status: SCHEDULED
[06:42:00.880] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:32:0) after 6m40s
[06:48:40.967] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:32:0)
[06:48:40.986] Build status: SCHEDULED
[06:48:40.986] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:33:0) after 4m42s
[06:53:23.014] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:33:0)
[06:53:23.036] Build status: SCHEDULED
[06:53:23.036] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:34:0) after 8m41s
[07:02:04.063] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:34:0)
[07:02:04.087] Build status: SCHEDULED
[07:02:04.087] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:35:0) after 7m27s
[07:09:31.108] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:35:0)
[07:09:31.127] Build status: SCHEDULED
[07:09:31.127] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:36:0) after 2m52s
[07:12:23.146] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:36:0)
[07:12:23.166] Build status: SCHEDULED
[07:12:23.166] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:37:0) after 3m22s
[07:15:45.318] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:37:0)
[07:15:45.371] Build status: SCHEDULED
[07:15:45.371] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:38:0) after 7m10s
[07:22:55.434] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:38:0)
[07:22:55.453] Build status: SCHEDULED
[07:22:55.453] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:39:0) after 9m12s
[07:32:07.475] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:39:0)
[07:32:07.498] Build status: SCHEDULED
[07:32:07.498] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:40:0) after 9m51s
[07:41:58.539] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:40:0)
[07:41:58.670] Build status: SCHEDULED
[07:41:58.670] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:41:0) after 6m34s
[07:48:32.708] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:41:0)
[07:48:32.750] Build status: SCHEDULED
[07:48:32.750] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:42:0) after 9m13s
[07:57:45.769] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:42:0)
[07:57:45.789] Build status: SCHEDULED
[07:57:45.789] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:43:0) after 1m53s
[07:59:38.809] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:43:0)
[07:59:38.850] Build status: SCHEDULED
[07:59:38.850] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:44:0) after 4m46s
[08:04:24.880] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:44:0)
[08:04:24.900] Build status: SCHEDULED
[08:04:24.900] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:45:0) after 3m59s
[08:08:23.928] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:45:0)
[08:08:23.946] Build status: SCHEDULED
[08:08:23.946] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:46:0) after 6m5s
[08:14:28.973] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:46:0)
[08:14:28.996] Build status: SCHEDULED
[08:14:28.996] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:47:0) after 8m45s
[08:23:14.142] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:47:0)
[08:23:14.164] Build status: SCHEDULED
[08:23:14.164] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:48:0) after 6m0s
[08:29:14.491] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:48:0)
[08:29:14.583] Build status: SCHEDULED
[08:29:14.583] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:49:0) after 5m29s
[08:34:43.591] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:49:0)
[08:34:43.619] Build status: SCHEDULED
[08:34:43.619] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:50:0) after 7m49s
[08:42:32.641] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:50:0)
[08:42:32.663] Build status: SCHEDULED
[08:42:32.663] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:51:0) after 7m16s
[08:49:48.768] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:51:0)
[08:49:48.789] Build status: SCHEDULED
[08:49:48.789] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:52:0) after 8m16s
[08:58:04.818] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:52:0)
[08:58:04.838] Build status: SCHEDULED
[08:58:04.838] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:53:0) after 4m44s
[09:02:48.962] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:53:0)
[09:02:48.981] Build status: SCHEDULED
[09:02:48.981] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:54:0) after 8m24s
[09:11:12.998] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:54:0)
[09:11:13.042] Build status: SCHEDULED
[09:11:13.042] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:55:0) after 8m42s
[09:19:55.066] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:55:0)
[09:19:55.133] Build status: SCHEDULED
[09:19:55.133] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:56:0) after 5m0s
[09:24:55.262] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:56:0)
[09:24:55.281] Build status: SCHEDULED
[09:24:55.281] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:57:0) after 6m51s
[09:31:46.349] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:57:0)
[09:31:46.373] Build status: SCHEDULED
[09:31:46.373] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:58:0) after 5m28s
[09:37:14.390] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:58:0)
[09:37:14.433] Build status: SCHEDULED
[09:37:14.433] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:59:0) after 4m41s
[09:41:55.437] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:59:0)
[09:41:55.453] Build status: SCHEDULED
[09:41:55.453] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:60:0) after 6m57s
[09:48:52.497] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:60:0)
[09:48:52.571] Build status: SCHEDULED
[09:48:52.571] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:61:0) after 5m25s
[09:54:17.764] Handling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:61:0)
[09:54:17.781] Build status: SCHEDULED
[09:54:17.781] Scheduling timer "check-buildbucket-build-status" (chromium/Mac deterministic:8949210759056587168:62:0) after 3m12s
[09:57:07.708] Received PubSub notification, asking Buildbucket for the build status
[09:57:07.760] Build:
{
"id": "8783780697041838001",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "Mac deterministic"
},
"number": 60579,
"createdBy": "project:chromium",
"createTime": "2023-04-15T03:56:10.494552252Z",
"endTime": "2023-04-15T09:57:06.619678Z",
"updateTime": "2023-04-15T09:57:06.619678Z",
"status": "INFRA_FAILURE",
"statusDetails": {
"resourceExhaustion": {
},
"timeout": {
}
},
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "158d090b76c878788f81a98cc1363b529e4c81ec",
"ref": "refs/heads/main"
}
}
}
[09:57:07.760] Invocation finished in 6h0m58.5713452s with status FAILED