[08:58:38.067] New invocation is queued and will start shortly
[08:58:39.233] Starting the invocation (attempt 1)
[08:58:39.267] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/5f58bd093d1bf0eae5024a2364572c499c28c4ad"
[08:58:39.267] Popped gitiles commit info from properties and tags
[08:58:39.267] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[08:58:39.267] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[08:58:39.267] Buildbucket request:
{
"requestId": "8878979079628622576",
"builder": {
"project": "chromium",
"bucket": "reclient",
"builder": "Windows Cross deterministic"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5f58bd093d1bf0eae5024a2364572c499c28c4ad",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8878979079628622576"
},
{
"key": "scheduler_job_id",
"value": "chromium/Windows Cross deterministic"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[08:58:39.745] Scheduled build:
{
"id": "8713549017634452129",
"builder": {
"project": "chromium",
"bucket": "reclient",
"builder": "Windows Cross deterministic"
},
"number": 3372,
"createdBy": "project:chromium",
"createTime": "2025-05-29T08:58:39.343909149Z",
"updateTime": "2025-05-29T08:58:39.343909149Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5f58bd093d1bf0eae5024a2364572c499c28c4ad",
"ref": "refs/heads/main"
}
}
}
[08:58:39.745] Task URL: https://cr-buildbucket.appspot.com/build/8713549017634452129
[08:58:39.745] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:2:0) after 9m31s
[08:58:40.767] Received PubSub notification, asking Buildbucket for the build status
[08:58:40.793] Build status: SCHEDULED
[09:04:13.833] Received PubSub notification, asking Buildbucket for the build status
[09:04:13.855] Build status: STARTED
[09:08:10.769] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:2:0)
[09:08:10.787] Build status: STARTED
[09:08:10.788] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:5:0) after 6m6s
[09:14:16.846] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:5:0)
[09:14:16.880] Build status: STARTED
[09:14:16.880] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:6:0) after 2m1s
[09:16:18.008] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:6:0)
[09:16:18.089] Build status: STARTED
[09:16:18.090] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:7:0) after 4m11s
[09:20:29.071] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:7:0)
[09:20:29.136] Build status: STARTED
[09:20:29.136] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:8:0) after 2m9s
[09:22:38.178] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:8:0)
[09:22:38.204] Build status: STARTED
[09:22:38.204] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:9:0) after 5m35s
[09:28:13.231] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:9:0)
[09:28:13.260] Build status: STARTED
[09:28:13.260] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:10:0) after 5m42s
[09:33:55.338] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:10:0)
[09:33:55.366] Build status: STARTED
[09:33:55.367] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:11:0) after 5m29s
[09:39:24.446] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:11:0)
[09:39:24.467] Build status: STARTED
[09:39:24.467] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:12:0) after 3m13s
[09:42:37.496] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:12:0)
[09:42:37.531] Build status: STARTED
[09:42:37.532] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:13:0) after 4m50s
[09:47:27.560] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:13:0)
[09:47:27.585] Build status: STARTED
[09:47:27.585] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:14:0) after 1m23s
[09:48:50.965] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:14:0)
[09:48:51.000] Build status: STARTED
[09:48:51.000] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:15:0) after 2m38s
[09:51:28.993] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:15:0)
[09:51:29.020] Build status: STARTED
[09:51:29.020] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:16:0) after 4m57s
[09:56:26.060] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:16:0)
[09:56:26.097] Build status: STARTED
[09:56:26.098] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:17:0) after 9m52s
[10:06:18.266] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:17:0)
[10:06:18.301] Build status: STARTED
[10:06:18.301] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:18:0) after 1m2s
[10:07:20.349] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:18:0)
[10:07:20.376] Build status: STARTED
[10:07:20.376] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:19:0) after 8m48s
[10:16:08.424] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:19:0)
[10:16:08.455] Build status: STARTED
[10:16:08.455] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:20:0) after 8m41s
[10:24:49.627] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:20:0)
[10:24:49.663] Build status: STARTED
[10:24:49.663] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:21:0) after 6m24s
[10:31:13.969] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:21:0)
[10:31:14.095] Build status: STARTED
[10:31:14.095] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:22:0) after 1m38s
[10:32:52.256] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:22:0)
[10:32:52.291] Build status: STARTED
[10:32:52.291] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:23:0) after 3m13s
[10:36:05.496] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:23:0)
[10:36:05.519] Build status: STARTED
[10:36:05.519] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:24:0) after 1m23s
[10:37:28.587] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:24:0)
[10:37:28.609] Build status: STARTED
[10:37:28.610] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:25:0) after 1m27s
[10:38:55.657] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:25:0)
[10:38:55.705] Build status: STARTED
[10:38:55.705] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:26:0) after 5m44s
[10:44:39.727] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:26:0)
[10:44:39.922] Build status: STARTED
[10:44:39.923] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:27:0) after 9m4s
[10:53:43.947] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:27:0)
[10:53:43.983] Build status: STARTED
[10:53:43.983] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:28:0) after 5m44s
[10:59:28.001] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:28:0)
[10:59:28.103] Build status: STARTED
[10:59:28.103] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:29:0) after 4m11s
[11:03:39.366] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:29:0)
[11:03:39.433] Build status: STARTED
[11:03:39.433] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:30:0) after 8m53s
[11:12:32.472] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:30:0)
[11:12:32.509] Build status: STARTED
[11:12:32.509] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:31:0) after 4m35s
[11:17:07.823] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:31:0)
[11:17:07.846] Build status: STARTED
[11:17:07.846] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:32:0) after 8m58s
[11:26:06.129] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:32:0)
[11:26:06.152] Build status: STARTED
[11:26:06.152] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:33:0) after 2m6s
[11:28:12.176] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:33:0)
[11:28:12.200] Build status: STARTED
[11:28:12.200] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:34:0) after 8m46s
[11:36:58.240] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:34:0)
[11:36:58.261] Build status: STARTED
[11:36:58.261] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:35:0) after 8m57s
[11:45:55.300] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:35:0)
[11:45:55.315] Build status: STARTED
[11:45:55.315] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:36:0) after 7m46s
[11:53:41.382] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:36:0)
[11:53:41.430] Build status: STARTED
[11:53:41.430] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:37:0) after 3m21s
[11:57:02.508] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:37:0)
[11:57:02.545] Build status: STARTED
[11:57:02.545] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:38:0) after 5m56s
[12:02:58.801] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:38:0)
[12:02:58.825] Build status: STARTED
[12:02:58.825] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:39:0) after 4m11s
[12:07:10.062] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:39:0)
[12:07:10.119] Build status: STARTED
[12:07:10.119] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:40:0) after 9m59s
[12:17:09.410] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:40:0)
[12:17:09.434] Build status: STARTED
[12:17:09.434] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:41:0) after 7m31s
[12:24:40.486] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:41:0)
[12:24:40.503] Build status: STARTED
[12:24:40.503] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:42:0) after 7m4s
[12:31:44.535] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:42:0)
[12:31:44.560] Build status: STARTED
[12:31:44.561] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:43:0) after 8m44s
[12:40:28.607] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:43:0)
[12:40:28.622] Build status: STARTED
[12:40:28.622] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:44:0) after 7m49s
[12:48:17.707] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:44:0)
[12:48:17.729] Build status: STARTED
[12:48:17.729] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:45:0) after 7m47s
[12:56:04.748] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:45:0)
[12:56:04.773] Build status: STARTED
[12:56:04.773] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:46:0) after 4m18s
[13:00:22.869] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:46:0)
[13:00:22.913] Build status: STARTED
[13:00:22.913] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:47:0) after 8m56s
[13:09:18.937] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:47:0)
[13:09:18.956] Build status: STARTED
[13:09:18.956] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:48:0) after 4m51s
[13:14:10.010] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:48:0)
[13:14:10.030] Build status: STARTED
[13:14:10.030] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:49:0) after 7m29s
[13:21:39.558] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:49:0)
[13:21:39.573] Build status: STARTED
[13:21:39.574] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:50:0) after 7m29s
[13:29:08.673] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:50:0)
[13:29:08.701] Build status: STARTED
[13:29:08.701] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:51:0) after 9m3s
[13:38:11.752] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:51:0)
[13:38:11.785] Build status: STARTED
[13:38:11.785] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:52:0) after 5m38s
[13:43:49.906] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:52:0)
[13:43:49.928] Build status: STARTED
[13:43:49.928] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:53:0) after 4m46s
[13:48:36.001] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:53:0)
[13:48:36.028] Build status: STARTED
[13:48:36.028] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:54:0) after 8m3s
[13:56:39.056] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:54:0)
[13:56:39.079] Build status: STARTED
[13:56:39.079] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:55:0) after 7m17s
[14:03:56.130] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:55:0)
[14:03:56.150] Build status: STARTED
[14:03:56.150] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:56:0) after 1m39s
[14:05:35.172] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:56:0)
[14:05:35.192] Build status: STARTED
[14:05:35.192] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:57:0) after 1m24s
[14:06:59.291] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:57:0)
[14:06:59.407] Build status: STARTED
[14:06:59.407] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:58:0) after 9m57s
[14:16:56.436] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:58:0)
[14:16:56.536] Build status: STARTED
[14:16:56.536] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:59:0) after 3m54s
[14:20:50.659] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:59:0)
[14:20:50.719] Build status: STARTED
[14:20:50.719] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:60:0) after 5m12s
[14:26:03.012] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:60:0)
[14:26:03.029] Build status: STARTED
[14:26:03.029] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:61:0) after 1m52s
[14:27:55.300] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:61:0)
[14:27:55.332] Build status: STARTED
[14:27:55.332] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:62:0) after 7m59s
[14:35:54.389] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:62:0)
[14:35:54.409] Build status: STARTED
[14:35:54.410] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:63:0) after 4m47s
[14:40:41.482] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:63:0)
[14:40:41.502] Build status: STARTED
[14:40:41.502] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:64:0) after 9m30s
[14:50:11.541] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:64:0)
[14:50:11.662] Build status: STARTED
[14:50:11.662] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:65:0) after 4m8s
[14:54:19.681] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:65:0)
[14:54:19.706] Build status: STARTED
[14:54:19.706] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:66:0) after 1m56s
[14:56:15.788] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:66:0)
[14:56:15.895] Build status: STARTED
[14:56:15.895] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:67:0) after 3m25s
[14:59:40.928] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:67:0)
[14:59:40.945] Build status: STARTED
[14:59:40.945] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:68:0) after 8m56s
[15:08:36.970] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:68:0)
[15:08:37.022] Build status: STARTED
[15:08:37.022] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:69:0) after 6m16s
[15:14:53.229] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:69:0)
[15:14:53.270] Build status: STARTED
[15:14:53.270] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:70:0) after 8m43s
[15:23:36.307] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:70:0)
[15:23:36.331] Build status: STARTED
[15:23:36.331] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:71:0) after 5m34s
[15:29:10.412] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:71:0)
[15:29:10.428] Build status: STARTED
[15:29:10.428] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:72:0) after 2m32s
[15:31:42.471] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:72:0)
[15:31:42.492] Build status: STARTED
[15:31:42.492] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:73:0) after 3m2s
[15:34:44.515] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:73:0)
[15:34:44.547] Build status: STARTED
[15:34:44.547] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:74:0) after 8m2s
[15:42:46.589] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:74:0)
[15:42:46.603] Build status: STARTED
[15:42:46.603] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:75:0) after 7m25s
[15:50:11.629] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:75:0)
[15:50:11.651] Build status: STARTED
[15:50:11.651] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:76:0) after 1m18s
[15:51:29.686] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:76:0)
[15:51:29.824] Build status: STARTED
[15:51:29.824] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:77:0) after 6m39s
[15:58:08.928] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:77:0)
[15:58:08.943] Build status: STARTED
[15:58:08.944] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:78:0) after 4m22s
[16:02:31.149] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:78:0)
[16:02:31.173] Build status: STARTED
[16:02:31.173] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:79:0) after 5m35s
[16:08:06.189] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:79:0)
[16:08:06.215] Build status: STARTED
[16:08:06.216] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:80:0) after 8m24s
[16:16:30.326] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:80:0)
[16:16:30.351] Build status: STARTED
[16:16:30.351] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:81:0) after 6m26s
[16:22:56.388] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:81:0)
[16:22:56.408] Build status: STARTED
[16:22:56.408] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:82:0) after 4m51s
[16:27:47.442] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:82:0)
[16:27:47.772] Build status: STARTED
[16:27:47.772] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:83:0) after 5m37s
[16:33:24.770] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:83:0)
[16:33:24.813] Build status: STARTED
[16:33:24.813] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:84:0) after 3m0s
[16:36:24.913] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:84:0)
[16:36:24.945] Build status: STARTED
[16:36:24.945] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:85:0) after 1m13s
[16:37:37.933] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:85:0)
[16:37:37.971] Build status: STARTED
[16:37:37.971] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:86:0) after 1m0s
[16:38:38.000] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:86:0)
[16:38:38.123] Build status: STARTED
[16:38:38.123] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:87:0) after 5m51s
[16:44:29.143] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:87:0)
[16:44:29.236] Build status: STARTED
[16:44:29.236] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:88:0) after 3m15s
[16:47:44.257] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:88:0)
[16:47:44.320] Build status: STARTED
[16:47:44.320] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:89:0) after 1m20s
[16:49:04.431] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:89:0)
[16:49:04.455] Build status: STARTED
[16:49:04.455] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:90:0) after 5m39s
[16:54:43.479] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:90:0)
[16:54:43.514] Build status: STARTED
[16:54:43.514] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:91:0) after 3m50s
[16:58:33.537] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:91:0)
[16:58:33.557] Build status: STARTED
[16:58:33.557] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:92:0) after 9m0s
[17:07:33.650] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:92:0)
[17:07:33.668] Build status: STARTED
[17:07:33.668] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:93:0) after 4m52s
[17:12:25.858] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:93:0)
[17:12:25.942] Build status: STARTED
[17:12:25.942] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:94:0) after 1m16s
[17:13:41.974] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:94:0)
[17:13:41.992] Build status: STARTED
[17:13:41.992] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:95:0) after 1m53s
[17:15:35.070] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:95:0)
[17:15:35.088] Build status: STARTED
[17:15:35.088] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:96:0) after 4m14s
[17:19:49.184] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:96:0)
[17:19:49.210] Build status: STARTED
[17:19:49.210] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:97:0) after 1m58s
[17:21:47.240] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:97:0)
[17:21:47.256] Build status: STARTED
[17:21:47.256] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:98:0) after 4m11s
[17:25:58.315] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:98:0)
[17:25:58.349] Build status: STARTED
[17:25:58.349] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:99:0) after 1m10s
[17:27:08.414] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:99:0)
[17:27:08.437] Build status: STARTED
[17:27:08.437] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:100:0) after 8m26s
[17:35:34.522] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:100:0)
[17:35:34.542] Build status: STARTED
[17:35:34.542] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:101:0) after 3m1s
[17:38:35.616] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:101:0)
[17:38:35.662] Build status: STARTED
[17:38:35.662] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:102:0) after 5m26s
[17:44:01.890] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:102:0)
[17:44:01.918] Build status: STARTED
[17:44:01.918] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:103:0) after 5m19s
[17:49:21.050] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:103:0)
[17:49:21.076] Build status: STARTED
[17:49:21.076] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:104:0) after 4m19s
[17:53:40.114] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:104:0)
[17:53:40.145] Build status: STARTED
[17:53:40.146] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:105:0) after 3m5s
[17:56:45.170] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:105:0)
[17:56:45.191] Build status: STARTED
[17:56:45.191] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:106:0) after 4m9s
[18:00:54.225] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:106:0)
[18:00:54.247] Build status: STARTED
[18:00:54.247] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:107:0) after 3m4s
[18:03:58.320] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:107:0)
[18:03:58.342] Build status: STARTED
[18:03:58.342] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:108:0) after 4m31s
[18:08:29.337] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:108:0)
[18:08:29.352] Build status: STARTED
[18:08:29.352] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:109:0) after 7m56s
[18:16:25.641] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:109:0)
[18:16:25.716] Build status: STARTED
[18:16:25.716] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:110:0) after 5m6s
[18:21:31.985] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:110:0)
[18:21:32.010] Build status: STARTED
[18:21:32.010] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:111:0) after 3m17s
[18:24:49.282] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:111:0)
[18:24:49.317] Build status: STARTED
[18:24:49.317] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:112:0) after 8m30s
[18:33:19.352] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:112:0)
[18:33:19.380] Build status: STARTED
[18:33:19.380] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:113:0) after 7m34s
[18:40:53.456] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:113:0)
[18:40:53.510] Build status: STARTED
[18:40:53.510] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:114:0) after 3m31s
[18:44:24.493] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:114:0)
[18:44:24.518] Build status: STARTED
[18:44:24.518] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:115:0) after 1m57s
[18:46:21.541] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:115:0)
[18:46:21.624] Build status: STARTED
[18:46:21.624] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:116:0) after 6m38s
[18:52:59.700] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:116:0)
[18:52:59.730] Build status: STARTED
[18:52:59.730] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:117:0) after 7m41s
[19:00:40.854] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:117:0)
[19:00:40.917] Build status: STARTED
[19:00:40.917] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:118:0) after 5m12s
[19:05:52.982] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:118:0)
[19:05:53.012] Build status: STARTED
[19:05:53.012] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:119:0) after 7m41s
[19:13:34.112] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:119:0)
[19:13:34.151] Build status: STARTED
[19:13:34.151] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:120:0) after 1m27s
[19:15:01.217] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:120:0)
[19:15:01.256] Build status: STARTED
[19:15:01.256] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:121:0) after 1m45s
[19:16:46.572] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:121:0)
[19:16:46.589] Build status: STARTED
[19:16:46.589] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:122:0) after 8m41s
[19:25:27.677] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:122:0)
[19:25:27.697] Build status: STARTED
[19:25:27.697] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:123:0) after 2m41s
[19:28:08.797] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:123:0)
[19:28:08.827] Build status: STARTED
[19:28:08.827] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:124:0) after 3m40s
[19:31:48.971] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:124:0)
[19:31:48.995] Build status: STARTED
[19:31:48.995] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:125:0) after 6m41s
[19:38:30.201] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:125:0)
[19:38:30.441] Build status: STARTED
[19:38:30.441] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:126:0) after 3m8s
[19:41:38.472] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:126:0)
[19:41:38.493] Build status: STARTED
[19:41:38.493] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:127:0) after 2m13s
[19:43:51.578] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:127:0)
[19:43:51.603] Build status: STARTED
[19:43:51.603] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:128:0) after 2m4s
[19:45:55.628] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:128:0)
[19:45:55.647] Build status: STARTED
[19:45:55.647] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:129:0) after 9m53s
[19:55:48.928] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:129:0)
[19:55:49.027] Build status: STARTED
[19:55:49.027] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:130:0) after 8m12s
[20:04:01.046] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:130:0)
[20:04:01.080] Build status: STARTED
[20:04:01.080] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:131:0) after 6m7s
[20:10:08.235] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:131:0)
[20:10:08.254] Build status: STARTED
[20:10:08.254] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:132:0) after 7m2s
[20:17:10.339] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:132:0)
[20:17:10.363] Build status: STARTED
[20:17:10.364] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:133:0) after 7m22s
[20:24:32.488] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:133:0)
[20:24:32.506] Build status: STARTED
[20:24:32.506] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:134:0) after 6m52s
[20:31:24.588] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:134:0)
[20:31:24.607] Build status: STARTED
[20:31:24.607] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:135:0) after 5m51s
[20:37:15.681] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:135:0)
[20:37:15.727] Build status: STARTED
[20:37:15.727] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:136:0) after 6m30s
[20:43:45.762] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:136:0)
[20:43:45.784] Build status: STARTED
[20:43:45.784] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:137:0) after 4m38s
[20:48:23.870] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:137:0)
[20:48:23.887] Build status: STARTED
[20:48:23.887] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:138:0) after 3m59s
[20:52:22.920] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:138:0)
[20:52:22.940] Build status: STARTED
[20:52:22.940] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:139:0) after 1m14s
[20:53:37.042] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:139:0)
[20:53:37.098] Build status: STARTED
[20:53:37.098] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:140:0) after 6m41s
[21:00:18.192] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:140:0)
[21:00:18.211] Build status: STARTED
[21:00:18.211] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:141:0) after 9m21s
[21:09:39.250] Handling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:141:0)
[21:09:39.267] Build status: STARTED
[21:09:39.267] Scheduling timer "check-buildbucket-build-status" (chromium/Windows Cross deterministic:8878979079628622576:142:0) after 7m18s
[21:10:30.873] Received PubSub notification, asking Buildbucket for the build status
[21:10:30.918] Build:
{
"id": "8713549017634452129",
"builder": {
"project": "chromium",
"bucket": "reclient",
"builder": "Windows Cross deterministic"
},
"number": 3372,
"createdBy": "project:chromium",
"createTime": "2025-05-29T08:58:39.343909149Z",
"startTime": "2025-05-29T09:04:12.025253680Z",
"endTime": "2025-05-29T21:10:30.405039101Z",
"updateTime": "2025-05-29T21:10:30.405039101Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "5f58bd093d1bf0eae5024a2364572c499c28c4ad",
"ref": "refs/heads/main"
}
}
}
[21:10:30.918] Invocation finished in 12h11m52.861795516s with status FAILED