[09:33:06.459] New invocation is queued and will start shortly
[09:33:07.724] Starting the invocation (attempt 1)
[09:33:07.754] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/c9fdc00c2e30514fc50d986f2fb68a156e1acb70"
[09:33:07.754] Popped gitiles commit info from properties and tags
[09:33:07.754] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[09:33:07.754] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[09:33:07.755] Buildbucket request:
{
"requestId": "8924094200034010592",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxASan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c9fdc00c2e30514fc50d986f2fb68a156e1acb70",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8924094200034010592"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxASan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[09:33:08.340] Scheduled build:
{
"id": "8758664137945445809",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxASan"
},
"number": 21841,
"createdBy": "project:chromium",
"createTime": "2024-01-17T09:33:07.820245178Z",
"updateTime": "2024-01-17T09:33:07.820245178Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c9fdc00c2e30514fc50d986f2fb68a156e1acb70",
"ref": "refs/heads/main"
}
},
"canary": true
}
[09:33:08.340] Task URL: https://cr-buildbucket.appspot.com/build/8758664137945445809
[09:33:08.340] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:2:0) after 4m3s
[09:33:13.012] Received PubSub notification, asking Buildbucket for the build status
[09:33:13.055] Build status: SCHEDULED
[09:34:58.198] Received PubSub notification, asking Buildbucket for the build status
[09:34:58.227] Build status: STARTED
[09:37:11.451] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:2:0)
[09:37:11.476] Build status: STARTED
[09:37:11.476] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:5:0) after 6m0s
[09:43:12.249] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:5:0)
[09:43:12.265] Build status: STARTED
[09:43:12.265] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:6:0) after 1m22s
[09:44:34.524] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:6:0)
[09:44:34.615] Build status: STARTED
[09:44:34.615] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:7:0) after 2m43s
[09:47:18.039] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:7:0)
[09:47:18.099] Build status: STARTED
[09:47:18.099] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:8:0) after 1m43s
[09:49:01.248] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:8:0)
[09:49:01.274] Build status: STARTED
[09:49:01.274] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:9:0) after 4m14s
[09:53:15.648] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:9:0)
[09:53:15.666] Build status: STARTED
[09:53:15.666] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:10:0) after 4m0s
[09:57:16.023] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:10:0)
[09:57:16.041] Build status: STARTED
[09:57:16.042] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:11:0) after 3m41s
[10:00:57.515] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:11:0)
[10:00:57.533] Build status: STARTED
[10:00:57.533] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:12:0) after 3m15s
[10:04:12.577] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:12:0)
[10:04:12.668] Build status: STARTED
[10:04:12.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:13:0) after 5m25s
[10:09:37.685] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:13:0)
[10:09:37.700] Build status: STARTED
[10:09:37.700] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:14:0) after 4m50s
[10:14:28.560] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:14:0)
[10:14:28.584] Build status: STARTED
[10:14:28.584] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:15:0) after 5m47s
[10:20:15.610] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:15:0)
[10:20:15.640] Build status: STARTED
[10:20:15.640] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:16:0) after 3m20s
[10:23:35.858] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:16:0)
[10:23:35.885] Build status: STARTED
[10:23:35.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:17:0) after 3m19s
[10:26:54.913] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:17:0)
[10:26:54.940] Build status: STARTED
[10:26:54.940] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:18:0) after 2m49s
[10:29:44.505] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:18:0)
[10:29:44.558] Build status: STARTED
[10:29:44.559] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:19:0) after 5m0s
[10:34:44.587] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:19:0)
[10:34:44.685] Build status: STARTED
[10:34:44.685] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:20:0) after 7m11s
[10:41:55.704] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:20:0)
[10:41:55.968] Build status: STARTED
[10:41:55.968] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:21:0) after 8m13s
[10:50:09.670] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:21:0)
[10:50:09.724] Build status: STARTED
[10:50:09.724] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:22:0) after 9m16s
[10:59:25.917] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:22:0)
[10:59:25.935] Build status: STARTED
[10:59:25.935] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:23:0) after 2m26s
[11:01:52.086] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:23:0)
[11:01:52.118] Build status: STARTED
[11:01:52.118] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:24:0) after 3m22s
[11:05:14.134] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:24:0)
[11:05:14.152] Build status: STARTED
[11:05:14.152] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:25:0) after 5m32s
[11:10:46.173] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:25:0)
[11:10:46.190] Build status: STARTED
[11:10:46.191] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:26:0) after 5m28s
[11:16:14.224] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:26:0)
[11:16:14.249] Build status: STARTED
[11:16:14.249] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:27:0) after 4m2s
[11:20:16.403] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:27:0)
[11:20:16.427] Build status: STARTED
[11:20:16.427] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:28:0) after 3m39s
[11:23:55.442] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:28:0)
[11:23:55.466] Build status: STARTED
[11:23:55.466] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:29:0) after 9m21s
[11:33:16.566] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:29:0)
[11:33:16.581] Build status: STARTED
[11:33:16.581] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:30:0) after 8m58s
[11:42:14.605] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:30:0)
[11:42:14.642] Build status: STARTED
[11:42:14.642] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:31:0) after 7m55s
[11:50:09.827] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:31:0)
[11:50:09.844] Build status: STARTED
[11:50:09.844] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:32:0) after 8m12s
[11:58:21.942] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:32:0)
[11:58:21.964] Build status: STARTED
[11:58:21.964] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:33:0) after 9m4s
[12:07:25.985] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:33:0)
[12:07:26.003] Build status: STARTED
[12:07:26.003] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:34:0) after 3m41s
[12:11:07.070] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:34:0)
[12:11:07.084] Build status: STARTED
[12:11:07.084] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:35:0) after 1m39s
[12:12:46.267] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:35:0)
[12:12:46.284] Build status: STARTED
[12:12:46.284] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:36:0) after 7m57s
[12:20:43.367] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:36:0)
[12:20:43.389] Build status: STARTED
[12:20:43.389] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:37:0) after 3m24s
[12:24:07.387] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:37:0)
[12:24:07.409] Build status: STARTED
[12:24:07.409] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:38:0) after 6m44s
[12:30:51.388] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:38:0)
[12:30:51.419] Build status: STARTED
[12:30:51.419] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:39:0) after 1m56s
[12:32:47.514] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:39:0)
[12:32:47.533] Build status: STARTED
[12:32:47.533] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:40:0) after 3m25s
[12:36:12.579] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:40:0)
[12:36:12.601] Build status: STARTED
[12:36:12.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:41:0) after 1m13s
[12:37:25.858] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:41:0)
[12:37:25.886] Build status: STARTED
[12:37:25.886] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:42:0) after 5m58s
[12:43:24.235] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:42:0)
[12:43:24.255] Build status: STARTED
[12:43:24.255] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:43:0) after 9m7s
[12:52:31.272] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:43:0)
[12:52:31.287] Build status: STARTED
[12:52:31.287] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:44:0) after 6m37s
[12:59:08.320] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:44:0)
[12:59:08.340] Build status: STARTED
[12:59:08.340] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:45:0) after 9m15s
[13:08:23.367] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:45:0)
[13:08:23.384] Build status: STARTED
[13:08:23.384] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:46:0) after 7m29s
[13:15:52.443] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:46:0)
[13:15:52.456] Build status: STARTED
[13:15:52.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:47:0) after 8m53s
[13:24:45.482] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:47:0)
[13:24:45.520] Build status: STARTED
[13:24:45.520] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:48:0) after 5m54s
[13:30:39.540] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:48:0)
[13:30:39.616] Build status: STARTED
[13:30:39.616] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:49:0) after 8m24s
[13:39:03.845] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:49:0)
[13:39:03.867] Build status: STARTED
[13:39:03.867] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:50:0) after 4m4s
[13:43:08.051] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:50:0)
[13:43:08.083] Build status: STARTED
[13:43:08.083] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:51:0) after 5m6s
[13:48:14.254] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:51:0)
[13:48:14.300] Build status: STARTED
[13:48:14.300] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:52:0) after 5m53s
[13:54:07.436] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:52:0)
[13:54:07.469] Build status: STARTED
[13:54:07.469] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:53:0) after 2m23s
[13:56:30.509] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:53:0)
[13:56:30.537] Build status: STARTED
[13:56:30.537] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:54:0) after 3m55s
[14:00:25.705] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:54:0)
[14:00:25.726] Build status: STARTED
[14:00:25.726] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:55:0) after 1m36s
[14:02:01.788] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:55:0)
[14:02:01.848] Build status: STARTED
[14:02:01.849] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:56:0) after 2m0s
[14:04:01.877] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:56:0)
[14:04:01.897] Build status: STARTED
[14:04:01.897] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:57:0) after 4m57s
[14:08:59.006] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:57:0)
[14:08:59.037] Build status: STARTED
[14:08:59.037] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:58:0) after 6m26s
[14:15:25.063] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:58:0)
[14:15:25.099] Build status: STARTED
[14:15:25.099] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:59:0) after 9m0s
[14:24:25.291] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:59:0)
[14:24:25.311] Build status: STARTED
[14:24:25.311] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:60:0) after 4m32s
[14:28:57.565] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:60:0)
[14:28:57.583] Build status: STARTED
[14:28:57.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:61:0) after 4m59s
[14:33:56.608] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:61:0)
[14:33:56.627] Build status: STARTED
[14:33:56.627] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:62:0) after 6m47s
[14:40:43.648] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:62:0)
[14:40:43.664] Build status: STARTED
[14:40:43.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:63:0) after 3m42s
[14:44:25.693] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:63:0)
[14:44:25.715] Build status: STARTED
[14:44:25.715] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:64:0) after 1m36s
[14:46:01.739] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:64:0)
[14:46:01.772] Build status: STARTED
[14:46:01.772] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:65:0) after 6m24s
[14:52:25.789] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:65:0)
[14:52:25.809] Build status: STARTED
[14:52:25.809] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxASan:8924094200034010592:66:0) after 3m5s
[14:54:29.841] Received PubSub notification, asking Buildbucket for the build status
[14:54:29.862] Build:
{
"id": "8758664137945445809",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxASan"
},
"number": 21841,
"createdBy": "project:chromium",
"createTime": "2024-01-17T09:33:07.820245178Z",
"startTime": "2024-01-17T09:34:48.369995386Z",
"endTime": "2024-01-17T14:54:28.813128Z",
"updateTime": "2024-01-17T14:54:28.813128Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "c9fdc00c2e30514fc50d986f2fb68a156e1acb70",
"ref": "refs/heads/main"
}
},
"canary": true
}
[14:54:29.862] Invocation finished in 5h21m23.417810019s with status FAILED