[10:09:00.607] New invocation is queued and will start shortly
[10:09:01.688] Starting the invocation (attempt 1)
[10:09:01.730] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/4b17509ec2a2cc8bb257c74bde35cfac96ec37e2"
[10:09:01.730] Popped gitiles commit info from properties and tags
[10:09:01.730] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[10:09:01.732] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[10:09:01.732] Buildbucket request:
{
"requestId": "8985697878406089104",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacASan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4b17509ec2a2cc8bb257c74bde35cfac96ec37e2",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8985697878406089104"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMacASan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[10:09:02.357] Scheduled build:
{
"id": "8820267816466797409",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacASan"
},
"number": 26578,
"createdBy": "project:chromium",
"createTime": "2022-03-08T10:09:01.818798014Z",
"updateTime": "2022-03-08T10:09:01.818798014Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4b17509ec2a2cc8bb257c74bde35cfac96ec37e2",
"ref": "refs/heads/main"
}
}
}
[10:09:02.357] Task URL: https://cr-buildbucket.appspot.com/build/8820267816466797409
[10:09:02.357] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:2:0) after 6m20s
[10:15:22.415] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:2:0)
[10:15:22.448] Build status: SCHEDULED
[10:15:22.448] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:3:0) after 6m6s
[10:21:28.468] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:3:0)
[10:21:28.486] Build status: SCHEDULED
[10:21:28.486] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:4:0) after 6m32s
[10:28:00.610] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:4:0)
[10:28:00.631] Build status: SCHEDULED
[10:28:00.631] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:5:0) after 1m17s
[10:29:17.654] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:5:0)
[10:29:17.675] Build status: SCHEDULED
[10:29:17.675] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:6:0) after 7m12s
[10:36:29.699] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:6:0)
[10:36:29.742] Build status: SCHEDULED
[10:36:29.742] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:7:0) after 1m46s
[10:38:15.853] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:7:0)
[10:38:15.974] Build status: SCHEDULED
[10:38:15.974] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:8:0) after 6m52s
[10:45:08.317] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:8:0)
[10:45:08.337] Build status: SCHEDULED
[10:45:08.337] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:9:0) after 2m49s
[10:47:57.473] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:9:0)
[10:47:57.493] Build status: SCHEDULED
[10:47:57.493] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:10:0) after 5m34s
[10:53:31.523] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:10:0)
[10:53:31.545] Build status: SCHEDULED
[10:53:31.545] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:11:0) after 2m22s
[10:55:53.529] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:11:0)
[10:55:53.554] Build status: SCHEDULED
[10:55:53.554] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:12:0) after 1m48s
[10:57:41.580] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:12:0)
[10:57:41.603] Build status: SCHEDULED
[10:57:41.603] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:13:0) after 8m5s
[11:05:46.626] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:13:0)
[11:05:46.664] Build status: SCHEDULED
[11:05:46.664] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:14:0) after 5m58s
[11:06:44.614] Received PubSub notification, asking Buildbucket for the build status
[11:06:44.657] Build status: STARTED
[11:11:44.779] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:14:0)
[11:11:44.803] Build status: STARTED
[11:11:44.803] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:16:0) after 1m5s
[11:12:50.140] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:16:0)
[11:12:50.177] Build status: STARTED
[11:12:50.177] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:17:0) after 5m48s
[11:18:38.255] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:17:0)
[11:18:38.277] Build status: STARTED
[11:18:38.277] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:18:0) after 1m29s
[11:20:07.303] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:18:0)
[11:20:07.347] Build status: STARTED
[11:20:07.347] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:19:0) after 1m16s
[11:21:23.430] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:19:0)
[11:21:23.450] Build status: STARTED
[11:21:23.450] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:20:0) after 3m59s
[11:25:22.687] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:20:0)
[11:25:22.716] Build status: STARTED
[11:25:22.716] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:21:0) after 7m47s
[11:33:10.207] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:21:0)
[11:33:10.228] Build status: STARTED
[11:33:10.228] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:22:0) after 9m42s
[11:42:52.294] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:22:0)
[11:42:52.347] Build status: STARTED
[11:42:52.347] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:23:0) after 3m16s
[11:46:08.621] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:23:0)
[11:46:08.654] Build status: STARTED
[11:46:08.655] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:24:0) after 6m21s
[11:52:29.682] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:24:0)
[11:52:29.777] Build status: STARTED
[11:52:29.777] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:25:0) after 4m18s
[11:56:47.932] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:25:0)
[11:56:47.954] Build status: STARTED
[11:56:47.954] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:26:0) after 6m41s
[12:03:29.061] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:26:0)
[12:03:29.082] Build status: STARTED
[12:03:29.082] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:27:0) after 8m22s
[12:11:51.107] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:27:0)
[12:11:51.153] Build status: STARTED
[12:11:51.153] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:28:0) after 5m50s
[12:17:41.179] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:28:0)
[12:17:41.202] Build status: STARTED
[12:17:41.202] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:29:0) after 4m22s
[12:22:03.294] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:29:0)
[12:22:03.316] Build status: STARTED
[12:22:03.316] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:30:0) after 5m37s
[12:27:40.350] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:30:0)
[12:27:40.388] Build status: STARTED
[12:27:40.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:31:0) after 2m16s
[12:29:56.414] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:31:0)
[12:29:56.478] Build status: STARTED
[12:29:56.478] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:32:0) after 8m37s
[12:38:33.506] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:32:0)
[12:38:33.571] Build status: STARTED
[12:38:33.571] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:33:0) after 1m5s
[12:39:38.671] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:33:0)
[12:39:38.699] Build status: STARTED
[12:39:38.699] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:34:0) after 3m13s
[12:42:52.026] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:34:0)
[12:42:52.052] Build status: STARTED
[12:42:52.052] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:35:0) after 5m59s
[12:48:51.081] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:35:0)
[12:48:51.103] Build status: STARTED
[12:48:51.103] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:36:0) after 4m46s
[12:53:37.127] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:36:0)
[12:53:37.148] Build status: STARTED
[12:53:37.148] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:37:0) after 3m56s
[12:57:33.177] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:37:0)
[12:57:33.217] Build status: STARTED
[12:57:33.217] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:38:0) after 5m51s
[13:03:24.318] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:38:0)
[13:03:24.342] Build status: STARTED
[13:03:24.342] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:39:0) after 1m45s
[13:05:09.511] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:39:0)
[13:05:09.538] Build status: STARTED
[13:05:09.538] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:40:0) after 3m26s
[13:08:35.731] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:40:0)
[13:08:35.778] Build status: STARTED
[13:08:35.778] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:41:0) after 9m38s
[13:18:13.982] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:41:0)
[13:18:14.008] Build status: STARTED
[13:18:14.008] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:42:0) after 5m47s
[13:24:01.077] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:42:0)
[13:24:01.100] Build status: STARTED
[13:24:01.100] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:43:0) after 2m51s
[13:26:52.153] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:43:0)
[13:26:52.177] Build status: STARTED
[13:26:52.177] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:44:0) after 8m40s
[13:35:32.201] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:44:0)
[13:35:32.239] Build status: STARTED
[13:35:32.239] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:45:0) after 1m46s
[13:37:18.269] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:45:0)
[13:37:18.293] Build status: STARTED
[13:37:18.293] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:46:0) after 4m32s
[13:41:50.469] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:46:0)
[13:41:50.501] Build status: STARTED
[13:41:50.501] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:47:0) after 8m49s
[13:50:39.534] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:47:0)
[13:50:39.566] Build status: STARTED
[13:50:39.566] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:48:0) after 7m39s
[13:58:18.590] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:48:0)
[13:58:18.616] Build status: STARTED
[13:58:18.616] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:49:0) after 2m16s
[14:00:34.676] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:49:0)
[14:00:34.726] Build status: STARTED
[14:00:34.726] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:50:0) after 4m45s
[14:05:19.807] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:50:0)
[14:05:19.829] Build status: STARTED
[14:05:19.829] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:51:0) after 7m5s
[14:12:24.959] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:51:0)
[14:12:24.987] Build status: STARTED
[14:12:24.987] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:52:0) after 3m40s
[14:16:05.169] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:52:0)
[14:16:05.204] Build status: STARTED
[14:16:05.204] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:53:0) after 9m34s
[14:25:39.229] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:53:0)
[14:25:39.315] Build status: STARTED
[14:25:39.315] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:54:0) after 1m21s
[14:27:00.360] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:54:0)
[14:27:00.385] Build status: STARTED
[14:27:00.385] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:55:0) after 3m46s
[14:30:46.511] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:55:0)
[14:30:46.536] Build status: STARTED
[14:30:46.536] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:56:0) after 3m39s
[14:34:25.581] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:56:0)
[14:34:25.745] Build status: STARTED
[14:34:25.745] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:57:0) after 4m9s
[14:38:34.872] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:57:0)
[14:38:34.896] Build status: STARTED
[14:38:34.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:58:0) after 7m51s
[14:46:25.949] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:58:0)
[14:46:25.992] Build status: STARTED
[14:46:25.992] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:59:0) after 1m3s
[14:47:29.092] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:59:0)
[14:47:29.111] Build status: STARTED
[14:47:29.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:60:0) after 8m18s
[14:55:47.321] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:60:0)
[14:55:47.380] Build status: STARTED
[14:55:47.380] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:61:0) after 7m42s
[15:03:29.582] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:61:0)
[15:03:29.606] Build status: STARTED
[15:03:29.606] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:62:0) after 8m54s
[15:12:23.753] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:62:0)
[15:12:23.780] Build status: STARTED
[15:12:23.780] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:63:0) after 8m43s
[15:21:06.810] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:63:0)
[15:21:06.833] Build status: STARTED
[15:21:06.833] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:64:0) after 5m24s
[15:26:30.817] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:64:0)
[15:26:30.855] Build status: STARTED
[15:26:30.855] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:65:0) after 7m27s
[15:33:57.988] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:65:0)
[15:33:58.033] Build status: STARTED
[15:33:58.033] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:66:0) after 4m30s
[15:38:28.270] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:66:0)
[15:38:28.298] Build status: STARTED
[15:38:28.298] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:67:0) after 3m44s
[15:42:12.321] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:67:0)
[15:42:12.343] Build status: STARTED
[15:42:12.343] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:68:0) after 5m24s
[15:47:36.548] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:68:0)
[15:47:36.572] Build status: STARTED
[15:47:36.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:69:0) after 5m42s
[15:53:18.601] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:69:0)
[15:53:18.665] Build status: STARTED
[15:53:18.665] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:70:0) after 5m14s
[15:58:32.858] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:70:0)
[15:58:32.892] Build status: STARTED
[15:58:32.892] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:71:0) after 2m0s
[16:00:33.310] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:71:0)
[16:00:33.331] Build status: STARTED
[16:00:33.332] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:72:0) after 1m41s
[16:02:14.367] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:72:0)
[16:02:14.391] Build status: STARTED
[16:02:14.391] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:73:0) after 6m20s
[16:08:34.680] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:73:0)
[16:08:34.702] Build status: STARTED
[16:08:34.702] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:74:0) after 1m9s
[16:09:43.857] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:74:0)
[16:09:43.891] Build status: STARTED
[16:09:43.891] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:75:0) after 6m12s
[16:15:55.918] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:75:0)
[16:15:55.938] Build status: STARTED
[16:15:55.938] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:76:0) after 7m35s
[16:23:31.105] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:76:0)
[16:23:31.154] Build status: STARTED
[16:23:31.154] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:77:0) after 8m17s
[16:31:48.517] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:77:0)
[16:31:48.549] Build status: STARTED
[16:31:48.549] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:78:0) after 8m15s
[16:40:03.758] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:78:0)
[16:40:03.816] Build status: STARTED
[16:40:03.816] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:79:0) after 5m24s
[16:45:27.913] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:79:0)
[16:45:27.952] Build status: STARTED
[16:45:27.952] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:80:0) after 3m38s
[16:49:06.069] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:80:0)
[16:49:06.105] Build status: STARTED
[16:49:06.105] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:81:0) after 9m12s
[16:58:18.288] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:81:0)
[16:58:18.310] Build status: STARTED
[16:58:18.310] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:82:0) after 1m44s
[17:00:02.299] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:82:0)
[17:00:02.324] Build status: STARTED
[17:00:02.324] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:83:0) after 3m43s
[17:03:45.360] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:83:0)
[17:03:45.388] Build status: STARTED
[17:03:45.388] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:84:0) after 6m22s
[17:10:07.418] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:84:0)
[17:10:07.452] Build status: STARTED
[17:10:07.452] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:85:0) after 5m29s
[17:15:36.490] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:85:0)
[17:15:36.511] Build status: STARTED
[17:15:36.511] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:86:0) after 2m45s
[17:18:21.537] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:86:0)
[17:18:21.778] Build status: STARTED
[17:18:21.778] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:87:0) after 7m24s
[17:25:45.927] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:87:0)
[17:25:45.950] Build status: STARTED
[17:25:45.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:88:0) after 4m21s
[17:30:07.108] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:88:0)
[17:30:07.135] Build status: STARTED
[17:30:07.135] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:89:0) after 8m36s
[17:38:43.388] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:89:0)
[17:38:43.415] Build status: STARTED
[17:38:43.415] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:90:0) after 1m39s
[17:40:22.570] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:90:0)
[17:40:22.608] Build status: STARTED
[17:40:22.608] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:91:0) after 9m29s
[17:49:51.680] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:91:0)
[17:49:51.702] Build status: STARTED
[17:49:51.702] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:92:0) after 3m25s
[17:53:16.828] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:92:0)
[17:53:16.851] Build status: STARTED
[17:53:16.851] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:93:0) after 2m5s
[17:55:21.902] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:93:0)
[17:55:21.927] Build status: STARTED
[17:55:21.927] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:94:0) after 7m19s
[18:02:41.222] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:94:0)
[18:02:41.289] Build status: STARTED
[18:02:41.289] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:95:0) after 9m53s
[18:12:34.769] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:95:0)
[18:12:34.793] Build status: STARTED
[18:12:34.793] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:96:0) after 6m10s
[18:18:44.846] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:96:0)
[18:18:44.882] Build status: STARTED
[18:18:44.882] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:97:0) after 5m33s
[18:24:17.908] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:97:0)
[18:24:17.946] Build status: STARTED
[18:24:17.946] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:98:0) after 3m29s
[18:27:46.944] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:98:0)
[18:27:46.965] Build status: STARTED
[18:27:46.965] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:99:0) after 8m9s
[18:35:56.114] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:99:0)
[18:35:56.145] Build status: STARTED
[18:35:56.145] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:100:0) after 6m43s
[18:42:39.420] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:100:0)
[18:42:39.456] Build status: STARTED
[18:42:39.456] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:101:0) after 1m16s
[18:43:55.580] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:101:0)
[18:43:55.604] Build status: STARTED
[18:43:55.604] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:102:0) after 6m16s
[18:50:11.650] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:102:0)
[18:50:11.676] Build status: STARTED
[18:50:11.676] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:103:0) after 2m14s
[18:52:25.854] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:103:0)
[18:52:25.877] Build status: STARTED
[18:52:25.877] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:104:0) after 2m35s
[18:55:01.182] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:104:0)
[18:55:01.212] Build status: STARTED
[18:55:01.212] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:105:0) after 2m6s
[18:57:07.261] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:105:0)
[18:57:07.289] Build status: STARTED
[18:57:07.289] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:106:0) after 4m55s
[19:02:02.381] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:106:0)
[19:02:02.417] Build status: STARTED
[19:02:02.417] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:107:0) after 2m44s
[19:04:46.442] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:107:0)
[19:04:46.464] Build status: STARTED
[19:04:46.464] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:108:0) after 6m29s
[19:11:15.499] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:108:0)
[19:11:15.521] Build status: STARTED
[19:11:15.521] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:109:0) after 4m9s
[19:15:24.600] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:109:0)
[19:15:24.628] Build status: STARTED
[19:15:24.628] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:110:0) after 2m33s
[19:17:57.750] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:110:0)
[19:17:57.769] Build status: STARTED
[19:17:57.769] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:111:0) after 5m0s
[19:22:58.039] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:111:0)
[19:22:58.072] Build status: STARTED
[19:22:58.072] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:112:0) after 9m2s
[19:32:00.295] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:112:0)
[19:32:00.328] Build status: STARTED
[19:32:00.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:113:0) after 8m32s
[19:40:32.700] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:113:0)
[19:40:32.723] Build status: STARTED
[19:40:32.723] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:114:0) after 8m55s
[19:49:27.799] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:114:0)
[19:49:27.824] Build status: STARTED
[19:49:27.824] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:115:0) after 4m55s
[19:54:22.852] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:115:0)
[19:54:22.874] Build status: STARTED
[19:54:22.874] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:116:0) after 9m11s
[20:03:33.901] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:116:0)
[20:03:33.924] Build status: STARTED
[20:03:33.924] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:117:0) after 2m33s
[20:06:06.960] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:117:0)
[20:06:06.983] Build status: STARTED
[20:06:06.983] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:118:0) after 2m0s
[20:08:07.390] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:118:0)
[20:08:07.419] Build status: STARTED
[20:08:07.419] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:119:0) after 8m9s
[20:16:16.584] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:119:0)
[20:16:16.626] Build status: STARTED
[20:16:16.626] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:120:0) after 1m18s
[20:17:34.846] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:120:0)
[20:17:34.888] Build status: STARTED
[20:17:34.888] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:121:0) after 8m45s
[20:26:19.918] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:121:0)
[20:26:19.976] Build status: STARTED
[20:26:19.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:122:0) after 1m0s
[20:27:19.992] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:122:0)
[20:27:20.031] Build status: STARTED
[20:27:20.031] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:123:0) after 7m3s
[20:34:23.193] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:123:0)
[20:34:23.233] Build status: STARTED
[20:34:23.233] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:124:0) after 8m24s
[20:42:47.328] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:124:0)
[20:42:47.361] Build status: STARTED
[20:42:47.361] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:125:0) after 6m52s
[20:49:39.387] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:125:0)
[20:49:39.419] Build status: STARTED
[20:49:39.419] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:126:0) after 1m10s
[20:50:49.488] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:126:0)
[20:50:49.513] Build status: STARTED
[20:50:49.513] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:127:0) after 3m34s
[20:54:23.619] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:127:0)
[20:54:23.654] Build status: STARTED
[20:54:23.654] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:128:0) after 7m10s
[21:01:33.767] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:128:0)
[21:01:33.810] Build status: STARTED
[21:01:33.810] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:129:0) after 3m1s
[21:04:34.915] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:129:0)
[21:04:34.942] Build status: STARTED
[21:04:34.942] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:130:0) after 3m39s
[21:08:14.111] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:130:0)
[21:08:14.195] Build status: STARTED
[21:08:14.195] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:131:0) after 9m50s
[21:18:04.350] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:131:0)
[21:18:04.371] Build status: STARTED
[21:18:04.371] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:132:0) after 2m42s
[21:20:46.400] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:132:0)
[21:20:46.450] Build status: STARTED
[21:20:46.450] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:133:0) after 6m17s
[21:27:03.585] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:133:0)
[21:27:03.620] Build status: STARTED
[21:27:03.620] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:134:0) after 4m21s
[21:31:24.665] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:134:0)
[21:31:24.743] Build status: STARTED
[21:31:24.743] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:135:0) after 7m32s
[21:38:57.402] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:135:0)
[21:38:57.441] Build status: STARTED
[21:38:57.441] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:136:0) after 7m30s
[21:46:27.620] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:136:0)
[21:46:27.773] Build status: STARTED
[21:46:27.773] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:137:0) after 5m0s
[21:51:27.780] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:137:0)
[21:51:27.801] Build status: STARTED
[21:51:27.801] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:138:0) after 4m5s
[21:55:33.083] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:138:0)
[21:55:33.146] Build status: STARTED
[21:55:33.146] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:139:0) after 1m11s
[21:56:44.310] Handling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:139:0)
[21:56:44.329] Build status: STARTED
[21:56:44.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMacASan:8985697878406089104:140:0) after 3m16s
[21:58:22.793] Received PubSub notification, asking Buildbucket for the build status
[21:58:22.838] Build:
{
"id": "8820267816466797409",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMacASan"
},
"number": 26578,
"createdBy": "project:chromium",
"createTime": "2022-03-08T10:09:01.818798014Z",
"startTime": "2022-03-08T11:06:43.684223Z",
"endTime": "2022-03-08T21:58:22.391963847Z",
"updateTime": "2022-03-08T21:58:22.391963847Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "4b17509ec2a2cc8bb257c74bde35cfac96ec37e2",
"ref": "refs/heads/main"
}
}
}
[21:58:22.838] Invocation finished in 11h49m22.253389598s with status FAILED