[05:04:56.797] New invocation is queued and will start shortly
[05:04:59.178] Starting the invocation (attempt 1)
[05:04:59.200] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/852c635f6b97f14401fe9a4f4157ed021c98ebb8"
[05:04:59.200] Popped gitiles commit info from properties and tags
[05:04:59.200] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[05:04:59.200] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[05:04:59.200] Buildbucket request:
{
"requestId": "8850999319383106032",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "852c635f6b97f14401fe9a4f4157ed021c98ebb8",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850999319383106032"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[05:04:59.875] Scheduled build:
{
"id": "8685569256089292321",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29979,
"createdBy": "project:chromium",
"createTime": "2026-04-03T05:04:59.307562733Z",
"updateTime": "2026-04-03T05:04:59.307562733Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "852c635f6b97f14401fe9a4f4157ed021c98ebb8",
"ref": "refs/heads/main"
}
}
}
[05:04:59.875] Task URL: https://cr-buildbucket.appspot.com/build/8685569256089292321
[05:04:59.875] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:2:0) after 8m55s
[05:05:00.935] Received PubSub notification, asking Buildbucket for the build status
[05:05:01.003] Build status: SCHEDULED
[05:06:03.686] Received PubSub notification, asking Buildbucket for the build status
[05:06:03.710] Build status: STARTED
[05:13:55.011] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:2:0)
[05:13:55.068] Build status: STARTED
[05:13:55.068] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:5:0) after 9m25s
[05:23:20.093] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:5:0)
[05:23:20.111] Build status: STARTED
[05:23:20.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:6:0) after 8m22s
[05:31:42.132] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:6:0)
[05:31:42.149] Build status: STARTED
[05:31:42.149] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:7:0) after 7m37s
[05:39:19.163] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:7:0)
[05:39:19.187] Build status: STARTED
[05:39:19.187] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:8:0) after 2m47s
[05:42:06.214] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:8:0)
[05:42:06.238] Build status: STARTED
[05:42:06.238] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:9:0) after 2m1s
[05:44:07.267] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:9:0)
[05:44:07.288] Build status: STARTED
[05:44:07.288] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:10:0) after 2m27s
[05:46:34.348] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:10:0)
[05:46:34.371] Build status: STARTED
[05:46:34.372] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:11:0) after 9m56s
[05:56:30.486] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:11:0)
[05:56:30.500] Build status: STARTED
[05:56:30.500] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:12:0) after 4m2s
[06:00:32.525] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:12:0)
[06:00:32.540] Build status: STARTED
[06:00:32.540] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:13:0) after 4m15s
[06:04:47.520] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:13:0)
[06:04:47.535] Build status: STARTED
[06:04:47.535] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:14:0) after 4m25s
[06:09:12.554] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:14:0)
[06:09:12.583] Build status: STARTED
[06:09:12.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:15:0) after 2m43s
[06:11:55.653] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:15:0)
[06:11:55.670] Build status: STARTED
[06:11:55.670] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:16:0) after 3m15s
[06:15:10.688] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:16:0)
[06:15:10.708] Build status: STARTED
[06:15:10.708] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:17:0) after 3m40s
[06:18:50.727] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:17:0)
[06:18:50.785] Build status: STARTED
[06:18:50.785] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:18:0) after 1m17s
[06:20:07.773] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:18:0)
[06:20:07.791] Build status: STARTED
[06:20:07.791] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:19:0) after 9m43s
[06:29:50.811] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:19:0)
[06:29:50.856] Build status: STARTED
[06:29:50.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:20:0) after 1m39s
[06:31:29.872] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:20:0)
[06:31:29.893] Build status: STARTED
[06:31:29.893] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:21:0) after 9m5s
[06:40:34.861] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:21:0)
[06:40:34.879] Build status: STARTED
[06:40:34.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:22:0) after 5m20s
[06:45:55.325] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:22:0)
[06:45:55.420] Build status: STARTED
[06:45:55.420] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:23:0) after 4m45s
[06:50:40.670] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:23:0)
[06:50:40.729] Build status: STARTED
[06:50:40.729] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:24:0) after 2m20s
[06:53:00.847] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:24:0)
[06:53:00.899] Build status: STARTED
[06:53:00.899] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:25:0) after 9m5s
[07:02:05.920] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:25:0)
[07:02:05.943] Build status: STARTED
[07:02:05.943] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:26:0) after 8m0s
[07:10:06.329] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:26:0)
[07:10:06.346] Build status: STARTED
[07:10:06.346] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:27:0) after 8m56s
[07:19:02.364] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:27:0)
[07:19:02.381] Build status: STARTED
[07:19:02.382] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:28:0) after 7m5s
[07:26:07.448] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:28:0)
[07:26:07.474] Build status: STARTED
[07:26:07.474] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:29:0) after 8m25s
[07:34:32.548] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:29:0)
[07:34:32.583] Build status: STARTED
[07:34:32.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:30:0) after 1m23s
[07:35:55.686] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:30:0)
[07:35:55.715] Build status: STARTED
[07:35:55.715] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:31:0) after 6m44s
[07:42:39.738] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:31:0)
[07:42:39.770] Build status: STARTED
[07:42:39.770] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:32:0) after 6m17s
[07:48:56.860] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:32:0)
[07:48:56.879] Build status: STARTED
[07:48:56.879] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:33:0) after 6m1s
[07:54:57.892] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:33:0)
[07:54:57.907] Build status: STARTED
[07:54:57.908] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:34:0) after 3m52s
[07:58:49.972] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:34:0)
[07:58:49.989] Build status: STARTED
[07:58:49.989] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:35:0) after 7m49s
[08:06:39.692] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:35:0)
[08:06:39.737] Build status: STARTED
[08:06:39.737] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:36:0) after 2m57s
[08:09:36.878] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:36:0)
[08:09:36.895] Build status: STARTED
[08:09:36.895] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:37:0) after 3m5s
[08:12:41.962] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:37:0)
[08:12:41.983] Build status: STARTED
[08:12:41.984] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:38:0) after 3m57s
[08:16:39.008] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:38:0)
[08:16:39.025] Build status: STARTED
[08:16:39.025] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8850999319383106032:39:0) after 7m36s
[08:18:42.090] Received PubSub notification, asking Buildbucket for the build status
[08:18:42.137] Build:
{
"id": "8685569256089292321",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 29979,
"createdBy": "project:chromium",
"createTime": "2026-04-03T05:04:59.307562733Z",
"startTime": "2026-04-03T05:06:03.504257509Z",
"endTime": "2026-04-03T08:18:40.714695342Z",
"updateTime": "2026-04-03T08:18:40.714695342Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "852c635f6b97f14401fe9a4f4157ed021c98ebb8",
"ref": "refs/heads/main"
}
}
}
[08:18:42.137] Invocation finished in 3h13m45.35381161s with status FAILED