[14:17:31.070] New invocation is queued and will start shortly
[14:17:32.175] Starting the invocation (attempt 1)
[14:17:32.204] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/fd869b3ef271ed5be020ac88eaac6b461032ae6b"
[14:17:32.204] Popped gitiles commit info from properties and tags
[14:17:32.204] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[14:17:32.204] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[14:17:32.204] Buildbucket request:
{
"requestId": "8923260933795631872",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "fd869b3ef271ed5be020ac88eaac6b461032ae6b",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8923260933795631872"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[14:17:32.767] Scheduled build:
{
"id": "8757830871856739889",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 19320,
"createdBy": "project:chromium",
"createTime": "2024-01-26T14:17:32.288843225Z",
"updateTime": "2024-01-26T14:17:32.288843225Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "fd869b3ef271ed5be020ac88eaac6b461032ae6b",
"ref": "refs/heads/main"
}
}
}
[14:17:32.767] Task URL: https://cr-buildbucket.appspot.com/build/8757830871856739889
[14:17:32.767] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:2:0) after 8m43s
[14:17:33.706] Received PubSub notification, asking Buildbucket for the build status
[14:17:33.727] Build status: SCHEDULED
[14:19:16.508] Received PubSub notification, asking Buildbucket for the build status
[14:19:16.525] Build status: STARTED
[14:26:17.021] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:2:0)
[14:26:17.073] Build status: STARTED
[14:26:17.073] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:5:0) after 6m10s
[14:32:27.136] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:5:0)
[14:32:27.170] Build status: STARTED
[14:32:27.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:6:0) after 1m25s
[14:33:52.709] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:6:0)
[14:33:52.766] Build status: STARTED
[14:33:52.766] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:7:0) after 7m38s
[14:41:30.789] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:7:0)
[14:41:30.808] Build status: STARTED
[14:41:30.808] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:8:0) after 6m34s
[14:48:04.930] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:8:0)
[14:48:04.980] Build status: STARTED
[14:48:04.980] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:9:0) after 2m58s
[14:51:03.872] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:9:0)
[14:51:03.896] Build status: STARTED
[14:51:03.896] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:10:0) after 9m51s
[15:00:54.933] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:10:0)
[15:00:54.993] Build status: STARTED
[15:00:54.993] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:11:0) after 5m5s
[15:06:00.351] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:11:0)
[15:06:00.375] Build status: STARTED
[15:06:00.376] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:12:0) after 4m47s
[15:10:47.395] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:12:0)
[15:10:47.424] Build status: STARTED
[15:10:47.424] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:13:0) after 4m43s
[15:15:30.624] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:13:0)
[15:15:30.659] Build status: STARTED
[15:15:30.659] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:14:0) after 3m41s
[15:19:11.684] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:14:0)
[15:19:11.715] Build status: STARTED
[15:19:11.715] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:15:0) after 5m36s
[15:24:47.732] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:15:0)
[15:24:47.750] Build status: STARTED
[15:24:47.750] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:16:0) after 7m15s
[15:32:03.068] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:16:0)
[15:32:03.088] Build status: STARTED
[15:32:03.088] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:17:0) after 5m17s
[15:37:20.141] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:17:0)
[15:37:20.162] Build status: STARTED
[15:37:20.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:18:0) after 9m21s
[15:46:41.576] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:18:0)
[15:46:41.594] Build status: STARTED
[15:46:41.594] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:19:0) after 6m30s
[15:53:11.620] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:19:0)
[15:53:11.643] Build status: STARTED
[15:53:11.643] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:20:0) after 6m30s
[15:59:41.719] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:20:0)
[15:59:41.738] Build status: STARTED
[15:59:41.738] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:21:0) after 4m35s
[16:04:16.760] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:21:0)
[16:04:16.915] Build status: STARTED
[16:04:16.915] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:22:0) after 1m13s
[16:05:29.986] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:22:0)
[16:05:30.012] Build status: STARTED
[16:05:30.012] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:23:0) after 6m8s
[16:11:38.255] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:23:0)
[16:11:38.430] Build status: STARTED
[16:11:38.430] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:24:0) after 4m3s
[16:15:41.894] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:24:0)
[16:15:41.977] Build status: STARTED
[16:15:41.977] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:25:0) after 5m5s
[16:20:47.354] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:25:0)
[16:20:47.371] Build status: STARTED
[16:20:47.371] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:26:0) after 8m32s
[16:29:19.385] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:26:0)
[16:29:19.412] Build status: STARTED
[16:29:19.412] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:27:0) after 6m6s
[16:35:25.485] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:27:0)
[16:35:25.527] Build status: STARTED
[16:35:25.527] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:28:0) after 1m20s
[16:36:45.604] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:28:0)
[16:36:45.632] Build status: STARTED
[16:36:45.632] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:29:0) after 5m51s
[16:42:36.656] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:29:0)
[16:42:36.676] Build status: STARTED
[16:42:36.676] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:30:0) after 4m41s
[16:47:17.744] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:30:0)
[16:47:17.787] Build status: STARTED
[16:47:17.787] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:31:0) after 3m55s
[16:51:12.864] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:31:0)
[16:51:12.909] Build status: STARTED
[16:51:12.909] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:32:0) after 7m41s
[16:58:53.960] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:32:0)
[16:58:53.976] Build status: STARTED
[16:58:53.976] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:33:0) after 4m18s
[17:03:12.084] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:33:0)
[17:03:12.109] Build status: STARTED
[17:03:12.109] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:34:0) after 2m37s
[17:05:49.186] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:34:0)
[17:05:49.211] Build status: STARTED
[17:05:49.211] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:35:0) after 6m11s
[17:12:00.303] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:35:0)
[17:12:00.328] Build status: STARTED
[17:12:00.328] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:36:0) after 3m17s
[17:15:18.114] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:36:0)
[17:15:18.141] Build status: STARTED
[17:15:18.141] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:37:0) after 3m45s
[17:19:03.160] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:37:0)
[17:19:03.180] Build status: STARTED
[17:19:03.180] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:38:0) after 1m41s
[17:20:44.204] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:38:0)
[17:20:44.223] Build status: STARTED
[17:20:44.223] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:39:0) after 2m20s
[17:23:04.306] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:39:0)
[17:23:04.326] Build status: STARTED
[17:23:04.326] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:40:0) after 7m43s
[17:30:47.600] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:40:0)
[17:30:47.619] Build status: STARTED
[17:30:47.619] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:41:0) after 2m47s
[17:33:34.676] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:41:0)
[17:33:34.694] Build status: STARTED
[17:33:34.694] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:42:0) after 8m38s
[17:42:12.795] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:42:0)
[17:42:12.814] Build status: STARTED
[17:42:12.814] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:43:0) after 5m7s
[17:47:19.796] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:43:0)
[17:47:19.939] Build status: STARTED
[17:47:19.939] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:44:0) after 3m37s
[17:50:56.961] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:44:0)
[17:50:56.977] Build status: STARTED
[17:50:56.977] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:45:0) after 1m15s
[17:52:11.994] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:45:0)
[17:52:12.017] Build status: STARTED
[17:52:12.017] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:46:0) after 2m27s
[17:54:39.098] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:46:0)
[17:54:39.115] Build status: STARTED
[17:54:39.115] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:47:0) after 2m48s
[17:57:27.134] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:47:0)
[17:57:27.199] Build status: STARTED
[17:57:27.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:48:0) after 5m23s
[18:02:50.311] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:48:0)
[18:02:50.337] Build status: STARTED
[18:02:50.337] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:49:0) after 3m39s
[18:06:29.369] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:49:0)
[18:06:29.392] Build status: STARTED
[18:06:29.392] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:50:0) after 7m1s
[18:13:30.468] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:50:0)
[18:13:30.510] Build status: STARTED
[18:13:30.510] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:51:0) after 4m0s
[18:17:30.547] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:51:0)
[18:17:30.635] Build status: STARTED
[18:17:30.635] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:52:0) after 7m47s
[18:25:17.709] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:52:0)
[18:25:17.735] Build status: STARTED
[18:25:17.735] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:53:0) after 7m56s
[18:33:13.760] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:53:0)
[18:33:13.837] Build status: STARTED
[18:33:13.837] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:54:0) after 3m20s
[18:36:33.906] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:54:0)
[18:36:33.928] Build status: STARTED
[18:36:33.928] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8923260933795631872:55:0) after 5m39s
[18:39:10.901] Received PubSub notification, asking Buildbucket for the build status
[18:39:10.946] Build:
{
"id": "8757830871856739889",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 19320,
"createdBy": "project:chromium",
"createTime": "2024-01-26T14:17:32.288843225Z",
"startTime": "2024-01-26T14:19:15.903229Z",
"endTime": "2024-01-26T18:39:10.191940Z",
"updateTime": "2024-01-26T18:39:10.191940Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "fd869b3ef271ed5be020ac88eaac6b461032ae6b",
"ref": "refs/heads/main"
}
}
}
[18:39:10.946] Invocation finished in 4h21m39.891021247s with status FAILED