[15:21:16.281] New invocation is queued and will start shortly
[15:21:17.347] Starting the invocation (attempt 1)
[15:21:17.374] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/726abb769a683a11e828f0893ba5bc8b42e5d895"
[15:21:17.374] Popped gitiles commit info from properties and tags
[15:21:17.375] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:21:17.375] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:21:17.375] Buildbucket request:
{
"requestId": "8900335890262022448",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "726abb769a683a11e828f0893ba5bc8b42e5d895",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8900335890262022448"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTLinuxTSan"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:21:18.016] Scheduled build:
{
"id": "8734905828393444449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 23005,
"createdBy": "project:chromium",
"createTime": "2024-10-05T15:21:17.441192432Z",
"updateTime": "2024-10-05T15:21:17.441192432Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "726abb769a683a11e828f0893ba5bc8b42e5d895",
"ref": "refs/heads/main"
}
},
"canary": true
}
[15:21:18.016] Task URL: https://cr-buildbucket.appspot.com/build/8734905828393444449
[15:21:18.016] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:2:0) after 6m11s
[15:21:18.941] Received PubSub notification, asking Buildbucket for the build status
[15:21:18.963] Build status: SCHEDULED
[15:22:27.260] Received PubSub notification, asking Buildbucket for the build status
[15:22:27.288] Build status: STARTED
[15:27:29.154] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:2:0)
[15:27:29.182] Build status: STARTED
[15:27:29.182] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:5:0) after 7m15s
[15:34:44.282] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:5:0)
[15:34:44.300] Build status: STARTED
[15:34:44.300] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:6:0) after 1m14s
[15:35:58.524] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:6:0)
[15:35:58.551] Build status: STARTED
[15:35:58.551] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:7:0) after 5m11s
[15:41:09.598] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:7:0)
[15:41:09.628] Build status: STARTED
[15:41:09.628] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:8:0) after 3m15s
[15:44:24.623] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:8:0)
[15:44:24.642] Build status: STARTED
[15:44:24.642] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:9:0) after 5m53s
[15:50:17.706] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:9:0)
[15:50:17.721] Build status: STARTED
[15:50:17.721] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:10:0) after 2m26s
[15:52:43.737] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:10:0)
[15:52:43.755] Build status: STARTED
[15:52:43.755] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:11:0) after 4m9s
[15:56:52.871] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:11:0)
[15:56:52.891] Build status: STARTED
[15:56:52.891] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:12:0) after 5m46s
[16:02:38.918] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:12:0)
[16:02:38.975] Build status: STARTED
[16:02:38.975] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:13:0) after 3m48s
[16:06:27.013] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:13:0)
[16:06:27.097] Build status: STARTED
[16:06:27.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:14:0) after 4m0s
[16:10:27.219] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:14:0)
[16:10:27.299] Build status: STARTED
[16:10:27.299] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:15:0) after 8m14s
[16:18:41.449] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:15:0)
[16:18:41.469] Build status: STARTED
[16:18:41.469] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:16:0) after 6m18s
[16:24:59.486] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:16:0)
[16:24:59.508] Build status: STARTED
[16:24:59.508] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:17:0) after 4m23s
[16:29:22.526] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:17:0)
[16:29:22.543] Build status: STARTED
[16:29:22.543] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:18:0) after 8m58s
[16:38:20.664] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:18:0)
[16:38:20.682] Build status: STARTED
[16:38:20.682] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:19:0) after 9m27s
[16:47:48.122] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:19:0)
[16:47:48.219] Build status: STARTED
[16:47:48.219] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:20:0) after 3m12s
[16:51:00.334] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:20:0)
[16:51:00.351] Build status: STARTED
[16:51:00.351] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:21:0) after 7m55s
[16:58:55.423] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:21:0)
[16:58:55.440] Build status: STARTED
[16:58:55.440] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:22:0) after 8m41s
[17:07:36.479] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:22:0)
[17:07:36.495] Build status: STARTED
[17:07:36.495] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:23:0) after 1m50s
[17:09:26.547] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:23:0)
[17:09:26.572] Build status: STARTED
[17:09:26.572] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:24:0) after 1m59s
[17:11:26.476] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:24:0)
[17:11:26.499] Build status: STARTED
[17:11:26.499] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:25:0) after 7m32s
[17:18:58.514] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:25:0)
[17:18:58.688] Build status: STARTED
[17:18:58.688] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:26:0) after 2m18s
[17:21:16.876] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:26:0)
[17:21:16.902] Build status: STARTED
[17:21:16.902] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:27:0) after 6m43s
[17:28:00.142] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:27:0)
[17:28:00.162] Build status: STARTED
[17:28:00.162] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:28:0) after 5m40s
[17:33:40.180] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:28:0)
[17:33:40.199] Build status: STARTED
[17:33:40.199] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:29:0) after 2m17s
[17:35:57.248] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:29:0)
[17:35:57.268] Build status: STARTED
[17:35:57.268] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:30:0) after 1m49s
[17:37:46.416] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:30:0)
[17:37:46.438] Build status: STARTED
[17:37:46.438] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:31:0) after 7m8s
[17:44:54.477] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:31:0)
[17:44:54.494] Build status: STARTED
[17:44:54.494] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:32:0) after 5m58s
[17:50:52.575] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:32:0)
[17:50:52.592] Build status: STARTED
[17:50:52.592] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:33:0) after 6m36s
[17:57:28.648] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:33:0)
[17:57:28.670] Build status: STARTED
[17:57:28.670] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:34:0) after 2m15s
[17:59:43.686] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:34:0)
[17:59:43.705] Build status: STARTED
[17:59:43.705] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:35:0) after 9m55s
[18:09:38.692] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:35:0)
[18:09:38.713] Build status: STARTED
[18:09:38.713] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:36:0) after 2m12s
[18:11:50.908] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:36:0)
[18:11:50.924] Build status: STARTED
[18:11:50.924] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:37:0) after 4m43s
[18:16:34.081] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:37:0)
[18:16:34.101] Build status: STARTED
[18:16:34.101] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:38:0) after 6m27s
[18:23:01.180] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:38:0)
[18:23:01.198] Build status: STARTED
[18:23:01.198] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:39:0) after 3m27s
[18:26:28.218] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:39:0)
[18:26:28.241] Build status: STARTED
[18:26:28.241] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:40:0) after 8m20s
[18:34:48.381] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:40:0)
[18:34:48.408] Build status: STARTED
[18:34:48.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:41:0) after 4m11s
[18:38:59.564] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:41:0)
[18:38:59.583] Build status: STARTED
[18:38:59.583] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:42:0) after 8m23s
[18:47:22.591] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:42:0)
[18:47:22.611] Build status: STARTED
[18:47:22.611] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:43:0) after 6m22s
[18:53:44.851] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:43:0)
[18:53:44.870] Build status: STARTED
[18:53:44.870] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:44:0) after 7m26s
[19:01:10.895] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:44:0)
[19:01:10.920] Build status: STARTED
[19:01:10.920] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:45:0) after 3m36s
[19:04:46.966] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:45:0)
[19:04:46.989] Build status: STARTED
[19:04:46.989] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:46:0) after 8m14s
[19:13:01.148] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:46:0)
[19:13:01.170] Build status: STARTED
[19:13:01.170] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:47:0) after 1m40s
[19:14:41.245] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:47:0)
[19:14:41.269] Build status: STARTED
[19:14:41.269] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:48:0) after 8m41s
[19:23:22.375] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:48:0)
[19:23:22.401] Build status: STARTED
[19:23:22.401] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:49:0) after 7m19s
[19:30:41.830] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:49:0)
[19:30:41.856] Build status: STARTED
[19:30:41.856] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:50:0) after 7m33s
[19:38:14.873] Handling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:50:0)
[19:38:14.891] Build status: STARTED
[19:38:14.891] Scheduling timer "check-buildbucket-build-status" (chromium/ToTLinuxTSan:8900335890262022448:51:0) after 6m11s
[19:38:40.621] Received PubSub notification, asking Buildbucket for the build status
[19:38:40.643] Build:
{
"id": "8734905828393444449",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTLinuxTSan"
},
"number": 23005,
"createdBy": "project:chromium",
"createTime": "2024-10-05T15:21:17.441192432Z",
"startTime": "2024-10-05T15:22:26.989291287Z",
"endTime": "2024-10-05T19:38:40.205374457Z",
"updateTime": "2024-10-05T19:38:40.205374457Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "726abb769a683a11e828f0893ba5bc8b42e5d895",
"ref": "refs/heads/main"
}
},
"canary": true
}
[19:38:40.643] Invocation finished in 4h17m24.36696263s with status FAILED