[15:49:48.519] New invocation is queued and will start shortly
[15:49:49.605] Starting the invocation (attempt 1)
[15:49:49.628] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d8a0c9b15903c3500fcb97504a3e0e451b425d58"
[15:49:49.628] Popped gitiles commit info from properties and tags
[15:49:49.629] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[15:49:49.629] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[15:49:49.629] Buildbucket request:
{
"requestId": "8850958748164794256",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d8a0c9b15903c3500fcb97504a3e0e451b425d58",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8850958748164794256"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTMac"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[15:49:50.189] Scheduled build:
{
"id": "8685528686276331793",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 40123,
"createdBy": "project:chromium",
"createTime": "2026-04-03T15:49:49.696981772Z",
"updateTime": "2026-04-03T15:49:49.696981772Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d8a0c9b15903c3500fcb97504a3e0e451b425d58",
"ref": "refs/heads/main"
}
}
}
[15:49:50.189] Task URL: https://cr-buildbucket.appspot.com/build/8685528686276331793
[15:49:50.190] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:2:0) after 7m57s
[15:49:51.144] Received PubSub notification, asking Buildbucket for the build status
[15:49:51.176] Build status: SCHEDULED
[15:50:32.774] Received PubSub notification, asking Buildbucket for the build status
[15:50:32.793] Build status: STARTED
[15:57:47.181] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:2:0)
[15:57:47.228] Build status: STARTED
[15:57:47.228] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:5:0) after 6m53s
[16:04:40.271] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:5:0)
[16:04:40.290] Build status: STARTED
[16:04:40.290] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:6:0) after 5m5s
[16:09:45.354] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:6:0)
[16:09:45.376] Build status: STARTED
[16:09:45.377] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:7:0) after 7m22s
[16:17:07.362] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:7:0)
[16:17:07.387] Build status: STARTED
[16:17:07.387] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:8:0) after 4m50s
[16:21:57.470] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:8:0)
[16:21:57.532] Build status: STARTED
[16:21:57.532] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:9:0) after 7m15s
[16:29:12.601] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:9:0)
[16:29:12.619] Build status: STARTED
[16:29:12.619] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:10:0) after 9m9s
[16:38:21.598] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:10:0)
[16:38:21.616] Build status: STARTED
[16:38:21.616] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:11:0) after 8m9s
[16:46:30.818] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:11:0)
[16:46:30.835] Build status: STARTED
[16:46:30.835] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:12:0) after 8m40s
[16:55:10.904] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:12:0)
[16:55:10.943] Build status: STARTED
[16:55:10.944] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:13:0) after 8m34s
[17:03:44.957] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:13:0)
[17:03:44.986] Build status: STARTED
[17:03:44.986] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:14:0) after 1m13s
[17:04:58.007] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:14:0)
[17:04:58.027] Build status: STARTED
[17:04:58.027] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:15:0) after 8m47s
[17:13:45.120] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:15:0)
[17:13:45.139] Build status: STARTED
[17:13:45.139] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:16:0) after 4m16s
[17:18:01.158] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:16:0)
[17:18:01.188] Build status: STARTED
[17:18:01.188] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:17:0) after 1m9s
[17:19:10.307] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:17:0)
[17:19:10.326] Build status: STARTED
[17:19:10.326] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:18:0) after 5m37s
[17:24:47.387] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:18:0)
[17:24:47.408] Build status: STARTED
[17:24:47.408] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:19:0) after 1m25s
[17:26:12.443] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:19:0)
[17:26:12.464] Build status: STARTED
[17:26:12.464] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:20:0) after 6m17s
[17:32:29.532] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:20:0)
[17:32:29.611] Build status: STARTED
[17:32:29.611] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:21:0) after 9m7s
[17:41:36.722] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:21:0)
[17:41:36.744] Build status: STARTED
[17:41:36.744] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:22:0) after 4m39s
[17:46:15.727] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:22:0)
[17:46:15.752] Build status: STARTED
[17:46:15.752] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:23:0) after 7m4s
[17:53:19.930] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:23:0)
[17:53:20.037] Build status: STARTED
[17:53:20.037] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:24:0) after 7m1s
[18:00:21.093] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:24:0)
[18:00:21.111] Build status: STARTED
[18:00:21.111] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:25:0) after 6m47s
[18:07:08.128] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:25:0)
[18:07:08.150] Build status: STARTED
[18:07:08.150] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:26:0) after 1m30s
[18:08:38.334] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:26:0)
[18:08:38.362] Build status: STARTED
[18:08:38.362] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:27:0) after 5m49s
[18:14:27.410] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:27:0)
[18:14:27.431] Build status: STARTED
[18:14:27.431] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:28:0) after 9m3s
[18:23:30.527] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:28:0)
[18:23:30.642] Build status: STARTED
[18:23:30.642] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:29:0) after 4m9s
[18:27:39.663] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:29:0)
[18:27:39.757] Build status: STARTED
[18:27:39.757] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:30:0) after 6m34s
[18:34:13.778] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:30:0)
[18:34:13.803] Build status: STARTED
[18:34:13.803] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:31:0) after 9m51s
[18:44:04.821] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:31:0)
[18:44:04.838] Build status: STARTED
[18:44:04.838] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:32:0) after 4m51s
[18:48:55.905] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:32:0)
[18:48:55.923] Build status: STARTED
[18:48:55.923] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:33:0) after 3m1s
[18:51:56.965] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:33:0)
[18:51:56.981] Build status: STARTED
[18:51:56.981] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:34:0) after 4m10s
[18:56:07.051] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:34:0)
[18:56:07.066] Build status: STARTED
[18:56:07.067] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:35:0) after 7m12s
[19:03:19.080] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:35:0)
[19:03:19.123] Build status: STARTED
[19:03:19.123] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:36:0) after 7m18s
[19:10:37.119] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:36:0)
[19:10:37.137] Build status: STARTED
[19:10:37.137] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:37:0) after 4m49s
[19:15:26.164] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:37:0)
[19:15:26.201] Build status: STARTED
[19:15:26.201] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:38:0) after 3m8s
[19:18:34.236] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:38:0)
[19:18:34.274] Build status: STARTED
[19:18:34.274] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:39:0) after 9m10s
[19:27:44.294] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:39:0)
[19:27:44.311] Build status: STARTED
[19:27:44.311] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:40:0) after 1m32s
[19:29:16.336] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:40:0)
[19:29:16.372] Build status: STARTED
[19:29:16.372] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:41:0) after 3m54s
[19:33:10.442] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:41:0)
[19:33:10.458] Build status: STARTED
[19:33:10.458] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:42:0) after 7m26s
[19:40:36.506] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:42:0)
[19:40:36.544] Build status: STARTED
[19:40:36.544] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:43:0) after 7m1s
[19:47:37.567] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:43:0)
[19:47:37.591] Build status: STARTED
[19:47:37.591] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:44:0) after 9m25s
[19:57:02.586] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:44:0)
[19:57:02.601] Build status: STARTED
[19:57:02.601] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:45:0) after 9m2s
[20:06:04.626] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:45:0)
[20:06:04.645] Build status: STARTED
[20:06:04.645] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:46:0) after 7m35s
[20:13:39.712] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:46:0)
[20:13:39.735] Build status: STARTED
[20:13:39.735] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:47:0) after 7m30s
[20:21:09.873] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:47:0)
[20:21:09.930] Build status: STARTED
[20:21:09.930] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:48:0) after 9m31s
[20:30:41.059] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:48:0)
[20:30:41.153] Build status: STARTED
[20:30:41.153] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:49:0) after 6m49s
[20:37:30.193] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:49:0)
[20:37:30.213] Build status: STARTED
[20:37:30.213] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:50:0) after 6m51s
[20:44:21.256] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:50:0)
[20:44:21.283] Build status: STARTED
[20:44:21.283] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:51:0) after 7m40s
[20:52:01.309] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:51:0)
[20:52:01.360] Build status: STARTED
[20:52:01.360] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:52:0) after 2m17s
[20:54:18.390] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:52:0)
[20:54:18.494] Build status: STARTED
[20:54:18.494] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:53:0) after 4m31s
[20:58:49.517] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:53:0)
[20:58:49.534] Build status: STARTED
[20:58:49.534] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:54:0) after 3m6s
[21:01:55.661] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:54:0)
[21:01:55.747] Build status: STARTED
[21:01:55.747] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:55:0) after 6m7s
[21:08:02.766] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:55:0)
[21:08:02.820] Build status: STARTED
[21:08:02.820] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:56:0) after 5m36s
[21:13:38.852] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:56:0)
[21:13:38.885] Build status: STARTED
[21:13:38.885] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:57:0) after 6m23s
[21:20:01.938] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:57:0)
[21:20:01.955] Build status: STARTED
[21:20:01.955] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:58:0) after 3m0s
[21:23:02.030] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:58:0)
[21:23:02.052] Build status: STARTED
[21:23:02.052] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:59:0) after 3m35s
[21:26:37.073] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:59:0)
[21:26:37.097] Build status: STARTED
[21:26:37.097] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:60:0) after 6m31s
[21:33:08.171] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:60:0)
[21:33:08.187] Build status: STARTED
[21:33:08.187] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:61:0) after 9m21s
[21:42:29.209] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:61:0)
[21:42:29.228] Build status: STARTED
[21:42:29.228] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:62:0) after 3m24s
[21:45:53.368] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:62:0)
[21:45:53.386] Build status: STARTED
[21:45:53.386] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:63:0) after 3m33s
[21:49:26.431] Handling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:63:0)
[21:49:26.447] Build status: STARTED
[21:49:26.447] Scheduling timer "check-buildbucket-build-status" (chromium/ToTMac:8850958748164794256:64:0) after 7m49s
[21:54:05.032] Received PubSub notification, asking Buildbucket for the build status
[21:54:05.061] Build:
{
"id": "8685528686276331793",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTMac"
},
"number": 40123,
"createdBy": "project:chromium",
"createTime": "2026-04-03T15:49:49.696981772Z",
"startTime": "2026-04-03T15:50:32.610365994Z",
"endTime": "2026-04-03T21:54:04.304834310Z",
"updateTime": "2026-04-03T21:54:04.304834310Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d8a0c9b15903c3500fcb97504a3e0e451b425d58",
"ref": "refs/heads/main"
}
}
}
[21:54:05.061] Invocation finished in 6h4m16.548414538s with status FAILED