[21:55:10.492] New invocation is queued and will start shortly
[21:55:13.490] Starting the invocation (attempt 1)
[21:55:13.517] Popped buildset tag "buildset:commit/gitiles/chromium.googlesource.com/chromium/src/+/d03fe6c60c6dbca023c4404004db25ab9ebed48d"
[21:55:13.517] Popped gitiles commit info from properties and tags
[21:55:13.517] Preparing PubSub topic for "https://cr-buildbucket.appspot.com"
[21:55:13.517] PubSub topic is "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
[21:55:13.517] Buildbucket request:
{
"requestId": "8953491527281354960",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"properties": {
},
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d03fe6c60c6dbca023c4404004db25ab9ebed48d",
"ref": "refs/heads/main"
},
"tags": [
{
"key": "scheduler_invocation_id",
"value": "8953491527281354960"
},
{
"key": "scheduler_job_id",
"value": "chromium/ToTiOSDevice"
},
{
"key": "user_agent",
"value": "luci-scheduler"
}
],
"notify": {
"pubsubTopic": "projects/luci-scheduler/topics/scheduler.buildbucket.cr-buildbucket~appspot.gserviceaccount.com"
}
}
[21:55:14.114] Scheduled build:
{
"id": "8788061463199826657",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29001,
"createdBy": "project:chromium",
"createTime": "2023-02-26T21:55:13.754275418Z",
"updateTime": "2023-02-26T21:55:13.754275418Z",
"status": "SCHEDULED",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d03fe6c60c6dbca023c4404004db25ab9ebed48d",
"ref": "refs/heads/main"
}
},
"canary": true
}
[21:55:14.114] Task URL: https://cr-buildbucket.appspot.com/build/8788061463199826657
[21:55:14.114] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:2:0) after 7m56s
[21:57:39.060] Received PubSub notification, asking Buildbucket for the build status
[21:57:39.092] Build status: STARTED
[22:03:10.128] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:2:0)
[22:03:10.146] Build status: STARTED
[22:03:10.146] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:4:0) after 8m14s
[22:11:24.306] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:4:0)
[22:11:24.335] Build status: STARTED
[22:11:24.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:5:0) after 3m23s
[22:14:47.528] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:5:0)
[22:14:47.555] Build status: STARTED
[22:14:47.555] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:6:0) after 9m31s
[22:24:18.644] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:6:0)
[22:24:18.667] Build status: STARTED
[22:24:18.668] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:7:0) after 3m4s
[22:27:23.145] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:7:0)
[22:27:23.176] Build status: STARTED
[22:27:23.176] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:8:0) after 3m12s
[22:30:35.202] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:8:0)
[22:30:35.226] Build status: STARTED
[22:30:35.226] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:9:0) after 3m9s
[22:33:44.549] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:9:0)
[22:33:44.575] Build status: STARTED
[22:33:44.575] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:10:0) after 2m26s
[22:36:10.597] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:10:0)
[22:36:10.618] Build status: STARTED
[22:36:10.618] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:11:0) after 6m19s
[22:42:29.636] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:11:0)
[22:42:29.950] Build status: STARTED
[22:42:29.950] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:12:0) after 3m23s
[22:45:52.984] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:12:0)
[22:45:53.006] Build status: STARTED
[22:45:53.006] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:13:0) after 9m34s
[22:55:27.124] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:13:0)
[22:55:27.172] Build status: STARTED
[22:55:27.172] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:14:0) after 7m5s
[23:02:32.193] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:14:0)
[23:02:32.214] Build status: STARTED
[23:02:32.214] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:15:0) after 6m30s
[23:09:02.278] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:15:0)
[23:09:02.298] Build status: STARTED
[23:09:02.298] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:16:0) after 8m31s
[23:17:33.349] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:16:0)
[23:17:33.369] Build status: STARTED
[23:17:33.369] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:17:0) after 7m46s
[23:25:19.443] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:17:0)
[23:25:19.466] Build status: STARTED
[23:25:19.466] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:18:0) after 4m32s
[23:29:51.555] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:18:0)
[23:29:51.573] Build status: STARTED
[23:29:51.573] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:19:0) after 9m7s
[23:38:58.787] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:19:0)
[23:38:58.808] Build status: STARTED
[23:38:58.808] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:20:0) after 9m14s
[23:48:12.943] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:20:0)
[23:48:12.964] Build status: STARTED
[23:48:12.964] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:21:0) after 6m52s
[23:55:05.207] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:21:0)
[23:55:05.225] Build status: STARTED
[23:55:05.225] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:22:0) after 8m10s
[00:03:15.246] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:22:0)
[00:03:15.268] Build status: STARTED
[00:03:15.268] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:23:0) after 2m39s
[00:05:54.308] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:23:0)
[00:05:54.329] Build status: STARTED
[00:05:54.329] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:24:0) after 1m54s
[00:07:48.402] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:24:0)
[00:07:48.423] Build status: STARTED
[00:07:48.423] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:25:0) after 1m38s
[00:09:26.441] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:25:0)
[00:09:26.466] Build status: STARTED
[00:09:26.466] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:26:0) after 6m3s
[00:15:29.499] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:26:0)
[00:15:29.519] Build status: STARTED
[00:15:29.519] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:27:0) after 6m13s
[00:21:42.606] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:27:0)
[00:21:42.627] Build status: STARTED
[00:21:42.627] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:28:0) after 2m4s
[00:23:46.658] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:28:0)
[00:23:46.682] Build status: STARTED
[00:23:46.682] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:29:0) after 4m11s
[00:27:57.700] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:29:0)
[00:27:57.720] Build status: STARTED
[00:27:57.720] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:30:0) after 4m15s
[00:32:12.701] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:30:0)
[00:32:12.732] Build status: STARTED
[00:32:12.732] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:31:0) after 9m9s
[00:41:21.750] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:31:0)
[00:41:21.904] Build status: STARTED
[00:41:21.904] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:32:0) after 4m7s
[00:45:28.921] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:32:0)
[00:45:28.942] Build status: STARTED
[00:45:28.942] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:33:0) after 7m42s
[00:53:11.020] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:33:0)
[00:53:11.043] Build status: STARTED
[00:53:11.043] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:34:0) after 9m39s
[01:02:50.114] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:34:0)
[01:02:50.136] Build status: STARTED
[01:02:50.136] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:35:0) after 1m42s
[01:04:32.141] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:35:0)
[01:04:32.180] Build status: STARTED
[01:04:32.180] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:36:0) after 4m24s
[01:08:56.265] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:36:0)
[01:08:56.285] Build status: STARTED
[01:08:56.285] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:37:0) after 5m0s
[01:13:56.312] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:37:0)
[01:13:56.334] Build status: STARTED
[01:13:56.335] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:38:0) after 9m12s
[01:23:08.355] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:38:0)
[01:23:08.381] Build status: STARTED
[01:23:08.381] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:39:0) after 8m37s
[01:31:45.610] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:39:0)
[01:31:45.628] Build status: STARTED
[01:31:45.628] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:40:0) after 1m35s
[01:33:20.741] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:40:0)
[01:33:20.762] Build status: STARTED
[01:33:20.762] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:41:0) after 6m16s
[01:39:37.073] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:41:0)
[01:39:37.094] Build status: STARTED
[01:39:37.094] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:42:0) after 4m5s
[01:43:42.119] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:42:0)
[01:43:42.153] Build status: STARTED
[01:43:42.153] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:43:0) after 3m58s
[01:47:40.170] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:43:0)
[01:47:40.190] Build status: STARTED
[01:47:40.190] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:44:0) after 8m40s
[01:56:20.206] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:44:0)
[01:56:20.227] Build status: STARTED
[01:56:20.227] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:45:0) after 9m26s
[02:05:46.246] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:45:0)
[02:05:46.266] Build status: STARTED
[02:05:46.266] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:46:0) after 7m54s
[02:13:40.369] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:46:0)
[02:13:40.393] Build status: STARTED
[02:13:40.393] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:47:0) after 6m22s
[02:20:02.786] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:47:0)
[02:20:02.806] Build status: STARTED
[02:20:02.806] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:48:0) after 1m11s
[02:21:13.824] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:48:0)
[02:21:13.845] Build status: STARTED
[02:21:13.845] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:49:0) after 7m24s
[02:28:38.057] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:49:0)
[02:28:38.158] Build status: STARTED
[02:28:38.158] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:50:0) after 5m7s
[02:33:45.318] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:50:0)
[02:33:45.341] Build status: STARTED
[02:33:45.341] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:51:0) after 4m5s
[02:37:50.434] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:51:0)
[02:37:50.455] Build status: STARTED
[02:37:50.455] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:52:0) after 6m26s
[02:44:16.471] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:52:0)
[02:44:16.496] Build status: STARTED
[02:44:16.496] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:53:0) after 2m33s
[02:46:49.556] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:53:0)
[02:46:49.606] Build status: STARTED
[02:46:49.606] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:54:0) after 4m25s
[02:51:14.625] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:54:0)
[02:51:14.645] Build status: STARTED
[02:51:14.645] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:55:0) after 6m27s
[02:57:41.764] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:55:0)
[02:57:41.792] Build status: STARTED
[02:57:41.792] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:56:0) after 9m2s
[03:06:43.824] Handling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:56:0)
[03:06:43.874] Build status: STARTED
[03:06:43.874] Scheduling timer "check-buildbucket-build-status" (chromium/ToTiOSDevice:8953491527281354960:57:0) after 9m2s
[03:10:15.125] Received PubSub notification, asking Buildbucket for the build status
[03:10:15.156] Build:
{
"id": "8788061463199826657",
"builder": {
"project": "chromium",
"bucket": "ci",
"builder": "ToTiOSDevice"
},
"number": 29001,
"createdBy": "project:chromium",
"createTime": "2023-02-26T21:55:13.754275418Z",
"startTime": "2023-02-26T21:57:38.373799Z",
"endTime": "2023-02-27T03:10:14.690128194Z",
"updateTime": "2023-02-27T03:10:14.690128194Z",
"status": "FAILURE",
"input": {
"gitilesCommit": {
"host": "chromium.googlesource.com",
"project": "chromium/src",
"id": "d03fe6c60c6dbca023c4404004db25ab9ebed48d",
"ref": "refs/heads/main"
}
},
"canary": true
}
[03:10:15.156] Invocation finished in 5h15m4.677731871s with status FAILED