If you don't want to remove the banner set but you want to hide it from your list of active banner sets you can instead Archive it. Select the banner set(s) you wish to Archive and then click on Archive in the action bar.
When the banner sets has been archived you can still access it if you filter the option to show archived banner sets.
An archived banner set can also easily be made active again. Just click on the corresponding row and then click Activate.