I haven't found a way to do so in the UI.
I asked a dev but haven't heard back yet.

You can delete them directly from the database.
They are saved in ost_queue.
You can locate it by its title.
You can find its title under Tickets -> Search on the agent panel.

In this example I created a search called "Abatha". When I go to the ost_queue table I see it there.

I just delete this row and its gone. I have not seen any bad behavior since deleting this one directly from the database.

AHA!

Go to the saved search.

Click on the search in the list.
Click on the gear and you will get a menu.

Very welcome.