Go to the tournament and click the withdraw button. If nobody is in the tournament than it will end immediately.
I created one since yesterday and still nobody joined, even if I showed the link in 3 teams and in the forum.
lichess.org/tournament/Zk4PaXTkThe private tournaments are no longer listed so unless you find friends to join, the world may not have seen the link.
It might of been better to have a tab like the lobby to display a few tournaments.
For the moment you can see your tournaments with this link:
lichess.org/@/PatM/tournaments/created