Many people start podcasts without any knowledge of where their audience is going to stem from. Getting anything new started can be difficult.
Your social media following is the first secure way to generate traffic to your podcast. The second way is to create value for others. People love learning things that can benefit their growth. Don't get overwhelmed trying to jump through hoops to get going. Start somewhere and grow as you go. Promote yourself and offer freebies. Somehow people love free stuff.
Check out ours and other podcasts that are huge to see what works and what doesn't work. Marketing and advertising is a big piece of the growth of your podcast and continuing growth of the podcast quality.