A computer with a headset...
... or earbuds with a microphone.
Recording software. There are some good options that are free, such as Audacity, WavePad or AudioDope.
Music and sound effects in the public domain (free and legal to download), such as FreePD for music and FreeSFX for sound effects.
A free podcast hosting service, such as Internet Archive and Anchor (unlimited), Podomatic (limited to 500 MB) and SoundCloud (limited to 180 minutes).