The spelling of /b/ is either <b> or <bb>. The /ð/ sound is spelled <th>.
Both sounds are voiced but they differ in manner and place of articulation. It is unlikely to be a listening problem, although many speakers will have difficulty articulating the /ð/ sound if it does not occur in their own language especially in syllable-final clusters in words such as bathes. For some reason the computer identified an unusually large number of contracted forms such as they'll and that'd in this list. Notice the homograph bow which appears in two different contrasts.
Interesting pairs include
battered that'd
blabber blather
The mean density value is 0.8% though with a wide disparity in the frequency of the sounds. The list makes 35 semantic distinctions, a loading of 76%.
babe bathe babes bathes bail they'll bale they'll barbs baths bare their bares theirs bare there bat that bats that's battered that'd battle that'll bay they bayed they'd bear their bears theirs bear there beau though beaux those be thee bee thee bees these ben then bine thine blabber blather blabbered blathered blabbering blathering blabbers blathers boob booth boobs booths bough thou bourbon burthen bourbons burthens bow thou bow though bows those bus thus buy thy by thy bye thy dibber dither dibbers dithers larboard lathered lobe loathe lobed loathed lobes loathes
John Higgins, Shaftesbury, January 2010.