The spelling of /p/ is <p> in this set. The /ʒ/ sound is spelled <ge> or <su>.
The contrast is between a voiceless stop and a voiced fricative, well separated in the mouth. It is unlikely to be a problem.
The consonant /p/ can occur in any position in the syllable, but /ʒ/ only occurs finally or intervocalically. /ʒ/ is in any case a very rare phoneme, found in only 334 words in the source dictionary. The figure for the mean density of this contrast is 3/7465, too small to signify, while the semantic loading is 100%, since there are no pairs which are inflections of the same root.
leopard leisured leper leisure loop luge
John Higgins, Shaftesbury, January 2010.