The spelling of /v/ is <v>. The /ʒ/ sound is spelled <su>.
The contrast is between two voiced fricatives fairly well separated in the mouth. It is unlikely to cause a problem.
The /v/ sound can occur in any position in the syllable, but /ʒ/ only occurs finally or intervocalically, other than in foreign names or loan words such as gendarme. /ʒ/ is in any case a very rare phoneme, found in only 334 words in the source dictionary.
The figure for the mean density is 0.1%. The semantic loading is 100%.
clover closure Trevor treasure
John Higgins, Shaftesbury, January 2010.