ext4 is faster than ext2 and ext3 for transferring large amount of small files, while vfat is good for transferring small amount of large files. Therefore, if the purpose of your external flash is to store media files, you may not need to reformat.

In addition, without wear levelling, ext4 is not the best choice for flash media. I'm not sure if the latest development has included wear levelling in ext4, you may search 'wear levelling' in t.m.o for detail discussion.