Refined taste is a long list of dislikes. The people that I admire most love their craft so much that they have a highly tuned bullshit filter. It's easy to be offended by good taste because one sees his favorite things impugned and even casual affections challenged but its healthy to be able to withstand passion and disagreement. Slavish devotion to anything is to be avoided at all cost. It leads to a blindness of the mind.
That said I think Flusser's position on wearability a bit stifling on first impression, but its not implausible. His attitude makes me homesick for New York. In the video below Flusser talks about how clueless Hollwood celebrities are about dressing and the difference between style and fashion. Enjoy.