Brahms the Baby?
"Really? The composer of the glorious First Symphony?" Erick was aghast.
"Oh," said JA, "who could dislike that gorgeous clarinet quintet?"
I said, "Actually, my reaction was that nobody could be deaf to the opus 51 quartets. Even Schoenberg liked them."
So you have three classical music fans reacting to a composer and we all chose radically different works from different periods of Brahms' career. That may be why people like Midgette "don't like" him—he just wrote so damn much and in so many forms.
"On the other hand," Erick said, "he did write too many Liebeslieder waltzes. I can't stand those."
See? He wrote so much that, while there's something for everyone, there's also almost certainly something that will tick off almost everyone.
For the record, Furtwangler agreed with Erick about the First Symphony.