I think the start of this development begins with the Frederic Wertham's campaign against violence and immorality in comics in the 1950's. In his book, Seduction of the Innocent, Wertham made many unfounded accusations against comics at the time, some of which included suggestions that Batman and Robin were a sort of homosexual fairy tale.
Mort Weisinger developed which worked well for the Man of Steel...
...but not as well for Batman.
So what you end up with is a cycle wherein the more imaginative/evocative covers sort of dictated Superman editorial for a while. These were covers that rarely featured Superman in a one on one struggle with a villain, as Red Kryptonite and Imaginary Stories provided a means to present more evocative storylines.
On the rare occasion that Superman was shown battling some worthy foe, it was usually tied to some gimmick.
And as the target audience was (at the time) a sort of ever replenishing pool of juvenile readers, the use of outlandish ideas worked for many years (decades even)
[ image from Superman in the Bronze Age blog ]
And while Batman got a revamp in the 60's which allowed writers to focus more on adventure/crime stories featuring such intriguing opponents as Ras Al Ghul, Superman never got a similar editorial makeover. The best new villains he got was Terra-Man and Vartox (both of which never really gained a fan following)
In this modern age of retro navel gazing and intellectual property hoarding, I doubt we will see many new inspiring rogues developed for Superman.
And that's sort of a shame.