But looking at this promo video of Star Trek Online: Borg Faction made me realize that what makes the Borg a brilliant plot device is that not only do they outclass Federation technology, but on top of that they’re pure evil. Now some fans have critiqued the Borg for just being a rip off of the Daleks from Dr. Who: But for me that’s a superficial comparison because Dr. Who (and the Time Lords) are on some equal footing with the Daleks. On the other hand the Federation isn’t in the same league as the Borg. In fact the irony of the first Star Trek: The Next Generation script is that the ever so annoying character Q is a rip off of sorts of Dr. Who.
I'm no Whovian, so I'll not address that point. But I do fully agree that the Borg were a necessary first step to adding actual risk into Federation ventures.
In classic Trek, the Federation had more or less naval parity with its two primary rivals. By the beginning of TNG, however, one foe had become variously friendly or neutral and the other was contained within its borders. There was no existential threat to the Federation -- in much the same way that the US presently enjoys worldwide naval supremacy.
Any pre-Borg threats were either godlike, such as Q, or subversive, such as the neural parasites. And I would argue that the Borg, at the beginning, were treated as godlike beings requiring magical solutions, rather than conventional threats requiring classic naval strategy. This changed by the later Voyager seasons when Borg/Federation naval parity was fairly close.
Now these plot lines could be quite interesting, but there was something missing in the extremes of impossible-to-stop threats against the Federation and defenseless Cardassian bandits. This narrative middle ground was found in the Dominion War of DS9 when the Federation faced a conventional foe that was militarily superior, but not overwhelmingly so (and the Cardassians were re-written as a more impressive force). From this balance emerged one of the most exciting story arcs in the Star Trek canon.