No, the tranny doesn't need to be in any certain gear, when you install the switch. I would put it in any gear other than reverse, when installing the switch (being in R, may make it harder to screw the switch in).
Some aftermarket switches have switch buttons on them that are too long and it will actually bind up the shift rail and make it hard, or impossible to shift into R. At least that I what I have read/heard...never experienced it myself.
You could try unscrewing the switch a turn or two and trying R again. If that fixes the problem, then you could either get a different switch, or put a washer between that switch and the tranny housing, so it can't screw in as far. If you put a washer in, you just have to make sure you have the switch in far enough that it will still activate, when you shift into R, but not too far as to bind up the shift rail.
Also, as suggested, it might be a good idea to remove the switch and see if you can physically push the little button on it in with your finger, just to make sure the switch isn't stuck.