I've ran into an odd scenario where I might need to use my pci voodoo 3 in a slot thats not closest to the CPU/northbridge. I know in general practice I've always put it in this slot since there is a distance factor that would contribute to latency but would it effect performance? I can't seem to find anything on this subject.
No, the latency thing is a weird myth, no idea where you heard it.
PC busses are fixed clock, there is no added latency the further something is located, its all taken into consideration in the standard with maximum allowable distance accounted for.
Besides PCI bus is parallel, all the cards in all the slots are seeing same signals. There is no way to send faster signals to closer card, the further ones wouldn be able to tell what is going on.
Many motherboards assign different IRQs and resources to different PCI slots, and the slots also tend to share resources with motherboard devices as well. Some slots tend to have more free resources available than others.
This can cause compatibility issues with almost any sort of PCI card depending on the driver's tolerance for shared resources. I often have to move cards around depending on the OS that I install.