Flagship cards can have good FPS/watt under load, but they all use more power at idle and at low GPU loads. Older games won't load the card at 100% so the FPS/watt will take a big hit. Cards with more memory and wider memory bus consume more power at idle.

Compare these:
GTX 970 4GB = 14.6W at idle
GTX 960 4GB = 11W
GTX 660 2GB = 11W
GTX 650 Ti 1GB = 8W

I don't know about Radeon cards of this generation, but the 4/5/6000 series can be hit or miss. For instance I have a 5570 1GB which throttles to 157MHz at idle and is around 10W maybe? But I have a 5670 1GB which only would throttle to 400MHz and clearly ran hotter at idle (though I never wrote down a measurement).

Some Radeon 3DMark efficiency benchmarks with vsync on at 60 Hz, max resolution and 32 bit. The HD 5770 couldn't go lower than 75 Hz at 2000's max of 1280 x 1024, so it's excluded. All cards averaged 60 FPS except in 2006 and the R7 360 E in 2000 AA. In 2000 the high detail benchmark was used, instead of low as last time. In 2006 the first SM3 benchmark was used.

Peak system Wattage, 1 second window:

Ratio of FPS to average system Wattage:

The R7 360 E was about 30-40% more efficient than the HD 7970 GHz and about 5-15% more efficient than the HD 5770. This time the 5770 wasn't undervolted.

In 2000 with AA, the R7 360 E averaged 57 FPS, so it didn't quite hit the cap. The HD 5770 could do 74.6, so overall for DX6-9 it tends to have similar efficiency plus better performance in the DX6-7 range. In DX9 the R7 starts being faster, but its driver breaks compatibility with 3DMark 99.

Quadro FX 1500. Good efficiency. Looks like good support for DX6 onwards with 90 series drivers, but 3DMark 06 is too demanding so at best no late DX9. Not enough performance for SSAA, besides the drivers don't seem to support pure SSAA.

FPS per Watt, vsynced to 75 Hz in 2000 and 60 Hz in 2003: