Yes to measure voltage, set multimeter to measure DC voltage and connect common (black) terminal of multimeter to common/ground (black) terminal from power supply.
Any black wire will do, for example one on molex connector, or even the metal case of the PSU.

With multimeter red lead, poke around metal contacts on P8&P9 connector. Use a paper clip or wire as a conductor if you can't reach connector pins with huge multimeter probe.

Having said that, the results are quite meaningless if you have no load on the PSU. It may not even start without proper load.
On one AT supply I had to connect two spare hard drives as dummy load before output voltages looked sane.

With too light load the voltages may be out of specification, either too high or too low.

But as much as you'd like to test if the capacitors are good or bad, you only see average DC voltages with a multimeter.
You can't see how much there is voltage ripple with a multimeter.

Fortunately if caps are in really bad shape then the DC voltages could show too high or too low values as well.

Edit: I just noticed you also wrote you want to measure amps too. As a word of warning, you can't check amperes by just poking the multimeter in ampere measurement mode directly to a PSU, that'll short the PSU through the multimeter and huge current will flow, blowing the (about 10A) safety fuse in the multimeter and possibly damaging the PSU as well. What you need is to connect a load to the PSU and you connect the multimeter between PSU and load to measure how much current flows. Also measuring currents is not so important, because if you put resistors as dummy load, you know the resistance and can measure voltage over them so you know how much current goes through the resistors. For example, 5 ohms resistor connected to 5 volts draws 1 amp load current so it's a 5 watt load. A 12V 50W halogen or car headlamp is a 50W load on the 12V line, but I can't recommend them as dummy loads - they are too bright to look at and if you soldered wires directly to the lamp, the lamp gets hot enough to melt solder.

The other concern with a power supply is ripple and transient voltage (both at startup and load change). To measure that, you will need an oscilloscope. If you don't have access to one, a second hand tube scope is very inexpensive to purchase. Hell, you might be able to find one for free. There are also good options available starting for <$500.