Maybe google on how to use a voltmeter ...
You'd want to pick the DCV (Volts, DC, all batteries [Technically cells in this case] are DC) setting for a battery - the different numbers just set the measurement range. For example, the 2000m setting will measure voltages up to 2000mV or 2V with a resolution of 1mV. The 20 setting will measure up to 20V (usually with a resolution of 10mV or 0.01V) and so on.
Higher end multimeters are auto-ranging so all you need to do is select DC Volts and hook it up - it'll do the rest for you.
Further edit: Keep in mind, you need to have the probes plugged into the correct socket. Usually it's a common ground socket with one socket for Voltage/Resistance/Diode testing and the other for Current. Current is typically (typically because clamp meters don't really work that way...) measured through a component while voltage is measured across a component. The socket for current will be low resistance (to ground) to allow that current to flow through the meter and be read, the socket for voltage will be high resistance (to ground) so that lots of current doesn't flow through the meter and destroy it when you go to take a voltage reading.
