Quick summary. Go buy a multimeter, they run ~$10, an autoparts store will have them along with homedepot, etc. Put the black probe in Common, the red in V(or volts), turn the meter to 20 VDC setting (usually counter clockwise from off where as VAC is clockwise from off) and stick the probes on each battery terminal. Doesn't really matter what order they are in, if they are backwards you'll just see a negative voltage but the magnitude is the same. Try it with the furnace off and then with the furnace trying to kick on.
If your battery is charged it should be 12.6V or higher. If it's really low the battery is discharged or possibly dead. If the voltage is okay with the furnace off but falls very low (ie under 12V) then your battery is probably worn out.
Check it out and throw up some numbers.