Those "barrel" batteries are usually a few ni-cad cells in series, count the bumps and multiply by 1.2 to get the voltage.
Ni-cads being rechargable will have a reverse "charging" voltage/current - You can replace them with AA(a) "dry" cells, but you need to add a diode to block this reverse current. Keep in mind that a diode will drop about 0.7v - I usually find that 3 AA cells = 4.5v = 0.7v works out to about 3.8v which is usually close enough and within spec for the devices in the circuit.
And do remove the old cells - they are notorious for corroding and leaking when they get old!
- Dave ; https://dunfield.themindfactory.com ; "Daves Old Computers" ; SW dev addict best known:
ImageDisk: rd/wr ANY floppy PChardware can ; Micro-C: compiler for DOS+ManySmallCPU ; DDLINK: simple/small FileTrans(w/o netSW)via Lan/Lpt/Serial