i guess that's just your monitor's refresh rate limiting it to 60Hz
you can check the real fps in a separate app drawing an fps chart, once you minimize dosbox

1. tiny change here first:
dosbox.conf
[sdl]
priority=higher,highest
(if you have windows control panel->power options->high performance,
then priority=higher,higher is enough)
[vsync]
vsyncmode=off

2. in dgvoodoo control panel
in "General" tab
Enumerate refresh rates on
in "Glide" tab
Resolution:
640x480, 999Hz
(or whatever resolution you want, the 999Hz is important here 😀 )
Miscellanneous:
Force vSync off

3. radeon's control pannel -> vsync off

4. once you got the game running in a window hit esc for the game's menu
5. now place your mouse cursor on minimize icon and hit esc to leave menu and immediately click minimize before the book animation disappears.

You should see real fps on the chart now 😀

(you can do the same in fullscreen, hit esc, and immediately after hit win+d to minimize)
(chart from MSI Afterburner works 😉 )