If your 320x200 modes are not openglhqified, you probably have "scaler=normal2x" (the default value). Set it to "none", because otherwise, OpenGL-HQ sees 4 pixels where one should be, and decides to keep the blockiness 😉
OpenGL-HQ asks your video driver if it runs the pixel shaders in hardware and refuses to run if it doesn't - there is no way of having it fall back to software or something.
If display seems sluggish in fullscreen, remember that higher resolutions are slower - even though most is done in hardware, output size still matters, and your video driver does a few calculations on the CPU, so your CPU speed also matters. Raising frameskip by 1 should help. But nevertheless, it's a little bit funny... Check that you don't force vsync-wait for all apps in the catalyst control center, that made dosbox a lot slower for me.
I have found out that fulldouble is generally problematic: don't use it with openglhq. There is a workaround if you really want it, read about SDL_OPENGLHQ_DOUBLEBUF in the README.
Now about your problems, WilliamC: If you've downloaded a new SDL.DLL and set SDL_VIDEODRIVER, you're using the correct release. If you set "ouptut=openglhq" in dosbox.conf, you're using the old release. Don't do that. Go to the download directory and fetch the newest SDL-1.3-<something>.zip, read the README, and replace the SDL.DLL of dosbox with the one from the zipfile. Get back to us if you need more help. Oh, and if you have any problems, first check a plain 0.63 release of dosbox: I found out that one of my own mouse patches doesn't work that well with SDL-openglhq.