Finally I was managed to get to a PC with ATI R9600 and Catalyst 4.9beta (or alpha or something like that). I tried dgv on that PC, and indeed, there were problems in VDD mode. There were different cases, once it worked fine, once again it just dropped me back to the desktop, and also I could experince those wonderful reboots, when an invalid page exception occured in ATI3DUAG.DLL (I don't know that exactly what this component is responsible for).
However, I had no problems with server mode!
Frankly, when I was about to port dos part of dgv to XP, I wasn't sure at all, if it is "healthy" or not to use DirectX inside the NTVDM process. Since NTVDM has its own virtual screen, the display driver may take care this process in an other way than a normal win32 process (I think of that you can switch to full scr text or mode when GDI doesn't work, etc.). So, I was afraid of that DirectX would do a crash somewhere in the videodriver along with handling console of NTVDM. I was searching some docs about it on the net, but all I found was that somebody wrote a VDD that provided VESA support via DirectDraw, however that wasn't enough for me, I would have liked to read an "official" doc from Microsoft, or something like that.
All in all, this is the only reason why I support server mode under XP. At the moment I have the feeling that this problem is brought in ATI's newest drivers, but, of course, maybe I'm totally wrong. It's only a guess.