First post, by FrankensteinTech
I initially searched around these forums (and tons of others online) regarding the topic of running DOS/Windows 98 under Proxmox/VirtManager (QEMU/KVM) with "passthrough" input/audio/video+acceleration, but I wasn't able to find all the answers to my questions in one place (or at all in some cases)... I spent months playing around with settings in Proxmox (my preferred hypervisor) until I found what worked well for me, and I have collected them here for everyone's reference.
TL;DR: Proxmox is capable of running DOS *or* Windows 98 as an emulated *or* virtualized machine (with virtualized legacy audio and video cards!), with virtualized PS/2 keyboard/mouse binding and passthrough PCI FX 5500 video (including 3D acceleration of VESA/DirectX/OpenGL, even over a Thunderbolt-connected PCI bridge)!
NOTE: I am not sure if this is the correct place for this topic... but it involves PCI passthrough of legacy hardware, so "General Old Hardware" felt appropriate.
My goals/hurdles for this project were the below (all of which were successful!):
Hurdle: I wanted to use the 'newest' Intel (or AMD) hardware available to run DOS/Windows 98 near-natively via virtualization and passthrough solutions (ideally no emulation)
Solution: Any *modern* (Nehalem and up) Intel (or AMD) hardware will work fine, but the below requirements need to be met:
*Ideally, you will want to use a non-Intel GPU as the hypervisor's primary video output (Intel GPUs cause 'VGA Arbitration' issues, resulting in no output from the passed-through legacy GPU)
NOTE: This can be worked around by using the i915 VGA Arbiter patch, but that patch is not present in the Proxmox kernel.
*Motherboard/CPU must support Virtualization (VT-x) and IOMMU (VT-d or AMD-v) for PCI passthrough of the legacy GPU
Hurdle: I wanted to use Proxmox (specifically) as the hypervisor (instead of VirtManager) as I prefer being able to Proxmox' Web UI for quickly controlling VMs via cell phone/etc)
Solution: Easy-peasy, Proxmox will boot on basically anything... CSM, EFI, doesn't matter as long as it was built in the last 15 years.
Hurdle: I wanted to have native keyboard+mouse (PS/2) within DOS/Windows 98 via EVDEV passthrough (USB, bluetooth, PS/2, etc)
Solution: Proxmox doesn't expose it, but QEMU/KVM do allow the PS/2 ports of VMs to be mapped to Linux EVDEV input devices (i.e. any input device within the host OS, i.e. USB/bluetooth/PS2/etc) via arguments.
Hurdle: I wanted to have native video+acceleration (VESA, DirectX and OpenGL) from DOS/Windows 98 via PCI passthrough of a real GPU
Solution: Despite it's poor reputation, I quite like the Nvidia Geforce FX 5500 (5200 Ultra) - the PCI version works with modern hosts, and can be used over Thunderbolt with an eGPU adapter and a PCIE->PCI bridge (ASM1083).
Hurdle: I wanted to have native audio (Adlib, SB16 and optionally AC'97) from DOS/Windows 98 outputting via ALSA/PulseAudio on the hypervisor (host)
Solution: Proxmox doesn't expose it, but QEMU/KVM allow virtualization of the below sound cards with output via ALSA/PulseAudio:
*SB16 (Creative Sound Blaster 16)
*ES1370 (ENSONIQ AudioPCI ES1370)
*AC97 (Intel 82801AA AC97 Audio)
*ADLIB (Yamaha YM3812/Adlib [OPL2])
*GUS (Gravis Ultrasound GF1)
*CS4231A (CS4231A)
*HDA (Intel HD Audio)
*PCSPK (PC speaker)
Bonus: You can also use PCI passthrough to use a real sound card if you want to - I have tested YMF-7xx and CMI8738 and they both work fine.
Hurdle: I wanted the above solution to be portable between Proxmox hypervisors (hosts) using Thunderbolt
Solution: Proxmox, combined with Thunderbolt-based VM storage (NVME) *plus* Thunderbolt-based passthrough eGPU (FX 5500 via bridges/etc) and EVDEV (PS/2) + ALSA/PulseAudio (ADLIB/SB16/AC97) results in a 'portable' DOS/Windows 98 VM.
I have been successful in booting DOS/Windows 98 under Proxmox with the above VM configuration on the below hardware:
*HP DL370 G6 (GPU ROM dump required, PCIE->PCI Bridge *and* Titan Ridge TB3->NVME->PCIE->PCI Bridge))
*HP DL380 G8 (GPU ROM dump required, PCIE->PCI Bridge *and* Titan Ridge TB3->NVME->PCIE->PCI Bridge)
*Gigabyte GA-X58A-UD3R (Native PCI *and* Titan Ridge TB3->NVME->PCIE->PCI Bridge)
*Dell T3610 (Native PCI *and* Alpine Ridge TB3->NVME->PCIE->PCI Bridge)
*Dell T5810 (Native PCI *and* Alpine Ridge TB3->NVME->PCIE->PCI Bridge)
*Apple MacBook Pro 2015 (TB3->NVME->PCIE->PCI Bridge)
*Apple MacBook Air 2018 (TB3->NVME->PCIE->PCI Bridge)
From a hypervisor preparation standpoint, this is the process I follow to stand up a new host (NOTE: I use my Proxmox hosts as 'gaming workstations'):
*Download Proxmox (https://proxmox.com/en/downloads/category/iso-images-pve) and burn to DVD/place on flash drive
*Install Proxmox on the host using CSM or EFI boto method (doesn't matter which)
*Ensure that the primary GPU in use by the Proxmox OS *is not* intended to be passed through to the VM (i.e. don't use the FX 5500 to run Proxmox)
*Don't install the Proxmox OS on a flash drive/SD card/etc (logs will kill the drive in no time)
*Add 'deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription' repo to file '/etc/apt/sources.list'
*Run command 'apt update' to... update apt!
*Run command 'apt dist-upgrade' to... upgrade the distro!
*Run command 'apt install xfce4 chromium lightdm' to "convert Proxmox into a workstation" (i.e. xfce4 GUI + Chromium browser)
*Run command 'adduser insertusernamehere' to create a lower-privileged user account for GUI access
*Optionally run command 'apt install sudo' to install SUDO (will also require editing /etc/
*Optionally run command 'apt install bolt' to install support for Thunderbolt adapters (i.e. Gigabyte Alpine/Titan Ridge, etc)
*Optionally run command 'apt install bluez' to install support for Bluetooth adapters (keyboards, mice, audio, etc)
*Add the below lines to file '/etc/modules' (enables VFIO modules required for PCI passthrough to work smoothly):
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
*Create file '/etc/modprobe.d/iommu_unsafe_interrupts.conf' with the below lines (configures VFIO to be less strict about interrupts):
options vfio_iommu_type1 allow_unsafe_interrupts=1
*Create file '/etc/modprobe.d/kvm_ignore_msrs.conf' with the below lines (configures KVM to ignore MSRs):
options kvm ignore_msrs=1
*Create file '/etc/modprobe.d/load-vfio-before-xxxx.conf' with the below lines (configures Kernel to load VFIO module *before* the GPU modules):
softdep radeon pre: vfio-pci
softdep amdgpu pre: vfio-pci
softdep nvidia pre: vfio-pci
softdep nouveau pre: vfio-pci
*Create file '/etc/modprobe.d/vfio.conf' with the below lines (configures VFIO module to 'hook' the listed PCI devices at boot time to allow easier passthrough - in this case, 10de:0326 is the PCI ID for the FX 5500):
options vfio-pci ids=10de:0326
NOTE: Do not list your Proxmox OS' primary video card in 'vfio.conf' - it should only contain PCI IDs for intended for PCI passthrough!
*Commit the changes to '/etc/modprobe.d' by running command 'update-initramfs -u -k all'
*Edit file '/etc/default/grub' with the below (enables/optimizes Intel's IOMMU and it's grouping, in addition to disabling PCI-E power management):
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction pcie_aspm=off"
*Commit the changes to '/etc/default/grub' by running command 'update-grub'
*Add the below line to file '/etc/pulse/default.pa' to create an 'open' PulseAudio socket in the /tmp folder:
load-module module-native-protocol-unix auth-anonymous=1 socket=/tmp/pulse-socket
*Install ALSA tools and initialize ALSA by running the below commands:
apt install alsa-utils
apt install alsamixergui
*Initialize ALSA and make note of which Audio adapter is assigned to which number:
alsactl init
NOTE: If your Proxmox host has multiple Audio adapters (assuming they have not been intentionally 'hooked' by 'vfio.conf') then you may need to force PCM to go to a specific audio device (based on the above output) by creating file '/etc/asound.conf' with the below contents:
pcm.!default {
type hw
card 0
}
ctl.!default {
type hw
card 0
}
*After performing the above, reboot! Proxmox should automatically load into the XFCE4 Window Manager (GUI).
*Load Chromium, and browse to 'https://localhost:8006' for easy management of Proxmox VMs (don't forget to bookmark it! 😀.
*NOTE: One of the benefits of having XFCE4/Chromium installed on the Proxmox host is the ability to administer Proxmox VMs when network access is down/unavailable.
*Argument '-machine hpet=off' is required to ensure VM performance remains consistent in 16-bit and 32-bit OSes
*Argument '-device adlib -device sb16' enables Adlib and Sound Blaster 16 devices in VM
*Argument '-audiodev id=audio0,driver=xxxx' tells QEMU/KVM where to route the VM's audio to (ALSA or PulseAudio)
*Argument '-object input-linux,id=kbd1,evdev=/dev/input/eventX' binds the VM's virtualized PS/2 keyboard port to a Linux EVDEV keyboard input device (found in /dev/input and /dev/input/by-path - make note of the symlinks in these directories!)
*Argument '-object input-linux,id=mouse1,evdev=/dev/input/eventX' binds the VM's virtualized PS/2 mouse port to a Linux EVDEV mouse input device (found in /dev/input and /dev/input/by-path - make note of the symlinks in these directories!)
*Parameter 'hostpci0' can use value 'romfile=gpuromfilegoeshere.rom' to define a dumped GPU ROM file located in '/usr/share/kvm/gpuromfilegoeshere.rom' (this is sometimes required for finicky Proxmox hosts)
*Parameter 'hostpci0' will will need value 'x-vga=1' if you are passing a legacy GPU through to the VM (FX 5500) to ensure the VM treats it as 'Primary'
*Parameter 'machine' needs value 'pc-i440fx-2.11' to ensure Proxmox loads the correct Machine Type (2.11 was the last version to include working legacy audio)
*Argument 'kvm' can have value '0' (hardware virtualization disabled) or '1' (hardware virtualization enabled)
NOTE: This setting does not actually affect PCI passthrough (works either way), but it *does* cause the HAL in the VM (Chipset/CPU/etc) to become emulated (instead of virtualized)... this is good from a compatibility standpoint, but bad from a performance standpoint. Toggle this setting if DOS/Windows 98 timing is too fast or too slow.
*Argument 'cpu' should have value 'qemu32' (artificially limits the CPU features in the VM to closely replicate a 32-bit only CPU) but optionally can instead have value 'host' to fully expose the Proxmox OS' CPU to the VM OS.
args: -machine hpet=off -device adlib -device sb16 -audiodev id=audio0,driver=pa,server=/tmp/pulse-socket
balloon: 0
boot: order=ide0
cores: 1
cpu: qemu32
ide0: local:xxx/os.raw,backup=0,size=10G
kvm: 1
machine: pc-i440fx-2.11
memory: 512
meta: creation-qemu=7.0.0,ctime=1669278954
name: DOSGames
numa: 0
ostype: other
scsihw: virtio-scsi-pci
smbios1: uuid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
sockets: 1
tablet: 0
vga: cirrus
vmgenid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
args: -machine hpet=off -device adlib -device sb16 -audiodev id=audio0,driver=pa,server=/tmp/pulse-socket -object input-linux,id=kbd1,evdev=/dev/input/event1 -object input-linux,id=mouse1,evdev=/dev/input/event2
balloon: 0
boot: order=ide0
cores: 1
cpu: qemu32
hostpci0: 0000:1b:04,x-vga=1
ide0: local:xxx/os.raw,backup=0,size=10G
kvm: 1
machine: pc-i440fx-2.11
memory: 512
meta: creation-qemu=7.0.0,ctime=1669278954
name: DOSGames
numa: 0
ostype: other
scsihw: virtio-scsi-pci
smbios1: uuid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
sockets: 1
tablet: 0
vga: none
vmgenid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
args: -machine hpet=off -device adlib -device sb16 -audiodev id=audio0,driver=pa,server=/tmp/pulse-socket -object input-linux,id=kbd1,evdev=/dev/input/event1 -object input-linux,id=mouse1,evdev=/dev/input/event2
balloon: 0
boot: order=ide0
cores: 1
cpu: qemu32
hostpci0: 0000:1b:04,romfile=gpuromfilegoeshere.rom,x-vga=1
ide0: local:xxx/os.raw,backup=0,size=10G
kvm: 1
machine: pc-i440fx-2.11
memory: 512
meta: creation-qemu=7.0.0,ctime=1669278954
name: DOSGames
numa: 0
ostype: other
scsihw: virtio-scsi-pci
smbios1: uuid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
sockets: 1
tablet: 0
vga: none
vmgenid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
args: -machine hpet=off -device adlib -device sb16 -audiodev id=audio0,driver=alsa
balloon: 0
boot: order=ide0
cores: 1
cpu: qemu32
ide0: local:xxx/os.raw,backup=0,size=10G
kvm: 1
machine: pc-i440fx-2.11
memory: 512
meta: creation-qemu=7.0.0,ctime=1669278954
name: DOSGames
numa: 0
ostype: other
scsihw: virtio-scsi-pci
smbios1: uuid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
sockets: 1
tablet: 0
vga: cirrus
vmgenid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
args: -machine hpet=off -device adlib -device sb16 -audiodev id=audio0,driver=alsa -object input-linux,id=kbd1,evdev=/dev/input/event1 -object input-linux,id=mouse1,evdev=/dev/input/event2
balloon: 0
boot: order=ide0
cores: 1
cpu: qemu32
hostpci0: 0000:1b:04,x-vga=1
ide0: local:xxx/os.raw,backup=0,size=10G
kvm: 1
machine: pc-i440fx-2.11
memory: 512
meta: creation-qemu=7.0.0,ctime=1669278954
name: DOSGames
numa: 0
ostype: other
scsihw: virtio-scsi-pci
smbios1: uuid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
sockets: 1
tablet: 0
vga: none
vmgenid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
args: -machine hpet=off -device adlib -device sb16 -audiodev id=audio0,driver=alsa -object input-linux,id=kbd1,evdev=/dev/input/event1 -object input-linux,id=mouse1,evdev=/dev/input/event2
balloon: 0
boot: order=ide0
cores: 1
cpu: qemu32
hostpci0: 0000:1b:04,romfile=gpuromfilegoeshere.rom,x-vga=1
ide0: local:xxx/os.raw,backup=0,size=10G
kvm: 1
machine: pc-i440fx-2.11
memory: 512
meta: creation-qemu=7.0.0,ctime=1669278954
name: DOSGames
numa: 0
ostype: other
scsihw: virtio-scsi-pci
smbios1: uuid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
sockets: 1
tablet: 0
vga: none
vmgenid: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
My go-to drivers for DOS are:
*JEMMEX
*HIMEM
*CTMOUSE
*MOSLO
*SHSUCD
*UNIVBE (works with virtualized Cirrus VGA!)
The only drivers needed in Windows 98 would be:
*Audio Driver (Adlib/SB16 drivers are already built-in to Windows 98 and use default ports)
*Audio Driver (Intel AC'97, attached as file "intel_ac97_windows98.zip"- hard to find!)
*Video Driver (Fx 5500, I use 81.98 from here: https://www.vogonsdrivers.com/getfile.php?fil … 952&menustate=0)
Nothing special here - upload a Windows 98 Setup ISO file to the Proxmox OS host, mount as CD-ROM in VM and install Windows 98 (or DOS) as normal.
Alrighty... so, you wanna get a legacy GPU (FX 5500 PCI *wink wink nudge nudge*) working on a DOS/Windows 98 VM running on a Thunderbolt-enabled Proxmox host eh?
Maybe something crazy like a MacBook Pro running Proxmox, with a Windows 98 VM passthrough via Thunderbolt? 😁
If so, here's the parts list you will need! 😀
Corsair CX750 (PSU) ~$50
EDIY eGPU Bracket ~$40
ADT-LINK R3G (PCI-E -> TB3) ~$150
Startech PEX1PCI1 (ASM1083 PCI -> PCI-E) ~$40
Nvidia Geforce FX 5500 256MB PCI ~$50
Total: $330
Proxmox host doesn't have Thunderbolt?
Optional: Gigabyte GC-TR (or GC-AR) w/SL0 Firmware from eBay/etc (TB3 -> PCI-E) ~$50
Once you have the above, you can use 'boltctl enroll' on the shell in Proxmox to 'trust' the Thunderbolt adapter and then use 'lspci -v' to confirm if the PCI device is visible on the PCI bus - once visible, it can be passed through to a VM as normal in the Proxmox UI.