Doesn't that just loop until f_vco is between min and max? And the found
value for r is used then.

I hadn't bothered to understand what it was doing, I just recognized the if logic; breaks only if both conditionals are true, if f_vco is less than or equal to min_vco then it's less than max_vco, as min_vco is less than max_vco, so either checking f_vco < max_vco is unecessary or it's supposed to be checking >=.

Care that it's

1MIN_VCO <= f_vco && f_vco < MAX_VCO

and not

1f_vco <= MIN_VCO && f_vco < MAX_VCO

The <MAX_VCO check might be not necessary in the loop, but could be
put after it, yet it looks cleaner that way and isn't speed relevant code.

"Care that it's" What? A more complete sentence perhaps?

As well, in your example I can see you swapped variable positions but I don't know what you're trying to show in doing so.

I don't find doing such cleaner, but confusing, and yes, of course it isn't speed relevent.

Speaking of optimizations, vga_determinmode can be size optimized by replacing all the vga_setmode calls with variable=immediate.

1void VGA_DetermineMode(void) {
2    int mode;    
3	/* Test for VGA output active or direct color modes */
4	if (vga.s3.misc_control_2 & 0xf0) {
5		switch (vga.s3.misc_control_2 >> 4) {
6		case 1:mode=M_LIN8;break;
7		case 3:mode=M_LIN15;break;
8		case 5:mode=M_LIN16;break;
9		case 13:mode=M_LIN32;break;
10		}
11	/* Test for graphics or alphanumeric mode */
12	} else if (vga.attr.mode_control & 1) {
13		if (vga.gfx.mode & 0x40) mode=M_VGA;
14		else if (vga.gfx.mode & 0x20) mode=M_CGA4;
15		else if ((vga.gfx.miscellaneous & 0x0c)==0x0c) mode=M_CGA2;
16		else {
17			if (vga.s3.reg_31 & 0x8) 
18				mode=M_LIN4;
19			else
20				mode=M_EGA;
21		}
22	} else {
23		mode=M_TEXT;
24	}	
25	VGA_SetMode(mode);
26}

A more complete sentence perhaps?

Yeh maybe. Send it to harekiet or so.