The problem with terms like "low density" and "high density" are entirely relative so not very useful.
The GA-5SMM has a SiS 530 chipset. This chipset supports a maximum memory density of 128Mb (same as the i440BX). In 1998 this was 'high density' (as opposed to 64Mb chips), by 2000 this was 'low denisty' (as opposed to 256Mb chips) That means that the maximum supported chips are 16Mx8 and 8Mx16. I don't know if it supports 32Mx4 chips (i440BX certainly doesn't but Via ApolloPro133(A) does).
A single memory bank is 64b wide, so the highest supported bank size is 128Mbx8=128MB. With double-sided DIMMs (128b wide) you can do 128Mbx16=256MB.
If your 256MB DIMMs are detected as 128MB, that means that they have 256Mb chips, either single sided modules with 8 32Mx8 chips, or double sided with 8 16Mx16. The 512MB DIMMs are double-sided with 16 32Mx8 chips. As the SiS530 can only address 128Mb per chip, you get half capacity on all these.
So, for maximum RAM you need double-sided 256MB DIMMs with 16 chips with 16Mx8 structure, same as for i440BX. That way you can certainly get 512MB, and possibly 768MB (depending on max total number of banks supported, either 4 or 6).