Edman wrote: I know it's hard to see reason behind my intentions.
Well, if it is a hobby, isn't that good enough reason 😀
About the supposedly wrong caps, yes, changing some of them with lower ESR can help with noise, and some don't, depending on what they do in the circuit. Changing some of them to larger capacitance than originally will help by allowing lower frequencies to pass the amplifier chip.
But, I would not directly say that the capacitors are of wrong value or of wrong type.
In the case of the TEA2025B amplifier chip, the manufacturer has suggested some example values for some application (like portable radio cassette players in this case), using it for something else (like home computer speaker amplifier) means it can use different component values. One thing is because their power supplies are different, portable stereos have batteries with some noticeable voltage sag during peak current consumption, while PCs have regulated power supply that can provide tens of watts without noticeable sag, so the amplifier chip just does not require that much bulk capacitance in a PC. Back in those days, if you connected passive speakers to a computer, they were typically very tiny and could not reproduce very low frequencies anyway, so why bother putting bigger and larger caps there to make the sound card bigger, heavier and more expensive. If you wanted more quality output, you would use connect the Line Out to home stereo system anyway and ignore the Speaker Out. Of course changing them to values suggested by amp datasheets will allow for lower frequencies to be heard better with headphones or large enough speakers connected to Speaker Out, that is a fact.
And the amplifier output DC decoupling capacitors that should be non-polarized electrolytics according to your source, well, he calls for low leakage current to protect earphones from DC, but I could easily find regular polarized electrolytics with lower leakage currents than what non-polarized electrolytics have. Non-polarized electrolytics are not so common to find in sizes you want easily, and they are more expensive. The thing is, since the amplifier chip is a single supply amplifier, the DC bias polarity over the capacitor never reverses, so the circuit does not even remotely seem to need non-polarized caps. Amplifier chip examples all have regular polarized electrolytics here, just to show that it does not need to be a special non-polarized electrolytic. And capacitor values required by the amplifier chip are so large the only reasonable standard option for these are electrolytic capacitors which are by default just standard polarized capacitors.
If the rant about polarized vs. non-polarized caps mean why there are 1uF polarized electrolytics instead of 0.22uF non-polarized capacitors as the TEA2025B input coupling capacitors, the answer is easy. If the design already had 1uF capacitors elsewhere, it will be OK use them there as well to reduce the amount of different components, instead of introducing new components. It could be cheaper to put for example ten 1uF capacitors per card, than to put eight 1uF capacitors and two 0.22uF capacitors per board. Increasing the coupling capacitance only makes the low frequency response better, and again, a polarized electrolytic is just fine there as long as the DC bias over it is positive or zero. The amplifier chip example puts a non-polarized capacitor there just to show it does not need to be polarized, and 0.22uF caps are so small in value that other types than electrolytics are usually used here and they are non-polarized by default. There is also talk about whether they should be removed because there is no volume slider, well, if there is DC bias voltage on the audio from mixer, then these capacitors must not be removed so that they remove the DC bias of course.
So yes, it is a typical consumer sound card for typical home use, not a high-end professional device for recording studios, that's why it is not perfect in every sense but just good enough, and it shows on the design and price of the device.
By all means, do the recap, I just wanted to show that putting extremely high quality capacitors everywhere makes little point and bigger is not always better. Modern average capacitors are still better than average capacitors back then when the card was made. Just replace with equal capacitance and voltage rating and it will still be improved up to a certain point. To reduce noise even further, swapping better electrolytics is not enough, I am surprised nobody has yet modded their sound card with extra inductors on supply voltage wires for extra noise filtering.