Label remover (contains limonene I think) will shift some things. Otherwise I would scrub with a kitchen/bathroom surface cleaner of some kind, though that's likely to work best if it contains some scouring component which may ruin the paint finish slightly.
Alternatively of course, put the network switch back where it was (!) or something similar like a KVM switch or a box of CDs, or cover it with a cloth. When I used to stack my PCs 2-high, I put cut-up cleaning cloths under the feet of the top ones. Probably I had had the same issue as you (long time ago) or was just paranoid about scratching things, but decaying rubber is a big annoyance in so many areas. Recently I found that the matt rubber finish on my best ps/2 keyboard had gone sticky (think it's cleaned up ok with diluted spray disinfectant), as well as a camera lens (likewise some manufacturers give their camera bodies this surface treatment); old film camera light seals decay too, and there is needless internal foam rubber padding in all kinds of things (an old IBM typewriter I obtained was full of foam that turned into sticky black mush when touched for example - presumably for sound muffling despite it being noisy as hell anyway). End of rant 😉