If you are sure you only need ARP.EXE, you can extract an updated version from the WinSock 2 Windows 95 Update. It's a free update distributed by Microsoft back then, so I think it's OK to upload it here. The "wsock2.exe" is a self-extracting archive, you can use 7-Zip ow WinRAR to open it and extract the ARP.EXE file. You will also need "SNMPAPI.DLL" and "INETMIB1.DLL", and depending on you system maybe others.
Keep in mind that Windows 95 requires the installation media for almost any system change, so depending on what you are trying to do, it's very likely you are going to need it sooner or later.