keenmaster486 wrote on 2024-09-30, 14:14:

Yeah, sorry, I need to get you much more detailed information for this to be useful to you. But these are real problems. I can d […]
Show full quote

mbbrutman wrote on 2024-09-28, 20:19:

1. When was the last time you defragmented you hard drive? Fragmented free space would cause similar behavior. 2. How old is th […]
Show full quote

1. When was the last time you defragmented you hard drive? Fragmented free space would cause similar behavior.
2. How old is this hard drive? Is it using a stepper motor or a voice coil? Stepper motor drives in particular need to recalibrate as they warm up, as the heat changes the relative placement of the head to the tracks on the platters. Wear from use also does this too. If it's an old timer you might want to backup and then do a low-level format before things start to become unreadable.
3. Did you turn on tracing or something and leave it turned on? Tracing is great for debugging, but it's hard on slower systems.
4. Do you have a sufficient number of buffers for DOS in your CONFIG.SYS?
5. What have you set the buffer sizes to in the mTCP config file? Specifically FTPSRV_FILEBUFFER_SIZE and FTPSRV_TCPBUFFER_SIZE
6. Are you transferring files one at a time or is your client opening multiple connections at a time? Multiple connections at a time is going to slow things down and cause extra work for the hard drive as it juggles the multiple connections.

Yeah, sorry, I need to get you much more detailed information for this to be useful to you. But these are real problems. I can document everything in a couple of weeks and even take some videos if that would be helpful.

1. Both issues happen consistently for me across multiple machines, fresh installs, over many years. I always use MTCP FTPSRV the most when I'm first setting up a machine, so fragmentation is not a concern.
2. Same answer as #1, happens no matter what drive or machine I'm using, the churning issue even happens on CF cards but it's less noticeable because of the low latency.
3. Nope. But I suppose that would be useful to diagnose the problem.
4. I don't know. I never set the buffers parameter. What should I set it to? I'm a little unfamiliar with how exactly that affects performance.
5. I don't modify these from the defaults. Should I?
6. Single connection. I did think of this already and always set FileZilla to open exactly one connection at a time and transfer files sequentially, which it does. It works great for the first 10-20 files.

Some info off the top of my head that may help:

-All of my machines use 3C509 cards, except for my 5150, which uses some other card... can't remember what it is off the top of my head right now but it's not 3Com
-Always IDE or MFM, never SCSI
-DOS 5.0 on 286- machines, DOS 7.1 on 386+
-Always most noticeable when transferring at least dozens of files a few KB in size, such as for games that have everything split into many small files
-The delay issue (not the churning issue) doesn't seem to happen on slow machines (8088-286)
-My home network is an unremarkable ethernet network with good quality Cat6 cables, recent Netgear router and switches, no WiFi bridges or anything weird like that
-Both issues have happened across multiple modern machines as well, through multiple motherboard and network card swaps, OS reinstalls, etc., so there is no common denominator there

Do you have full duplex enabled on your 3c509 cards? Turn it off if you do using 3c5x9cfg, I noticed almost identical behavior when I tried enabling FD

thepirategamerboy12 wrote on 2024-09-30, 13:59:

mbbrutman wrote on 2024-09-29, 04:13:

Can you even get network cards for their bus?

There's a ton of C-Bus Ethernet cards and most models from the mid-90s onward have PCI slots too where you could plug in a PCI one. Atm you're basically stuck with Microsoft LAN Manager, Novell Netware, or doing stuff through Windows.

By not working properly I mean stuff like this, which does happen on real hardware as well. I didn't really have any expectations since it's not something you know much about at all, but couldn't hurt to ask I guess.

PC98 isn't that different where this is not possible, but I don't have a machine to work with. And I also need more time too ... On the plus side, I can add this to the list of retirement projects in a few years.

If anybody wants to take a stab at modifying mTCP to run on PC98 machines I'd be happy to consult and help debug.

Keenmaster486

I have 3509 cards and I've never seen this. I would:

• Read the mTCP docs on how to set the performance parameters. Your machines and card might need bigger buffers.
• Read the documentation about how to collect traces. Videos really don't help me understand what might be broken. I need to see real data.
• maxtherabbit suggested moving to a half-duplex connection to see if that changes anything, as he has noticed similar symptoms.
• Check your DOS BUFFERS settings in config.sys.
• Your transfer hiccups might be totally unrelated to the disk .. it might be something like excessive ARP traffic on your network or lost packets. Only a trace can show me that.

Please send a bug report ... let's leave this thread open for feature requests and less complex bug reports.

Netdrive server for older Windowses than 10/11.

Thanks, and I have that on the todo list. It will be a simpler C program that people can compile for whatever platform they need. But as a result it won't be multi-threaded or support the advanced features, like journaled disks.

Like the Unix guys said, you can make 80% of functionality with 20% effort. Or something like that. Those advanced stuff is not needed. But it would be nice to feed your retro DOS computer with an old little retro XP/7 machine.

OT, but the Pareto principle predates Unix by quite a bit ... https://en.wikipedia.org/wiki/Pareto_principle

Wow my precetages were right, i just took them from top of my head. Read it from a Unix critic trolling book somewhere. But yeah nice to have a simple app for oldbie windowses. By the way i now use usb to ftp server capable old WAN and DSL and WIFI routers, they are plenty in the trash.

GigAHerZ wrote on 2024-09-28, 09:54:

I haven't recently checked - Does the telnet program support Z-modem? There are some telnet BBS'es that don't support X- nor Y-modem protocol, unfortunately...
Maybe the modem capability can be implemented as a proxy program for packet driver as i think NetDrive has (or will have) the capability? I remember that one reason for lack of z-modem protocol was that telnet binary was getting too big?

mbbrutman wrote on 2024-09-28, 16:58:

Sorry, no Z-modem. That would make the size of the Telnet program much larger. I'm surprised that there are Telnet BBSes that don't support X or Y modem; that seems to be a configuration error. If there was enough interest I would just have three flavors of Telnet; one with no file transfer, as even X modem and Y modem added quite a bit of code, the current version with X and Y modem, and then a super deluxe version with Z modem too.

I don't have any need for this, but had some thoughts. They may be far less practical than I imagine, so feel free to ignore!

DOS terminal emulators which used serial ports would let you configure some external file transfer software which you could start by picking a menu item or which would be started automatically if its special transfer initiation string was detected. I see that I apparently had Telemate configured to run Moby/GSZ, MPt, BiModem, HS/Link, HydraCom, Lynx, ZedZap and Ice-ZM, most of which I don't remember anything about any more 😁 I guess that was easy for Telemate to support because the serial port state doesn't change just because you run another program, and you might not even need to pass the COM port number to the transfer program because it might just be configured separately because one tended to have just one modem and not move it around!

Perhaps it would be nice if something similar could be done for running external processes for file transfers in telnet, so that the user can choose how much disk space to take up based on which protocols they want support for?

Would it be prohibitively difficult to pass an open socket between processes in mTCP? I imagine it depends on how much of the connection state is in the stack and how much is in the client application.

Perhaps another option would be for telnet to hook some interrupts in order to emulate the FOSSIL interface, in which case some of the file transfer protocols/software I mentioned above might "just work" if they are able to use that interface rather than just talking to the serial port directly. Perhaps this couldn't coexist with someone using SLIP though, and I'm not sure how complex the FOSSIL interface is. Surely it's a lot simpler than a TCP/IP stack though 😁

If not FOSSIL, then perhaps some proprietary interrupt-based interface? Perhaps a proprietary interface is better since I imagine the caller (the transfer software) to have a reasonably large stack so that telnet can call into mTCP?

It is an interesting topic.

The state of a TCP socket is basically some large counters and some flag bits. If you use shared memory and you define a way to pass those back and forth, Telnet or other programs could shell out to a different program, have it pick up and do something, and then return back to Telnet and let it pick back up.

Where things break is how to handle incoming packets. When a program like Telnet first connects to a packet driver, it asks to receive packets and it gives the address of a callback function that will handle incoming packets. That callback function is inside of the Telnet code.

To have another piece of code take over means getting the packet driver to call it instead. The correct way to do that is to have Telnet unregister from the packet driver and have the new code register with the packet driver. And then you miss any packets that came in during that window. And then the new program would have to unregister and Telnet would have to register again upon return. Or you could use self modifying code to have the Telnet incoming packet handler call off to the new code with each incoming packet, but that requires some handshaking because the new code has to be up and running first before you can make those changes. And if you have receive data that is meant for the other program you have to find a way to pass those buffers back and forth too. Serial programs generally don't have these concerns.

As you suggest you could do something more elegant and define a more formal API for programs that will cooperate with an existing program. I actually do something very similar with NetDrive.SYS and NetDrive.EXE and other mTCP programs, which has a similar problem. NetDrive.SYS has no idea where the packet driver is because it loads before DOS is loaded. Those two programs use shared memory to communicate. For the upcoming mTCP I solve the "multiple programs one packet driver" problem by creating a fake packet driver on the fly in NetDrive.SYS and have it pass packets from the real packet driver to the fake one. That is some very active trickery to make that work.

It is interesting and it would make Telnet extensible, but there would be exactly one or two file transfer plugins that would use it, and I'd have to code those because it is unlikely that anybody else would. So given the todo list and the amount of time available, it's probably not worth doing.

mbbrutman wrote on 2024-10-02, 17:21:

To have another piece of code take over means getting the packet driver to call it instead. The correct way to do that is to have Telnet unregister from the packet driver and have the new code register with the packet driver. And then you miss any packets that came in during that window. And then the new program would have to unregister and Telnet would have to register again upon return. Or you could use self modifying code to have the Telnet incoming packet handler call off to the new code with each incoming packet, but that requires some handshaking because the new code has to be up and running first before you can make those changes. And if you have receive data that is meant for the other program you have to find a way to pass those buffers back and forth too.

Thanks for the reply. Ouch, yes, that sounds like it'd be painful. I suppose that due to memory constraints, DOS doesn't seem like the kind of environment where you want to buffer a bunch of data for when the new program launches, but you don't know quite how long that might take. Life was probably easier for launching another program that was going to take over the serial port for a while, because nobody - and presumably no software - expected the serial connection to be perfectly reliable, but we do tend to expect data to not be mysteriously dropped from a TCP connection!

Does the mTCP FTP server support resuming? I guess not as I see this in the output from my FTP client:

Opdracht: REST 1212416
Antwoord: 502 Command not implemented

If not is it possible to implement this?

I can add it to the long term list to investigate adding. Looking at the RFC it does seem irritating if you are in ASCII mode because the restart point is now dependent on how many newline characters are in the file, which is something that you have to scan the file first to find.

How often are you actually getting a broken data connection and needing to resume where you left off?

mbbrutman wrote on 2024-10-06, 16:46:

I can add it to the long term list to investigate adding. Looking at the RFC it does seem irritating if you are in ASCII mode because the restart point is now dependent on how many newline characters are in the file, which is something that you have to scan the file first to find.

How often are you actually getting a broken data connection and needing to resume where you left off?

At the moment quitte often as the wireless part of my PicoMem is not very stable. 😀

Maybe it's an idea to implement it for binary files first or only for binary files? I see for example in the Filezilla client that the default is not to do it for ASCII files as it could indeed be a problem with newlines between different platforms etc. Also most of the times ASCII files are not that big. So with the upcoming overwrite option it would cover all bases (at least for me 😜).

I think the lack of an antenna on the PicoMem is going to be a longer term problem ... I love what he did there, but putting a WiFi device inside of an old metal box just isn't going to work well.

mbbrutman wrote on 2024-10-06, 23:48:

I think the lack of an antenna on the PicoMem is going to be a longer term problem ... I love what he did there, but putting a WiFi device inside of an old metal box just isn't going to work well.

A little off-topic but got this problem also with an open case. Seems it's a firmware problem as it works okay with the fast ram firmware but then you loose the some options like EMS (which I want to keep). As Freddy can't reproduce it it's difficult for him to fix.

Nothing to do with mTCP of course. 😀