BTW: wasn't there an alternative to ODIPKT called PKTODI ?
I would like to try that, but can't find...

mbbrutman wrote on 2025-02-11, 20:04:

I might craft up some debug code later ...

Amazing, we would be very grateful. Hopefully it’s something simple that can be adjusted.

Please try http://www.brutman.com/mTCP/download/netdrive.sys_debug and let me know what it does.

Mike

mbbrutman wrote on 2025-02-11, 21:27:

Please try http://www.brutman.com/mTCP/download/netdrive.sys_debug and let me know what it does.

Same story - it connects the remote image, can list the directory, but fails when reading a file.

Meanwhile, I tried to replace ODIPKT with PKT2ODI - https://home.mnet-online.de/willybilly/fdhelp … ork/pkt2odi.htm
FTP client seems to work fine.
NetDrive - using the original NETDRIVE.SYS - hangs already on the "netdrive connect..." command.

One more time please ... just download that file again. (I replaced it.)

If this doesn't work I'll have to look into recreating the problem myself.

mbbrutman wrote on 2025-02-11, 22:41:

One more time please ... just download that file again. (I replaced it.)

This time it fails on "netdrive connect...", but doesn't hang.
Tried with both ODIPKT and PKT2ODI - same effect:

[/s]

You might want to clear things out and try again ... I didn't change the "connect" flow, and you are using the same EXE as before so that should have worked. If it connected before, it should connect again.

And just a technical note: the reason you were able to see the directory listing before is because after the connect the EXE program does a filesystem command to force DOS to see the disk change. The root directory was probably buffered by DOS at that point. Further attempts to do reads would be failing once the SYS is given control, probably due to something mismatched in the ODI conversion.

Oops, indeed - I killed the server, my previous post is total nonsense.
Obviously, I should go to sleep already...

After starting the server, the situation is back to "normal":
- with ODIPKT, it connects the image, lists the directory, but fails while reading a file
- with PKT2ODI, it hangs on "netdrive connect..."

Good night!

mbbrutman wrote on 2025-02-11, 21:27:

Please try http://www.brutman.com/mTCP/download/netdrive.sys_debug and let me know what it does.

Mike

Just getting a 404 error when I go to that URL 🙁

Is there a debug log file generated that I should send back to you?

It didn't work for Grzyb so I removed it.

There is no debug log file available for this; a 5.5KB device driver doesn't do things like that. I'll have to recreate it here, but it's going to take a while because I generally don't use ODI and I'll have to get up to speed on it.

mbbrutman wrote on 2023-12-10, 22:06:

DOS 3.31 and up can use remote images up to 2GB in size.

This made me confused - other sources claim that the limit for DOS 3.31 is 512 MB.
So I'm experimenting...

- netdrive_linux_amd64 create hd 2047 FAT16B test2g.dsk
- mounted it in MS-DOS 6.20
- filled it with RAR archives
- booted MS-DOS 3.31 from a floppy, and mounted the image
- rar t *.rar
- most of the files aren't even recognized as RAR archives!

Too big, after all?
Next try...

- netdrive_linux_amd64 create hd 450 FAT16B test450m.dsk
- mounted it in MS-DOS 6.20
- filled it with RAR archives
- booted MS-DOS 3.31 from a floppy, and mounted the image
- rar t *.rar
- same story: CRC failed, or not recognized at all!

What's the problem?
Is the FAT16B format different in 3.31 vs. 6.20 ?

Sorry - it's Compaq MS DOS 3.31 that supports FAT16B .. there might be another OEM version that also supports FAT16B, but I think almost everything else is still limited to FAT16 at that version.

Also, the limit for FAT16 is just 32MB assuming 512 byte sectors. (The number of sectors field is only 16 bits in FAT16, just like it is in FAT12.)

mbbrutman wrote on 2025-02-12, 22:25:

It didn't work for Grzyb so I removed it.

There is no debug log file available for this; a 5.5KB device driver doesn't do things like that. I'll have to recreate it here, but it's going to take a while because I generally don't use ODI and I'll have to get up to speed on it.

No worries. Really appreciate you looking into it. I'm really surprised no one else is storing ISO's on network drives. I must be the only person doing it 😉

I'm pretty sure you are not the first: https://bitbang.social/@1Bit/111560440401536780

There is somebody using SHSUCD, mounting an ISO and playing it. Basically on the same day I released the original NetDrive!

mbbrutman wrote on 2025-02-13, 03:46:

I'm pretty sure you are not the first: https://bitbang.social/@1Bit/111560440401536780

There is somebody using SHSUCD, mounting an ISO and playing it. Basically on the same day I released the original NetDrive!

Oops, I should have been more specific. First person wanting to play a game with a mounted ISO from a NETDRIVE + IPX multiplayer at the same time. The ISO's I've mounted so far via NETDRIVE have been working perfectly when just doing single player in the games.

And I'm confused again...

I was inclined to believe that SHSUCDHD+SHSUCDX can't work properly with an ISO image located on a remote drive...
becasue of this post - Re: DOS/SMB/Mounting images
and my subsequent experiments with 386DX-40 and Microsoft Network Client.

Now, I'm trying to reproduce the problem on a 486DX2-66 - but I can't!
Seems to work fine with both Microsoft Network Client and NetDrive.

However, the NetDrive+SHSUCDHD+SHSUCDX combo seems much slower.
I will post benchmarks later...

The test data:

1-rw-r--r--   1 user user 218368237 Feb 13 03:39 aaa.rar
2-rw-r--r--   1 user user 218368237 Feb 13 03:56 bbb.rar
3-rw-r--r--   1 user user 218368237 Feb 13 04:13 ccc.rar
4-rw-r--r--   1 user user 655466496 Feb 13 05:04 obraz.iso

The ISO image also contains the RAR archives.
It's all located in both the Samba directory, and in the NetDrive disk image.

Test procedure is:

rar t *.rar

First testing the archives outside the ISO, then loading SHSUCDHD+SHSUCDX, and testing the archives inside the ISO.
All the archives are zero-compression, so the test is easy for the CPU - just read it all, and check the CRC.
Using RAR 2.50, which displays the "Elapsed time".

Microsoft Network Client: 27 minutes
Microsoft Network Client + SHSUCDHD + SHSUCDX: 37 minutes
NetDrive: 27 minutes
NetDrive + SHSUCDHD + SHSUCDX: 261 minutes (yes, 4 hours 21 minutes!)

Interesting.

I looked inside of the SHSUCDHD code and it's just doing DOS level file seeks and reads to the file.

The difference has to be in caching or read-ahead behavior in the MS Network client. How much memory is the MS Network client using compared to the packet driver and NetDrive combination? Extra memory being is probably used for caching.

You can try to increase the number of buffers available to DOS and rerun the test; that might equalize things. To fully understand what is going on I'd have to see the pattern of sectors that RAR is making. If the test feature is causing a lot of jumping around and rereading of data, then something with a cache is going to be far better.

mbbrutman wrote on 2025-02-13, 23:31:

You can try to increase the number of buffers available to DOS and rerun the test; that might equalize things.

Default for BUFFERS is 15.
So I added BUFFERS=50

NetDrive: still 27 minutes
NetDrive + SHSUCDHD + SHSUCDX: 3 hours 32 minutes

Ok, so that helps prove that buffering is part of the issue.

You started with 7.5K of buffering and went to 25K. How much memory is the SMB client using? (I'm betting quite a bit of it is buffering.)

NetDrive has one inherent disadvantage compared to SMB - it's block based, not filesystem based. So it can only use 1KB of a 1.5KB Ethernet packet, potentially reducing it's throughput by 50% because only 2 512 byte blocks fit in a packet. But your initial numbers showed that was not an issue - NetDrive was reading just as fast as the MS Network client.

Adding the overhead of SHSUCDHD AND SHSUCDX adds a layer of indirection, but that's CPU cost. The rest of the difference has to be buffering. I'm not sure how ISO filesystems are laid out but if you have to keep seeking back to the file metadata to do a seek operation and that metadata is getting knocked out of cache, then every operation becomes far more expensive.

Do you have access to the "L" disk cache? (L stood for "Lightening". It was a small executable named l.com and it was pretty good. Probably far more effective than bumping BUFFERS.)