Is it DOS that gave the issue, or some app? Because the 2GB limit might come as the result of using signed integer for file pointers within the app.
The half of the theoretically supported size would be lost to the sign bit within the app's implementation.

vstrakh wrote on 2022-05-14, 09:49:

Is it DOS that gave the issue, or some app? Because the 2GB limit might come as the result of using signed integer for file pointers within the app.
The half of the theoretically supported size would be lost to the sign bit within the app's implementation.

I used MTCP's FTP tool to transfer the combined ISO to my DOS PC, FTP complains it can't complete the transfer because the disk might be full. As an alternative i connected the hard drive to my Windows PC and i managed to copy the file to the disk. Back at the DOS PC the dir command reports the file size correctly, but i'm not able to COPY, MOVE, DEL or do whatever with the file. SHSUCDHD also refuses to mount the ISO. So i'm leaning more towards that maybe the DOS kernel can't deal with files bigger than 2GB.

Wikipedia reports the following for FAT32:

1Max. file size	2,147,483,647 bytes (2 GB – 1) (without LFS)
24,294,967,295 bytes (4 GB – 1)[1] (with LFS)
3274,877,906,943 bytes (256 GB – 1) (only with FAT32+[34])

LFS stands for Large File Support, and it seems indeed that the issue is that DOS uses 32bit signed integers to represent file sizes and positions.

I've been confused by, and trying to solve this problem for years now. It seems the DOS 7.1 Kernel does not support 4GB-1 (long. unsigned files), but it will support both FAT32 partitions (max filesize 4GB -1) and also 64-kb cluster sizes on FAT16 partitions which means those partitions can also (theoretically!) contain also files of up to 4GB -1.

However DOS 7.1 does NOT support fileseeks or work with files in excess of (2GB-1) because of the awful longint *signed* filesize limitation. I've been trying to find a version of DOS -- *any* version which does allow *unsigned* longint fileseeks and native support for such files of up to (4GB -1)

What matters most is that the *kernel* support these unsigned longint files.

I do know, for example, that the Windows NTVDM DOS "emualtor" supports unsigned longint fileseeks and also sees files up to (4GB -1) size as perfectly normal and acceptable.

The DOS compile of, MPLAYER video player does NOT work with filesizes > 2 GiB in regular DOS 7.1, but it DOES work with files 4 GiB-1 in NTDVDM... Not a Windows compilation, but the *DOS* compile.

Does any version of DOS (FreeDOS, ROMDOS, Caldara DOS, etc.) of native, 16-bit DOS COMMAND.COM support 32bit unsigned integer filesizes? The question is very important, as it effectively *doubles* the max filesize which is useable in DOS.....

Why so little attention in 'fixing' this limitation and so much confusion has resulted over the decades is really surprising to me. It seems like it could be patched and / or fixed on the Kernel basis. Patching command.com and / or MSDOS.sys.

Even Mr. Rudolph R. Loew, the famous expert of DOS software, didn't address this issue. He wrote MANY excellent DOS programs and utilities but they all still have the 2GB filesize limit, due to working with an unpatched kernel, I believe.

BTW: What is FAT32+[34]? Probably it isn't supported in baremetal DOS, anyway. I know R. Loew also wrote utilities which extended FAT32 capacity, but he never addressed the awful signed 32-bit integer limit in Native DOS.

zyzzle wrote on 2022-05-15, 00:55:

I've been confused by, and trying to solve this problem for years now. It seems the DOS 7.1 Kernel does not support 4GB-1 (long. […]
Show full quote

I've been confused by, and trying to solve this problem for years now. It seems the DOS 7.1 Kernel does not support 4GB-1 (long. unsigned files), but it will support both FAT32 partitions (max filesize 4GB -1) and also 64-kb cluster sizes on FAT16 partitions which means those partitions can also (theoretically!) contain also files of up to 4GB -1.

However DOS 7.1 does NOT support fileseeks or work with files in excess of (2GB-1) because of the awful longint *signed* filesize limitation. I've been trying to find a version of DOS -- *any* version which does allow *unsigned* longint fileseeks and native support for such files of up to (4GB -1)

What matters most is that the *kernel* support these unsigned longint files.

I do know, for example, that the Windows NTVDM DOS "emualtor" supports unsigned longint fileseeks and also sees files up to (4GB -1) size as perfectly normal and acceptable.

The DOS compile of, MPLAYER video player does NOT work with filesizes > 2 GiB in regular DOS 7.1, but it DOES work with files 4 GiB-1 in NTDVDM... Not a Windows compilation, but the *DOS* compile.

Does any version of DOS (FreeDOS, ROMDOS, Caldara DOS, etc.) of native, 16-bit DOS COMMAND.COM support 32bit unsigned integer filesizes? The question is very important, as it effectively *doubles* the max filesize which is useable in DOS.....

Why so little attention in 'fixing' this limitation and so much confusion has resulted over the decades is really surprising to me. It seems like it could be patched and / or fixed on the Kernel basis. Patching command.com and / or MSDOS.sys.

Even Mr. Rudolph R. Loew, the famous expert of DOS software, didn't address this issue. He wrote MANY excellent DOS programs and utilities but they all still have the 2GB filesize limit, due to working with an unpatched kernel, I believe.

BTW: What is FAT32+[34]? Probably it isn't supported in baremetal DOS, anyway. I know R. Loew also wrote utilities which extended FAT32 capacity, but he never addressed the awful signed 32-bit integer limit in Native DOS.

It's not that simple to fix without breaking older software.
The way seeking is done for a file under DOS is typically using INT 21 function 42. This function takes a signed 32-bit offset as it has to support 3 different modes: seeking relative to the beginning of the file, relative to the current position and relative to the end of the file. So changing the offset to be unsigned (to support values larger than 2GB) simply wouldn't work; negative values are needed to seek backwards from the current position or the end of the file.

LFS presumably adds new functions to handle large offsets rather than change the behaviour of the existing ones and breaking older software.

zyzzle wrote on 2022-05-15, 00:55:

However DOS 7.1 does NOT support fileseeks or work with files in excess of (2GB-1) because of the awful longint *signed* filesize limitation. I've been trying to find a version of DOS -- *any* version which does allow *unsigned* longint fileseeks and native support for such files of up to (4GB -1)

Note that the Interrupt List for the seek function says

for FAT32 drives, the file must have been opened with AX=6C00h with the "extended size" flag in order to expand the file beyond 2GB

https://fd.lod.bz/rbil/interrup/dos_kernel/2142.html

And the 6C00h function is called "EXTENDED OPEN/CREATE"
https://fd.lod.bz/rbil/interrup/dos_kernel/216c00.html

BH = flags
bit 4 = (FAT32) extended size (allow 4GB files instead of 2GB)

So the support seems to be there. I may make some tests later.
But it has to be supported by the application, if it doesn't set this "I am ready for files bigger than 2GB" flag, it won't get files bigger than 2GB.

Open source tools could be fixed to use this flag if someone wills to take the effort..

Azarien wrote on 2022-05-15, 11:46:

Note that the Interrupt List for the seek function says […]
Show full quote

Note that the Interrupt List for the seek function says

for FAT32 drives, the file must have been opened with AX=6C00h with the "extended size" flag in order to expand the file beyond 2GB

https://fd.lod.bz/rbil/interrup/dos_kernel/2142.html

And the 6C00h function is called "EXTENDED OPEN/CREATE"
https://fd.lod.bz/rbil/interrup/dos_kernel/216c00.html

BH = flags
bit 4 = (FAT32) extended size (allow 4GB files instead of 2GB)

So the support seems to be there. I may make some tests later.
But it has to be supported by the application, if it doesn't set this "I am ready for files bigger than 2GB" flag, it won't get files bigger than 2GB.

Open source tools could be fixed to use this flag if someone wills to take the effort..

This is interesting, as the last DOS compile of 7zip archiver (v. 16.02), *does* work with and extract files of up to 4GiB -1 in native, unpatched, baremetal "DOS 7.1", and it appears to set the "extended size" flag. However, it also works with such files on FAT16 drives with 64kb cluster sizes. The files > 2 GiB are able to be compressed and extracted *only* with this DJGPP compilation of 7zip, which was compiled under gcc C++ 2.81. Could CWSDMPI.EXE DOS extended automatically also set the "extended size" flag? Such big files aren't seekable or viewable by any other archivers (pkzip, ARJ, RAR, etc), nor are they "viewable" by 16-bit DOS software such as Xtree Gold or Norton's utilities. These applications give phantom error messages (eg, "Disk Full","unknown error", "can't create file", etc) when trying to work with > 2GiB files, obviously because they can't seek in the file due to reasons posted above.

I'm using 4DOS version 7.50, despite my DOS 7.1 installation recognizing files bigger than 2GB i can't do anything with them, MOVE or COPY commands don't work for instance. I guess these commands need to be patched to support the INT 21 AX=6C00 function.

I wonder if Jason Hood, author of SHSUCDX and SHSUCDHD, could upgrade his tools. However, he does have the SHSUDVHD which is able to load multiple image files that have been firstly split into several 512MB images exactly because of the 2GB DOS file size limit. I wonder now if he is aware about the AX=6C00h function.

zyzzle wrote on 2022-05-16, 00:56:

This is interesting, as the last DOS compile of 7zip archiver (v. 16.02), *does* work with and extract files of up to 4GiB -1 in native, unpatched, baremetal "DOS 7.1", and it appears to set the "extended size" flag. However, it also works with such files on FAT16 drives with 64kb cluster sizes. The files > 2 GiB are able to be compressed and extracted *only* with this DJGPP compilation of 7zip, which was compiled under gcc C++ 2.81. Could CWSDMPI.EXE DOS extended automatically also set the "extended size" flag? Such big files aren't seekable or viewable by any other archivers (pkzip, ARJ, RAR, etc), nor are they "viewable" by 16-bit DOS software such as Xtree Gold or Norton's utilities. These applications give phantom error messages (eg, "Disk Full","unknown error", "can't create file", etc) when trying to work with > 2GiB files, obviously because they can't seek in the file due to reasons posted above.

I would say possibly, I remember DJGPP doing automatic things with LFN's, so would not surprise me if it supported extended file handle functionality.