Those ebooks are probably UTF-8.

DOS uses ASCII codepage 437: https://en.wikipedia.org/wiki/Code_page_437

Typically, text editors designed for programming can convert between character encoding schemes.

On *nix iconv is the tool for converting between character sets.

When reading old DOS README files with box drawing characters and such, you can read them in a modern way via iconv -f cp437 -t utf-8
For the reverse, you'd reverse the to/from.
There has to be a windows/dos equivalent or cross-compiled version out there.

Converting between character sets was also a specialty of Kermit which is all open source now.

jakethompson1 wrote on Today, 15:29:

On *nix iconv is the tool for converting between character sets. […]
Show full quote

On *nix iconv is the tool for converting between character sets.

When reading old DOS README files with box drawing characters and such, you can read them in a modern way via iconv -f cp437 -t utf-8
For the reverse, you'd reverse the to/from.
There has to be a windows/dos equivalent or cross-compiled version out there.

Converting between character sets was also a specialty of Kermit which is all open source now.

Iconv seemed like a great option for me until I started going down the rabbit hole. I’m using TinyCoreLinux which is probably the issue, iconv is not available by default and I had to install glibc apps and conversion tools.

But even then it spits out the error- illegal input sequence at position 0, so I thought maybe they weren’t legit UTF-8 format.

Apparently: iconv file name

Is supposed to tell you the encoding but same error.
Illegal input sequence at position 0. It’s a shame I can’t just force a conversation without knowing the original format.

Maybe I’ll download a more standard Live Distro and try this again later .

keenmaster486 wrote on Today, 15:21:

Those ebooks are probably UTF-8.

DOS uses ASCII codepage 437: https://en.wikipedia.org/wiki/Code_page_437

Typically, text editors designed for programming can convert between character encoding schemes.

Do you have an example of one? The issues I’m hitting with using most dos word processors is that some of these files are two large, and I’d rather not break them into small pieces.

Oh, I was just referring to modern editors such as Sublime or VSCode, or even xed. Not sure what utility you would use that runs in DOS.

I do have a little DOS tool I wrore years ago to do adjustments like this to text files.

1Convert Text File
2
3use:    CTF  infile outfile[or .] [options]
4
5opts:   -Nc         - Don't output char 'c'
6        c1=c2       - change char c1 to be c2
7        c1-c2=c3    -       ""    from c1 to c2 to be c3
8        c1-c2=c3+   -                    ""             (and increment)
9
10character values:
11    'x' `x` "x"     = character matching x
12    n..         = characger with hex value: nnn
13        %n=binary       @n=octal    .n=decimal      $n=hexidecimal (default)
14
15outfile = . means write to console
16
17eg: ctf file . '$'='?'                     		 all '$'s changed to '?'s
18    ctf file . 'a'-'z'='A'+                 	 lowercase converted to upper
19    ctf f1 f2 -N00                          	 remove 00 chars
20    ctf f1 f2 00-1F=0 7F-FF=0 0A=0A 0D=0D -n0    remove all non-printable ASCII
21                               ^-----^----- replace LF and CR removed by 00-1F

If it would help I could dig it out. clean it up a bit and make it available.

I've seen linux commands that will strip out single characters too, I was hoping for something that would basically force cp437 compliance. Delete all unsupported characters.

This 16-bit DOS command line tool will remove all none DOS characters from a text file (of any size).

usage:
txtclean input.txt output.txt

BinaryDemon wrote on Today, 17:10:

Iconv seemed like a great option for me until I started going down the rabbit hole. I’m using TinyCoreLinux which is probably th […]
Show full quote

jakethompson1 wrote on Today, 15:29:

On *nix iconv is the tool for converting between character sets. […]
Show full quote

On *nix iconv is the tool for converting between character sets.

When reading old DOS README files with box drawing characters and such, you can read them in a modern way via iconv -f cp437 -t utf-8
For the reverse, you'd reverse the to/from.
There has to be a windows/dos equivalent or cross-compiled version out there.

Converting between character sets was also a specialty of Kermit which is all open source now.

Iconv seemed like a great option for me until I started going down the rabbit hole. I’m using TinyCoreLinux which is probably the issue, iconv is not available by default and I had to install glibc apps and conversion tools.

But even then it spits out the error- illegal input sequence at position 0, so I thought maybe they weren’t legit UTF-8 format.

Apparently: iconv file name

Is supposed to tell you the encoding but same error.
Illegal input sequence at position 0. It’s a shame I can’t just force a conversation without knowing the original format.

Maybe I’ll download a more standard Live Distro and try this again later .

That's unfortunate that it doesn't work, as it (and Kermit, which maybe deserves a second look since it has a native DOS version) is smart enough to do things like substitute curly " and ' with ASCII equivalents rather than just stripping them out.

cookertron wrote on 33 minutes ago:

This 16-bit DOS command line tool will remove all none DOS characters from a text file (of any size). […]
Show full quote

This 16-bit DOS command line tool will remove all none DOS characters from a text file (of any size).

The attachment txtclean.zip is no longer available

usage:
txtclean input.txt output.txt

This works perfectly , thanks! The first document I tested it on, it did 2114 corrections and the resulting document is much easier to read.