I guess knowing ASM is a first step. I, too, want to pimp my 8088 with stuff on an EEPROM, and might wanna learn a bit about that.

Stojke wrote:

How does one acquire knowledge on modifying programs based on hex disassembly and viewers?
How do you identify certain portions of a program?

It's mostly a matter of experience: if you've written certain code before, you know more or less what it will look like, and what to look for.
It often comes down to finding 'strategic' calls to certain APIs, which you can do either by setting breakpoints in a debugger while you run the code, or by using a disassembler and looking for these functions in the deadlisting.

Stojke wrote:

Where to start to learn about advanced stuff like that?

Start learning to code, both assembly language and C/C++.
Assembly language so you know exactly what things look like at the instruction level, and C/C++ so you know what it looks like at the level that most programmers code at (the overall structure and patterns of programs).
If you can write it, you can reverse-engineer it.

I am interested. Can you suggest some resources for this? I got inspired for reverse engineering because we have one HP laptop here in the shop that has an locked bios. The bios unlock application wont accept the required parameters to run properly and I tried to insert them manually via an hex editor thinking that they would be clearly visible. Sadly the entire program is just random rubbish to my eyes 😀

I always liked how certain applications like HDAT2 and MHDD look and feel like. Especially if an dos app has mouse support. I always thought that DOS apps could use dual screen support to offload the main window and make work more efficient.

Be prepared to spend years to learn programming.

Indeed. "21 days" is a myth.

Write simple programs in C, compile & link them with a relevant compiler (e.g., if you intend to reverse DOS games, use Borland or Watcom), and execute your program in a debugger and disassemble it with IDA. Study those. You'll learn what kind of machine code compilers generate and learn to recognize the basic stuff like saving and restoring the stack.

I don't think you need to actually learn how to write assembly code, just read it.

You don't just read the hex directly unless you are reversing custom formats. Instead you convert those Hex bytes into their assembly language, and then start modifying the hex data to adjust the assembly code flow.

For a current reference when I traced the code flow of American McGee's Alice to fix the A3D2 detection bug, I used IDA and set breakpoints near where the soundcard detection routines would be. Then I stepped through the code until it was making a decision to enable A3D2 or not, by changing the code branch to the opposite logic you can force it to take the enable path after it fails to detect the Vortex2 cards. You don't just jump into Alice.exe with a hex editor and start making changes.

So to learn to read hex, there isn't anything to it. 0-9,A-F base 16 counting. Each byte represents some form of data or code, disassemblers attempt to determine what is what and produce proper readable/functional ASM code. Read the ASM code by understanding the cpu's instruction set of which there should be online documentation widely available. Once you understand the ASM you can translate to a higher language like C for reverse engineering purposes.

Or you can learn C and work your way down to understanding how your C code was compiled to ASM, and then look at the HEX produced in the binary output file.
If you want to learn ASM, I started with the MOS6502 CPU as it has a very simple instruction set, and is used in the Atari2600 and many other devices/consoles.

Well, there's a lot of separate things you must know a bit about to get started.

For high level languages like C or Pascal, and when doing some higher level stuff like reading/writing files or doing standard input/output with keyboard/screen, I think they are a bit like spoken languages in the sense than you must know the syntax how to write senteces so the compiler won't give errors, and then you must know what words to use to say something (like how to open, read and close a file). For instance in this case there is little difference if you read files in C language on a 386 CPU running DOS or on an ARM CPU running Linux.

If you want to write low level stuff that directly accesses the hardware, then you need to know the hardware platform regardless of the language used. Usually a nice practical example is how to use video bios to change into VGA 320x200 graphics mode, plot a few pixels on screen, wait for user keypress, and change back to VGA text mode and exit program without hanging the PC, not many more than 25 lines even in assembly language.

But assembly language also has a syntax to learn, and in this case you must have a "programmer's model" of the CPU, meaning what are the CPU register names and how you can use them, and what kind of instructions there are available and on what CPU registers they can be used. Higher level languages get compiled into assembly, so even a complex subroutine made in C or Pascal is just a series of small, simple assembly opcodes, so if you reverse-engineer something, it will take time to "see the matrix" what is the big picture the software does. Even if you understand each and every assembly opcode what it does, it is more important to understand why it does the things it does to achieve something.

Well, I still find HelpPC tech reference and PCGPE (PC Game Programmer's Encyclopedia) useful. You can get an old Borland C compiler from Borland Museum for your own use, also some C++ compiler available, and very old Turbo Pascal version which does not support assembler syntax. Also many DOS programming sites on the net, and they even look like old BBS 😀

Jepael wrote:

But assembly language also has a syntax to learn, and in this case you must have a "programmer's model" of the CPU, meaning what are the CPU register names and how you can use them, and what kind of instructions there are available and on what CPU registers they can be used. Higher level languages get compiled into assembly, so even a complex subroutine made in C or Pascal is just a series of small, simple assembly opcodes, so if you reverse-engineer something, it will take time to "see the matrix" what is the big picture the software does. Even if you understand each and every assembly opcode what it does, it is more important to understand why it does the things it does to achieve something.

Yes, that is why I think you need to be able to write assembly code (to a certain extent) in order to be able to read it.
Aside from also being fluent in structural programming in a language such as C or Pascal.
It will help you to identify patterns in code, and know what to look for, and what to ignore.

I recall reverse-engineering a memory dump of a Canon PowerShot-camera with a friend. This is a camera with a 186-compatible CPU running ROMDOS. When looking through the disassembly, at some point I said: "Look, we must be in the libc here, this is strlen()". My friend was amazed at how I saw that in the blink of an eye. But I guess it's just because I've written such code in asm many times before, so the bit of code with a rep scasb stood out to me.

I learned C/C++ in around 3 months. I consider my success to be only moderate because 3 months is really not enough to learn all the advanced stuff. I'm proud of what i achieved in 3 months, but i feel like it was just a tear drop in an ocean. One of the things i regret was never learning how to design a GUI, and running all my programs in command line.

I would love to learn more low level programming, having a more deep knowledge of the PC architecture so that i could do stuff to run in DOS. That's why i would love to learn x86 assembly, it is impressive the stuff you can do when you are proficient at it.

I realized though that i would need some sort of mentor to help me out and to guide me.

If you are interested in low-level hardware access from assembly in a retro context, the IBM PC & XT BIOS' are a great resource. The code listing is heavily commented.
https://sites.google.com/site/pcdosretro/ibmpcbios
Look at the commented assembly and the raw hex side-by-side to get a feel for how it works.

Regarding languages, definitely x86 assembly and C. C maps very closely to asm.
Starting in the retro domain is not a bad idea, as 16-bit/DOS is much less complicated than more modern technologies.
Personally I like MASM and Microsoft C 6.0. They have some decent online help for their vintage.
If you run the CodeView debugger in source+disassembly mode, it will show C, asm and hex opcodes together.
The C Programming Language - K&R is a good reference for C.

Just be aware you are learning niche skills and obsolete technologies here. Most web developers etc know nothing of this.
It is a great foundation for learning higher-level/more modern though. These kinds of skills are dying out in the workplace,
apart from the likes of embedded/OS dev/security research. Even in my area (game dev) there are fewer opportunites to hit the metal than 10 years ago. I fully appreciate this probably leads to better products though.

leileilol wrote:

Indeed. "21 days" is a myth.

Agreed

I understand that I am required to posses basic programing knowledge. I know how to code in C# and I am familiar with C syntax as I've coded some simple applications in it before.
I know that some members here did modifications and also code low level programs. Such programing has always been my interest. I am edger to learn and understand such processes.

As with everything in life I understand that it takes time and practice to understand, that's why I am asking for some guide lines and where to start.