ATAPI is more demanding for a microcontroller than SCSI ? Because there are people that made a SCSI emulator just with a stm32 https://github.com/erichelgeson/BlueSCSI
https://m.youtube.com/watch?v=81tLrZeMP8U
(Which I think can also emulate a SCSI CD-ROM drive)

Just a reminder that SCSI was introduced in '86 and predates ATA. Also, what SCSI level, surely not the Ultra modes?
If you keep to SCSI-1, or subset of SCSI-2, then I'd argue it is easier. Reason being each transaction is always ack'ed so if you can't keep up it will simply slow down the transfer, but not break it. ATA modes, on the other hand, are timed on both ends and if you do not present data in that window, you just completly glitched it.

So, in other words, PIO 0 mode might seem slower than SCSI-1 on paper but you have to be able to present the data in 600ns (less than that actually to account for various line delay issues). Miss it just once on any occasion and that's it. In SCSI you just take more time to ACK and this causes a single clock strech, you can easily make it up with the rest of the stream and end up with higher transfer rate overall. Downside of SCSI is more signals and addressing, and several other bus modes but these are optional so the software side is more complicated but not that much.

SScorpio wrote on 2021-10-13, 22:28:

I have to wonder if a simple board that has a 40pin connector for IDE, molex power connector, analog and digital internal audio headers, and 1/8 jack that ends up being a Pi hat or allow connecting to an ESP32 module would be enough? Both a Pi and ESP32 can run with 5V so the earlier comment about some chips being 3.3V nowadays goes away. It could even have headers to support a cheap small screen and buttons you attach to a 5 1/4 bay insert.

At the very least you need resistors to safely drop voltage, like here https://github.com/Manawyrm/ATAPIHat-SMI/blob … TAPIHat-SMI.pdf (this is controller emulation, not device)
For device emulation you also need to adhere to very strict timings hence additional cpld/fpga requirements like here https://www.retrotronics.org/home-page/netpi-ide/
and its SPI-IDE bridge implementation Verilog code https://www.retrotronics.org/svn/jride/trunk/ … rc/bridge_top.v

Building an IDE interface that allows connecting IDE/ATAPI devices (master) is easier because being the BUS controller you control timings and can take your sweet time (like ATAPIHat-SMI).

The problem with building IDE DEVICE (not controller) is it has strict timing requirements because deep down it started its life as an ISA BUS device. First IDE drives just integrated ISA ST506 controller electronics, and IDE connection was a simplified ISA bus. You cant just reply when you feel like it, you need to reliably react to bus requests even when you are busy (by signaling that you are not ready or busy).
We can do this without CPLD/FPGA with bitbanging. This is how Tattiebogle does it, 180MHz Cortex-M3 12 cycles interrupt latency is quiet comfy at meeting timing requirements. ESP32 also should be able to do it. http://www.hardwarebook.info/ISA#Shortening_o … g_the_bus_cycle
As a bonus same code can be reused to emulate any IDE device (HDD/ZIP/Streamer) and even ISA cards (vga, sound, maybe even crazy video decoders).

One of the nice things about ESP32 is its speed - its fast and capable enough to not only play CD audio off of raw CD image, but it could also decode audio from mp3/ogg files on the fly = much smaller disk images.

Deunan wrote on 2021-10-13, 23:08:

I really wish it was as easy as slapping a dumb hat with few connectors on top of a Pi and calling it a day.

you mean like this https://cdn.hackaday.io/files/20774888794816/ … C_schematic.pdf? not quite dumb, but not rocket science either. Lets not make a big deal out of a couple 245 buffers.

Deunan wrote on 2021-10-13, 23:08:

I mean no offense but this is another one of those "Why don't we just emulate the laser head?" type of things.

Isnt that exactly what xstation is doing? injecting data at the point of laser output, similar to how modchips injected data (ATIP wobble) directly into tracking output pins?

Deunan wrote on 2021-10-14, 08:59:

AFAIK the ESP32 is sort of virtual machine, that is the user code is running under RTOS kernel layer that has priority, so interrupt latency sucks. It might be so bad that even reliable PIO 0 is not possible.

I already linked to experiment showing 200ns latency being doable by bypassing interrupts

Deunan wrote on 2021-10-14, 08:59:

What would be perfect is a chip that, like modern FPGAs, can have it's I/O banks work at different voltages. And at least some would have to be true 5V I/O, that's where the problem lies.

or you know, a dollar worth of $0.3 SN74LVC8T245 buffers

SpocksBeer wrote on 2021-10-14, 00:10:

Meanwhile, in Soviet Russia...
https://www.youtube.com/watch?v=fFdFw1K25Js&a … channel=sinowin

nice find! STM32 + cpld and dude already implemented a ton of IDE devices. Clearly a passion project, first emulated CDROM (Mitsumi FX400E) on the list is the one Author had in his computer >20 years ago 😀

rasz_pl wrote on 2021-10-14, 10:34:

you mean like this https://cdn.hackaday.io/files/20774888794816/ … C_schematic.pdf? not quite dumb, but not rocket science either. Lets not make a big deal out of a couple 245 buffers.

Take another look. I think you missed an FGPA - with all the logic in it. So no, not just a couple of '245 buffers.
Also, what can this do, exactly? Somehow I doubt it properly emulates ATA commands to switch the PIO modes, rather it has one or two baked-in timings and hopes the host will not attempt anything faster. Will this work? Sure. With every retro PC? With 60cm long cable and another ATA device already connected to it? I have my doubts.

rasz_pl wrote on 2021-10-14, 10:34:

Isnt that exactly what xstation is doing? injecting data at the point of laser output, similar to how modchips injected data (ATIP wobble) directly into tracking output pins?

Frankly I do not know exactly how these work, but chances are it's a layer above the direct laser stream. I mean, the console itself would need a fast ADC and logic to process such data to tell if it's legit or not, why do that if you can have an ASIC return it in already digitized and somewhat processed way, like what Saturn does. But even if that was a laser stream - it's for a console, and not even every mobo is supported. You can take some shorcuts if you know the system well, the emulation doesn't need to be perfect, or even correct as such. It only needs to fool the mobo, by whatever means it takes, to accept the "CD-ROM" as valid. Maybe they found a simple glitch that somehow works.

rasz_pl wrote on 2021-10-14, 10:34:

I already linked to experiment showing 200ns latency being doable by bypassing interrupts
(...)
or you know, a dollar worth of $0.3 SN74LVC8T245 buffers

You clearly know better but I have to ask, if it's so simple why isn't there a ton of ODEs for PC based on ESP32 or RPi already on the market?

Deksor wrote on 2021-10-14, 09:41:

ATAPI is more demanding for a microcontroller than SCSI ? Because there are people that made a SCSI emulator just with a stm32 https://github.com/erichelgeson/BlueSCSI
https://m.youtube.com/watch?v=81tLrZeMP8U
(Which I think can also emulate a SCSI CD-ROM drive)

*No Synchronous transfers.
*No Ultra modes.
Read, slower than PIO4.

Any attempt to go for these seems to need extra chips or an FPGA. I've seen for example, SCSI2SD v6 prototypes had an Lattice FPGA. Production models have what it looks like an ASIC (that Silicon Blue chip) coupled with an STM32 CPU, thus is magnitude more expensive than simpler asynchronous/no ultra speed v5's.

rasz_pl wrote on 2021-10-14, 10:34:

nice find! STM32 + cpld and dude already implemented a ton of IDE devices. Clearly a passion project, first emulated CDROM (Mitsumi FX400E) on the list is the one Author had in his computer >20 years ago 😀

So he uses the 2 chip CPLD+CPU approach... Using an STM32 is natural, as you don't need computation horsepower for this task, as I've been saying... Well, at least if you don't emulate DPM copy protections with MDF image dumps. And I suppose there should be beefier STM32s for the last one. Still, remains to know how "cheap" his approach is. Dunno is cheaper than SCSI2SD v6 which also uses an STM32 + CPLD/ASIC which does the physical layer hardwork.. But who knows...

Deunan wrote on 2021-10-14, 11:12:

Take another look. I think you missed an FGPA

I was replying to weird insinuations about crossing 3.3-5V barrier somehow being an insurmountable challenge

Deunan wrote on 2021-10-14, 11:12:

- with all the logic in it.

As already stated this can be bitbanged. Afaik no programmable logic in LPC18xx https://shop.tattiebogle.net/images/idesim_normal.png

Deunan wrote on 2021-10-14, 11:12:

Also, what can this do, exactly? Somehow I doubt it properly emulates ATA commands to switch the PIO modes

afaik it doesnt work like that. you dont switch device speeds, you read device capabilities returned by Identify command "Word 51: PIO Data Transfer Cycle Timing Mode." and set your controlled to an appropriate PIO mode

Deunan wrote on 2021-10-14, 11:12:

rather it has one or two baked-in timings and hopes the host will not attempt anything faster

see above, host has to query speed and set itself to comply

Deunan wrote on 2021-10-14, 11:12:

Will this work? Sure.

so where is the problem then? 😀 why are you arguing so vehemently about something being impossible/uber difficult when it was already done twice

Deunan wrote on 2021-10-14, 11:12:

rasz_pl wrote on 2021-10-14, 10:34:

Isnt that exactly what xstation is doing? injecting data at the point of laser output, similar to how modchips injected data (ATIP wobble) directly into tracking output pins?

Frankly I do not know exactly how these work

wait, didnt you make like 3 of those emulators already ? 😀 That makes it even more impressive!

Deunan wrote on 2021-10-14, 11:12:

, but chances are it's a layer above the direct laser stream. I mean, the console itself would need a fast ADC and logic to process such data to tell if it's legit or not

???
There is no ADC, data on the disk is already digital. CD ASICs work in a similar manner to floppy controllers - there is some kind of RF amp, then PLL and comparator - all of this is used to recover original clock and clean up the data flow. Then you have EFM decoder, descrambling, reformatting, CRC error checks etc.

Deunan wrote on 2021-10-14, 11:12:

why do that if you can have an ASIC return it in already digitized and somewhat processed way, like what Saturn does.

Saturn. Saturn is a great example of design by a committee (or a few). Its a hodge podge of everything, so many chips, busses, processors. I think I tried counting all of the separate busses at some point and ended up at 144bits or something crazy like that 😮. No DFM, no integration. In Saturn it looks like HD49232 handles this preliminary decoding +4 other chips to arrive at actual data. Looking at schematics you inject data into YGR019A 124-131 pins, + obviously monitor laser head position.
In early PSX you have small RF amplifier + big Sony ASIC consuming RFAC and doing everything from clock recovery, thru decoding cdrom data to sound DSP.

Deunan wrote on 2021-10-14, 11:12:

But even if that was a laser stream - it's for a console, and not even every mobo is supported.

You are the best person to answer why. Isnt it just a matter of pinouts and connectors (20/21) in your case?

Deunan wrote on 2021-10-14, 11:12:

You can take some shorcuts if you know the system well, the emulation doesn't need to be perfect, or even correct as such. It only needs to fool the mobo, by whatever means it takes, to accept the "CD-ROM" as valid. Maybe they found a simple glitch that somehow works.

There is no glitch required. You need to scramble and EFM encode CD data before injecting to emulate stream from the laser diode, same procedure as in CD Writing.
https://en.wikipedia.org/wiki/Eight-to-fourteen_modulation
https://www.ecma-international.org/wp-content … n_june_1996.pdf
https://github.com/simoninns/EFM-Decoder
https://debugmo.de/2007/07/read-your-dvds-the-raw-way/
"Dissecting the CD Player: How to Turn Shiny Plastic into Music" by Technology Connections https://www.youtube.com/watch?v=3yJqlD9RxD4 https://www.youtube.com/watch?v=7olNiMCz9to

Deunan wrote on 2021-10-14, 11:12:

You clearly know better but I have to ask, if it's so simple why isn't there a ton of ODEs for PC based on ESP32 or RPi already on the market?

cheeky, probably same reason ordering gdemu is a smooth and easy experience 😀

rasz_pl wrote on 2021-10-14, 17:02:

cheeky, probably same reason ordering gdemu is a smooth and easy experience 😀

Depends who you ask I guess. I'd argue I have a few happy customers, I take you aren't one. FYI you can hack together an IDE interface with a decent AVR, you don't need ESP32 cores for that. It's not fast and doesn't really handle any kind of errors but it works - most of the time. Is it good enough for an actual, stable ODE? Nope.

If I sound overly pessimistic it's because I don't want anyone to underestimate such project, including the non-technical side of things. Point is though, I now see it's personal. I did mention this subject can get toxic and I know where this argument is heading. Fine, I'll just leave it in your capable hands, I'm done here.

Wow, this thread has changed a bit since I last looked.

Anyway yes, to address some comments I just read over that have appeared:

1) Screens and buttons: I sadly didn't add any extra I/O. I definitely won't make this mistake again - also some LEDs 😀 I thought at first "USB is fine, dual USB means it can do literally anything" but of course that means it can only speak to actual USB devices. The Crystalfontz solution was basically a firmware-only solution to at least give people something fairly "out the box" (since they make the drive bay mount and their screen has buttons and USB): it produces something entirely workable by only having me add (lots of) code. There's a few cheap $5 USB dev boards you can buy from places like eBay that could be connected to one of those OLED screens and some buttons and I had the notion of offering that just as some source code/binary, BoM and instructions on my wiki, but I haven't had time. Maybe I can take a look at this again soon, though time is scarce just now (soon to be a new dad, new job, other extracurricular activities).

2) Regarding PC support in particular, old school IDE/ATA is a super pain on PCs - they're where it came from so it turns out to be way less "correct" than the ATA/ATAPI specification documents you can find online would have you believe. Some Super I/O chip datasheets even happily admit to timing specs completely incompatible with the claimed ATA ones, so it's basically just an exercise in reverse engineering, but there's a million variations out there from a million old PCs. Either way at this point I have it working on all the 386s and 486s in my possession, with various Super I/Os (both ISA and VLB). Some users still report some sort of edge-case timing issue on some hardware I can't replicate based on reports on my issue tracker, so I'm still just ordering weird looking Super I/Os from eBay when I spot them and testing them out. Support for non PCs (and PCs with a Southbridge, where the IDE port is just pure I/O) has been a lot simpler, even weird Roland synths were pretty well behaved as their IDE ports are not really just ISA.

3) I did also have the notion of attempting to do a cost reduction step, possibly by switching to a combi FPGA/MCU chip, and larger (maybe almost 5.25") to get single sided for cheaper assembly instead of having the FPGA on the back as well as maybe also squeeze SPDIF output some people requested on, but the chip shortage has postponed me from getting prototypes (all the good chips are just "52 week lead time" just now, at least at US companies). The original version was small as it was meant to plug straight into the IDE socket on a Konami System 573, after all.

electric_monk wrote on 2021-10-17, 05:48:

1) Screens and buttons: I sadly didn't add any extra I/O. I definitely won't make this mistake again - also some LEDs 😀 I thought at first "USB is fine, dual USB means it can do literally anything" but of course that means it can only speak to actual USB devices. The Crystalfontz solution was basically a firmware-only solution to at least give people something fairly "out the box" (since they make the drive bay mount and their screen has buttons and USB): it produces something entirely workable by only having me add (lots of) code. There's a few cheap $5 USB dev boards you can buy from places like eBay that could be connected to one of those OLED screens and some buttons and I had the notion of offering that just as some source code/binary, BoM and instructions on my wiki, but I haven't had time. Maybe I can take a look at this again soon, though time is scarce just now (soon to be a new dad, new job, other extracurricular activities).

Enjoy the chaos ! 😀 & congratulations.

electric_monk wrote on 2021-10-17, 05:48:

2) Regarding PC support in particular, old school IDE/ATA is a super pain on PCs - they're where it came from so it turns out to be way less "correct" than the ATA/ATAPI specification documents you can find online would have you believe. Some Super I/O chip datasheets even happily admit to timing specs completely incompatible with the claimed ATA ones, so it's basically just an exercise in reverse engineering, but there's a million variations out there from a million old PCs. Either way at this point I have it working on all the 386s and 486s in my possession, with various Super I/Os (both ISA and VLB). Some users still report some sort of edge-case timing issue on some hardware I can't replicate based on reports on my issue tracker, so I'm still just ordering weird looking Super I/Os from eBay when I spot them and testing them out. Support for non PCs (and PCs with a Southbridge, where the IDE port is just pure I/O) has been a lot simpler, even weird Roland synths were pretty well behaved as their IDE ports are not really just ISA.

Is there a list of all the chipsets/implementations that you've tested ?

electric_monk wrote on 2021-10-17, 05:48:

3) I did also have the notion of attempting to do a cost reduction step, possibly by switching to a combi FPGA/MCU chip, and larger (maybe almost 5.25") to get single sided for cheaper assembly instead of having the FPGA on the back as well as maybe also squeeze SPDIF output some people requested on,

If you're willing to go bigger will you also consider a laptop cdrom version? allthough that would be pretty annoying given all the different connectors manufacturers used to connect the cd/dvdrom drives to the laptops. 😉

electric_monk wrote on 2021-10-17, 05:48:

but the chip shortage has postponed me from getting prototypes (all the good chips are just "52 week lead time" just now, at least at US companies). The original version was small as it was meant to plug straight into the IDE socket on a Konami System 573, after all.

Does that mean that this time is not a good moment to order one?

Great work & thank you for your service.

weedeewee wrote on 2021-10-17, 08:10:

Enjoy the chaos ! 😀 & congratulations.

Thanks 😉 I am fully expecting chaos 😛

weedeewee wrote on 2021-10-17, 08:10:

Is there a list of all the chipsets/implementations that you've tested ?

Interesting idea - I'll see what I can do. I suspect (as the IDE cards just connect some signals from the ISA bus fairly directly onto the IDE bus) that'd be best as motherboard + card combinations, I'll just need to figure out what the motherboards are (they don't seem to be well labelled, especially the little low profile 386s)

weedeewee wrote on 2021-10-17, 08:10:

electric_monk wrote on 2021-10-17, 05:48:

3) I did also have the notion of attempting to do a cost reduction step, possibly by switching to a combi FPGA/MCU chip, and larger (maybe almost 5.25") to get single sided for cheaper assembly instead of having the FPGA on the back as well as maybe also squeeze SPDIF output some people requested on,

If you're willing to go bigger will you also consider a laptop cdrom version? allthough that would be pretty annoying given all the different connectors manufacturers used to connect the cd/dvdrom drives to the laptops. 😉

Haha, it's an interesting idea, though like you say, it may be a bit heavy going if there's too many connectors 😉 Probably worth a Google anyway, to see how nightmarish it might be.

weedeewee wrote on 2021-10-17, 08:10:

electric_monk wrote on 2021-10-17, 05:48:

but the chip shortage has postponed me from getting prototypes (all the good chips are just "52 week lead time" just now, at least at US companies). The original version was small as it was meant to plug straight into the IDE socket on a Konami System 573, after all.

Does that mean that this time is not a good moment to order one?

Great work & thank you for your service.

I wouldn't let it stop anyone thinking about it from picking one up now - the chip shortage means I've put it on the back burner, though I do periodically check for parts to at least try a prototype (I note even the parts on the current version are out of stock, so I might not be able to do another batch of the current design until things calm down). It'll be a bit of an R&D exercise anyway, like if the parts I substitute end up just requiring other support parts the savings might be cancelled out. It may end up just being a similarly priced board with the extra I/O people requested (e.g. SPDIF and buttons/I2C for OLED), I wouldn't want to make any promises 😉

Until the global market sorts itself out (I'm hoping next year?) I was mostly just planning to work on the software 😊

Hello guys!
I'm an author of ATA/ATAPI2SD emulator. I saw some traffic from vogons on my YT videos so I searched and came here.
Actually this thread inspired me to begin my project in the first place. I also was long curious about ATA (ATAPI) protocol and wanted to know how it really works, so I decided to make my own emulator.
But instead of making heavily specialized device I wanted to make a versatile gadget to cover most (if not all) possible scenarios.
This is the list of initial features that I wanted to implement:

• Emulation of ATA HDD of any size with CHS or LBA translation
• Emulation of ATAPI CD-ROM. Support for analog output of AUDIO tracks. SPDIF-out as an option.
• Emulation of other ATAPI devices like Iomega ZIP, MO, etc.
• Support for transfer protocols: PIO, MW-DMA, UDMA.
• Support for host mode where ATA/ATAPI drive is connected to emulator for device id dumping, sector viewing, making of drive image.
• Storage for images: SD-card. No requirement to convert images to some special format.
• USB interface for configuration and debug output via emulated serial port.
• Ability to connect generic OLED I2C screen for status display (as on Gotek)
• Minimum three physical buttons to select image or change settings.
• Ability to work with very old computers as well as with modern computers or devices.
• Compact size to fit standard 3.5" bay.

As of today I have done most of this among a lots of other things.

The most important new feature is a support for two emulated devices on the same cable which could be selected from any combination of ATA/ATAPI drives.
My approach for drive emulation is based on real device id info extracted from real drives, so it is not some generic "ATAPI emulator", but a real clone of any ATA HDD or ATAPI CD-ROM at least from the point of ids and capabilities saved in device id dump. Thereby emulated drive and image are a two different entities. When HDD image is loaded to some drive, device id is updated to match image size and transfer mode limitations.
Emulator has confirmed support for Iomega ZIP-100 and ZIP250 drives.
Another interesting feature is an ability to override device id data using per-drive config files. In other words, you can change any option you want on per-drive basis, for example change CHS parameters or serial number to match some specific piece of hardware.
On current board I use PCM5102 I2S DAC module for analog audio output. My MCU supports direct SPDIF output but it is not implemented yet.

Picture of rev1 proto board:

sintech wrote on 2021-10-26, 21:07:

Hello guys! I'm an author of ATA/ATAPI2SD emulator. I saw some traffic from vogons on my YT videos so I searched and came here. […]
Show full quote

Hello guys!
I'm an author of ATA/ATAPI2SD emulator. I saw some traffic from vogons on my YT videos so I searched and came here.
Actually this thread inspired me to begin my project in the first place. I also was long curious about ATA (ATAPI) protocol and wanted to know how it really works, so I decided to make my own emulator.
But instead of making heavily specialized device I wanted to make a versatile gadget to cover most (if not all) possible scenarios.

Welcome, and please keep us posted on any progress.

Your project looked exciting from what was shown in the video and what Google Translate was able to not mangle. But seeing all that spelled out is very impressive.

Hurrah! More options! 😀

I really like that it's designed to be an everything solution. Will there be a way to add, say, CD images to the SD card while the unit is running either via USB or even WLAN? Could be a little inconvenient if you're using the device as your primary HD and you have to shut everything down to pull the card and add new files.

Ok, that was unexpected.. And cool.

Great work on that ATA/ATAPI2SD emulator. That target feature list looks really exciting and it even includes SPDIF *thumbs up* 😀 I'm really curious about it's further development.

Would it be possible to use it as an IDE-XT (also known as: XT-IDE, XTA) hard drive? Those are becoming very difficult to obtain, and some systems like the PS/1 will only accept drives matching the exact parameters of the original drive.

It looks very cool - I don't see CD audio among the options however. I assume that will be supported as well?

indeed this is amazing!

red_avatar wrote on 2021-11-05, 20:10:

It looks very cool - I don't see CD audio among the options however. I assume that will be supported as well?

you overlooked it...

Emulation of ATAPI CD-ROM. Support for analog output of AUDIO tracks. SPDIF-out as an option.

sintech wrote on 2021-10-26, 21:07:

I saw some traffic from vogons on my YT videos so I searched and came here.

Huzzah, my plan worked!

Your product sounds amazing. People are still going to quibble over the price, but I fully expect it, when announced, to fall in line with the other estimates in this thread (unless proven wrong!)