So... What if you used the internals of a real CD-ROM drive and just replaced the laser unit with an emulator and then also had a way to make the drive think that a CD had been ejected and a new one put in. That seems like it would be a simple way to make it work and not be too expensive as a lot of the hardware would already be there and you wouldn't have to worry about driver support either.

cyclone3d wrote on 2021-09-18, 04:38:

So... What if you used the internals of a real CD-ROM drive and just replaced the laser unit with an emulator and then also had a way to make the drive think that a CD had been ejected and a new one put in. That seems like it would be a simple way to make it work and not be too expensive as a lot of the hardware would already be there and you wouldn't have to worry about driver support either.

Short version: if I understand you correctly, few to none optical pick-up units (OPUs) are understood/reverse-engineered to the point where they can be "replaced with an emulator" (most if not all are undocumented)
See: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6066758/ for work on three very specific OPUs.

And even if they were, no dumps of discs at that level exist. If such image files existed, they'd probably be pretty huge anyways, much larger than a typical disc image. (I will ask a friend for a few more specifics.)

[EDIT] Because the transitions of pits and lands are actually analog, as the laser slowly moves into and out of pits, due to how the non return to zero PLL and the zero cross detection work, if stored at 8-bit 10 MSPS (mega samples per second), a typical audio CD would be encoded at approximately 34GB/hour.

It would make more sense to store the EFM after decoding, which would be about 2-3 times the size of the data, but the EFM is decoded after the OPU, usually by the Servo DSP. EFM can contain the errors from copy protection and extra length runs that break the EFM format used in copy protection too. (thanks to smally_dd86 for the details here).

[EDIT] Here's a block diagram for the Panasonic CD-R/RW Drive CW-7586-B, an 8x CD-R 4x CD-RW 32x CD drive, picked at random. This is typically around the depth of documentation you can get on a typical optical drive, as you can see, it doesn't explain near what you need to do to replace part of it with an emulator. You don't stand a chance of getting thorough detailed info on the servo DSP without signing an NDA, and you would have had to have done so when the drive was new. Anyhow, it also kinda shows you what's going on inside. And as drives get newer, more and more of these functions get condensed into fewer and fewer chips. So very quickly, whether by intent/design or by evolution, obtaining access to the "raw data" becomes rather impossible, and you always have to deal with how the drive's firmware, plus the drivers, plus the OS, decodes and interprets the data. So... emulating at the level of ISO/BIN+CUE/NRG/etc. is by far the easier option.

[EDIT] http://bugworkshop.blogspot.com/2015/05/mitsu … b-cd-rw_31.html for a teardown of the CW-7586-B, which lists a BOM for the controller board. Good luck finding datasheets! 😀

Reverse engineering is pretty much the only direction this can go in, and there's too many drives, all similar but not identical at the functional level, let alone a lower level. You'd have to pick a very specific drive to emulate, and even then, "replacing the laser with an emulator" would tie your production to a limited supply of obsolete drive controller boards.

Someone is making a rpi based scuzzy adaptor for macs surely it could emulate atapi at 4x speeds.

Some folks here ask why there are good ODE for consoles and none for PC.
What I answer is: there is something when someone think there is a market for it. It improves over time when there is indeed a market.
Thread like this one, with people talking of it can make things change one day because we show there is a potential market.
I previously described here what I think what must be an ODE for a PC that could sell. The final price of course is the key. Because, again as mentioned earlier, functional solutions already exists, but they are good but not great and absolutely not cheap ($50-70 have to be the price).
I'm really for a solution with an attached media (CF/SD/USB key) instead of using command line tools to mount images or Wi-Fi or Bluetooth with internal storage or whatever else because external media are cheap, easy to use and easy to manage (multiple media organized by genre / editor / year).

Some other folks try to find new ideas about what could be a solution for emulating CD reading in general. That's the spirit and I like that. And maybe through such thinking good solutions will emerge. That's what I call dreaming. Bringing ideas and see if they can be relevant.

BTW, a bit off-topic but the reason I know some of the optical drive info I mentioned in my post above because MAME is (incredibly slowly) working towards actual optical drive emulation at the chip/instruction level.

IE. Emulate the drive servo DSP, and the drive controller, decode and interpret the drive firmware, have a RAM area for the controller to use, allowing you to debug and watch the program flow, etc.

Eventually for complete execution, this will require disc samples at the raw level, which no one has created and basically isn't possible yet. Or more likely we could have data at the EFM level, and modulate back to the raw data on the fly.

Not really sure what the main lead on this initiative thus far, Olivier Galibert, is thinking where we'll end up using for disc images. But he's been focusing on Macintosh (PowerMac) thus far.

Incredibly early days though, all we've documented thus far are some firmware dumps and the chips found on some drive controller boards (thanks to Bitsavers), no actual emulation. I've been my typical documentation-seeking self. Best finds so far have been some service schematics, but there's not enough on some of the chips. Regardless, I daresay we'll get there before any other Macintosh or PC emulator, since MAMEdev most enjoys tackling the things others won't even think of doing.

Examples:
https://github.com/mamedev/mame/blob/master/s … si/crd254sh.cpp - Skeleton device for Sony/Apple CRD-254SH 4x CD-ROM reader
https://github.com/mamedev/mame/blob/master/s … scsi/cdu415.cpp - Skeleton device for Sony CDU415 CD-ROM reader
https://github.com/mamedev/mame/blob/master/s … scsi/cdu75s.cpp - Skeleton device for Sony/Apple CDU75S 4x CD-ROM reader

Stiletto wrote on 2021-09-18, 08:16:

BTW, a bit off-topic but the reason I know some of the optical drive info I mentioned in my post above because MAME is (incredib […]
Show full quote

BTW, a bit off-topic but the reason I know some of the optical drive info I mentioned in my post above because MAME is (incredibly slowly) working towards actual optical drive emulation at the chip/instruction level.

IE. Emulate the drive servo DSP, and the drive controller, decode and interpret the drive firmware, have a RAM area for the controller to use, allowing you to debug and watch the program flow, etc.

Eventually for complete execution, this will require disc samples at the raw level, which no one has created and basically isn't possible yet. Or more likely we could have data at the EFM level, and modulate back to the raw data on the fly.

Not really sure what the main lead on this initiative thus far, Olivier Galibert, is thinking where we'll end up using for disc images. But he's been focusing on Macintosh (PowerMac) thus far.

Incredibly early days though, all we've documented thus far are some firmware dumps and the chips found on some drive controller boards (thanks to Bitsavers), no actual emulation. I've been my typical documentation-seeking self. Best finds so far have been some service schematics, but there's not enough on some of the chips. Regardless, I daresay we'll get there before any other Macintosh or PC emulator, since MAMEdev most enjoys tackling the things others won't even think of doing.

Examples:
https://github.com/mamedev/mame/blob/master/s … si/crd254sh.cpp - Skeleton device for Sony/Apple CRD-254SH 4x CD-ROM reader
https://github.com/mamedev/mame/blob/master/s … scsi/cdu415.cpp - Skeleton device for Sony CDU415 CD-ROM reader
https://github.com/mamedev/mame/blob/master/s … scsi/cdu75s.cpp - Skeleton device for Sony/Apple CDU75S 4x CD-ROM reader

MAME goal is to emulate the hardware the most accurately as possible at a very low level with little to no consideration to know if the software will run on it on nowadays hardware. If the driver is done correctly the software will work as a consequence but it is done at the cost of a huge CPU consumption. For a few years now, JIT implementation and heavy code reworking has allowed a faster emulation but still, it require a lot of power even for simple 70's machines and some tradeoffs with timing must done anyway. I don't think such an approach is possible with a low cost FPGA. But it can be interesting to see and learn of it, if a project using such principle see the light of day in the future.

Datadrainer wrote on 2021-09-18, 09:08:

MAME goal is to emulate the hardware the most accurately as possible at a very low level with little to no consideration to know if the software will run on it on nowadays hardware. If the driver is done correctly the software will work as a consequence but it is done at the cost of a huge CPU consumption. For a few years now, JIT implementation and heavy code reworking has allowed a faster emulation but still, it require a lot of power even for simple 70's machines and some tradeoffs with timing must done anyway. I don't think such an approach is possible with a low cost FPGA. But it can be interesting to see and learn of it, if a project using such principle see the light of day in the future.

In case it's not clear, Datadrainer, I've worked on MAME for 21 years (off and on) as a developer on its internal development team, I am fully aware of its intended purpose and its shortcomings. And did not intend to imply that its approach could be used on a low cost FPGA. 😉

Stiletto wrote on 2021-09-18, 09:56:

Datadrainer wrote on 2021-09-18, 09:08:

MAME goal is to emulate the hardware the most accurately as possible at a very low level with little to no consideration to know if the software will run on it on nowadays hardware. If the driver is done correctly the software will work as a consequence but it is done at the cost of a huge CPU consumption. For a few years now, JIT implementation and heavy code reworking has allowed a faster emulation but still, it require a lot of power even for simple 70's machines and some tradeoffs with timing must done anyway. I don't think such an approach is possible with a low cost FPGA. But it can be interesting to see and learn of it, if a project using such principle see the light of day in the future.

In case it's not clear, Datadrainer, I've worked on MAME for 21 years (off and on) as a developer on its internal development team, I am fully aware of its intended purpose and its shortcomings. And did not intend to imply that its approach could be used on a low cost FPGA. 😉

Yep, I know and I appreciate your work a lot 😀 I was just saying, for anyone that read the thread, that the approach used by MAME is possibly not the approach that will allow what could become an accessible low cost solution. What is what people wants. Anyway, I think it is an interesting way to go and I encourage it (I said) 😉

ps: I have nothing against MAME. On the contrary, I'm absolutely fond of the project, following its development from almost the beginning and having spent a lot of time reading parts of the source code. I learnt a lot through it, most of the time it is well commented and is an inestimable and valuable source of information. I don't want to be the hasshole of service, just wanted to deliver my point of view so I think it was important to clarify.

Can't you just take the UniPCemu(hardware/ide.c, which is the most accurate one afaik)/Bochs/Dosbox/etc. IDE byte interface(for the IDE I/O ports) and somehow interface it to the real CPU using a (simple) translation layer like a simple linux-running chip like a Pi?
Although you'd need to need to find a way to interface the 32 used IDE pins into simple read, write and interrupt and DMA access lines? And since the Pi doesn't have that much GPIO pins, you'll need some kind of serializer to make it only use a few IO pins?

On a hunch, I searched the Internet for "ATAPI2SD" and found these videos from just last month by "sinowin".
https://www.youtube.com/watch?v=fFdFw1K25Js
https://www.youtube.com/watch?v=tCxNaDNNLMA
Doesn't show the hardware and could easily be faked, but the UI mentions "Unmount SD Card" so perhaps they are working on a hardware project headed for release.

cyclone3d wrote on 2021-09-18, 04:38:

So... What if you used the internals of a real CD-ROM drive and just replaced the laser unit with an emulator and then also had a way to make the drive think that a CD had been ejected and a new one put in.

The XStation does something like that... And still needs this Intel Cyclone 10 FPGA thing to do the IO and data processing... Which isn't a cheap chip. XStation has to have an agreeement with their suppliers to get the chip in a price that can get customers an $90USD XStation, which doesn't precisely emulate a demon speed drive (Playstation drives aren't).
TLDR; You will not get lower price advantage walking that path. If any, more headaches, and you will still need an FPGA. Better to just cut the middle man.

superfury wrote on 2021-09-18, 10:48:

Can't you just take the UniPCemu(hardware/ide.c, which is the most accurate one afaik)/Bochs/Dosbox/etc. IDE byte interface(for the IDE I/O ports) and somehow interface it to the real CPU using a (simple) translation layer like a simple linux-running chip like a Pi?

Few words... IO packet realtime processing. Specially when some users aren't just asking data only pio1-2 speed drive emulation, but also want DMA speeds with Copy Protection emulation and Audio.

I like the NetPi-IDE idea, though it seems to have no progress since 2014 🙁
NetPi-IDE | retrotonics.org

weedeewee wrote on 2021-09-18, 11:45:

I like the NetPi-IDE idea, though it seems to have no progress since 2014 🙁
NetPi-IDE | retrotonics.org

I still see there an FPGA... Which is the main price point driver in this type of solutions....
As I've said before... The Pi doesn't really matter than much when you are using an FPGA to do you realtime IO packet processing. Is the price which matters to many people it seems.

hyoenmadan wrote on 2021-09-18, 11:23:

The XStation does something like that... And still needs this Intel Cyclone 10 FPGA thing to do the IO and data processing... Wh […]
Show full quote

cyclone3d wrote on 2021-09-18, 04:38:

So... What if you used the internals of a real CD-ROM drive and just replaced the laser unit with an emulator and then also had a way to make the drive think that a CD had been ejected and a new one put in.

The XStation does something like that... And still needs this Intel Cyclone 10 FPGA thing to do the IO and data processing... Which isn't a cheap chip. XStation has to have an agreeement with their suppliers to get the chip in a price that can get customers an $90USD XStation, which doesn't precisely emulate a demon speed drive (Playstation drives aren't).
TLDR; You will not get lower price advantage walking that path. If any, more headaches, and you will still need an FPGA. Better to just cut the middle man.

superfury wrote on 2021-09-18, 10:48:

Can't you just take the UniPCemu(hardware/ide.c, which is the most accurate one afaik)/Bochs/Dosbox/etc. IDE byte interface(for the IDE I/O ports) and somehow interface it to the real CPU using a (simple) translation layer like a simple linux-running chip like a Pi?

Few words... IO packet realtime processing. Specially when some users aren't just asking data only pio1-2 speed drive emulation, but also want DMA speeds with Copy Protection emulation and Audio.

Well, won't it be enough to just map DMA to IO port 1F0 and then map the DMA to normal reads/writes to that port?
Audio is already supported by UniPCemu's interface. You just would need to run the audio output blocks through a 44.1kHz timer and put it on the audio output pins? ATAPI_renderAudioSample in UniPCemu receives each sample at a rate of 44.1kHz. So simply make it send it's stereo 16-bit sample to some serializer or output pin DAC for the connected sound card?

Copy protection on the other hand I've yet to see emulated as a hardware interface? Isn't that usually done in software on the machine itself?

superfury wrote on 2021-09-18, 11:57:

Well, won't it be enough to just map DMA to IO port 1F0 and then map the DMA to normal reads/writes to that port?
Audio is already supported by UniPCemu's interface. You just would need to run the audio output blocks through a 44.1kHz timer and put it on the audio output pins? ATAPI_renderAudioSample in UniPCemu receives each sample at a rate of 44.1kHz. So simply make it send it's stereo 16-bit sample to some serializer or output pin DAC for the connected sound card?

Copy protection on the other hand I've yet to see emulated as a hardware interface? Isn't that usually done in software on the machine itself?

The net-pi mentioned by weedeewee does something like that... And still needs an FPGA for the IO realtime packet processing (In this case a Lattice one) while doing just PIO speeds (and no copy protect emulation it seems). As I said to him... The PI (or any software engine doing the backend work) doesn't matter that much when you have an FPGA doing your IO packet processing. But it will "hurt" the price of the result.

Stiletto wrote on 2021-09-18, 11:16:

On a hunch, I searched the Internet for "ATAPI2SD" and found these videos from just last month by "sinowin". https://www.youtube […]
Show full quote

On a hunch, I searched the Internet for "ATAPI2SD" and found these videos from just last month by "sinowin".
https://www.youtube.com/watch?v=fFdFw1K25Js
https://www.youtube.com/watch?v=tCxNaDNNLMA
Doesn't show the hardware and could easily be faked, but the UI mentions "Unmount SD Card" so perhaps they are working on a hardware project headed for release.

Neither he shows links to some development process or code example. But when I was describing an all-in-one solution of dreams. I was thinking of such a think but in 3.5" floppy form factor, managed through a small OLED screen. That would mean 2 floppy drives and 4 [or 2] IDE drives. The drives, parameters, and images folders could be set in a config file, then images can be mounted from the menu/buttons. That is from an HID perspective. Sure, such hardware would be quite expensive and I don't if its even possible to do something like that through an FGPA as bus bandwidths can be saturated with 4 133 MB/s HDD working at the same time, so maybe, just one IDE with M/S port would be more adequate.
Possibly, such a thing would be a near perfect solution.

Datadrainer wrote on 2021-09-18, 12:31:

Stiletto wrote on 2021-09-18, 11:16:

On a hunch, I searched the Internet for "ATAPI2SD" and found these videos from just last month by "sinowin". https://www.youtube […]
Show full quote

On a hunch, I searched the Internet for "ATAPI2SD" and found these videos from just last month by "sinowin".
https://www.youtube.com/watch?v=fFdFw1K25Js
https://www.youtube.com/watch?v=tCxNaDNNLMA
Doesn't show the hardware and could easily be faked, but the UI mentions "Unmount SD Card" so perhaps they are working on a hardware project headed for release.

Neither he shows links to some development process or code example. But when I was describing an all-in-one solution of dreams. I was thinking of such a think but in 3.5" floppy form factor, managed through a small OLED screen. That would mean 2 floppy drives and 4 [or 2] IDE drives. The drives, parameters, and images folders could be set in a config file, then images can be mounted from the menu/buttons. That is from an HID perspective. Sure, such hardware would be quite expensive and I don't if its even possible to do something like that through an FGPA as bus bandwidths can be saturated with 4 133 MB/s HDD working at the same time, so maybe, just one IDE with M/S port would be more adequate.
Possibly, such a thing would be a near perfect solution.

FYI, the 133MB/s is for the bus, so that would be max 2 of those, primary & secondary ide controller, though more than likely only one, since who would want to hook up two 80 pin ide cables and a floppy cable to one device to manage images (don't answer that :-p I'd do it in a heartbeat or a few seconds more )
And that is without the bus overhead.

I'd settle for anything that can do pio & dma transfers, UDMA33 is plenty, which I think an rpi with some gluelogic should be able to manage.

weedeewee wrote on 2021-09-18, 12:45:

I'd settle for anything that can do pio & dma transfers, UDMA33 is plenty, which I think an rpi with some gluelogic should be able to manage.

If with "some gluelogic" do you mean an FPGA frontend doing all the nasty bus talking work, then yes. But again, FPGA == Cost (aka, not below 90USD).
Please, read the NetPI post again... Even with the FPGA, they didn't managed to get past PIO4 in such configuration. Man, why is so difficult to accept it? Sure, the RPi can do lots of amazing things... But fast realtime IO processing isn't one of them. It is a different deal, in a way you can have a tiny JMicron with a 8042-like MCU translating from SATA to IDE. Why? Because the processing CPU raw power doesn't matter as much as the IO processing power and latency, which these things do all in specialized ASIC logic. You can throw all the raw processing powers of an ARM Cortex on the task, and still don't match JMicron's, because your IO is deficient.

I keep seeing people mention FPGAs and them blowing out the budget for a cost-efficient device.

Yes, some FPGAs can be expensive but there are others that aren't. It depends on how much processing is needed. I can't find which FPGA is on the mentioned XStation ODE, I can only find pictures of the top which just has a WiFi/Bluetooth chip shown. But in the Intel Cyclone 10 series, there are big boy GX series chips that are over $100 and the lowest end chip has 85,000 LE. But there is also the Intel Cyclone 10 LP series where non-bulk discount you can get a single chip for just over $7 and it has 6,800 LE.

To put things into perspective, the older MiST FPGA runs off a Cyclone III EP3C25 with 24,000 LE and can run a full Amiga, Atari ST, etc. Those computer cores don't fully simulate the internals of a CD-ROM, but the processor, video, audio, etc add up to much, much more than what's in a lowly CD-ROM.

We probably won't see something like a $16 GoTek Floppy, but we also aren't looking at something that requires a $100+ chip.

With a working device, the logic designed on the FPGA could be taken to create an ASIC which would do the same thing while costing less once the manufacturing world returns to normal. This is outside the realm of hobbiest but maybe a smaller company would see this as an unexplored market.

weedeewee wrote on 2021-09-18, 12:45:

FYI, the 133MB/s is for the bus, so that would be max 2 of those, primary & secondary ide controller, though more than likely on […]
Show full quote

Datadrainer wrote on 2021-09-18, 12:31:

Stiletto wrote on 2021-09-18, 11:16:

On a hunch, I searched the Internet for "ATAPI2SD" and found these videos from just last month by "sinowin". https://www.youtube […]
Show full quote

On a hunch, I searched the Internet for "ATAPI2SD" and found these videos from just last month by "sinowin".
https://www.youtube.com/watch?v=fFdFw1K25Js
https://www.youtube.com/watch?v=tCxNaDNNLMA
Doesn't show the hardware and could easily be faked, but the UI mentions "Unmount SD Card" so perhaps they are working on a hardware project headed for release.

Neither he shows links to some development process or code example. But when I was describing an all-in-one solution of dreams. I was thinking of such a think but in 3.5" floppy form factor, managed through a small OLED screen. That would mean 2 floppy drives and 4 [or 2] IDE drives. The drives, parameters, and images folders could be set in a config file, then images can be mounted from the menu/buttons. That is from an HID perspective. Sure, such hardware would be quite expensive and I don't if its even possible to do something like that through an FGPA as bus bandwidths can be saturated with 4 133 MB/s HDD working at the same time, so maybe, just one IDE with M/S port would be more adequate.
Possibly, such a thing would be a near perfect solution.

FYI, the 133MB/s is for the bus, so that would be max 2 of those, primary & secondary ide controller, though more than likely only one, since who would want to hook up two 80 pin ide cables and a floppy cable to one device to manage images (don't answer that :-p I'd do it in a heartbeat or a few seconds more )
And that is without the bus overhead.

I'd settle for anything that can do pio & dma transfers, UDMA33 is plenty, which I think an rpi with some gluelogic should be able to manage.

I was saying 4 133 MB/s HDD, not 4 HDD working at 133 MB/s which is only a theoretical speed anyway. I know it's the speed of an Ultra ATA/133 bus. That was an example to say that if high speed devices are transferring together that could be a real problem for an FPGA, and that is not considering buffers management. That said, for a 386/486 even Pentium, ATA-4 is enough indeed, but for PII and especially PIII is not enough at all, Ultra ATA/66 at least is required for an optimal experience of software from this generation.

About pluging three cables on one device, I'll not answer then :p Because, it will need 1 audio output too with a mixer to emulate 4 CD-ROM drives with audio. So that is 4 cables to the very minimum. «all-in-one solution of dreams» I said, not the most doable thing ever 😁

But to came back to the Pi, I don't know. I said I don't like it and I see it as not viable for a every day use. But taken as a DIY project it is a very interesting one and maybe by coding a library able stream data through the GPIO port, ATA-4 at 33 MB/s may be achieved. I don't know. I never developed on Pi and I don't know what transfer speed can be really done with it. I find all this ideas (using a micro computer as IDE drive, modding a CD-ROM drive to emulate it's controllers, etc) very fascinating 😀