MPU-401/Compatible UART-mode Buffer Reference

Discussion about old sound cards, MIDI devices and sound related accessories.

MPU-401/Compatible UART-mode Buffer Reference

Postby Cloudschatze » 2017-7-14 @ 16:17

Here's some research/testing I'd put together regarding MPU-401 UART-mode buffer sizes. Yes, my life is that exciting.

Image

A couple of notes:
  • Transmit Buffer = Host to MPU
  • Receive Buffer = External MIDI Input to MPU
  • Populated fields represent information that is documented, mostly in datasheets and programming references, where the fields highlighted in green have been personally determined and/or verified.
  • Mega-Em employs software-based circular-buffering. I'm not sure what the impact on the transmit buffer is, if any, but it bumps the receive buffer from 16-bytes (hardware) to 255-bytes.
  • Windows' MMSYSTEM.DLL, while not represented in or by this data, also seems to employ circular-buffering.
  • Where a dedicated, non-MCU-based UART is used, 1-byte buffering is adequate (the Atari ST being another such example).
User avatar
Cloudschatze
Oldbie
 
Posts: 943
Joined: 2005-6-16 @ 14:32

Re: MPU-401/Compatible UART-mode Buffer Reference

Postby Scali » 2017-7-15 @ 11:08

That's interesting. So at the very least, some MPU401 implementations appear to get away with the minimal 1-byte buffer.

Cloudschatze wrote:[*]Where a dedicated, non-MCU-based UART is used, 1-byte buffering is adequate (the Atari ST being another such example).[/list]


Yes, I suppose when you have a (slow) microcontroller on the board, the buffer can be used for asynchronous communication, so you won't deadlock the host waiting for the MCU to process the byte.
I guess that's the entire reason why you'd want a buffer for sending MIDI.
For receiving MIDI, a larger buffer makes more sense, because you basically can't lock on the MIDI data lines as far as I know. That is, you only have one data line, there's no additional handshaking. So you can't signal to a device that tries to send MIDI that it has to wait.
So a MIDI interface will have to store anything it receives in a buffer, so that it can wait there for the host to read it. Using a 1-byte buffer on that side is quite a risk on very slow computers I'd say.
On modern computers you'd get away with it, because you could put a host-side buffer on it:
When a byte is received, the device generates an interrupt. The interrupt handler copies the byte to an internal buffer ASAP, so that the device is ready to receive the next byte.
The MIDI applications will then read from the internal buffer, not the device buffer, so as not to deadlock the interface.
In the Windows-age MIDI devices always work that way anyway, because MIDI applications always talk to the MIDI mapper, not directly to the hardware.

I'd want to ask though: How exactly do you test for specific receive and transmit buffer sizes? As in, how can you verify the exact size of the buffers?
Scali
l33t
 
Posts: 2803
Joined: 2014-12-13 @ 14:24

Re: MPU-401/Compatible UART-mode Buffer Reference

Postby Jepael » 2017-7-15 @ 13:31

Scali wrote:So a MIDI interface will have to store anything it receives in a buffer, so that it can wait there for the host to read it. Using a 1-byte buffer on that side is quite a risk on very slow computers I'd say.
On modern computers you'd get away with it, because you could put a host-side buffer on it:
When a byte is received, the device generates an interrupt. The interrupt handler copies the byte to an internal buffer ASAP, so that the device is ready to receive the next byte.


Yes, and also given the byte rate of 3125 bytes per second, when sending bytes back to back, and if there would be no HW buffering, you have only 320 microseconds time to read the first byte or the second will overrun it. A motherboard I have with dump MPU interface (no DOS drivers installed), it does not give me MIDI byte receive interrupts, so if there was no FIFO, I'd have to set a timer interrupt to be generated at a rate faster than 3125 Hz so I could poll received bytes in the timer interrupt. Fortunately, the chipset has 16-byte FIFO, so any timer rate over 200Hz would be enough to poll so no bytes are lost. Assuming nothing causes latency in the interrupt processing, like performing some longish operations in the timer interrupt like playing OPL music.

Scali wrote:I'd want to ask though: How exactly do you test for specific receive and transmit buffer sizes? As in, how can you verify the exact size of the buffers?


For determining receive FIFO, you first flush the input so FIFO is empty, by reading buffered MIDI bytes until there's no more to read. Then you send a bunch of MIDI bytes like by pushing few keys on a MIDI keyboard, and stop sending. Now again, you check how many bytes you can read from the FIFO until there is no more to read. You can also connect MIDI OUT to MIDI IN and send bytes to yourself, one byte at a time.

It's kind of similar to determine the transmit FIFO size, I connected MIDI OUT to MIDI IN on the motherboard and again flushed the input FIFO. Then in a FOR loop wrote letters from 'A' to 'Z' to the MIDI output port, and since filling the output FIFO is faster than transferring bytes, I waited for bytes to start appearing at the input, and checked what letters I got back. I recall getting back 16 letters from 'A' onwards, and 17th letter was 'Z', so I determined the TX FIFO is 16 bytes. That's because 'A' starts to be transferred out so 16 letters from 'B' onwards go to TX FIFO, and when the FIFO is full, the last letter gets overwritten, so it's 'Z'.
Jepael
Oldbie
 
Posts: 1072
Joined: 2005-6-15 @ 19:28
Location: Finland

Re: MPU-401/Compatible UART-mode Buffer Reference

Postby Cloudschatze » 2017-7-15 @ 22:08

Jepael wrote:For determining receive FIFO, you first flush the input so FIFO is empty, by reading buffered MIDI bytes until there's no more to read. Then you send a bunch of MIDI bytes like by pushing few keys on a MIDI keyboard, and stop sending. Now again, you check how many bytes you can read from the FIFO until there is no more to read.

This describes the process I'd used near-verbatim. To help facilitate things, I'd written a dumb little routine in QBasic to read the input and provide a count at the end.

It's kind of similar to determine the transmit FIFO size, I connected MIDI OUT to MIDI IN on the motherboard and again flushed the input FIFO. Then in a FOR loop wrote letters from 'A' to 'Z' to the MIDI output port, and since filling the output FIFO is faster than transferring bytes, I waited for bytes to start appearing at the input, and checked what letters I got back. I recall getting back 16 letters from 'A' onwards, and 17th letter was 'Z', so I determined the TX FIFO is 16 bytes. That's because 'A' starts to be transferred out so 16 letters from 'B' onwards go to TX FIFO, and when the FIFO is full, the last letter gets overwritten, so it's 'Z'.

I'd tried something similar, but was unable to send data to the MPU faster than it could transmit it, using either debug or QBasic routines. I am not a programmer. :)

Validating the UART-mode transmit buffer of an actual MPU-401 or variant yet needs to occur, as it's more-or-less the reference standard, but remains mostly anecdotal at 256-bytes until it can be tested, or determined through ROM disassembly otherwise.

Testing the transmit buffer of the SB16 chipset proved easy enough though. Here's what I'd written about that in related correspondence with NewRisingSun:
Cloudschatze wrote:The undocumented DSP command FAh is described exactly nowhere, but it's behavior is such that the DSP is put into a "busy" state, and in which mode it responds to nothing other than a Reset command. Consequently, the MPU behavior is also affected, whereby, after sending exactly one data byte to the MPU (which does not get output), the DRR flag is raised. Continuing to send data to the MPU at this point simply causes the one-byte buffer to be overwritten (LIFO behavior). Upon resetting the DSP, this buffered MPU byte is automatically output. I've tested this behavior with all of the cards in my little Excel spreadsheet, and they all behave similarly. In short, I've been able to confirm and verify that the MPU output buffer of the entire SB16 family is indeed only a single byte.


After further testing, it was determined that the FAh command would "hang" the DSP because it expected a further two bytes of data, and was therefore incomplete, but this behavior worked for the purposes described.
User avatar
Cloudschatze
Oldbie
 
Posts: 943
Joined: 2005-6-16 @ 14:32

Re: MPU-401/Compatible UART-mode Buffer Reference

Postby Jepael » 2017-7-15 @ 23:36

Cloudschatze wrote:Validating the UART-mode transmit buffer of an actual MPU-401 or variant yet needs to occur, as it's more-or-less the reference standard, but remains mostly anecdotal at 256-bytes until it can be tested, or determined through ROM disassembly otherwise.


From manual: UART mode 85 bytes for firmware versions below 1.3, approximately 1700 bytes for versions 1.3 and above.

I don't own one, I haven't tested it.
Jepael
Oldbie
 
Posts: 1072
Joined: 2005-6-15 @ 19:28
Location: Finland

Re: MPU-401/Compatible UART-mode Buffer Reference

Postby Cloudschatze » 2017-7-15 @ 23:46

Jepael wrote:From manual: UART mode 85 bytes for firmware versions below 1.3, approximately 1700 bytes for versions 1.3 and above.

Right, that refers to the UART-mode receive buffer, and it's exactly 1744 bytes. :)

The transmit buffer, on the other hand, isn't documented anywhere that I've seen. The sole reference I've found is a USENET post, wherein the 256-byte figure was supposedly provided by Roland. The problem with this information is that it may actually refer to the total buffering used in "smart" mode, rather than UART mode.
User avatar
Cloudschatze
Oldbie
 
Posts: 943
Joined: 2005-6-16 @ 14:32

Re: MPU-401/Compatible UART-mode Buffer Reference

Postby Scali » 2017-7-17 @ 10:29

Jepael wrote:It's kind of similar to determine the transmit FIFO size, I connected MIDI OUT to MIDI IN on the motherboard and again flushed the input FIFO. Then in a FOR loop wrote letters from 'A' to 'Z' to the MIDI output port, and since filling the output FIFO is faster than transferring bytes, I waited for bytes to start appearing at the input, and checked what letters I got back. I recall getting back 16 letters from 'A' onwards, and 17th letter was 'Z', so I determined the TX FIFO is 16 bytes. That's because 'A' starts to be transferred out so 16 letters from 'B' onwards go to TX FIFO, and when the FIFO is full, the last letter gets overwritten, so it's 'Z'.


Yes, but that seems to imply that the state management does not take into account that a buffer can be full.
As in, I would expect a robust implementation to not signal the DRR-flag once the FIFO buffer is full, so you simply cannot overwrite values in the buffer.
But perhaps the implementation is simplified, and DRR is only intended as a synchronization mechanism from host to FIFO buffer, and does not extend to the FIFO buffer waiting to output its data.

Because if it was implemented in a robust way, the only way to measure it would be to measure the delay between writes, I suppose.
That is, we can assume that we can write faster than 3125 bytes per second to the FIFO buffer, and once it's full, it will be throttled to exactly 3125 bytes per second, because we will be in a busy-wait for DRR until a byte has been sent out of the FIFO buffer, and we have room to send a new byte.
It will be difficult to get an exact number this way, since bytes will be sent out of the buffer while you're trying to fill it up.
So if you see a delay after 20 bytes, the buffer may only be 16 bytes large, but it managed to send out 4 bytes while you were filling it.

The idea of the FIFO might be that it's merely a 'helper' to throttle writes a bit. That is, the application is expected to only write 3125 bytes per second to the interface nominally, but since MIDI commands are generally just 2 or 3 bytes, it can write these out in small bursts.
Which would mean that sending SysEx commands would still be a problem, because they're generally much larger than the FIFO size.
Scali
l33t
 
Posts: 2803
Joined: 2014-12-13 @ 14:24


Return to Sound

Who is online

Users browsing this forum: No registered users and 1 guest