IBM Music Feature Card/Yamaha FB-01

Developer's Forum, for discussion of bugs, code, and other developmental aspects of DOSBox.

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-01 @ 00:37

Studying the code some more, the implementation is indeed not like a real FB-01 at all.
It uses a bruteforce approach where it allocates one YM2151 emulator for each MIDI channel, and does a 1:1 mapping.
Which also means that you get a polyphony of 8 notes per channel.
Apparently the VFB-01 project was never more than 'make something that can sound FB-01-like'. Good enough if you want to use it as a sound module when composing music, but not very useful if you want an actual replacement for the real thing.
Likewise, it implements its own SysEx command for setting voice parameters, but no actual SysEx commands for the real FB-01.

I think my next tasks will be:
1) Set up some stubs for all the FB-01 SysEx commands, so they can be implemented later
2) Rewrite the MIDI-to-YM2151 code so that it uses the same mapping strategy as the real FB-01: You have instruments which can have a polyphony of 0 or more, with a maximum of 8. Each instrument can be assigned to 0 or more MIDI channels. Oh, and don't forget that you also have low and high note ranges for each voice, so you could have multiple voices on the same MIDI channel.

Once the remapping works, I can at least implement the SysEx commands that manipulate the mapping or that manipulate either an instrument directly, or manipulate an instrument through an associated MIDI channel.

So, while we have sound, it's not actually the 'right' sound yet :)
It will still take some work before the SysEx commands can be implemented, and the thing can start behaving as a real FB-01.

Another thing that I wonder about is the way the FB-01 implements 'streaming' SysEx commands.
It has certain 'list' commands that can contain a number of command sequences, which the real thing will play in realtime. It includes things like custom note on/off commands and such, so being realtime is very important (the Sierra games actually upload their voice data this way, which leads to the large 6k SysEx commands that broke DOSBox earlier).

The way most software implements MIDI on Windows is to send SysEx as simple 'long messages' via the API (which is why DOSBox broke).
That isn't going to work in this case.
For the voice data upload I already had to make a workaround, since that particular SysEx command was only implemented on the IMFC, not on a real FB-01. So I capture the 'streaming' SysEx command when it comes in on the IMFC interface, and I split it up into smaller SysEx commands for the FB-01.
I suppose I could also do that for other types of 'list' commands, so that at least DOSBox works correctly (albeit at a slightly higher bandwidth requirement for MIDI).
But on the receiving end, the FB-01 should be able to handle it as well. I'm not sure at this moment whether the MUNT MIDI parser can do this. First impression was that it cannot.
The parser would need to be stateful: when you receive a 'long' MIDI message, it is not necessarily a complete SysEx command, it could be a fragment. Then the parser would need to keep streaming the data to the backend until it reaches the end of the SysEx command. Basically the same trick as what I implemented in the IMFC code.
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby gdjacobs » 2017-7-01 @ 04:28

Regarding legal matters, I definitely recommend consulting primary sources and/or a lawyer for authoritative answers.

I can say that the LGPL was explicitly intended for use by libraries to permit dynamic linking without requiring free software licensing from the non-LGPL portions of the overall application. It carves out a specific exception for dynamic linking and (with certain limitations) static linking that does not exist in the standard GPL.

So, LGPL allows you to link (via DLLs, object files, etc) with other code to create an overall work which is not LGPL licensed. Any derived work of the LGPL portion still has to be LGPL licensed, though (or it can be relicensed under the GPL per the terms of the LGPL). Trying to link with GPL licensed modules, objects, or libraries requires GPL compatible licensing for everything.
User avatar
gdjacobs
l33t
 
Posts: 4181
Joined: 2015-11-03 @ 05:51
Location: The Great White North

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-01 @ 11:02

gdjacobs wrote:Regarding legal matters, I definitely recommend consulting primary sources and/or a lawyer for authoritative answers.


Well, I think there is no real authorative answer on what is or isn't a 'derivative work': https://en.wikipedia.org/wiki/GNU_Gener ... ived_works

gdjacobs wrote:So, LGPL allows you to link (via DLLs, object files, etc) with other code to create an overall work which is not LGPL licensed.


Yes, technically I'm not doing that. I am building a DLL, but that is the 'end product' for that particular project. I don't link it with other code myself. It's just a Windows driver.

gdjacobs wrote:Trying to link with GPL licensed modules, objects, or libraries requires GPL compatible licensing for everything.


Yes, since some parts of the code are GPL rather than LGPL, I suppose the DLL itself would have to be treated as GPL.
I think that's very impractical though, I'd rather have it the other way around: that GPL-code can be used in DLLs, where you merely force all code in the DLL to be GPL, but LPGL then 'takes over', so the GPL-demands don't extend beyond those limits.
That would be more similar to how GPL acts inside binaries: only the code used within those binaries has to be GPL. The GPL doesn't extend to the part of the code that actually loads and executes the binary (the OS basically), nor to any other code that tries to connect to it in some way (sockets, shared memory objects, pipes etc).

But as I say, I think in my case I can treat the DLL as a binary. It's a driver, not a programming library meant to be linked to directly. In fact, I don't even supply the .lib file to link to it directly.
It is invoked by the OS when you do a midiOutOpen() call, specifying the ID of that driver.
So you don't talk to the DLL directly, you talk to the midiOut-APIs of the OS.

'Dynamic linking'... Yes, technically the OS will be loading my DLL. But it is more of a 'plugin' type of DLL: the interface is specified, the implementation is not. So the OS that loads my DLL is not specifically written to use *my* library. In fact, the OS was written long before my library existed, so you cannot claim it is a 'derivative work' just because it dynamically links to my DLL.

Good enough for me. Or does anyone have other suggestions? :)
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby gdjacobs » 2017-7-01 @ 20:00

For the case of a DLL which is GPL licensed, whatever links to the library doesn't necessarily have to be GPL licensed, but it does have to be GPL compatible. And, yes, the tools required for dynamic linking do not have to be GPL licensed. However, you may want to check the licensing of any system libraries you pull in.
User avatar
gdjacobs
l33t
 
Posts: 4181
Joined: 2015-11-03 @ 05:51
Location: The Great White North

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-01 @ 20:14

gdjacobs wrote:However, you may want to check the licensing of any system libraries you pull in.


I don't think that's a 'thing' though? It's pretty much impossible to write a Windows application without at least pulling some system libraries in, such as kernel32.dll and user32.dll.
Yet there's tons of GPL-ed Windows applications out there (such as DOSBox itself).
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby gdjacobs » 2017-7-01 @ 21:20

Maybe the wrong word. I was thinking more of the C library you're using.
User avatar
gdjacobs
l33t
 
Posts: 4181
Joined: 2015-11-03 @ 05:51
Location: The Great White North

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-01 @ 22:59

Okay, here we go...
I've put the current quick-and-dirty code for the fb01emu.dll driver, based on MUNT up on Github:
https://github.com/Scalibq/FB-01-Emulator

The code is still a bit sloppy here and there, so don't be surprised if you still see some references to MT-32 here and there :)
Anyway, for me it was important to set up a proper repository. I've been developing on my private repository so far. Now I can continue working directly on Github, and others can contribute as well, if they like.

I've also put the IMFC.cpp for DOSBox on there, so people can add it to their own DOSBox builds, at least until it's part of the DOSBox SVN.
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-02 @ 14:25

I've just waded through both the FB-01 Service Manual and the User Manual, and I think I've come up with the following list of SysEx commands...
And a first subdivision to aid parsing them:
Code: Select all
static int fb01_exclusive( MidiEvent* ev )
{
   /*
   SysEx Messages (see Yamaha FB-01 Service Manual page 12):
   A: Instrument message
   1) Parameter Change (1 byte) by System Channel + Instrument Number
      F0 43 75 <0000ssss> <00011iii> <00pppppp> <0ddddddd> F7
      s = system No.
      i = instrument No.
      d = data
   2) Parameter Change (2 byte) by System Channel + Instrument Number
      F0 43 75 <0000ssss> <00011iii> <01pppppp> <0000dddd> <0000dddd> F7
      s = system No.
      i = instrument No.
      d = data
   3) Voice bulk data dump
      F0 43 75 <0000ssss> <00101iii> 40 00 F7
      s = system No.
      i = instrument No.
   4) Store into voice RAM
      F0 43 75 <0000ssss> <00101iii> 00 <00dddddd> F7
      s = system No.
      i = instrument No.
      d = voice No.

   B: System message
   1) System parameter change
      F0 43 75 <0000ssss> 10 <0ppppppp> <0ddddddd> F7
      s = system No.
      p = parameter No.
      d = data
   2) Voice RAM 1 bulk data dump
      F0 43 <0010ssss> 0C F7
   3) Each voice bank bulk data dump
      F0 43 75 <0000ssss> 20 00 <00000xxx> F7
   4) Current configuration data dump
      F0 43 75 <0000ssss> 20 01 00 F7 (NB: 00 and 01 reversed in Service Manual?)
   5) Configuration data dump
      F0 43 75 <0000ssss> 20 02 <000xxxxx> F7
   6) 16 configuration data dump
      F0 43 75 <0000ssss> 20 03 00 F7
   7) Unit ID number dump
      F0 43 75 <0000ssss> 20 04 00 F7
   8) Configuration data store
      F0 43 75 <0000ssss> 20 40 <000ddddd> F7

   Other system messages
   C: Bulk data
   1) 48 voice bulk data (voice RAM 1)
      F0 43 <0000ssss> 0C 20 00 <0000dddd> <0000dddd> ... <0000dddd> <0000dddd> <0eeeeeee> 10 40 <0000dddd> <0000dddd> ... <0000dddd> <0000dddd> <0eeeeeee> F7
   2) 48 voices bulk data (to specific bank)
      F0 43 75 <0000ssss> 0C 00 00 <00000xxx> 20 10 <0000dddd> <0000dddd> ... <0000dddd> <0000dddd> <0eeeeeee> 10 40 <0000dddd> <0000dddd> ... <0000dddd> <0000dddd> <0eeeeeee> F7
   3) Current configuration
      F0 43 75 <0000ssss> 00 01 00 01 20 <0ddddddd> ... <0ddddddd> <0eeeeeee> F7
   4) Configuration memory
      F0 43 75 <0000ssss> 00 02 <000xxxxx> 01 20 <0ddddddd> ... <0ddddddd> <0eeeeeee> F7
   5) 16 configuration memory
      F0 43 75 <0000ssss> 00 03 00 14 00 <0ddddddd> ... <0ddddddd> <0eeeeeee> F7
   6) 1 voice bulk data
      F0 43 75 <0000ssss> <00001iii> 00 00 01 00 <0000dddd> <0000dddd> ... <0000dddd> <0000dddd> <0eeeeeee> F7

   D: Channel message
   1) Parameter Change (1 byte) by MIDI channel specification
      F0 43 <0001nnnn> 15 <00pppppp> <0ddddddd> F7
   2) Parameter Change (2 byte) by MIDI channel specification
      F0 43 <0001nnnn> 15 <01pppppp> <0000dddd> <0000dddd> F7

   E: Event list
   1) Event list
      F0 43 75 70 <0eeeeeee> ... <0eeeeeee> F7

   Event data embedded in event list (not separate SysEx messages):
   1) Key OFF
      <0000nnnn> <0kkkkkkk> <0fffffff>
   2) Key ON/OFF
      <0001nnnn> <0kkkkkkk> <0fffffff> <0vvvvvvv>
   3) Key ON/OFF (with duration)
      <0010nnnn> <0kkkkkkk> <0fffffff> <0vvvvvvv> <0ddddddd> <0ddddddd>
   4) Parameter change (1 byte)
      <0111nnnn> <00pppppp> <0ddddddd>
   5) Parameter change (2 byte)
      <0111nnnn> <01pppppp> <0000dddd> <0000dddd> (NB: mistakenly listed as <00pppppp> in Service and User Manuals?)

   (not in Service Manual, only in user manual)
   6) Control Change
      <0011nnnn> <0ccccccc> <0vvvvvvv>
   7) Program Change
      <0100nnnn> <0ppppppp>
   8) After Touch
      <0101nnnn> <0vvvvvvv>
   9) Pitch Bend
      <0110nnnn> <0yyyyyyy> <0xxxxxxx>
   */

   if ( ev->ex_buf[1] == 0x75 ) { /* Sub-status */
      // The following SysEx messages can be recognized by Sub-status 75H as third byte in the SysEx message:
      // A1, A2, A3, A4, B1, B3, B4, B5, B6, B7, B8, C2, C3, C4, C5, C6, E1
   }
   else
   {
      // The following SysEx messages do not have 75H as third byte:
      // B2, C1, D1, D2
   }

   return 0;
}


At some point I'll have to do the same for the IMFC to verify that there aren't any other differences aside from the Parameter List SysEx command that is currently being translated in IMFC.cpp on the DOSBox side.
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby NewRisingSun » 2017-7-02 @ 17:25

IMFC.cpp is just the MIDI interface part, right? Because MUNT's DOSBox patch puts the sound generation part into DOSBox as well, removing the need to install a Windows driver.
NewRisingSun
Oldbie
 
Posts: 774
Joined: 2005-9-02 @ 02:26

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-02 @ 19:29

NewRisingSun wrote:IMFC.cpp is just the MIDI interface part, right?


Yes, it is for now.
Perhaps in the future the emulation itself can be built into DOSBox. Technically it should be like that, because unlike an MPU-401 + MT-32 setup, you actually have an onboard FB-01 AND a MIDI in/out port.
So if you want a realistic emulation, you should implement the routing, and route to the onboard FB-01 and/or the MIDI out based on the configuration.
Currently it just routes all data to the MIDI out.

In its current form, that already gives you IMFC-emulation if you connect it to a real FB-01 (as said, above, the code translates IMFC-specific commands to be FB-01-compatible).
And hopefully the FB-01 emulator will be complete/accurate enough to take over this role in the near future.
But the FB-01 may also have other applications than just getting IMFC-support, so it makes sense to make it available as a MIDI driver.
I wouldn't mind if it goes the same way as MUNT, having multiple variations, such as an integrated DOSBox version, a standalone driver, and running in the MUNT host app.
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-02 @ 19:46

Stiletto wrote:But for you, for instance, we have someone on our team who plans to update every FM synth core based upon information gained from chip decaps (previously it's been mostly reverse-engineering based on documentation and readings from test rigs) and do a complete rewrite, which should make for an even deeper level of accuracy, and it might be good for you to be able to update should that project ever finish. (His to-do list's pretty monstrous so it may be a while...)


I have two small questions regarding this:
1) Do you know if this chip decapping project also takes the YM2164 into account? Nobody seems to know exactly what an YM2164 is, not even Yamaha (I asked them for documentation, they said they no longer had any available for such old discontinued chips). Common knowledge on the internet is that the YM2164 is identical to the YM2151, save for some undocumented registers.
I wonder if that's true... and I also wonder what these undocumented registers do, and what the differences are.
For now I'm assuming it's true, since I can get an YM2151 emulator, but there is no YM2164 emulator.

Thing is, common knowledge on the internet is also that the IMFC is an FB-01 on an ISA card. Which is sorta true, but not entirely. It uses different firmware, which means that:
a) It supports some SysEx commands that a vanilla FB-01 does not. Sierra tripped up on this in their FB-01 driver.
b) Some parameters in the configuration stored in the ROM are slightly different, so certain default sounds may be slightly different.

2) Do you happen to know what the status of OPL3 emulation is? I don't know what DOSBox uses, but if I run the TEST-SBP.EXE program, the 4-operator test sounds quite distorted and bad compared to the real thing.
Is DOSBox just using an outdated emulation core, or is the state-of-the-art in OPL3 emulation still not quite there yet?
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-02 @ 20:51

Scali wrote:1) Set up some stubs for all the FB-01 SysEx commands, so they can be implemented later


This is now complete.

Scali wrote:2) Rewrite the MIDI-to-YM2151 code so that it uses the same mapping strategy as the real FB-01: You have instruments which can have a polyphony of 0 or more, with a maximum of 8. Each instrument can be assigned to 0 or more MIDI channels. Oh, and don't forget that you also have low and high note ranges for each voice, so you could have multiple voices on the same MIDI channel.


This is what I will be doing next: reading the FB-01 manual closely again, and trying to understand exactly how it maps MIDI channels to YM2164 voices in all possible scenarios, and then design an implementation for that.
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-03 @ 10:57

As far as I understand it, the 8 'instruments' are the core of the MIDI mapping on the FB-01.
An instrument contains the following settings:
- Voice data
- MIDI channel
- Low and high MIDI note range
- Polyphony (0-8)

Multiple instruments can use the same MIDI channel. In this case, a note on the MIDI channel is played sequentially on all instruments from low to high.

So I think the algorithm would be something like this:
Given a MIDI command on channel N, iterate over all 8 instruments, and for each instrument where MIDI channel == N, and the low < note < high, play the note.

Playing notes then is done via a polyphony routine. The instruments do not necessarily map 1:1 to the hardware voices. When polyphony is more than 1, multiple voices are allocated to the same instrument (so the voice data of the instrument is set to these hardware voices), and the note will be played either on the first empty voice, or on the note that has been active the longest.

The sum of polyphony of all instruments can never be more than 8. When a polyphony is set on an instrument that would result in a sum of more than 8, then the polyphony of other instruments is reduced, starting with the instrument that has the highest polyphony value (I am not sure how this works in detail yet... do they mean this instrument will first get turned down to 0, then the next instrument etc... or do they mean that they step the polyphony down by 1 at a time, then re-check which instrument has the highest polyphony now? Will investigate later).
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby Cloudschatze » 2017-7-03 @ 14:41

Scali wrote:When a polyphony is set on an instrument that would result in a sum of more than 8, then the polyphony of other instruments is reduced, starting with the instrument that has the highest polyphony value

This would seem to be yet another difference between the IMFC and FB-01. Through use of the FB-01's front-panel operations, once all voices have been assigned, that count has to be manually reduced from a given instrument before voice allocation can occur elsewhere. I don't suspect any difference in this behavior when the voices are allocated via SysEx, but will validate later.

(I am not sure how this works in detail yet... do they mean this instrument will first get turned down to 0, then the next instrument etc... or do they mean that they step the polyphony down by 1 at a time, then re-check which instrument has the highest polyphony now? Will investigate later).

I would guess the latter, but then, what is the behavior when two instruments have the same number of voices assigned?
User avatar
Cloudschatze
Oldbie
 
Posts: 953
Joined: 2005-6-16 @ 14:32

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-03 @ 15:18

Cloudschatze wrote:I would guess the latter, but then, what is the behavior when two instruments have the same number of voices assigned?


My guess is that it would go from lowest to highest instrument number.
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby Cloudschatze » 2017-7-04 @ 06:17

Cloudschatze wrote:
Scali wrote:When a polyphony is set on an instrument that would result in a sum of more than 8, then the polyphony of other instruments is reduced, starting with the instrument that has the highest polyphony value

This would seem to be yet another difference between the IMFC and FB-01. Through use of the FB-01's front-panel operations, once all voices have been assigned, that count has to be manually reduced from a given instrument before voice allocation can occur elsewhere. I don't suspect any difference in this behavior when the voices are allocated via SysEx, but will validate later.

And... validated. The FB-01's (inferior?) voice-assignment behavior is the same whether configured through SysEx, or by front-panel operations.
User avatar
Cloudschatze
Oldbie
 
Posts: 953
Joined: 2005-6-16 @ 14:32

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-04 @ 08:53

Cloudschatze wrote:And... validated. The FB-01's (inferior?) voice-assignment behavior is the same whether configured through SysEx, or by front-panel operations.


Meaning what exactly? That it just rejects the SysEx command if you try to allocate more voices than it has available?
I suppose we could solve that in the IMFC-emulation part... let the IMFC code keep track of voice allocation, and adjust the voice allocation by sending additional SysEx commands.
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby lchiocca » 2017-7-04 @ 10:11

Just to keep you up-to-date: I'm reverse engineering the IMF ROM code. It's really simple; no assembly optimizations :). I'm not finished yet, but getting there.
lchiocca
Newbie
 
Posts: 17
Joined: 2006-9-17 @ 22:35

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-04 @ 10:23

lchiocca wrote:Just to keep you up-to-date: I'm reverse engineering the IMF ROM code. It's really simple; no assembly optimizations :). I'm not finished yet, but getting there.


I was just doing the same thing :)
But I only just started, so perhaps I'll just leave it to you for now.
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

Re: IBM Music Feature Card/Yamaha FB-01

Postby Scali » 2017-7-04 @ 17:04

I'm working out the instrument logic now. Most things in the FB-01 seem to revolve around that.
I will need the 8 instruments to do the MIDI mapping, voice allocation and that sort of thing.
A configuration builds on that, it's basically a set of 8 instruments, a name string, and two global states (COMBINE and key receive mode... the latter is to increase polyphony when using two FB-01s, to configure the unit to either only play the even or odd keys. Not really relevant for us, but we can put it in without much extra effort).

Once the instruments and the configuration are done, I'll try to rewrite the voice allocation so that you can use each of the 16 channels on a single YM2151.
I've already stripped the code somewhat so that it only uses a single YM2151 now. But currently that means only MIDI channel 0 can be used.
Scali
l33t
 
Posts: 2867
Joined: 2014-12-13 @ 14:24

PreviousNext

Return to DOSBox Development

Who is online

Users browsing this forum: No registered users and 3 guests