Difference between pages "FF7" and "MIDI format"

From Final Fantasy Inside
(Difference between pages)
Jump to navigation Jump to search
my_wiki>Nax
m (LZS -> LZSS)
 
my_wiki>Halkun
 
Line 1: Line 1:
<small> This is a Stub article. A Wiki version of Gears should go here. For more information, select the "discussion" tab above so we can best architecture the data. For those who want to start converting, download the Gears pdf [[User:Halkun|Halkun]] 20:18, 5 Mar 2005 (CST) </small>
+
== Standard MIDI File (SMF) Format ==
  
<br />
+
The <B>Standard MIDI File</B> (SMF) is a file format used to store MIDI data (plus some
 +
other kinds of data typically needed by a sequencer.
  
==  Contents  ==
+
This format stores the standard MIDI messages (ie, status bytes with appropriate data bytes)
 +
plus a time-stamp for each message (ie, a series of bytes that represent how many clock pulses
 +
to wait before "playing" the event). The format also allows saving information about tempo,
 +
time and key signatures, the names of tracks and patterns, and other information typically
 +
needed by a sequencer. One SMF can store information for numerous patterns and tracks so that any
 +
sequencer can preserve these structures when loading the file.
  
* [[FF7/History|History]]
+
<B><FONT COLOR=RED>NOTE:</FONT></B> A <B>track</B> usually is analogous to one musical part,
* [[FF7/Engine_basics|Engine Basics]]
+
such as a Trumpet part. A <B>pattern</B> would be analogous to all of the musical parts (ie,
** Parts of the Engine
+
Trumpet, Drums, Piano, etc) for one song.
** Generic Program Flow
+
 
* [[FF7/Kernel|The Kernel]]
+
The format was designed to be generic so that the most important data can be read by all
** [[FF7/Kernel/Overview|Kernel Overview]]
+
sequencers. Think of a MIDI file as a musical version of an ASCII text file (except
*** History
+
that the MIDI file contains binary data), and the various sequencer programs as text editors
*** Kernel Functionality
+
all capable of reading that file. But, unlike ASCII, MIDI file format saves data in <B>chunks</B>
** [[FF7/Kernel/Memory_management|Memory Management]]
+
(ie, groups of bytes preceded by an ID and size) which can be parsed, loaded, skipped, etc.
*** RAM Management
+
Therefore, SMF format is flexible enough for a particular sequencer to store
*** VRAM Management
+
its own proprietary, "extra" data in such a way that another sequencer won't be confused when
*** PSX CD-ROM management
+
loading the file and can safely ignore this extra stuff that it doesn't need.
** [[FF7/Kernel/Kernel.bin|Kernel.bin]]
+
For example, maybe a sequencer wants to save a "flag byte"
*** The KERNEL.BIN Archive
+
that indicates whether the user has turned on an audible metronome click. The sequencer can save
*** The KERNEL2.BIN Archive
+
this flag byte in such a way that another sequencer can skip this byte without
** [[FF7/Kernel/Low_level_libraries|Low Level Libraries]]
+
having to understand what that byte is for. In the future, the SMF format can also be
*** PC to PSX comparison
+
extended to include new "official" chunks that all sequencer programs may elect to load and use.
*** Data Archives
+
This can be done without making old data files obsolete, nor making old sequencers no longer
**** BIN Archive data format
+
able to load the new files. So, the format is designed to be
**** [[FF7/LZSS_format| LZSS Archives]]
+
extensible in a backwardly compatible way.
**** [[FF7/LGP_format| LGP Archives]]
+
 
*** Textures
+
Of course, SMF files may be used by other MIDI software than just sequencers.
**** [[PSX/TIM_format| TIM texture data format for PSX]]
+
Since SMF files can store any and all types of MIDI messages, including System Exclusive
**** [[FF7/TEX_format| TEX texture data format for the PC]]
+
messages, they may be used to store/load data by all kinds of MIDI software, such as a Patch
*** File formats for 3D models
+
Editor that wants to save some System Exclusive messages it received from a MIDI module. (The
**** [[FF7/Kernel/Low_level_libraries#Model_formats_for_PSX|Model Formats for PSX]]
+
"timestamp" for each message may be irrelevant to such a Patch Editor. But it's easily
**** Model Formats for PC
+
ignored for programs that don't really need it).
* [[FF7/Menu_Module|The Menu Module]]
+
 
** Menu Overview
+
In conclusion, any software that saves or loads MIDI data should use SMF format for its data
** Menu Initialization
+
files.
** Menu Modules
+
 
** Calling the various menus
+
==== MIDI File Format: Chunks ====
** Menu dependencies
+
 
** [[FF7/Savemap|The Save Game format]]
+
Data is always saved within a <B>chunk</B>. There can be many chunks inside of a MIDI file.
* [[FF7/Field_Module| The Field Module]]
+
 
** Field Overview
+
Each chunk can be a different size (and likely will be). A chunk's size is how many (8-bit)
** Field Format (PC)
+
bytes are contained in the chunk.
*** General PC Field File Format
+
 
*** PC Field File Header
+
The data bytes in a chunk are typically related in some way. For example, all of the bytes in one chunk may be for one particular sequencer track. The bytes for another sequencer track may be put in
*** File Section Details
+
a different chunk. So, a chunk is simply a group of related bytes.
** [[FF7/Field_Module#Field_Format_.28PSX.29|Field Format (PSX)]]
+
 
*** [[FF7/Field_Module#PSX_DAT_Format|PSX DAT Format]]
+
Each chunk must begin with a 4 character (ie, 4 ascii bytes) <B>ID</B> which tells what "type"
*** [[FF7/Field_Module#PSX_MIM_Format|PSX MIM Format]]
+
of chunk this is.
*** [[FF7/Field_Module#PSX_BSX_Format|PSX BSX Format]]
+
 
*** [[FF7/Field_Module#PSX_BCX_Format|PSX BCX Format]]
+
The next 4 bytes must form a 32-bit length (ie, size) of the chunk.
* The Battle Module
+
 
** Battle Overview
+
<U>All chunks must begin with these two fields</U> (ie, 8 bytes), which are referred to as the
** [[FF7/Battle/Battle_Mechanics| Battle Mechanics]]
+
<B>chunk header</B>.
** [[FF7/Battle/Battle_Field| Battle Field]]
+
 
** [[FF7/Battle/Battle_Scenes| Battle Scenes (scene.bin)]]
+
Here's what a chunk's header looks like if you defined it in C:
*** [[FF7/Battle/Battle_Scenes/Battle_Script| Battle Scripts]]
+
<pre>
** [[FF7/Playstation_Battle_Model_Format| Battle Models (PSX)]]
+
struct CHUNK_HEADER
** [[FF7/Battle/Battle_Animation_(PC)| Battle Animation (PC)]]
+
{
* [[FF7/WorldMap_Module| The World Map Module]]
+
  char            ID[4];
** [[FF7/World_Map/BSZ| World Map BSZ model file format (PSX)]]
+
  unsigned long    Length;
** [[FF7/World_Map/TXZ| World Map TXZ file format (PSX)]]
+
};
** [[FF7/WorldMap_Module/Script| World Map Script Engine]]
+
</pre>
* Sound
+
<B><FONT COLOR=RED>NOTE:</FONT></B> The <B>Length</B> does not include the 8 byte chunk
** [[FF7/PSX/PSX_Sound|PSX Sound]]
+
header. It simply tells you how many bytes of data are in the chunk <U>following this header</U>.
*** [[FF7/PSX/Sound/Overview|Overview]]
+
 
*** [[FF7/PSX/Sound/INSTRx.DAT|INSTRx.DAT]]
+
And here's an example chunk header (with bytes expressed in hex) if you examined it with a hex editor:
*** [[FF7/PSX/Sound/INSTRx.ALL|INSTRx.ALL]]
+
 
*** [[FF7/PSX/Sound/AKAO_frames|AKAO frames]]
+
4D 54 68 64 00 00 00 06
** PC Sound
+
 
* [[FF7/Technical| Technical Help]]
+
Note that the first 4 bytes make up the ascii ID of <B>MThd</B> (ie, the first four
* [[FF7/Technical/Customising|Tools and patches]]
+
bytes are the ascii values for 'M', 'T', 'h', and 'd'). The next 4 bytes tell us that there should
* [[FF7/Technical/Source|Source Code Forensics]]
+
be 6 more data bytes in the chunk (and after that we should find the next chunk header or the end
 +
of the file).
 +
 
 +
=== MIDI File Format: MThd Chunk ===
 +
 
 +
The MThd header has an ID of <B>MThd</B>, and a Length of <B>6</B>.
 +
 
 +
Let's examine the 6 data bytes (which follow the MThd header) in an MThd chunk.
 +
 
 +
The first two data bytes tell the <B>Format</B> (which I prefer to call "type").
 +
There are actually 3 different types (ie, formats) of MIDI files. A type of 0 means that the file
 +
contains one single track containing midi data on possibly all 16 midi channels. If your sequencer
 +
sorts/stores all of its midi data in one single block of memory with the data in the order that it's
 +
"played", then it should read/write this type. A type of 1 means that the file contains one or
 +
more simultaneous (ie, all start from an assumed time of 0) tracks, perhaps each on a single midi
 +
channel. Together, all of these tracks are considered one sequence or pattern. If your
 +
sequencer separates its midi data (i.e. tracks) into different blocks of memory but plays them back
 +
simultaneously (ie, as one "pattern"), it will read/write this type. A type of 2 means that the
 +
file contains one or more sequentially independant single-track patterns.  If your sequencer
 +
separates its midi data into different blocks of memory, but plays only one block at a time (ie,
 +
each block is considered a different "excerpt" or "song"), then it will read/write this type.
 +
 
 +
The next 2 bytes tell how many tracks are stored in the file, <B>NumTracks</B>. Of course,
 +
for format type 0, this is always 1. For the other 2 types, there can be numerous tracks.
 +
 
 +
The last two bytes indicate how many Pulses (i.e. clocks) Per Quarter Note (abbreviated as PPQN) resolution the time-stamps are based upon, <B>Division</B>. For example, if your sequencer has 96 ppqn, this field would be (in hex):
 +
 
 +
00 60
 +
 
 +
<B><FONT COLOR=RED>NOTE:</FONT></B> The 4 bytes that make up the <B>Length</B> are stored in (Motorola) "Big Endian" byte order, not (Intel) "Little Endian" reverse byte order. (ie, The 06 is the fourth byte instead
 +
of the first of the four).
 +
 
 +
In fact, all MIDI files begin with the above <B>MThd header</B> (and that's how you know that it's
 +
a MIDI file).
 +
 
 +
Alternately, if the first byte of Division is negative, then this represents the division
 +
of a second that the time-stamps are based upon. The first byte will be -24, -25, -29, or -30,
 +
corresponding to the 4 SMPTE standards representing frames per second. The second byte (a
 +
positive number) is the resolution within a frame (ie, subframe). Typical values may be 4 (MIDI
 +
Time Code), 8, 10, 80 (SMPTE bit resolution), or 100.
 +
 
 +
You can specify millisecond-based timing by the data bytes of -25 and 40 subframes.
 +
 
 +
Here's what an MThd chunk looks like if you defined it in C:
 +
 
 +
<pre>
 +
struct MTHD_CHUNK
 +
{
 +
  /* Here's the 8 byte header that all chunks must have */
 +
  char          ID[4];  /* This will be 'M','T','h','d' */
 +
  unsigned long  Length; /* This will be 6 */
 +
 
 +
  /* Here are the 6 bytes */
 +
  unsigned short Format;
 +
  unsigned short NumTracks;
 +
  unsigned short Division;
 +
};
 +
</pre>
 +
 
 +
And here's an example of a complete MThd chunk (with header) if you examined it in a hex editor:
 +
<pre>
 +
4D 54 68 64    MThd ID
 +
00 00 00 06    Length of the MThd chunk is always 6.
 +
00 01          The Format type is 1.
 +
00 02          There are 2 MTrk chunks in this file.
 +
E7 28          Each increment of delta-time represents a millisecond.
 +
</pre>

Revision as of 16:47, 12 March 2005

Standard MIDI File (SMF) Format

The Standard MIDI File (SMF) is a file format used to store MIDI data (plus some other kinds of data typically needed by a sequencer.

This format stores the standard MIDI messages (ie, status bytes with appropriate data bytes) plus a time-stamp for each message (ie, a series of bytes that represent how many clock pulses to wait before "playing" the event). The format also allows saving information about tempo, time and key signatures, the names of tracks and patterns, and other information typically needed by a sequencer. One SMF can store information for numerous patterns and tracks so that any sequencer can preserve these structures when loading the file.

NOTE: A track usually is analogous to one musical part, such as a Trumpet part. A pattern would be analogous to all of the musical parts (ie, Trumpet, Drums, Piano, etc) for one song.

The format was designed to be generic so that the most important data can be read by all sequencers. Think of a MIDI file as a musical version of an ASCII text file (except that the MIDI file contains binary data), and the various sequencer programs as text editors all capable of reading that file. But, unlike ASCII, MIDI file format saves data in chunks (ie, groups of bytes preceded by an ID and size) which can be parsed, loaded, skipped, etc. Therefore, SMF format is flexible enough for a particular sequencer to store its own proprietary, "extra" data in such a way that another sequencer won't be confused when loading the file and can safely ignore this extra stuff that it doesn't need. For example, maybe a sequencer wants to save a "flag byte" that indicates whether the user has turned on an audible metronome click. The sequencer can save this flag byte in such a way that another sequencer can skip this byte without having to understand what that byte is for. In the future, the SMF format can also be extended to include new "official" chunks that all sequencer programs may elect to load and use. This can be done without making old data files obsolete, nor making old sequencers no longer able to load the new files. So, the format is designed to be extensible in a backwardly compatible way.

Of course, SMF files may be used by other MIDI software than just sequencers. Since SMF files can store any and all types of MIDI messages, including System Exclusive messages, they may be used to store/load data by all kinds of MIDI software, such as a Patch Editor that wants to save some System Exclusive messages it received from a MIDI module. (The "timestamp" for each message may be irrelevant to such a Patch Editor. But it's easily ignored for programs that don't really need it).

In conclusion, any software that saves or loads MIDI data should use SMF format for its data files.

MIDI File Format: Chunks

Data is always saved within a chunk. There can be many chunks inside of a MIDI file.

Each chunk can be a different size (and likely will be). A chunk's size is how many (8-bit) bytes are contained in the chunk.

The data bytes in a chunk are typically related in some way. For example, all of the bytes in one chunk may be for one particular sequencer track. The bytes for another sequencer track may be put in a different chunk. So, a chunk is simply a group of related bytes.

Each chunk must begin with a 4 character (ie, 4 ascii bytes) ID which tells what "type" of chunk this is.

The next 4 bytes must form a 32-bit length (ie, size) of the chunk.

All chunks must begin with these two fields (ie, 8 bytes), which are referred to as the chunk header.

Here's what a chunk's header looks like if you defined it in C:

struct CHUNK_HEADER
{
   char             ID[4];
   unsigned long    Length; 
};

NOTE: The Length does not include the 8 byte chunk header. It simply tells you how many bytes of data are in the chunk following this header.

And here's an example chunk header (with bytes expressed in hex) if you examined it with a hex editor:

4D 54 68 64 00 00 00 06

Note that the first 4 bytes make up the ascii ID of MThd (ie, the first four bytes are the ascii values for 'M', 'T', 'h', and 'd'). The next 4 bytes tell us that there should be 6 more data bytes in the chunk (and after that we should find the next chunk header or the end of the file).

MIDI File Format: MThd Chunk

The MThd header has an ID of MThd, and a Length of 6.

Let's examine the 6 data bytes (which follow the MThd header) in an MThd chunk.

The first two data bytes tell the Format (which I prefer to call "type"). There are actually 3 different types (ie, formats) of MIDI files. A type of 0 means that the file contains one single track containing midi data on possibly all 16 midi channels. If your sequencer sorts/stores all of its midi data in one single block of memory with the data in the order that it's "played", then it should read/write this type. A type of 1 means that the file contains one or more simultaneous (ie, all start from an assumed time of 0) tracks, perhaps each on a single midi channel. Together, all of these tracks are considered one sequence or pattern. If your sequencer separates its midi data (i.e. tracks) into different blocks of memory but plays them back simultaneously (ie, as one "pattern"), it will read/write this type. A type of 2 means that the file contains one or more sequentially independant single-track patterns. If your sequencer separates its midi data into different blocks of memory, but plays only one block at a time (ie, each block is considered a different "excerpt" or "song"), then it will read/write this type.

The next 2 bytes tell how many tracks are stored in the file, NumTracks. Of course, for format type 0, this is always 1. For the other 2 types, there can be numerous tracks.

The last two bytes indicate how many Pulses (i.e. clocks) Per Quarter Note (abbreviated as PPQN) resolution the time-stamps are based upon, Division. For example, if your sequencer has 96 ppqn, this field would be (in hex):

00 60

NOTE: The 4 bytes that make up the Length are stored in (Motorola) "Big Endian" byte order, not (Intel) "Little Endian" reverse byte order. (ie, The 06 is the fourth byte instead of the first of the four).

In fact, all MIDI files begin with the above MThd header (and that's how you know that it's a MIDI file).

Alternately, if the first byte of Division is negative, then this represents the division of a second that the time-stamps are based upon. The first byte will be -24, -25, -29, or -30, corresponding to the 4 SMPTE standards representing frames per second. The second byte (a positive number) is the resolution within a frame (ie, subframe). Typical values may be 4 (MIDI Time Code), 8, 10, 80 (SMPTE bit resolution), or 100.

You can specify millisecond-based timing by the data bytes of -25 and 40 subframes.

Here's what an MThd chunk looks like if you defined it in C:

struct MTHD_CHUNK
{
   /* Here's the 8 byte header that all chunks must have */
   char           ID[4];  /* This will be 'M','T','h','d' */
   unsigned long  Length; /* This will be 6 */

   /* Here are the 6 bytes */
   unsigned short Format;
   unsigned short NumTracks;
   unsigned short Division;
};

And here's an example of a complete MThd chunk (with header) if you examined it in a hex editor:

4D 54 68 64     MThd ID
00 00 00 06     Length of the MThd chunk is always 6.
00 01           The Format type is 1.
00 02           There are 2 MTrk chunks in this file.
E7 28           Each increment of delta-time represents a millisecond.