Anonymous

Changes

From Final Fantasy Inside

FF7/Kernel/Low level libraries

3,393 bytes removed, 05:19, 23 May 2019
m
59 revisions imported
== PC to PSX comparison ==
The files and data formats used in the PSX version of FF7 and it's PC port are conceptually the same thing, and accomplish the same tasks. That being said, they both have wildly different formats. Both ofwhich , both of which were derived from a third original format that is also somewhat different that to the first two.
The original PSX FF7 was created in part using Sony's Psy-Q development library. This library usescommon uses common formats that are "native" to the PSX. Often times , a toolkit was used to convert commondevelopmentcommon development- based formats, such as a TGA bitmap or a palleted GIF file, to something a little moresuited more suited to Psy-Q, which would be a [[PSX/TIM_format|TIM file]].
During the porting process to the PC, some of the original artwork, (and artists for that matter), wereno were no longer available. This resulted in the port team having to use the Psy-Q versions of many files,which were ill suited for the PC architecture. In our example, the [[PSX/TIM_format|TIM file ]] was converted to a TEX file,which would be manipulated in the PC's video memory a little more efficiently. Sometimes theoriginal the original artwork was available, such as the pictures of the characters within the menu, or the original MIDI files. Most often times it was not. To make things a little more confusing, both systems also archive their data files in different ways, making the extraction and rendering of each file a bit of a bear. For the most part the data within each file is the same thing, just a little switched around. Here, we will cover the more generic files first, and then common files used in each module.
To make things a little more confusing, both systems also archive their data files in different ways,making the extraction and rendering of each file a bit of a bear. For the most part the data within eachfile is the same thing, just a little switched around. This manual will cover the more generic files first,and then common files used in each module.
== Data Archives ==
=== BIN archive data format ===
The BIN format comes as two different types. They both have the same extension, so one must open the file to see which format is which. They are best described as BIN Types and BinBIN-GZIP types. ==== BIN Type Archives ====
==== BIN Types ====These are uncompressed archives. The header is 4 bytes long and gives the length of the file without the header and then the data beyond that.
These are uncompressed archives. The header consists of a 4 byte header that gives the length of the file without the header and then the data beyond that.
==== BIN-GZIP Types Type Archives ====
Unless otherwise noted, these have a 6 byte header. After this are many gziped sections concatenated together.
{| borderclass="0" cellpadding="3" cellspacing="1" style="background: rgb(0,0,0)" align="centerwikitable"! style="background:rgb(204,204,204); width:50px;" align="center" | Offset! style="background:rgb(204,204,204); width:50px;" | Length! style="background:rgb(204,204,204); width:250px;" | Description
|-
|style="background:rgb(255,255,255);" align="center" | 0x0000|style="background:rgb(255,255,255);" | 2 bytes|style="background:rgb(255,255,255);" | Length of gzipped section 1
|-
|style="background:rgb(255,255,204)" align="center" | 0x0002|style="background:rgb(255,255,204);" | 2 bytes|style="background:rgb(255,255,204);" | UnknownLength of ungzipped section 1
|-
|style="background:rgb(255,255,255);" align="center" | 0x0004|style="background:rgb(255,255,255);" | 2 bytes|style="background:rgb(255,255,255);" | File numbertype*
|-
|style="background:rgb(255,255,255);" align="center" | 0x0006|style="background:rgb(255,255,255);" | Varies|style="background:rgb(255,255,255);" | [0x1F8B080000000000...] - Gzip header 1 and data
|-
|style="background:rgb(255,255,255);" align="center" | Varies|style="background:rgb(255,255,255);" | 2 bytes|style="background:rgb(255,255,255);" | Length of gzipped section 2
|-
|style="background:rgb(255,255,204);" align="center" | Varies|style="background:rgb(255,255,204);" | 2bytes|style="background:rgb(255,255,204);" | UnknownLength of ungzipped section 2
|-
|style="background:rgb(255,255,255);" align="center" | Varies|style="background:rgb(255,255,255);" | 2 bytes|style="background:rgb(255,255,255);" | File numbertype*
|-
|style="background:rgb(255,255,255);" align="center" | Varies|style="background:rgb(255,255,255);" | Varies|style="background:rgb(255,255,255);" | [0x1F8B080000000000...] - Gzip header 2 and data
|-
|style="background:rgb(255,255,255);" colspan="3" align="center" | ...
|}
<br />
''*'' This particular value might be ignored by the whatever method is decompressing these archive types. Within archives it declares that the compressed file is a particular type. These values seem to be unique to the particular archive that is being opened and is not consistent between archives.<p />
Example 1: Within the [[FF7/Kernel/Kernel.bin|KERNEL.BIN]] the first nine files are all different data sets so are numbered sequentially 0-8. All remaining files are text types and get labeled as type 9.<br />
Example 2: Within the WINDOW.BIN file there are three files. The first two are type "0" and are textures. The third file is type "1" and not a texture.
=== LZS Archives ===
== LZS Archive ==Under construction. I'm placing a The [[FF7/LZS format|link hereLZS format]]is used throughout the PSX version of Final Fantasy 7, often ending with the .lzs extension. LZS itself stands for Lempel-Ziv-Shannon-Fano, Statistical plus Arithmetic. It was originally developed by [http://oku.edu.mie-u.ac.jp/~okumura/index-e.html Professor Haruhiko Okumura] so I can move things based on the work of [http://www.hpl.hp.com/about/bios/abraham_lempel.html Abraham Lempel] and [http://www.marconifoundation.org/pages/dynamic/fellows/fellow_details.php?roster_id=23 Jacob Ziv]
=== LGP Archive format for PC by [[User:Ficedula|Ficedula]] Archives ===
This section explains how The LGP file format is only used for the LGP PC port of Final Fantasy 7. These are large "volume" type archives from FF7PC are constructed. Therethat hold most of the game's probably no reason why you'd need to know data. These archives can hold thousands of files. Unlike the BIN or LZS type files, this (Plug: Use my archive does reference the data within it by filename. Its file format is explained [http:[FF7//www.ficedula.com LGP Editorformat|here]] !) but the file format might be useful to SOMEBODY.
Essentially the LGP file is split up into four (maybe less, depending on how you count it) sections.== Textures ==
# File header/Table of contents# CRC code# Actual data# File terminator ==== Section 1: File Header ==== This contains two parts: A header of fixed size, then the table of contents. The first item is 12 bytes containing the file creator. This texture is just a standard string, except it picture that is "rightaligned"placed into video memory. In other words It is later manipulated by the blank space comes BEFORE engine and displayed on the actual text, not after. Oh: In FF7 it's always "SQUARESOFT" preceded by two nulls to make it 12 bytesscreen. The only other thing you might see is the header "FICEDULA-LGP", which I use to indicate a file is an LGP *patch* one native format of my programs has constructed, not a complete archive. Next is a four-byte integer saying how many files the archive contains. Following this is the table of contents (TOC): One entry per file. Each entry in texture was the TOC has the following structure: {| border="0" cellpadding="3" cellspacing="1" style="background: rgb(0,0,0)" align="center"! style="background:rgb(204,204,204); width:80px;" align="center" | Offset! style="background:rgb(204,204,204); width:200px;" | Length|Psy-|style="background:rgb(255,255,255);" | 20 bytes|style="background:rgb(255,255,255);" | Null terminated string, giving filename |-|style="background:rgb(255,255,255);" | 4 byte integer|style="background:rgb(255,255,255);" | Position in this file where data starts for the file|-Q [[PSX/TIM format|style="background:rgbTIM]] (255,255,255);" | 3 bytes|style="background:rgb(255,255,204Texture Image Map);" | Some sort of check code. Normally seems to be<br />14,0,0 but it does vary. Unsure about this. |} Simple! ==== Section 2: CRC Code ==== This code is used to validate as the LGP archive. The bad news is I have no idea how to make it (I've figured out how to decode itnative format for the PSX version as well, ie. find out whether the archive is valid ... but I can't create my own)with a few caveats explained below. The good news is you don't need to! The ONLY thing this CRC is based on is the number of files in the archive (maybe the filenames too ... haven't checked that). Anyway, the TOC is the only thing this check relates to. So if you're replicating an archive from FF7 for use in the game with the same number of files and filenames (and what ELSE would you use LGP archives for?) you file can just copy the CRC section from an existing filehold multiple color look up tables. Cheap but effective :) Normally it's 3602 bytes long. I think one archive This was different? Maybe MAGIC.LGP - can't remember. Anyway, one normally-safe way of calculating the CRC size is to find reasons why a video card on the end of the TOC and the beginning of the first file. Anything in between is probably CRC code. (Not guaranteed to work! It works with "official" archives but editors - such as mine - can screw around with the TOC to achieve extra things).  ==== Section 3: Actual Data ==== The PC that could do palleted data from the filesat high color depths was needed. However it's not that simple: the TOC doesn't list how long each file is (somewhat useful!). It's done here. The offset in the TOC is actually the position of yet another file header. Format is:  {| border="0" cellpadding="3" cellspacing="1" style="background: rgb(0,0,0)" align="center"! style="background:rgb(204,204,204); width:80px;" align="center" | Size! style="background:rgb(204,204,204); width:200px;" | Description|-|style="background:rgb(255,255,255);" | 20 bytes|style="background:rgb(255,255,255);" | Null terminated string, giving filename |-|style="background:rgb(255,255,255);" | 4 bytes|style="background:rgb(255,255,255);" | File length|-|style="background:rgb(255,255,255);" | Varies|style="background:rgb(255,255,255);" | The file data itself|}
Simple!=== TIM texture data format for PSX ===
The [[PSX/TIM_format|TIM files]] are found both on raw format and also within several archives, including [[FF7/Kernel/Low_level_libraries#BIN_archive_data_format|BIN]], [[FF7/Kernel/Low_level_libraries#LZS_Archives|LZS]], or even MNU. The format proper has the ability to contain 24 bit bitmaps, but is not used in FF7. The format was created because the PSX does not have direct access to it's VRAM, and must go through the [[PSX/GPU|GPU]] for any graphic access. [[PSX/TIM format|A TIM file]] is a clean way to load a texture and color look up table into VRAM.
==== Section 4: Terminator =TEX texture data format for the PC ===
After the last piece of data comes the file descriptor. This is a simple string, except instead of being null-terminated it's terminated by the end of TEX files are texture files for the filePC. It's "FINAL FANTASY 7" The format for all archives, except LGP patches, where it's "LGP PATCH FILE"these files are located [[FF7/TEX format|here]].
==== Notes ====
The game is remarkably flexible about LGP archives. So long as the TOC and the CRC data is intact it'll accept just about anything.== File formats for 3D models ==
* Example 1: The filename in During the TOC and in development process, 3D models contain a good deal of information needed by the actual file header don't have to match. It only checks artist every time they save or load the TOCmodel. * Example 2: You can point two entries in When the TOC at the same data model is finished, it is often exported and it worksbroken up into smaller files with many unneeded attributes stripped from them. * Example 3: You can have ANY junk in When the models for FF7 were created, they were exported into Psy-Q's 3D library formats. These include [[PSX/RSD|resource data section so long as all the TOC entries point to a valid file header(. Not every piece of RSD)]], polygon data has to be "accounted" for by the TOC(.PLY), polygon groups (.GRP), materials (.MAT), [[PSX/TIM_file|textures (.TIM)]], [[PSX/HRC|skeletal hierarchy (.HRC)]], and animation (. There can be data not usedANM).
My LGP Editor uses this to its advantage The models are handled differently between modules. The models in the Advanced Editor. If you want to replace "battle" modules have a file in an LGP archive with your own copy, it just sticks different animation system than the file on field models. When the end of models were converted to the LGP, writes a new file terminatorPC version, and updates they were taken from the TOC Psy-Q formats to point at the new file. (Advantage: Fast)a more PC-friendly one. It Some are even lets you link two TOC entries to the same data :) or have "inactive" original, uncompiled, Psy-Q files in the archive that aren't referenced by any TOC entry.
I don't know whether the file terminator has to be intact, but === Model formats for safety's sake my editor preserves it. The CRC DEFINTELY has to be present and correct. Also, if you're replacing an archive with you're own custom version make sure it has filenames in the TOC matching the ones in the old one, ne?PSX ===
Oh: The game doesn't check archive sizes so long as all filenames Playstation models are presentstored in the following directories, \ENEMY1 \ENEMY2 \ENEMY3 \ENEMY4 \ENEMY5 \ENEMY6 (battle models), \FIELD (field models and field character models), \MAGIC (Summon magic), and \STAGE1 \STAGE2 (battle scenes). So if you want Battle model names for special characters and party characters are stored in \ENEMY6, you could replace all models of this type end in an archive containing 95 files .LZS extension. The same goes with summon magic used they are stored with there animations etc. in \MAGIC with a 98-file archive.LZS extension. The only exception to this extension is the FIELD models, so long as 95 of those 98 names matched those present in which use the original 95-file archive! (There's no point in doing this - after allextensions BSX and BCX for scene models and character models respectively. The [[FF7/Playstation Battle Model Format|Playstation battle model format]], is different than the game won't use any files OTHER than [[FF7/Field/BSX|Playstation field model format]], also the 95 it's expecting [[FF7/Playstation battle scene format]], is similiar but not identical to find)the [[FF7/Playstation Battle Model Format|Playstation battle model format]]. The [[FF7/Playstation magic model|Playstation magic model]] format is a work in progress.
Other point: I've heard reports on Qhimm's message board that once you've f***ed an archive and the game refuses to read it, it won't EVER read it until you reinstall - even if you fix the problem/restore from a backup. The idea was generally scorned and ignored, but I'll mention it because something like that happened to me. Then again, it COULD have been because I upgraded basically everything in my === Model Formats for PC; so no solid conclusion to be drawn here.===
Further point: (due to changes The PC models are stored in my the LGP Tools/Cosmo programs) Sometimes, there are data "gaps" files in the file that don't appear to be referenced by any file - even by an inactive file/DATA directory. This happens due to The names for the way my programs update archivesmodels were obfuscated a little. If you're only using The data can be found in the TOC method to get at [[PSX/HRC|Hierarchy files (the easy way.HRC) then you won't notice this anyway. However]], if you're stepping through the file header by header, even reading the unused ones, this can cause problems[[PSX/RSD|Resource data files (. If you use my program to update a file with one that's smaller than the original (can happenRSD) then it writes it in]], but leaves a gap after it and [[FF7/P|Polygon files (of course). However, to help you out, after the end of the file, it writes a 4 byte integer saying how much more space to skip over to reach the next file header. This really doesn't affect many things - only tools (like my Advanced LGP EditorP) that bypass the TOC to construct their own file lists. FF7 never notices a thing]].