Difference between revisions of "FF7/Kernel/Low level libraries"

From Final Fantasy Inside
< FF7‎ | Kernel
Jump to navigation Jump to search
my_wiki>Halkun
(LZS Archive)
my_wiki>Halkun
Line 67: Line 67:
  
 
=== LZS Archive ===
 
=== LZS Archive ===
Under construction. I'm placing a [[FF7/LZS format|link here]] so I can move things about.
+
Under construction. I'm placing a [[FF7/LZS format|LZS link here]] so I can move things about.
  
 
+
=== LGP Archive ===
 
+
Under construction. I'm placing a [[FF7/LGP format|LGP link here]] so I can move things about.
=== LGP Archive format for PC by [[User:Ficedula|Ficedula]] ===
 
 
 
This section explains how the LGP archives from FF7PC are constructed. There's probably no reason why you'd need to know this (Plug: Use my [http://www.ficedula.com LGP Editor] !) but the file format might be useful to SOMEBODY.
 
 
 
Essentially the LGP file is split up into four (maybe less, depending on how you count it) sections.
 
 
 
# File header/Table of contents
 
# CRC code
 
# Actual data
 
# File terminator
 
 
 
==== Section 1: File Header ====
 
 
 
This contains two parts: A header of fixed size, then the table of contents.
 
 
 
The first item is 12 bytes containing the file creator. This is a standard string, except it is "rightaligned". In other words the blank space comes BEFORE the actual text, not after. Oh: In FF7 it's always "SQUARESOFT" preceded by two nulls to make it 12 bytes. The only other thing you might see is the header "FICEDULA-LGP", which I use to indicate a file is an LGP *patch* one of my programs has constructed, not a complete archive.
 
 
 
Next is a four-byte integer saying how many files the archive contains.
 
 
 
Following this is the table of contents (TOC): One entry per file.
 
 
 
Each entry in the TOC has the following structure:
 
 
 
{| border="0" cellpadding="3" cellspacing="1" style="background: rgb(0,0,0)" align="center"
 
! style="background:rgb(204,204,204); width:80px;" align="center" | Offset
 
! style="background:rgb(204,204,204); width:200px;" | Length
 
|-
 
|style="background:rgb(255,255,255);" | 20 bytes
 
|style="background:rgb(255,255,255);" | Null terminated string, giving filename
 
|-
 
|style="background:rgb(255,255,255);" | 4 byte integer
 
|style="background:rgb(255,255,255);" | Position in this file where data starts for the file
 
|-
 
|style="background:rgb(255,255,255);" | 3 bytes
 
|style="background:rgb(255,255,204);" | Some sort of check code. Normally seems to be<br />14,0,0 but it does vary. Unsure about this.
 
|}
 
 
 
Simple!
 
 
 
==== Section 2: CRC Code ====
 
 
 
This code is used to validate the LGP archive. The bad news is I have no idea how to make it (I've figured out how to decode it, ie. find out whether the archive is valid ... but I can't create my own). The good news is you don't need to! The ONLY thing this CRC is based on is the number of files in the archive (maybe the filenames too ... haven't checked that). Anyway, the TOC is the only thing this check relates to. So if you're replicating an archive from FF7 for use in the game with the same number of files and filenames (and what ELSE would you use LGP archives for?) you can just copy the CRC section from an existing file. Cheap but effective :)
 
 
 
Normally it's 3602 bytes long. I think one archive was different? Maybe MAGIC.LGP - can't remember. Anyway, one normally-safe way of calculating the CRC size is to find the end of the TOC and the beginning of the first file. Anything in between is probably CRC code. (Not guaranteed to work! It works with "official" archives but editors - such as mine - can screw around with the TOC to achieve extra things).
 
 
 
==== Section 3: Actual Data ====
 
 
 
The data from the files. However it's not that simple: the TOC doesn't list how long each file is (somewhat useful!). It's done here. The offset in the TOC is actually the position of yet another file header. Format is:
 
 
 
{| border="0" cellpadding="3" cellspacing="1" style="background: rgb(0,0,0)" align="center"
 
! style="background:rgb(204,204,204); width:80px;" align="center" | Size
 
! style="background:rgb(204,204,204); width:200px;" | Description
 
|-
 
|style="background:rgb(255,255,255);" | 20 bytes
 
|style="background:rgb(255,255,255);" | Null terminated string, giving filename
 
|-
 
|style="background:rgb(255,255,255);" | 4 bytes
 
|style="background:rgb(255,255,255);" | File length
 
|-
 
|style="background:rgb(255,255,255);" | Varies
 
|style="background:rgb(255,255,255);" | The file data itself
 
|}
 
 
 
Simple!
 
 
 
 
 
==== Section 4: Terminator ====
 
 
 
After the last piece of data comes the file descriptor. This is a simple string, except instead of being null-terminated it's terminated by the end of the file. It's "FINAL FANTASY 7" for all archives, except LGP patches, where it's "LGP PATCH FILE".
 
 
 
==== Notes ====
 
 
 
The game is remarkably flexible about LGP archives. So long as the TOC and the CRC data is intact it'll accept just about anything.
 
 
 
* Example 1: The filename in the TOC and in the actual file header don't have to match. It only checks the TOC.
 
* Example 2: You can point two entries in the TOC at the same data and it works.
 
* Example 3: You can have ANY junk in the data section so long as all the TOC entries point to a valid file header. Not every piece of data has to be "accounted" for by the TOC. There can be data not used.
 
 
 
My LGP Editor uses this to its advantage in the Advanced Editor. If you want to replace a file in an LGP archive with your own copy, it just sticks the file on the end of the LGP, writes a new file terminator, and updates the TOC to point at the new file. (Advantage: Fast). It even lets you link two TOC entries to the same data :) or have "inactive" files in the archive that aren't referenced by any TOC entry.
 
 
 
I don't know whether the file terminator has to be intact, but for safety's sake my editor preserves it. The CRC DEFINTELY has to be present and correct. Also, if you're replacing an archive with you're own custom version make sure it has filenames in the TOC matching the ones in the old one, ne?
 
 
 
Oh: The game doesn't check archive sizes so long as all filenames are present. So if you want, you could replace an archive containing 95 files with a 98-file archive, so long as 95 of those 98 names matched those present in the original 95-file archive! (There's no point in doing this - after all, the game won't use any files OTHER than the 95 it's expecting to find).
 
 
 
Other point: I've heard reports on Qhimm's message board that once you've f***ed an archive and the game refuses to read it, it won't EVER read it until you reinstall - even if you fix the problem/restore from a backup. The idea was generally scorned and ignored, but I'll mention it because something like that happened to me. Then again, it COULD have been because I upgraded basically everything in my PC; so no solid conclusion to be drawn here.
 
 
 
Further point: (due to changes in my LGP Tools/Cosmo programs) Sometimes, there are data "gaps" in the file that don't appear to be referenced by any file - even by an inactive file. This happens due to the way my programs update archives. If you're only using the TOC method to get at files (the easy way) then you won't notice this anyway. However, if you're stepping through the file header by header, even reading the unused ones, this can cause problems. If you use my program to update a file with one that's smaller than the original (can happen) then it writes it in, but leaves a gap after it (of course). However, to help you out, after the end of the file, it writes a 4 byte integer saying how much more space to skip over to reach the next file header. This really doesn't affect many things - only tools (like my Advanced LGP Editor) that bypass the TOC to construct their own file lists. FF7 never notices a thing.
 

Revision as of 10:14, 11 March 2005

PC to PSX comparison

The files and data formats used in the PSX version of FF7 and it's PC port are conceptually the same thing, and accomplish the same tasks. That being said, they both have wildly different formats. Both ofwhich were derived from a third original format that is also somewhat different that the first two.

The original PSX FF7 was created in part using Sony's Psy-Q development library. This library usescommon formats that are "native" to the PSX. Often times a toolkit was used to convert commondevelopment- based formats, such as a TGA bitmap or a palleted GIF file, to something a little moresuited to Psy-Q, which would be a TIM file.

During the porting process to the PC, some of the original artwork, (and artists for that matter), wereno longer available. This resulted in the port team having to use the Psy-Q versions of many files,which were ill suited for the PC architecture. In our example, the TIM file was converted to a TEX file,which would be manipulated in the PC's video memory a little more efficiently. Sometimes theoriginal artwork was available, such as the pictures of the characters within the menu, or the original MIDI files. Most often times it was not.

To make things a little more confusing, both systems also archive their data files in different ways,making the extraction and rendering of each file a bit of a bear. For the most part the data within eachfile is the same thing, just a little switched around. This manual will cover the more generic files first,and then common files used in each module.

Data Archives

To save space, quicken access time, and to obfuscate the file structure a little, most of the data files are stored in some kind of archive format. The archives remove such useful items as subdirectories and logical data placement. There is no real "native" format these are based on.

BIN archive data format

The BIN format comes as two different types. They both have the same extension, so one must open the file to see which format is which. They are best described as BIN Types and Bin-GZIP types

BIN Types

These are uncompressed archives. The header consists of a 4 byte header that gives the length of the file without the header and then the data beyond that.

BIN-GZIP Types

Unless otherwise noted, these have a 6 byte header. After this are many gziped sections concatenated together.

Offset Length Description
0x0000 2 bytes Length of gzipped section 1
0x0002 2 bytes Unknown
0x0004 2 bytes File number
0x0006 Varies [0x1F8B080000000000...] - Gzip header 1 and data
Varies 2 bytes Length of gzipped section 2
Varies 2 Unknown
Varies 2 bytes File number
Varies Varies [0x1F8B080000000000...] - Gzip header 2 and data
...


LZS Archive

Under construction. I'm placing a LZS link here so I can move things about.

LGP Archive

Under construction. I'm placing a LGP link here so I can move things about.