Difference between revisions of "FF7/LGP format"

From Final Fantasy Inside
< FF7
Jump to navigation Jump to search
(Section 2: Section formerly designated as "CRC Code")
m (Introduction)
 
(2 intermediate revisions by 2 users not shown)
Line 1: Line 1:
===  LGP Archive format for PC by [[User:Ficedula|Ficedula]]  ===
+
== Introduction ==
  
This section explains how the LGP archives from FF7 PC are constructed. If you're looking for a tool that already manages LGP archives, try [[User:Ficedula|Ficedula]]'s [http://sylphds.net/f2k3/index.html LGP Editor].
+
LGP is an archive format used by Final Fantasy VII to store game assets. The format is used in the PC version of the game to package various game resources including textures, models, scripts, and other data files.
  
Essentially the LGP file is split up into four (maybe less, depending on how you count it) sections.
+
An LGP archive consists of a header, table of contents (TOC), hash lookup table, optional path table for directories, and the actual file data. All multi-byte integers in the format are stored in '''little-endian''' format.
  
# File header/Table of contents
+
The archive terminates with the magic string "FINAL FANTASY7" to mark the end of the file.
# CRC code
 
# Actual data
 
# File terminator
 
  
====  Section 1: File Header  ====
+
== File Structure Overview ==
  
This contains two parts: A header of fixed size, then the table of contents.
+
An LGP archive is divided into six sections in the following order:
  
The first item is 12 bytes containing the file creator. This is a standard string, except it is "rightaligned". In other words the blank space comes before the actual text, not after. In FF7 it's always "SQUARESOFT" preceded by two nulls to make it 12 bytes. The only other thing you might see is the header "FICEDULA-LGP", which I use to indicate a file is an LGP *patch* one of my programs has constructed, not a complete archive.
+
{| class="wikitable"
 +
! Section !! Size !! Description
 +
|-
 +
| Header || 16 bytes || Archive metadata and file count
 +
|-
 +
| Table of Contents || 27 bytes × file count || File entries with names and offsets
 +
|-
 +
| Hash Table || 3600 bytes (fixed) || Lookup table for fast file access
 +
|-
 +
| Path Table || Variable || Optional directory paths for files
 +
|-
 +
| File Data || Variable || Actual file contents with individual headers
 +
|-
 +
| Footer || 14 bytes || "FINAL FANTASY7" terminator string
 +
|}
 +
 
 +
== Header ==
 +
 
 +
The header is always 16 bytes and contains the archive's magic identifier and file count.
 +
 
 +
{| class="wikitable"
 +
! Offset !! Size !! Type !! Description
 +
|-
 +
| 0x00 || 2 bytes || uint16 || Reserved (always 0)
 +
|-
 +
| 0x02 || 10 bytes || char[10] || Magic string: "SQUARESOFT" (ASCII, no null terminator)
 +
|-
 +
| 0x0C || 2 bytes || uint16 || Number of files in archive
 +
|-
 +
| 0x0E || 2 bytes || uint16 || Reserved (always 0)
 +
|}
 +
 
 +
'''Validation:''' The magic string at offset 0x02 must exactly match "SQUARESOFT" (ASCII, no null terminator within the 10 bytes).
 +
 
 +
== Table of Contents (TOC) ==
 +
 
 +
Immediately follows the header and contains one 27-byte entry for each file in the archive.
 +
 
 +
{| class="wikitable"
 +
! Offset !! Size !! Type !! Description
 +
|-
 +
| 0x00 || 20 bytes || char[20] || Filename (null-padded, no path)
 +
|-
 +
| 0x14 || 4 bytes || uint32 || Absolute file offset in archive
 +
|-
 +
| 0x18 || 1 byte || uint8 || File type (always 0x0E / 14)
 +
|-
 +
| 0x19 || 2 bytes || uint16 || Path index (0 = no path, 1+ = path table index)
 +
|}
  
Next is a four-byte integer saying how many files the archive contains.
+
'''Notes:'''
 +
* Filenames are stored without directory paths
 +
* Filenames are null-terminated within the 20-byte field
 +
* The offset field points to the start of the File Header for this file
 +
* The path field references the Path Table (1-indexed); 0 means the file is in the root directory
 +
* Maximum filename length is 19 characters plus null terminator
  
Following this is the table of contents (TOC): One entry per file.
+
== Hash Table ==
  
Each entry in the TOC has the following structure:
+
The hash table immediately follows the TOC and is always exactly 3600 bytes (900 entries × 4 bytes). It provides fast O(1) lookup of files by filename.
 +
 
 +
=== Hash Table Entry ===
 +
 
 +
Each entry is 4 bytes:
  
 
{| class="wikitable"
 
{| class="wikitable"
! Offset
+
! Offset !! Size !! Type !! Description
! Length
 
 
|-
 
|-
| 20 bytes
+
| 0x00 || 2 bytes || uint16 || Index into TOC (1-indexed, 0 = empty bucket)
| Null terminated string, giving filename
 
 
|-
 
|-
| 4 byte integer
+
| 0x02 || 2 bytes || uint16 || Count of consecutive entries in this bucket
| Position in this file where data starts for the file
+
|}
 +
 
 +
=== Hash Function ===
 +
 
 +
The hash is computed from the filename (without path or extension) using only the first two characters of the file stem:
 +
 
 +
<pre>
 +
hash_value = hash(first_char) × 30 + hash(second_char) + 1
 +
</pre>
 +
 
 +
For filenames with only one character in the stem:
 +
 
 +
<pre>
 +
hash_value = hash(first_char) × 30
 +
</pre>
 +
 
 +
=== Character Hash Values ===
 +
 
 +
The hash function maps characters to numeric values as follows:
 +
 
 +
{| class="wikitable"
 +
! Character !! Hash Value
 
|-
 
|-
| 1 byte
+
| a-z (case insensitive) || 0-25
| style="background: rgb(255,255,204)" | Some sort of check code. File attributes? Normally seems to be<br />14 but it does vary.
 
 
|-
 
|-
| 2 byte short
+
| 0-9 || 0-9
| style="background: rgb(255,255,204)" | Something to do with duplicate file names. If a name is unique it is 0, otherwise it is assigned a value based on existing duplicates. (Hard to explain)
+
|-
 +
| _ (underscore) || 10 (same as 'k')
 +
|-
 +
| - (hyphen) || 11 (same as 'l')
 
|}
 
|}
  
====  Section 2: Section formerly designated as "CRC Code"  ====
+
'''Note:''' The hash function is case-insensitive, treating 'A' and 'a' identically.
 +
 
 +
=== Lookup Algorithm ===
  
This section is 3600 bytes.  It is 30 sets of 30 entries containing two 16-bit words each (30 x 30 x 2 x 2 = 3600)
+
# Compute hash from the filename (first two characters of stem)
 +
# Read HashTable[hash]
 +
# If index is 0, file not found (empty bucket)
 +
# Search TOC entries starting at index-1 (converting to 0-indexed) for count entries
 +
# Match by exact filename comparison
  
The sets contain file-group information which is based on the first two letters of each file name.
+
'''Example:''' To find "test.dat":
 +
# Compute hash: hash('t') × 30 + hash('e') + 1 = 19 × 30 + 4 + 1 = 575
 +
# Read HashTable[575]: {index: 42, count: 3}
 +
# Search TOC entries 41-43 (0-indexed: index-1)
 +
# Find entry where name == "test.dat"
  
The first letter, minus the value for ascii 'a' (0x61) is the index of the set to which the file belongs.
+
== Path Table ==
  
The second letter, minus the value for ascii ' ' ' (0x60) is the index of the entry within the set. Since the second letter in all of the file names is ascii 'a' or greater, it means that the lowest entry index is 1, so the first entry in every group is always zero (0x0000).
+
The path table immediately follows the hash table and stores directory paths for files. It uses a variable-length structure organized into "path groups" for files that share the same filename but exist in different directories.
  
Each entry is two words.
+
=== Path Table Header ===
  
The first word is the 1-based index of the directory entry for the first file in the set.
+
{| class="wikitable"
 +
! Offset !! Size !! Type !! Description
 +
|-
 +
| 0x00 || 2 bytes || uint16 || Number of path groups
 +
|}
  
The second word is the number of files in the set, most of which are 0x003c (60).  There are a few entries after the bulk which have fewer entries.
+
=== Path Group ===
  
The meaning of these sets and why they're divided in this manner is yet to be determined.
+
Each path group contains all directory paths for files sharing the same filename:
  
There is one 16-bit word with the value of 0 (0x0000) at the end of this data which may belong to this section or the next.
+
{| class="wikitable"
 +
! Offset !! Size !! Type !! Description
 +
|-
 +
| 0x00 || 2 bytes || uint16 || Number of paths in this group
 +
|-
 +
| 0x02 || Variable || Path[] || Array of Path entries
 +
|}
  
====  Section 3: Actual Data  ====
+
=== Path Entry ===
  
The data from the files. However it's not that simple: the TOC doesn't list how long each file is (somewhat useful). It's done here. The offset in the TOC is actually the position of yet another file header. Format is:
+
Each path entry is 130 bytes:
  
 
{| class="wikitable"
 
{| class="wikitable"
! style="background: rgb(204,204,204); width: 80px" align="center" | Size
+
! Offset !! Size !! Type !! Description
! style="background: rgb(204,204,204); width: 200px" | Description
 
 
|-
 
|-
| 20 bytes
+
| 0x00 || 128 bytes || char[128] || Directory path (null-padded, no trailing slash)
| Null terminated string, giving filename
+
|-
 +
| 0x80 || 2 bytes || uint16 || TOC index this path belongs to
 +
|}
 +
 
 +
'''Notes:'''
 +
* Path index in TOC entries is 1-indexed into path groups
 +
* Multiple files with the same name but different paths share a path group
 +
* Empty path string means root directory
 +
* Maximum path length is 127 characters plus null terminator
 +
 
 +
== File Data ==
 +
 
 +
File data blocks follow the path table. Each file consists of a 24-byte header followed by the raw file content.
 +
 
 +
=== File Header ===
 +
 
 +
{| class="wikitable"
 +
! Offset !! Size !! Type !! Description
 +
|-
 +
| 0x00 || 20 bytes || char[20] || Filename (same as in TOC)
 +
|-
 +
| 0x14 || 4 bytes || uint32 || File size in bytes
 +
|}
 +
 
 +
=== File Content ===
 +
 
 +
Immediately follows the file header. The size is specified in the header's file size field.
 +
 
 +
{| class="wikitable"
 +
! Offset !! Size !! Type !! Description
 
|-
 
|-
| 4 bytes
+
| 0x00 || file_size || byte[] || Raw file data
| File length
+
|}
 +
 
 +
== Footer ==
 +
 
 +
The archive ends with a 14-byte terminator string:
 +
 
 +
{| class="wikitable"
 +
! Offset !! Size !! Type !! Description
 
|-
 
|-
| Varies
+
| 0x00 || 14 bytes || char[14] || "FINAL FANTASY7" (no null terminator)
| The file data itself
 
 
|}
 
|}
  
====  Section 4: Terminator  ====
+
== Reading Algorithm ==
  
After the last piece of data comes the file descriptor. This is a simple string, except instead of being null-terminated it's terminated by the end of the file. It's "FINAL FANTASY 7" for all archives, except LGP patches, where it's "LGP PATCH FILE".
+
To read an LGP archive:
 +
 
 +
# '''Read Header''' - Validate magic string "SQUARESOFT" and extract file count
 +
# '''Read TOC''' - Read file_count entries of 27 bytes each, storing filename, offset, type, and path index
 +
# '''Skip Hash Table''' - Skip 3600 bytes (or read for validation/fast lookup)
 +
# '''Read Path Table''' - Read path group count, then for each group read path count and path entries
 +
# '''Resolve Full Paths''' - For each TOC entry with path index > 0, look up path group at path_index-1, find path entry matching TOC index, and prepend path to filename
 +
# '''Read File Data''' (on demand) - Seek to TOC entry's offset, read 24-byte file header, read file_size bytes of content
 +
 
 +
== Writing Algorithm ==
 +
 
 +
To create an LGP archive:
 +
 
 +
# '''Build Metadata''' - Compute hash for each file, group files by hash bucket, identify files needing path entries
 +
# '''Write Header''' - Write magic string, file count, and reserved fields
 +
# '''Write TOC''' - Write entries with placeholder offsets (will be updated later)
 +
# '''Write Hash Table''' - Populate 900 entries based on file hash groupings
 +
# '''Write Path Table''' - Group paths by filename collisions and write path groups
 +
# '''Write File Data''' - For each file write header and content, recording actual offset
 +
# '''Update TOC''' - Seek back to TOC section and update offsets with actual values
 +
# '''Write Footer''' - Append "FINAL FANTASY7" terminator
 +
 
 +
== Size Limits ==
 +
 
 +
The LGP format has the following technical limitations:
 +
 
 +
{| class="wikitable"
 +
! Limit !! Maximum Value !! Reason
 +
|-
 +
| Files per archive || 65,535 || uint16 in header
 +
|-
 +
| File size || 4 GB || uint32 in file header
 +
|-
 +
| Archive size || 4 GB || uint32 offsets in TOC
 +
|-
 +
| Filename length || 19 characters || 20 bytes with null terminator
 +
|-
 +
| Path length || 127 characters || 128 bytes with null terminator
 +
|-
 +
| Hash buckets || 900 || Fixed 30 × 30 grid (letters × letters)
 +
|}
  
====  Notes  ====
+
==  Notes  ==
  
 
The game is remarkably flexible about LGP archives. So long as the TOC and the CRC data is intact it'll accept just about anything.
 
The game is remarkably flexible about LGP archives. So long as the TOC and the CRC data is intact it'll accept just about anything.

Latest revision as of 02:29, 14 December 2025

Introduction

LGP is an archive format used by Final Fantasy VII to store game assets. The format is used in the PC version of the game to package various game resources including textures, models, scripts, and other data files.

An LGP archive consists of a header, table of contents (TOC), hash lookup table, optional path table for directories, and the actual file data. All multi-byte integers in the format are stored in little-endian format.

The archive terminates with the magic string "FINAL FANTASY7" to mark the end of the file.

File Structure Overview

An LGP archive is divided into six sections in the following order:

Section Size Description
Header 16 bytes Archive metadata and file count
Table of Contents 27 bytes × file count File entries with names and offsets
Hash Table 3600 bytes (fixed) Lookup table for fast file access
Path Table Variable Optional directory paths for files
File Data Variable Actual file contents with individual headers
Footer 14 bytes "FINAL FANTASY7" terminator string

Header

The header is always 16 bytes and contains the archive's magic identifier and file count.

Offset Size Type Description
0x00 2 bytes uint16 Reserved (always 0)
0x02 10 bytes char[10] Magic string: "SQUARESOFT" (ASCII, no null terminator)
0x0C 2 bytes uint16 Number of files in archive
0x0E 2 bytes uint16 Reserved (always 0)

Validation: The magic string at offset 0x02 must exactly match "SQUARESOFT" (ASCII, no null terminator within the 10 bytes).

Table of Contents (TOC)

Immediately follows the header and contains one 27-byte entry for each file in the archive.

Offset Size Type Description
0x00 20 bytes char[20] Filename (null-padded, no path)
0x14 4 bytes uint32 Absolute file offset in archive
0x18 1 byte uint8 File type (always 0x0E / 14)
0x19 2 bytes uint16 Path index (0 = no path, 1+ = path table index)

Notes:

  • Filenames are stored without directory paths
  • Filenames are null-terminated within the 20-byte field
  • The offset field points to the start of the File Header for this file
  • The path field references the Path Table (1-indexed); 0 means the file is in the root directory
  • Maximum filename length is 19 characters plus null terminator

Hash Table

The hash table immediately follows the TOC and is always exactly 3600 bytes (900 entries × 4 bytes). It provides fast O(1) lookup of files by filename.

Hash Table Entry

Each entry is 4 bytes:

Offset Size Type Description
0x00 2 bytes uint16 Index into TOC (1-indexed, 0 = empty bucket)
0x02 2 bytes uint16 Count of consecutive entries in this bucket

Hash Function

The hash is computed from the filename (without path or extension) using only the first two characters of the file stem:

hash_value = hash(first_char) × 30 + hash(second_char) + 1

For filenames with only one character in the stem:

hash_value = hash(first_char) × 30

Character Hash Values

The hash function maps characters to numeric values as follows:

Character Hash Value
a-z (case insensitive) 0-25
0-9 0-9
_ (underscore) 10 (same as 'k')
- (hyphen) 11 (same as 'l')

Note: The hash function is case-insensitive, treating 'A' and 'a' identically.

Lookup Algorithm

  1. Compute hash from the filename (first two characters of stem)
  2. Read HashTable[hash]
  3. If index is 0, file not found (empty bucket)
  4. Search TOC entries starting at index-1 (converting to 0-indexed) for count entries
  5. Match by exact filename comparison

Example: To find "test.dat":

  1. Compute hash: hash('t') × 30 + hash('e') + 1 = 19 × 30 + 4 + 1 = 575
  2. Read HashTable[575]: {index: 42, count: 3}
  3. Search TOC entries 41-43 (0-indexed: index-1)
  4. Find entry where name == "test.dat"

Path Table

The path table immediately follows the hash table and stores directory paths for files. It uses a variable-length structure organized into "path groups" for files that share the same filename but exist in different directories.

Path Table Header

Offset Size Type Description
0x00 2 bytes uint16 Number of path groups

Path Group

Each path group contains all directory paths for files sharing the same filename:

Offset Size Type Description
0x00 2 bytes uint16 Number of paths in this group
0x02 Variable Path[] Array of Path entries

Path Entry

Each path entry is 130 bytes:

Offset Size Type Description
0x00 128 bytes char[128] Directory path (null-padded, no trailing slash)
0x80 2 bytes uint16 TOC index this path belongs to

Notes:

  • Path index in TOC entries is 1-indexed into path groups
  • Multiple files with the same name but different paths share a path group
  • Empty path string means root directory
  • Maximum path length is 127 characters plus null terminator

File Data

File data blocks follow the path table. Each file consists of a 24-byte header followed by the raw file content.

File Header

Offset Size Type Description
0x00 20 bytes char[20] Filename (same as in TOC)
0x14 4 bytes uint32 File size in bytes

File Content

Immediately follows the file header. The size is specified in the header's file size field.

Offset Size Type Description
0x00 file_size byte[] Raw file data

Footer

The archive ends with a 14-byte terminator string:

Offset Size Type Description
0x00 14 bytes char[14] "FINAL FANTASY7" (no null terminator)

Reading Algorithm

To read an LGP archive:

  1. Read Header - Validate magic string "SQUARESOFT" and extract file count
  2. Read TOC - Read file_count entries of 27 bytes each, storing filename, offset, type, and path index
  3. Skip Hash Table - Skip 3600 bytes (or read for validation/fast lookup)
  4. Read Path Table - Read path group count, then for each group read path count and path entries
  5. Resolve Full Paths - For each TOC entry with path index > 0, look up path group at path_index-1, find path entry matching TOC index, and prepend path to filename
  6. Read File Data (on demand) - Seek to TOC entry's offset, read 24-byte file header, read file_size bytes of content

Writing Algorithm

To create an LGP archive:

  1. Build Metadata - Compute hash for each file, group files by hash bucket, identify files needing path entries
  2. Write Header - Write magic string, file count, and reserved fields
  3. Write TOC - Write entries with placeholder offsets (will be updated later)
  4. Write Hash Table - Populate 900 entries based on file hash groupings
  5. Write Path Table - Group paths by filename collisions and write path groups
  6. Write File Data - For each file write header and content, recording actual offset
  7. Update TOC - Seek back to TOC section and update offsets with actual values
  8. Write Footer - Append "FINAL FANTASY7" terminator

Size Limits

The LGP format has the following technical limitations:

Limit Maximum Value Reason
Files per archive 65,535 uint16 in header
File size 4 GB uint32 in file header
Archive size 4 GB uint32 offsets in TOC
Filename length 19 characters 20 bytes with null terminator
Path length 127 characters 128 bytes with null terminator
Hash buckets 900 Fixed 30 × 30 grid (letters × letters)

Notes

The game is remarkably flexible about LGP archives. So long as the TOC and the CRC data is intact it'll accept just about anything.

  • Example 1: The filename in the TOC and in the actual file header don't have to match. It only checks the TOC.
  • Example 2: You can point two entries in the TOC at the same data and it works.
  • Example 3: You can have ANY junk in the data section so long as all the TOC entries point to a valid file header. Not every piece of data has to be "accounted" for by the TOC. There can be data not used.

LGP Editor uses this to its advantage in the Advanced Editor. If you want to replace a file in an LGP archive with your own copy, it just puts the file on the end of the LGP, writes a new file terminator, and updates the TOC to point at the new file. It even lets you link two TOC entries to the same data or have "inactive" files in the archive that aren't referenced by any TOC entry.

I don't know whether the file terminator has to be intact, but for safety's sake my editor preserves it. The CRC must be present and correct. Also, if you're replacing an archive with you're own custom version make sure it has filenames in the TOC matching the ones in the old one.

The game doesn't check archive sizes as long as all filenames are present. So if you want, you could replace an archive containing 95 files with a 98-file archive, so long as 95 of those 98 names matched those present in the original 95-file archive. (However there's no point in doing this when the game won't use any files other than the 95 it's expecting to find).

There are reports on Qhimm's board that once you've altered an archive and the game refuses to read it, it won't ever read it until you reinstall - even if you fix the problem/restore from a backup. The idea was generally scorned and ignored, but I'll mention it because something like that happened to me. No solid conclusion can be drawn here.

Sometimes, there are data "gaps" in the file that don't appear to be referenced by any file - even by an inactive file. If you're only using the TOC method to get at files (the easy way) then you won't notice this anyway. However, if you're stepping through the file header by header, even reading the unused ones, this can cause problems. If you use my program to update a file with one that's smaller than the original (can happen) then it writes it in, but leaves a gap after it (of course). However, to help you out, after the end of the file, it writes a 4 byte integer saying how much more space to skip over to reach the next file header. This really doesn't affect many things - only tools (like my Advanced LGP Editor) that bypass the TOC to construct their own file lists. FF7 never notices a thing.

Useful downloads

Below there are links to known programs that are capable to edit LGP archives:

  • LGP Tools - with an Advanced LGP Editor allowing edit archive thoughoutly
  • Emerald - has mass extracting/repacking function
  • Unmass - general file extractor with LGP archives support