FF7/LZS format - Revision history

BukTop: Redirected page to FF7/LZSS format

2019-07-14T22:01:51Z

Redirected page to FF7/LZSS format

BukTop: 1 revision imported

2019-07-14T22:00:41Z

1 revision imported

BukTop: 7 revisions imported

2019-05-23T02:19:30Z

7 revisions imported

my_wiki>Nax: Nax moved page FF7/LZS format to FF7/LZSS format: The format is really LZSS and not LZS, despite the extension (probably to make it 3 letters)

2019-04-23T08:48:43Z

Nax moved page FF7/LZS format to FF7/LZSS format: The format is really LZSS and not LZS, despite the extension (probably to make it 3 letters)

my_wiki>BukTop: 5 revisions imported

2018-04-04T09:34:50Z

5 revisions imported

my_wiki>Albeoris: Created page with "=== LZS Compressed archive for PSX by Ficedula === ==== Format ==== The LZS archive has a very small header at 0x00 that has the length of the compress..."

2018-03-23T23:41:40Z

Created page with "=== LZS Compressed archive for PSX by Ficedula === ==== Format ==== The LZS archive has a very small header at 0x00 that has the length of the compress..."

my_wiki>NFITC1: /* Format */

2009-03-09T20:45:53Z

‎Format

my_wiki>NFITC1: /* Example */

2008-11-17T16:29:17Z

‎Example

my_wiki>NFITC1: /* Example */

2008-11-17T16:28:57Z

‎Example

my_wiki>Synergy Blades: Minor formalisation

2005-03-11T10:28:33Z

Minor formalisation

← Older revision		Revision as of 22:01, 14 July 2019
Line 1:		Line 1:
−	~~=== LZS Compressed archive for PSX by~~ [[~~User:Ficedula\|Ficedula]] ===~~	+	#REDIRECT [[FF7/LZSS format]]
−
−	~~==== Format ====~~
−
−	~~The LZS archive has a very small header at 0x00 that has the length of the compressed file as an unsigned 32 bit integer. After that is the compressed data.~~
−
−	~~==== LZS compression ====~~
−
−	FF7 uses LZS compression on some of their files - more properly, a slightly modified version of LZSS compression as devised by Professor Haruhiko Okumura. LZS data works on a control byte scheme. So each block in the file begins with a single byte indicating how much of the block is uncompressed ('literal data'), and how much is compressed ('references'). You read the byte right-to-left, with 1=literal, 0=reference.
−
−	~~Literal data means just that: read one byte in from the source (compressed) data, and write it straight to the output.~~
−
−	~~References take up two bytes, and are essentially a pointer to a piece of data that's been written out (i.e. is part of the data you've already decompressed).~~ LZSS ~~uses a 4K buffer, so it can only reference data in the last 4K of data.~~
−
−	~~==== Reference~~ format ~~====~~
−
−	A reference takes up two bytes, and has two pieces of information in it: offset (where to find the data, or which piece of data is going to be repeated), and length (how long the piece of data is going to be). The two reference bytes look like this:
−
−	~~OOOO OOOO OOOO LLLL~~
−
−	~~(O = Offset, L = Length)~~
−
−	So you get a 12-bit offset and a 4-bit length, but both of these values need modifying to work on directly. The length is easy to work with: just add 3 to it. This is because if a piece of repeated data was less than 3 bytes long, you wouldn't bother repeating it - it'd take up no more space to actually just put literal data in. So all references are at least 3 in length. So a length of 0 means 3 bytes repeated, 1 means 4 bytes repeated, so on.
−
−	Since we have 4 bits available, that gives us a final length ranging from 3-18 bytes long. That also means the absolute maximum compression we can ever get using LZSS is a touch under 9:1, since the best possible is to replace 18 bytes of data with two bytes of reference, and then you have to add control bytes as well.
−
−	Offset needs a bit work doing on it, depending on how you're actually holding your data. If all you have is an input buffer and an output buffer, what you really need is an output position in your buffer to start reading data from. In other words, if you've already written 10,000 bytes to your output, you want to know where to retrieve the repeated data from - it could fall anywhere in the past 4K of data (i.e. from 5904 through to 9999 bytes).
−
−	~~Here's how you get it:~~
−
−	~~real_offset = tail - ((tail - 18 - raw_offset) mod 4096)~~
−
−	Here, 'tail' is your current output position (eg. 10,000), 'raw_offset' is the 12-bit data value you've retrieved from the compressed reference, and 'real_offset' is the position in your output buffer you can begin reading from. This is a bit complex because it's not exactly the way LZSS traditionally does (de)compression; it uses a 4K circular buffer; if you do that, the offset is more or less usable directly.
−
−	~~Once you've got to the start position for your reference, you just copy the appropriate length of data over to your output, and you've dealt with that piece of data.~~
−
−	~~==== Example ====~~
−
−	~~If we're at position 1000 in our output, and we need to read in a new control byte because we've finished with the last one. The next data to look it is:~~
−
−	~~0xFC, 0x53, 0x12 .....~~
−
−	We read in a control byte: 0xFC. In binary, that's 11111100. That informs us that the current block of data has two compressed offsets (@ 2 bytes each), followed by 6 literal data bytes. Once we'd read in the next 10 bytes (the compressed data plus the literal data), we'd be ready to read in our next control byte and start again.
−
−	Looking at the first compressed reference, we read in $53 $12. That gives us a base offset of $153 (the 53 from the first byte, and the '1' from the second byte makes up the higher nybble). The base length is $2 (we just take the low nybble of the second byte).
−
−	~~Our final length is obviously just 5.~~
−
−	~~Our position in output is still 1000. So our final offset is:~~
−
−	~~<nowiki>= 1000 - ((1000 - 18 - 339) and $FFF)~~
−	~~</nowiki>~~
−
−	~~The 339 is just $153 in decimal. The (and $FFF) is a quick way to do modulus 4096.~~
−
−	~~<nowiki>= 1000 - (643 and 0xFFF)~~
−	~~= 1000 - 643~~
−	~~= 357~~
−	~~</nowiki>~~
−
−	So our final offset is 357. We go to position 357 in our output data, read in 5 bytes (remember the length?), then write those 5 bytes out to our output. Now we're ready to read in the next bit of data (another compressed reference), and do the procedure again.
−
−	~~==== Complications ====~~
−
−	~~Unfortunately, that doesn't quite cover everything - there's two more things to be aware of when decompressing data that will ruin you when using FF7 files, since they do use these features.~~
−
−	First, if you end up with an negative offset, i.e. reading data from 'before the beginning of the file', write out nulls (zero bytes). That's because the compression buffer is, by default, initialized to zeros; so it's possible, if the start of the file contains a run of zeros, that the file may reference a block you haven't written. For example, if you're at position 50 in your output, it's possible you may get an offset indicating to go back 60 bytes to offset -10. If you have to read 5 bytes from there, you just write out 5 nulls. However, you could have to read 15 bytes from there. In that case, you write out 10 nulls (the part of the data 'before' the file start), then the 5 bytes from the beginning of the file.
−
−	Secondly, you can have a repeated run. This is almost the opposite problem: when you go off the end of your output. Say you're at offset 100 in your output, and you have to go to offset 95 to read in a reference. This is okay, but if the reference length is >5, you loop the output. So if you had to write out 15 bytes, you'd write out the five bytes that were available, then write them out again, then again, to make up the 15 bytes you needed.
−
−	~~The FF7 files use both of these 'tricks', so you can't ignore them.~~

← Older revision		Revision as of 02:19, 23 May 2019
Line 1:		Line 1:
−	~~#REDIRECT~~ [[FF7/LZSS format]]	+	=== LZS Compressed archive for PSX by [[User:Ficedula\|Ficedula]] ===
		+
		+	==== Format ====
		+
		+	The LZS archive has a very small header at 0x00 that has the length of the compressed file as an unsigned 32 bit integer. After that is the compressed data.
		+
		+	==== LZS compression ====
		+
		+	FF7 uses LZS compression on some of their files - more properly, a slightly modified version of LZSS compression as devised by Professor Haruhiko Okumura. LZS data works on a control byte scheme. So each block in the file begins with a single byte indicating how much of the block is uncompressed ('literal data'), and how much is compressed ('references'). You read the byte right-to-left, with 1=literal, 0=reference.
		+
		+	Literal data means just that: read one byte in from the source (compressed) data, and write it straight to the output.
		+
		+	References take up two bytes, and are essentially a pointer to a piece of data that's been written out (i.e. is part of the data you've already decompressed). LZSS uses a 4K buffer, so it can only reference data in the last 4K of data.
		+
		+	==== Reference format ====
		+
		+	A reference takes up two bytes, and has two pieces of information in it: offset (where to find the data, or which piece of data is going to be repeated), and length (how long the piece of data is going to be). The two reference bytes look like this:
		+
		+	OOOO OOOO OOOO LLLL
		+
		+	(O = Offset, L = Length)
		+
		+	So you get a 12-bit offset and a 4-bit length, but both of these values need modifying to work on directly. The length is easy to work with: just add 3 to it. This is because if a piece of repeated data was less than 3 bytes long, you wouldn't bother repeating it - it'd take up no more space to actually just put literal data in. So all references are at least 3 in length. So a length of 0 means 3 bytes repeated, 1 means 4 bytes repeated, so on.
		+
		+	Since we have 4 bits available, that gives us a final length ranging from 3-18 bytes long. That also means the absolute maximum compression we can ever get using LZSS is a touch under 9:1, since the best possible is to replace 18 bytes of data with two bytes of reference, and then you have to add control bytes as well.
		+
		+	Offset needs a bit work doing on it, depending on how you're actually holding your data. If all you have is an input buffer and an output buffer, what you really need is an output position in your buffer to start reading data from. In other words, if you've already written 10,000 bytes to your output, you want to know where to retrieve the repeated data from - it could fall anywhere in the past 4K of data (i.e. from 5904 through to 9999 bytes).
		+
		+	Here's how you get it:
		+
		+	real_offset = tail - ((tail - 18 - raw_offset) mod 4096)
		+
		+	Here, 'tail' is your current output position (eg. 10,000), 'raw_offset' is the 12-bit data value you've retrieved from the compressed reference, and 'real_offset' is the position in your output buffer you can begin reading from. This is a bit complex because it's not exactly the way LZSS traditionally does (de)compression; it uses a 4K circular buffer; if you do that, the offset is more or less usable directly.
		+
		+	Once you've got to the start position for your reference, you just copy the appropriate length of data over to your output, and you've dealt with that piece of data.
		+
		+	==== Example ====
		+
		+	If we're at position 1000 in our output, and we need to read in a new control byte because we've finished with the last one. The next data to look it is:
		+
		+	0xFC, 0x53, 0x12 .....
		+
		+	We read in a control byte: 0xFC. In binary, that's 11111100. That informs us that the current block of data has two compressed offsets (@ 2 bytes each), followed by 6 literal data bytes. Once we'd read in the next 10 bytes (the compressed data plus the literal data), we'd be ready to read in our next control byte and start again.
		+
		+	Looking at the first compressed reference, we read in $53 $12. That gives us a base offset of $153 (the 53 from the first byte, and the '1' from the second byte makes up the higher nybble). The base length is $2 (we just take the low nybble of the second byte).
		+
		+	Our final length is obviously just 5.
		+
		+	Our position in output is still 1000. So our final offset is:
		+
		+	<nowiki>= 1000 - ((1000 - 18 - 339) and $FFF)
		+	</nowiki>
		+
		+	The 339 is just $153 in decimal. The (and $FFF) is a quick way to do modulus 4096.
		+
		+	<nowiki>= 1000 - (643 and 0xFFF)
		+	= 1000 - 643
		+	= 357
		+	</nowiki>
		+
		+	So our final offset is 357. We go to position 357 in our output data, read in 5 bytes (remember the length?), then write those 5 bytes out to our output. Now we're ready to read in the next bit of data (another compressed reference), and do the procedure again.
		+
		+	==== Complications ====
		+
		+	Unfortunately, that doesn't quite cover everything - there's two more things to be aware of when decompressing data that will ruin you when using FF7 files, since they do use these features.
		+
		+	First, if you end up with an negative offset, i.e. reading data from 'before the beginning of the file', write out nulls (zero bytes). That's because the compression buffer is, by default, initialized to zeros; so it's possible, if the start of the file contains a run of zeros, that the file may reference a block you haven't written. For example, if you're at position 50 in your output, it's possible you may get an offset indicating to go back 60 bytes to offset -10. If you have to read 5 bytes from there, you just write out 5 nulls. However, you could have to read 15 bytes from there. In that case, you write out 10 nulls (the part of the data 'before' the file start), then the 5 bytes from the beginning of the file.
		+
		+	Secondly, you can have a repeated run. This is almost the opposite problem: when you go off the end of your output. Say you're at offset 100 in your output, and you have to go to offset 95 to read in a reference. This is okay, but if the reference length is >5, you loop the output. So if you had to write out 15 bytes, you'd write out the five bytes that were available, then write them out again, then again, to make up the 15 bytes you needed.
		+
		+	The FF7 files use both of these 'tricks', so you can't ignore them.

← Older revision		Revision as of 08:48, 23 April 2019
Line 1:		Line 1:
−	~~=== LZS Compressed archive for PSX by~~ [[~~User:Ficedula\|Ficedula]] ===~~	+	#REDIRECT [[FF7/LZSS format]]
−
−	~~==== Format ====~~
−
−	~~The LZS archive has a very small header at 0x00 that has the length of the compressed file as an unsigned 32 bit integer. After that is the compressed data.~~
−
−	~~==== LZS compression ====~~
−
−	FF7 uses LZS compression on some of their files - more properly, a slightly modified version of LZSS compression as devised by Professor Haruhiko Okumura. LZS data works on a control byte scheme. So each block in the file begins with a single byte indicating how much of the block is uncompressed ('literal data'), and how much is compressed ('references'). You read the byte right-to-left, with 1=literal, 0=reference.
−
−	~~Literal data means just that: read one byte in from the source (compressed) data, and write it straight to the output.~~
−
−	~~References take up two bytes, and are essentially a pointer to a piece of data that's been written out (i.e. is part of the data you've already decompressed).~~ LZSS ~~uses a 4K buffer, so it can only reference data in the last 4K of data.~~
−
−	~~==== Reference~~ format ~~====~~
−
−	A reference takes up two bytes, and has two pieces of information in it: offset (where to find the data, or which piece of data is going to be repeated), and length (how long the piece of data is going to be). The two reference bytes look like this:
−
−	~~OOOO OOOO OOOO LLLL~~
−
−	~~(O = Offset, L = Length)~~
−
−	So you get a 12-bit offset and a 4-bit length, but both of these values need modifying to work on directly. The length is easy to work with: just add 3 to it. This is because if a piece of repeated data was less than 3 bytes long, you wouldn't bother repeating it - it'd take up no more space to actually just put literal data in. So all references are at least 3 in length. So a length of 0 means 3 bytes repeated, 1 means 4 bytes repeated, so on.
−
−	Since we have 4 bits available, that gives us a final length ranging from 3-18 bytes long. That also means the absolute maximum compression we can ever get using LZSS is a touch under 9:1, since the best possible is to replace 18 bytes of data with two bytes of reference, and then you have to add control bytes as well.
−
−	Offset needs a bit work doing on it, depending on how you're actually holding your data. If all you have is an input buffer and an output buffer, what you really need is an output position in your buffer to start reading data from. In other words, if you've already written 10,000 bytes to your output, you want to know where to retrieve the repeated data from - it could fall anywhere in the past 4K of data (i.e. from 5904 through to 9999 bytes).
−
−	~~Here's how you get it:~~
−
−	~~real_offset = tail - ((tail - 18 - raw_offset) mod 4096)~~
−
−	Here, 'tail' is your current output position (eg. 10,000), 'raw_offset' is the 12-bit data value you've retrieved from the compressed reference, and 'real_offset' is the position in your output buffer you can begin reading from. This is a bit complex because it's not exactly the way LZSS traditionally does (de)compression; it uses a 4K circular buffer; if you do that, the offset is more or less usable directly.
−
−	~~Once you've got to the start position for your reference, you just copy the appropriate length of data over to your output, and you've dealt with that piece of data.~~
−
−	~~==== Example ====~~
−
−	~~If we're at position 1000 in our output, and we need to read in a new control byte because we've finished with the last one. The next data to look it is:~~
−
−	~~0xFC, 0x53, 0x12 .....~~
−
−	We read in a control byte: 0xFC. In binary, that's 11111100. That informs us that the current block of data has two compressed offsets (@ 2 bytes each), followed by 6 literal data bytes. Once we'd read in the next 10 bytes (the compressed data plus the literal data), we'd be ready to read in our next control byte and start again.
−
−	Looking at the first compressed reference, we read in $53 $12. That gives us a base offset of $153 (the 53 from the first byte, and the '1' from the second byte makes up the higher nybble). The base length is $2 (we just take the low nybble of the second byte).
−
−	~~Our final length is obviously just 5.~~
−
−	~~Our position in output is still 1000. So our final offset is:~~
−
−	~~<nowiki>= 1000 - ((1000 - 18 - 339) and $FFF)~~
−	~~</nowiki>~~
−
−	~~The 339 is just $153 in decimal. The (and $FFF) is a quick way to do modulus 4096.~~
−
−	~~<nowiki>= 1000 - (643 and 0xFFF)~~
−	~~= 1000 - 643~~
−	~~= 357~~
−	~~</nowiki>~~
−
−	So our final offset is 357. We go to position 357 in our output data, read in 5 bytes (remember the length?), then write those 5 bytes out to our output. Now we're ready to read in the next bit of data (another compressed reference), and do the procedure again.
−
−	~~==== Complications ====~~
−
−	~~Unfortunately, that doesn't quite cover everything - there's two more things to be aware of when decompressing data that will ruin you when using FF7 files, since they do use these features.~~
−
−	First, if you end up with an negative offset, i.e. reading data from 'before the beginning of the file', write out nulls (zero bytes). That's because the compression buffer is, by default, initialized to zeros; so it's possible, if the start of the file contains a run of zeros, that the file may reference a block you haven't written. For example, if you're at position 50 in your output, it's possible you may get an offset indicating to go back 60 bytes to offset -10. If you have to read 5 bytes from there, you just write out 5 nulls. However, you could have to read 15 bytes from there. In that case, you write out 10 nulls (the part of the data 'before' the file start), then the 5 bytes from the beginning of the file.
−
−	Secondly, you can have a repeated run. This is almost the opposite problem: when you go off the end of your output. Say you're at offset 100 in your output, and you have to go to offset 95 to read in a reference. This is okay, but if the reference length is >5, you loop the output. So if you had to write out 15 bytes, you'd write out the five bytes that were available, then write them out again, then again, to make up the 15 bytes you needed.
−
−	~~The FF7 files use both of these 'tricks', so you can't ignore them.~~

@@ Line 1: / Line 1: @@
-=== LZS Compressed archive for PSX by [[User:Ficedula|Ficedula]] ===
+===  LZS Compressed archive for PSX by [[User:Ficedula|Ficedula]]  ===
-==== Format ====
+====  Format  ====
 The LZS archive has a very small header at 0x00 that has the length of the compressed file as an unsigned 32 bit integer. After that is the compressed data.
-==== LZS compression ====
+====  LZS compression  ====
-FF7 uses LZS compression on some of their files - more properly, a slightly modified version of LZSS compression as devised by Professor Haruhiko Okumura.
+FF7 uses LZS compression on some of their files - more properly, a slightly modified version of LZSS compression as devised by Professor Haruhiko Okumura. LZS data works on a control byte scheme. So each block in the file begins with a single byte indicating how much of the block is uncompressed ('literal data'), and how much is compressed ('references'). You read the byte right-to-left, with 1=literal, 0=reference.
 Literal data means just that: read one byte in from the source (compressed) data, and write it straight to the output.
 References take up two bytes, and are essentially a pointer to a piece of data that's been written out (i.e. is part of the data you've already decompressed). LZSS uses a 4K buffer, so it can only reference data in the last 4K of data.
-==== Reference format ====
+====  Reference format  ====
 A reference takes up two bytes, and has two pieces of information in it: offset (where to find the data, or which piece of data is going to be repeated), and length (how long the piece of data is going to be). The two reference bytes look like this:
   OOOO OOOO  OOOO LLLL
@@ Line 32: / Line 31: @@
   real_offset = tail - ((tail - 18 - raw_offset) mod 4096)
 Here, 'tail' is your current output position (eg. 10,000), 'raw_offset' is the 12-bit data value you've retrieved from the compressed reference, and 'real_offset' is the position in your output buffer you can begin reading from. This is a bit complex because it's not exactly the way LZSS traditionally does (de)compression; it uses a 4K circular buffer; if you do that, the offset is more or less usable directly.
 Once you've got to the start position for your reference, you just copy the appropriate length of data over to your output, and you've dealt with that piece of data.
-==== Example ====
+====  Example  ====
 If we're at position 1000 in our output, and we need to read in a new control byte because we've finished with the last one. The next data to look it is:
 xFC, 0x53, 0x12 .....
@@ Line 50: / Line 49: @@
 Our position in output is still 1000. So our final offset is:
-  = 1000 - ((1000 - 18 - 339) and $FFF)
+  <nowiki>= 1000 - ((1000 - 18 - 339) and $FFF)
-The 339 is just $153 in decimal.
+The 339 is just $153 in decimal. The (and $FFF) is a quick way to do modulus 4096.
-  = 1000 - (643 and 0xFFF)
+  <nowiki>= 1000 - (643 and 0xFFF)
   = 1000 - 643
   = 357
 So our final offset is 357. We go to position 357 in our output data, read in 5 bytes (remember the length?), then write those 5 bytes out to our output. Now we're ready to read in the next bit of data (another compressed reference), and do the procedure again.
-==== Complications ====
+====  Complications  ====
 Unfortunately, that doesn't quite cover everything - there's two more things to be aware of when decompressing data that will ruin you when using FF7 files, since they do use these features.
 First, if you end up with an negative offset, i.e. reading data from 'before the beginning of the file', write out nulls (zero bytes). That's because the compression buffer is, by default, initialized to zeros; so it's possible, if the start of the file contains a run of zeros, that the file may reference a block you haven't written. For example, if you're at position 50 in your output, it's possible you may get an offset indicating to go back 60 bytes to offset -10. If you have to read 5 bytes from there, you just write out 5 nulls. However, you could have to read 15 bytes from there. In that case, you write out 10 nulls (the part of the data 'before' the file start), then the 5 bytes from the beginning of the file.
 Secondly, you can have a repeated run. This is almost the opposite problem: when you go off the end of your output. Say you're at offset 100 in your output, and you have to go to offset 95 to read in a reference. This is okay, but if the reference length is >5, you loop the output. So if you had to write out 15 bytes, you'd write out the five bytes that were available, then write them out again, then again, to make up the 15 bytes you needed.
 The FF7 files use both of these 'tricks', so you can't ignore them.

@@ Line 3: / Line 3: @@
 ==== Format ====
-The LZS archive has a very small header at 0x00 that has the length of the decompressed file as an unsigned 32 bit integer. After that is the compressed data.
+The LZS archive has a very small header at 0x00 that has the length of the compressed file as an unsigned 32 bit integer. After that is the compressed data.
 ==== LZS compression ====

← Older revision	Revision as of 22:00, 14 July 2019
(No difference)

← Older revision	Revision as of 09:34, 4 April 2018
(No difference)

@@ Line 40: / Line 40: @@
 If we're at position 1000 in our output, and we need to read in a new control byte because we've finished with the last one. The next data to look it is:
-x03, 0x53, 0x12 .....
+xFC, 0x53, 0x12 .....
 We read in a control byte: 0xFC. In binary, that's 11111100. That informs us that the current block of data has two compressed offsets (@ 2 bytes each), followed by 6 literal data bytes. Once we'd read in the next 10 bytes (the compressed data plus the literal data), we'd be ready to read in our next control byte and start again.

@@ Line 42: / Line 42: @@
 x03, 0x53, 0x12 .....
-We read in a control byte: $03. In binary, that's 00000011. That informs us that the current block of data has two compressed offsets (@ 2 bytes each), followed by 6 literal data bytes. Once we'd read in the next 10 bytes (the compressed data plus the literal data), we'd be ready to read in our next control byte and start again.
+We read in a control byte: 0xFC. In binary, that's 11111100. That informs us that the current block of data has two compressed offsets (@ 2 bytes each), followed by 6 literal data bytes. Once we'd read in the next 10 bytes (the compressed data plus the literal data), we'd be ready to read in our next control byte and start again.
 Looking at the first compressed reference, we read in $53 $12. That gives us a base offset of $153 (the 53 from the first byte, and the '1' from the second byte makes up the higher nybble). The base length is $2 (we just take the low nybble of the second byte).

@@ Line 22: / Line 22: @@
   (O = Offset, L = Length)
-So you get a 12-bit offset and a 4-bit length, but both of these values need modifying to work on directly. The length is easy to work with: just add 3 to it. Why? Well, if a piece of repeated data was less than 3 bytes long, you wouldn't bother repeating it - it'd take up no more space to actually just put literal data in. So all references are at least 3 in length. So a length of 0 means 3 bytes repeated, 1 means 4 bytes repeated, so on.
+So you get a 12-bit offset and a 4-bit length, but both of these values need modifying to work on directly. The length is easy to work with: just add 3 to it. This is because if a piece of repeated data was less than 3 bytes long, you wouldn't bother repeating it - it'd take up no more space to actually just put literal data in. So all references are at least 3 in length. So a length of 0 means 3 bytes repeated, 1 means 4 bytes repeated, so on.
-Since we have 4 bits available, that gives us a final length ranging from 3-18 bytes long. (That also means the absolute maximum compression we can ever get using LZSS is a touch under 9:1, since the best possible is to replace 18 bytes of data with two bytes of reference, and then you have to add control bytes as well).
+Since we have 4 bits available, that gives us a final length ranging from 3-18 bytes long. That also means the absolute maximum compression we can ever get using LZSS is a touch under 9:1, since the best possible is to replace 18 bytes of data with two bytes of reference, and then you have to add control bytes as well.
 Offset needs a bit work doing on it, depending on how you're actually holding your data. If all you have is an input buffer and an output buffer, what you really need is an output position in your buffer to start reading data from. In other words, if you've already written 10,000 bytes to your output, you want to know where to retrieve the repeated data from - it could fall anywhere in the past 4K of data (i.e. from 5904 through to 9999 bytes).
@@ Line 32: / Line 32: @@
   real_offset = tail - ((tail - 18 - raw_offset) mod 4096)
-Here, 'tail' is your current output position (eg. 10,000), 'raw_offset' is the 12-bit data value you've retrieved from the compressed reference, and 'real_offset' is the position in your output buffer you can begin reading from. This is a bit complex because it's not exactly the way LZSS traditionally does (de) compression; it uses a 4K circular buffer; if you do that, the offset is more or less usable directly.
+Here, 'tail' is your current output position (eg. 10,000), 'raw_offset' is the 12-bit data value you've retrieved from the compressed reference, and 'real_offset' is the position in your output buffer you can begin reading from. This is a bit complex because it's not exactly the way LZSS traditionally does (de)compression; it uses a 4K circular buffer; if you do that, the offset is more or less usable directly.
 Once you've got to the start position for your reference, you just copy the appropriate length of data over to your output, and you've dealt with that piece of data.
@@ Line 59: / Line 59: @@
   = 357
-So our final offset is 357. We go to position 357 in our output data, read in 5 bytes (remember the length?), then write those 5 bytes out to our output. Now we're ready to read in the next bit of data (another compressed reference), and do the procedure again...
+So our final offset is 357. We go to position 357 in our output data, read in 5 bytes (remember the length?), then write those 5 bytes out to our output. Now we're ready to read in the next bit of data (another compressed reference), and do the procedure again.
 ==== Complications ====
-Unfortunately, that doesn't quite cover everything - there's two more things to be aware of when decompressing data that *will* ruin you when using FF7 files, since they do use these features.
+Unfortunately, that doesn't quite cover everything - there's two more things to be aware of when decompressing data that will ruin you when using FF7 files, since they do use these features.
-First, if you end up with an negative offset, i.e. reading data from 'before the beginning of the file', write out nulls (zero bytes). That's because the compression buffer is, by default, initialized to zeros; so it's possible, if the start of the file contains a run of zeros, that the file may reference a block you haven't written... EG: If you're at position 50 in your output, it's possible you may get an offset indicating to go back 60 bytes to offset -10! If you have to read 5 bytes from there, it's simple: you just write out 5 nulls. However, you *could* have to read 15 bytes from there. In that case, you write out 10 nulls (the part of the data 'before' the file start), then the 5 bytes from the beginning of the file.
+First, if you end up with an negative offset, i.e. reading data from 'before the beginning of the file', write out nulls (zero bytes). That's because the compression buffer is, by default, initialized to zeros; so it's possible, if the start of the file contains a run of zeros, that the file may reference a block you haven't written. For example, if you're at position 50 in your output, it's possible you may get an offset indicating to go back 60 bytes to offset -10. If you have to read 5 bytes from there, you just write out 5 nulls. However, you could have to read 15 bytes from there. In that case, you write out 10 nulls (the part of the data 'before' the file start), then the 5 bytes from the beginning of the file.
-Secondly, you can have a repeated run. This is almost the opposite problem: when you go off the end of your output. Say you're at offset 100 in your output, and you have to go to offset 95 to read in a reference. That's OK ... but what if the reference length is >5? In that case, you loop the output. So if you had to write out 15 bytes, you'd write out the five bytes that *were* available ... and then write them out again ... then again, to make up the 15 bytes you needed.
+Secondly, you can have a repeated run. This is almost the opposite problem: when you go off the end of your output. Say you're at offset 100 in your output, and you have to go to offset 95 to read in a reference. This is okay, but if the reference length is >5, you loop the output. So if you had to write out 15 bytes, you'd write out the five bytes that were available, then write them out again, then again, to make up the 15 bytes you needed.
-The FF7 files use both of these 'tricks', so you can't ignore them!
+The FF7 files use both of these 'tricks', so you can't ignore them.