Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New sprite compressor #5627

Open
wants to merge 124 commits into
base: upcoming
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
124 commits
Select commit Hold shift + click to select a range
bf4c933
Testing decompression
Oct 2, 2024
0fc2270
Savepoint
Oct 3, 2024
76e3426
Saving
Oct 3, 2024
8a5d3aa
save point
Oct 4, 2024
0fd9de7
Before branching to test
Oct 4, 2024
5d774bc
Reading stage 1 works
Oct 4, 2024
1bb0cdf
Saving
Oct 4, 2024
1dc423b
Progress
Oct 9, 2024
c115253
Progress
Oct 17, 2024
add3925
It works
Oct 22, 2024
c6a875b
Minor cleanup
Oct 22, 2024
7c2b83b
Merge branch 'upcoming' into smolCompress-test
Oct 22, 2024
5d68264
More cleanup
Oct 22, 2024
c02de1a
Moved all calls to decompression functions into their wrapper functions
Oct 22, 2024
6ea4156
Removed extra line
Oct 22, 2024
e2489cf
Merge branch 'compression-wrapper' into smolCompress-test
Oct 23, 2024
dcf1754
small fix
Oct 23, 2024
33b5152
Split DecompressData into V/Wram versions
Oct 23, 2024
4534623
Pokemon sprites work
Oct 30, 2024
5e950b9
Merge branch 'upcoming' into smolCompress-test
Oct 30, 2024
84b41d5
New image compressor, first public reveal
Oct 31, 2024
406ee24
Some cleanup
Nov 1, 2024
6cde528
Fixed stuff
Nov 3, 2024
fb33a2a
plase stop using outdated gcc
Nov 3, 2024
09596ee
Some cleanup and performance increase
Nov 8, 2024
05ceee4
tANS decoding improvement
Nov 10, 2024
0c80310
Some progress
Nov 11, 2024
c3fd449
Progress
Nov 19, 2024
e8ee526
Merge branch 'upcoming' into smolCompress-test
Dec 1, 2024
b824ffd
Tests and progress
Dec 5, 2024
9a0e3ce
Cleanup
Dec 5, 2024
6f4e6fe
Cleanup and test additions
Dec 5, 2024
21adec3
start
DizzyEggg Dec 5, 2024
8886cf7
595120
DizzyEggg Dec 5, 2024
e06bdec
558724
DizzyEggg Dec 5, 2024
ae73816
559906
DizzyEggg Dec 5, 2024
d663cdb
528282
DizzyEggg Dec 5, 2024
f74fa5b
514820
DizzyEggg Dec 5, 2024
68270e3
minor cleanup
Dec 5, 2024
1f76371
511676
DizzyEggg Dec 5, 2024
e73ef3c
509011
DizzyEggg Dec 5, 2024
e444b00
476660
DizzyEggg Dec 6, 2024
62e7c66
476660 it is....
DizzyEggg Dec 6, 2024
50722eb
Merge pull request #4 from DizzyEggg/compresss
hedara90 Dec 6, 2024
89d18ce
Merge branch 'smolCompress-test' of github.com:hedara90/pokeemerald-e…
Dec 6, 2024
3268e6a
Fix alignement of LO/Sym data
Dec 6, 2024
4d80738
395766
DizzyEggg Dec 6, 2024
dde78fd
merge
DizzyEggg Dec 6, 2024
f1732a9
remove
DizzyEggg Dec 6, 2024
630100c
Merge pull request #5 from DizzyEggg/compresss
hedara90 Dec 6, 2024
563a7d2
331597
DizzyEggg Dec 6, 2024
1cafefd
328614
DizzyEggg Dec 6, 2024
84c8f0b
299175
DizzyEggg Dec 6, 2024
3529d6d
281124
DizzyEggg Dec 6, 2024
c89e96b
Merge pull request #6 from DizzyEggg/compresss
hedara90 Dec 6, 2024
94c114b
268710
DizzyEggg Dec 7, 2024
239964a
265422
DizzyEggg Dec 7, 2024
4438440
cleanup
DizzyEggg Dec 7, 2024
83c0a08
265230
DizzyEggg Dec 7, 2024
9601fe2
Merge pull request #7 from DizzyEggg/compresss
hedara90 Dec 7, 2024
0f65304
Stuff
Dec 7, 2024
97b923e
Merge branch 'smolCompress-test' of github.com:hedara90/pokeemerald-e…
Dec 7, 2024
657d50d
Stuff again
Dec 7, 2024
796a0d4
Fixed accidental breaking of non-tANS encoded images
Dec 7, 2024
2053844
move sReadIndex write 0
DizzyEggg Dec 7, 2024
f06ff69
Size acquisition fixes
Dec 7, 2024
3fd2bf4
Merge pull request #8 from DizzyEggg/compresss
hedara90 Dec 7, 2024
54ad44f
Minor cleanup
Dec 8, 2024
5c7ab67
Cleaned out wrong printf's
Dec 8, 2024
832699f
Changed image size alignment requirement
Dec 8, 2024
a2bd014
remove dma
DizzyEggg Dec 9, 2024
e34fb5d
remove dma
DizzyEggg Dec 9, 2024
58058a0
Merge pull request #9 from DizzyEggg/compresss
hedara90 Dec 9, 2024
db40868
lucky's suggestions
DizzyEggg Dec 11, 2024
fa29eac
Merge pull request #10 from DizzyEggg/compresss
hedara90 Dec 11, 2024
f3b9014
278173
DizzyEggg Dec 30, 2024
26d306c
Merge branch 'upcoming' into smolCompress-test
Jan 9, 2025
bfeda1f
Merge branch 'smolCompress-test' of https://github.com/hedara90/pokee…
DizzyEggg Jan 13, 2025
e8c3e8e
Merge pull request #11 from DizzyEggg/compresss
hedara90 Jan 13, 2025
7ba40bf
271215
DizzyEggg Jan 14, 2025
05957d7
271215
DizzyEggg Jan 14, 2025
3e40236
Implemented .fastSmol for faster smol decoding at the cost of file size
Jan 14, 2025
b116dac
Merge pull request #12 from DizzyEggg/compresss
hedara90 Jan 14, 2025
3368016
Added lz comparison tests
Jan 14, 2025
a7c1eaf
Optimize DecodeSymtANSLoop
DizzyEggg Jan 14, 2025
5527090
Replaced LZDecompress calls with the new wrapper functions
Jan 14, 2025
df0ff55
Added some comments explaining some of the decompression code
Jan 14, 2025
d449d87
Corrected some function arguments in header
Jan 15, 2025
3b88fcb
Merge branch 'smolCompress-test' of https://github.com/hedara90/pokee…
DizzyEggg Jan 17, 2025
3c4378d
BuildDecompressionTable - giant switch
DizzyEggg Jan 19, 2025
6e44797
make functions static
DizzyEggg Jan 19, 2025
707dede
248968
DizzyEggg Jan 19, 2025
41c7d51
244167
DizzyEggg Jan 19, 2025
c20e7dc
Merge pull request #13 from DizzyEggg/compresss
hedara90 Jan 19, 2025
b545787
241703
DizzyEggg Jan 19, 2025
6727b24
Cleaned up compressor a bit
Jan 19, 2025
fff5d66
remove sReadIndex - 241460
DizzyEggg Jan 19, 2025
478ab02
Added tests for Mr. Mime and Ledian
Jan 19, 2025
f97458e
241488
DizzyEggg Jan 19, 2025
d6a2a08
Merge branch 'smolCompress-test' of https://github.com/hedara90/pokee…
DizzyEggg Jan 19, 2025
9fe3a9e
233900
DizzyEggg Jan 20, 2025
27debf7
Forgot to add pngs
Jan 20, 2025
a8f1bb1
233900
DizzyEggg Jan 20, 2025
c01858b
Merge branch 'smolCompress-test' of https://github.com/hedara90/pokee…
DizzyEggg Jan 20, 2025
3f1cb9d
Merge pull request #14 from DizzyEggg/compresss
hedara90 Jan 20, 2025
42429a5
216171
DizzyEggg Jan 20, 2025
899f889
Added table generation tests
Jan 20, 2025
6107e13
210900
DizzyEggg Jan 20, 2025
7571b89
208831
DizzyEggg Jan 20, 2025
eb2ae41
208823
DizzyEggg Jan 20, 2025
e6267a1
Merge pull request #15 from DizzyEggg/compresss
hedara90 Jan 20, 2025
cc63d6e
Changed compression test builders
Jan 20, 2025
6aa2418
Merge branch 'smolCompress-test' of github.com:hedara90/pokeemerald-e…
Jan 20, 2025
2e4d006
Merge branch 'upcoming' into smolCompress-test
Jan 20, 2025
66805b7
163010
DizzyEggg Jan 21, 2025
8084d7d
156021
DizzyEggg Jan 21, 2025
2dd103d
Merge pull request #16 from DizzyEggg/compresss
hedara90 Jan 21, 2025
fa88129
Secondary tileset test
Jan 21, 2025
2920559
Cleaned up test debug prints, fixed make clean
Jan 21, 2025
c3a182b
147063
DizzyEggg Jan 21, 2025
e94f718
146976
DizzyEggg Jan 22, 2025
b9d9851
Merge pull request #17 from DizzyEggg/compresss
hedara90 Jan 22, 2025
7739957
Shuffled around some bits in the header
Jan 22, 2025
d3ef329
Merge branch 'smolCompress-test' of github.com:hedara90/pokeemerald-e…
Jan 22, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@
*.8bpp
*.gbapal
*.lz
*.smol
*.fastSmol
*.rl
*.latfont
*.hwjpnfont
Expand Down Expand Up @@ -41,3 +43,4 @@ prefabs.json
/pokeemerald-*.png
src/data/map_group_count.h
tools/trainerproc/trainerproc
tools/compresSmol/compresSmol
19 changes: 11 additions & 8 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,7 @@ endif
AUTO_GEN_TARGETS :=
include make_tools.mk
# Tool executables
SMOL := $(TOOLS_DIR)/compresSmol/compresSmol$(EXE)
GFX := $(TOOLS_DIR)/gbagfx/gbagfx$(EXE)
AIF := $(TOOLS_DIR)/aif2pcm/aif2pcm$(EXE)
MID := $(TOOLS_DIR)/mid2agb/mid2agb$(EXE)
Expand Down Expand Up @@ -294,7 +295,7 @@ clean-assets:
rm -f $(DATA_ASM_SUBDIR)/layouts/layouts.inc $(DATA_ASM_SUBDIR)/layouts/layouts_table.inc
rm -f $(DATA_ASM_SUBDIR)/maps/connections.inc $(DATA_ASM_SUBDIR)/maps/events.inc $(DATA_ASM_SUBDIR)/maps/groups.inc $(DATA_ASM_SUBDIR)/maps/headers.inc $(DATA_SRC_SUBDIR)/map_group_count.h
find sound -iname '*.bin' -exec rm {} +
find . \( -iname '*.1bpp' -o -iname '*.4bpp' -o -iname '*.8bpp' -o -iname '*.gbapal' -o -iname '*.lz' -o -iname '*.rl' -o -iname '*.latfont' -o -iname '*.hwjpnfont' -o -iname '*.fwjpnfont' \) -exec rm {} +
find . \( -iname '*.1bpp' -o -iname '*.4bpp' -o -iname '*.8bpp' -o -iname '*.gbapal' -o -iname '*.lz' -o -iname '*.smol' -o -iname '*.fastSmol' -o -iname '*.rl' -o -iname '*.latfont' -o -iname '*.hwjpnfont' -o -iname '*.fwjpnfont' \) -exec rm {} +
find $(DATA_ASM_SUBDIR)/maps \( -iname 'connections.inc' -o -iname 'events.inc' -o -iname 'header.inc' \) -exec rm {} +

tidy: tidymodern tidycheck tidydebug
Expand Down Expand Up @@ -328,13 +329,15 @@ generated: $(AUTO_GEN_TARGETS)
%.pal: ;
%.aif: ;

%.1bpp: %.png ; $(GFX) $< $@
%.4bpp: %.png ; $(GFX) $< $@
%.8bpp: %.png ; $(GFX) $< $@
%.gbapal: %.pal ; $(GFX) $< $@
%.gbapal: %.png ; $(GFX) $< $@
%.lz: % ; $(GFX) $< $@
%.rl: % ; $(GFX) $< $@
%.1bpp: %.png ; $(GFX) $< $@
%.4bpp: %.png ; $(GFX) $< $@
%.8bpp: %.png ; $(GFX) $< $@
%.gbapal: %.pal ; $(GFX) $< $@
%.gbapal: %.png ; $(GFX) $< $@
%.lz: % ; $(GFX) $< $@
%.fastSmol: % ; $(SMOL) -w $< $@ false false false
%.smol: % ; $(SMOL) -w $< $@
%.rl: % ; $(GFX) $< $@

clean-generated:
-rm -f $(AUTO_GEN_TARGETS)
Expand Down
3 changes: 3 additions & 0 deletions include/config/test.h
Original file line number Diff line number Diff line change
Expand Up @@ -1137,4 +1137,7 @@
#undef B_FLAG_SLEEP_CLAUSE
#define B_FLAG_SLEEP_CLAUSE FLAG_SPECIAL_FLAG_UNUSED_0x4003

// Compression DebugPrintf switch
#define T_COMPRESSION_SHOULD_PRINT FALSE

#endif // GUARD_CONFIG_TEST_H
62 changes: 60 additions & 2 deletions include/decompress.h
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,69 @@

#include "sprite.h"

#define TANS_TABLE_SIZE 64
#define PACKED_FREQ_MASK 0x3F
#define PARTIAL_FREQ_MASK 0xC0000000

#define FIRST_LO_MASK 0x7f
#define CONTINUE_BIT 0x80

#define SMOL_IMAGE_SIZE_MULTIPLIER 4


extern u8 ALIGNED(4) gDecompressionBuffer[0x4000];

void LZDecompressWram(const u32 *src, void *dest);
void LZDecompressVram(const u32 *src, void *dest);
struct LZ77Header {
u32 lz77IdBits:5;
u32 padding:3;
u32 size:24;
};

struct SmolHeader {
u32 mode:4;
u32 imageSize:14;
u32 symSize:14;
u32 initialState:6;
u32 bitstreamSize:13;
u32 loSize:13;
};

union CompressionHeader {
struct LZ77Header lz77;
struct SmolHeader smol;
};

struct SpriteSheetHeader {
u32 mode:4;
u32 numComponents:12;
u32 framesPerComponent:16;
};

enum CompressionMode {
MODE_LZ77 = 0,
BASE_ONLY = 1,
ENCODE_SYMS = 2,
ENCODE_DELTA_SYMS = 3,
ENCODE_LO = 4,
ENCODE_BOTH = 5,
ENCODE_BOTH_DELTA_SYMS = 6,
IS_FRAME_CONTAINER = 7,
};


void DecompressDataWithHeaderVram(const u32 *src, void *dest);
void DecompressDataWithHeaderWram(const u32 *src, void *dest);

// For decompressing a single part of a multi-part spritesheet
//void DecompressSubFrame(const u32 *src, void *dest, u32 frameId);

void SmolDecompressData(const struct SmolHeader *header, const u32 *data, void *dest);

bool32 isModeLoEncoded(enum CompressionMode mode);
bool32 isModeSymEncoded(enum CompressionMode mode);
bool32 isModeSymDelta(enum CompressionMode mode);

// Default Decompression functions are below here
u32 IsLZ77Data(const void *ptr, u32 minSize, u32 maxSize);

u16 LoadCompressedSpriteSheet(const struct CompressedSpriteSheet *src);
Expand Down
1 change: 1 addition & 0 deletions include/global.h
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
#include "gba/gba.h"
#include "fpmath.h"
#include "metaprogram.h"
#include "decompress.h"
#include "constants/global.h"
#include "constants/flags.h"
#include "constants/vars.h"
Expand Down
2 changes: 2 additions & 0 deletions include/sprite.h
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,8 @@ struct Sprite
u8 subspriteMode:2;

/*0x43*/ u8 subpriority;
//u16 prevFrame;
//const u32 *spriteArray;
};

struct OamMatrix
Expand Down
2 changes: 1 addition & 1 deletion make_tools.mk
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ MAKEFLAGS += --no-print-directory

# Inclusive list. If you don't want a tool to be built, don't add it here.
TOOLS_DIR := tools
TOOL_NAMES := aif2pcm bin2c gbafix gbagfx jsonproc mapjson mid2agb preproc ramscrgen rsfont scaninc trainerproc
TOOL_NAMES := aif2pcm bin2c gbafix gbagfx jsonproc mapjson mid2agb preproc ramscrgen rsfont scaninc trainerproc compresSmol
CHECK_TOOL_NAMES = patchelf mgba-rom-test-hydra

TOOLDIRS := $(TOOL_NAMES:%=$(TOOLS_DIR)/%)
Expand Down
8 changes: 4 additions & 4 deletions src/battle_anim.c
Original file line number Diff line number Diff line change
Expand Up @@ -1574,18 +1574,18 @@ void LoadMoveBg(u16 bgId)
void *dmaSrc;
void *dmaDest;

LZDecompressWram(tilemap, gDecompressionBuffer);
DecompressDataWithHeaderWram(tilemap, gDecompressionBuffer);
RelocateBattleBgPal(GetBattleBgPaletteNum(), (void *)gDecompressionBuffer, 0x100, FALSE);
dmaSrc = gDecompressionBuffer;
dmaDest = (void *)BG_SCREEN_ADDR(26);
DmaCopy32(3, dmaSrc, dmaDest, 0x800);
LZDecompressVram(gBattleAnimBackgroundTable[bgId].image, (void *)BG_SCREEN_ADDR(4));
DecompressDataWithHeaderVram(gBattleAnimBackgroundTable[bgId].image, (void *)BG_SCREEN_ADDR(4));
LoadCompressedPalette(gBattleAnimBackgroundTable[bgId].palette, BG_PLTT_ID(GetBattleBgPaletteNum()), PLTT_SIZE_4BPP);
}
else
{
LZDecompressVram(gBattleAnimBackgroundTable[bgId].tilemap, (void *)BG_SCREEN_ADDR(26));
LZDecompressVram(gBattleAnimBackgroundTable[bgId].image, (void *)BG_CHAR_ADDR(2));
DecompressDataWithHeaderVram(gBattleAnimBackgroundTable[bgId].tilemap, (void *)BG_SCREEN_ADDR(26));
DecompressDataWithHeaderVram(gBattleAnimBackgroundTable[bgId].image, (void *)BG_CHAR_ADDR(2));
LoadCompressedPalette(gBattleAnimBackgroundTable[bgId].palette, BG_PLTT_ID(2), PLTT_SIZE_4BPP);
}
}
Expand Down
2 changes: 1 addition & 1 deletion src/battle_anim_effects_2.c
Original file line number Diff line number Diff line change
Expand Up @@ -3095,7 +3095,7 @@ void AnimTask_LoadMusicNotesPals(u8 taskId)
paletteNums[i] = AllocSpritePalette(ANIM_SPRITES_START - i);

gMonSpritesGfxPtr->buffer = AllocZeroed(MON_PIC_SIZE * MAX_MON_PIC_FRAMES);
LZDecompressWram(gBattleAnimSpritePal_MusicNotes2, gMonSpritesGfxPtr->buffer);
DecompressDataWithHeaderWram(gBattleAnimSpritePal_MusicNotes2, gMonSpritesGfxPtr->buffer);
for (i = 0; i < NUM_MUSIC_NOTE_PAL_TAGS; i++)
LoadPalette(&gMonSpritesGfxPtr->buffer[i * 32], (u16)(OBJ_PLTT_ID(paletteNums[i])), PLTT_SIZE_4BPP);

Expand Down
2 changes: 1 addition & 1 deletion src/battle_anim_mons.c
Original file line number Diff line number Diff line change
Expand Up @@ -955,7 +955,7 @@ void ClearBattleAnimBg(u32 bgId)
void AnimLoadCompressedBgGfx(u32 bgId, const u32 *src, u32 tilesOffset)
{
CpuFill32(0, gBattleAnimBgTileBuffer, 0x2000);
LZDecompressWram(src, gBattleAnimBgTileBuffer);
DecompressDataWithHeaderWram(src, gBattleAnimBgTileBuffer);
LoadBgTiles(bgId, gBattleAnimBgTileBuffer, 0x2000, tilesOffset);
}

Expand Down
Loading
Loading