summaryrefslogtreecommitdiff
path: root/Allow-map-tiles-to-appear-above-sprites-(so-NPCs-can-walk-behind-tiles)-with-PRIORITY-colors.md
blob: d46c932eba66f398192cf4603a617d73768dc21d (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
Usually on overworld maps, sprites go on top of tiles. But there are exceptions: grass tiles overlap you as you walk, and the popup signs with location names (made of tiles) appear above all the NPC sprites.

Is it possible to make *any* tile in a tileset appear above sprites? Yes.

(The code for this feature was adapted from [Pokémon Polished Crystal](https://github.com/Rangi42/polishedcrystal/).)


## Contents

1. [Some background information](#1-some-background-information)
2. [Define `PAL_BG_PRIORITY_*` constants](#2-define-pal_bg_priority_-constants)
3. [Use one byte per color for tileset palettes](#3-use-one-byte-per-color-for-tileset-palettes)
4. [Fix the skipped space in palette_map.asm files](#4-fix-the-skipped-space-in-palette_mapasm-files)
5. [Fix the bank overflow](#5-fix-the-bank-overflow)
6. [Correctly read the extended palette data](#6-correctly-read-the-extended-palette-data)


## 1. Some background information

We can extend the tileset palette system to enable tiles above sprites. The key is in [constants/hardware_constants.asm](../blob/master/constants/hardware_constants.asm):

```asm
; OAM attribute flags
OAM_TILE_BANK EQU 3
OAM_OBP_NUM   EQU 4 ; non CGB Mode Only
OAM_X_FLIP    EQU 5
OAM_Y_FLIP    EQU 6
OAM_PRIORITY  EQU 7 ; 0: OBJ above BG, 1: OBJ behind BG (colors 1-3)

; BG Map attribute flags
PALETTE_MASK EQU %111
VRAM_BANK_1  EQU 1 << OAM_TILE_BANK ; $08
OBP_NUM      EQU 1 << OAM_OBP_NUM   ; $10
X_FLIP       EQU 1 << OAM_X_FLIP    ; $20
Y_FLIP       EQU 1 << OAM_Y_FLIP    ; $40
PRIORITY     EQU 1 << OAM_PRIORITY  ; $80
```

Every tile on the screen has an **attribute** byte. The lowest three bits define the color, which is why there's only room for eight colors (from `PAL_BG_GRAY`, 0, to `PAL_BG_TEXT`, 7). The other bits control other properties. In particular, the high bit controls **tile priority**. So if the [gfx/tilesets/\*\_palette\_map.asm](../blob/master/gfx/tilesets/) files could define tiles' priority as well as color, you could make any tile have priority over sprites.


## 2. Define `PAL_BG_PRIORITY_*` constants

Edit [constants/tileset_constants.asm](../blob/master/constants/tileset_constants.asm):

```diff
 ; bg palette values (see gfx/tilesets/*_palette_map.asm)
 ; TilesetBGPalette indexes (see gfx/tilesets/bg_tiles.pal)
 	const_def
 	const PAL_BG_GRAY   ; 0
 	const PAL_BG_RED    ; 1
 	const PAL_BG_GREEN  ; 2
 	const PAL_BG_WATER  ; 3
 	const PAL_BG_YELLOW ; 4
 	const PAL_BG_BROWN  ; 5
 	const PAL_BG_ROOF   ; 6
 	const PAL_BG_TEXT   ; 7

+const_value set $80
+	const PAL_BG_PRIORITY_GRAY   ; 80
+	const PAL_BG_PRIORITY_RED    ; 81
+	const PAL_BG_PRIORITY_GREEN  ; 82
+	const PAL_BG_PRIORITY_WATER  ; 83
+	const PAL_BG_PRIORITY_YELLOW ; 84
+	const PAL_BG_PRIORITY_BROWN  ; 85
+	const PAL_BG_PRIORITY_ROOF   ; 86
+	const PAL_BG_PRIORITY_TEXT   ; 87
```

(The exact `PAL_BG_PRIORITY_` names are important: [Polished Map](https://github.com/Rangi42/polished-map) supports them when editing tilesets.)

But we can't just start using colors like `PRIORITY_RED` in the tilesets' palette_map.asm files. The `tilepal` macro packs two tile color definitions into each byte, using four bits per tile: three for the color (`PALETTE_MASK`), one for the bank (`VRAM_BANK_1`). So we need to add space for the new priority data.


## 3. Use one byte per color for tileset palettes

Edit [gfx/tileset_palette_maps.asm](../blob/master/gfx/tileset_palette_maps.asm):

```diff
 tilepal: MACRO
 ; used in gfx/tilesets/*_palette_map.asm
 ; vram bank, pals
 x = \1 << OAM_TILE_BANK
-rept (_NARG +- 1) / 2
+rept _NARG +- 1
-	dn (x | PAL_BG_\3), (x | PAL_BG_\2)
+	db (x | PAL_BG_\2)
-	shift
 	shift
 endr
 ENDM
```


## 4. Fix the skipped space in palette_map.asm files

The [gfx/tilesets/\*\_palette\_map.asm](../blob/master/gfx/tilesets/) define tile palettes in order: first for tiles $00 to $5F, then for tiles $80 to $DF. Tiles $60 to $7F are skipped because those IDs are used for font graphics. But the skipping is done with a count of bytes, not of colors, so we need to double the counts.

Edit all the [gfx/tilesets/\*\_palette\_map.asm](../blob/master/gfx/tilesets/) files:

```diff
-rept 16
+rept 32
 	db $ff
 endr
```


## 5. Fix the bank overflow

Now the tileset palette data will take up twice as much space—one byte per tile instead of half a byte—so it won't fit in its ROM bank. Edit [main.asm](../blob/master/main.asm):

```diff
-SECTION "bank13", ROMX
+SECTION "Tileset Palettes", ROMX

 INCLUDE "engine/tilesets/map_palettes.asm"
 INCLUDE "gfx/tileset_palette_maps.asm"
+
+
+SECTION "bank13", ROMX
+
 INCLUDE "data/collision_permissions.asm"
 INCLUDE "engine/menus/empty_sram.asm"
 INCLUDE "engine/menus/savemenu_copytilemapatonce.asm"
 INCLUDE "engine/events/checksave.asm"
 INCLUDE "data/maps/scenes.asm"
 INCLUDE "engine/overworld/load_map_part.asm"
 INCLUDE "engine/phone/phonering_copytilemapatonce.asm"
```

Since we don't specify a bank for "Tileset Palettes" in [pokecrystal.link](../blob/master/pokecrystal.link), rgblink will place it in any bank that has enough room.


## 6. Correctly read the extended palette data

Edit [engine/tilesets/map_palettes.asm](../blob/master/engine/tilesets/map_palettes.asm):

```diff
 SwapTextboxPalettes::
 	hlcoord 0, 0
 	decoord 0, 0, wAttrMap
 	ld b, SCREEN_HEIGHT
 .loop
 	push bc
 	ld c, SCREEN_WIDTH
+	call GetBGMapTilePalettes
-.innerloop
-	ld a, [hl]
-	push hl
-	srl a
-	jr c, .UpperNybble
-	ld hl, TilesetPalettes
-	add [hl]
-	ld l, a
-	ld a, [TilesetPalettes + 1]
-	adc 0
-	ld h, a
-	ld a, [hl]
-	and $f
-	jr .next
-
-.UpperNybble:
-	ld hl, wTilesetPalettes
-	add [hl]
-	ld l, a
-	ld a, [wTilesetPalettes + 1]
-	adc 0
-	ld h, a
-	ld a, [hl]
-	swap a
-	and $f
-
-.next
-	pop hl
-	ld [de], a
-	res 7, [hl]
-	inc hl
-	inc de
-	dec c
-	jr nz, .innerloop
 	pop bc
 	dec b
 	jr nz, .loop
 	ret

 ScrollBGMapPalettes::
 	ld hl, wBGMapBuffer
 	ld de, wBGMapPalBuffer
+	; fallthrough
+GetBGMapTilePalettes:
 .loop
 	ld a, [hl]
 	push hl
-	srl a
-	jr c, .UpperNybble
-
-; .LowerNybble
 	ld hl, wTilesetPalettes
 	add [hl]
 	ld l, a
 	ld a, [wTilesetPalettes + 1]
 	adc 0
 	ld h, a
 	ld a, [hl]
-	and $f
-	jr .next
-
-.UpperNybble:
-	ld hl, wTilesetPalettes
-	add [hl]
-	ld l, a
-	ld a, [wTilesetPalettes + 1]
-	adc 0
-	ld h, a
-	ld a, [hl]
-	swap a
-	and $f
-
-.next
 	pop hl
 	ld [de], a
 	res 7, [hl]
 	inc hl
 	inc de
 	dec c
 	jr nz, .loop
 	ret
```

Notice how `SwapTextboxPalettes` now reuses the loop it shares with `ScrollBGMapPalettes`, and then the whole decision of which nybble to read is no longer necessary because the whole byte defines one tile's attributes.

Anyway—at this point you are done! Now when you edit a palette_map.asm file, you can use the names `PRIORITY_GRAY`, `PRIORITY_BROWN`, etc., and the corresponding tile will appear above any NPC.

![Screenshot](screenshots/tile-priority.png)

*However*, the lightest hue (that's white when you're editing the monochrome tileset PNG) will be transparent. That's how tall grass works: you see only the parts of the player sprite that overlap "white" pixels (actually light green, using the standard outdoor color palette). So design your overhead tiles carefully.