Local/GlobalState custom keys & descriptions #111

achidlow · 2024-03-01T08:22:31Z

Add the ability to specify a custom key and/or provide a description for LocalState and GlobalState objects. The resolution of a member name to the key has thus been deferred to the IR phase. To support this, additional nodes have been introduced for dealing with state keys by member name, which has also allowed for an approach that significantly improves the generated code in certain cases.

Solve the issue of requiring module level constructs (such as constants) to come before their usage

scratch_slots reservation fixes:

allow module constants args, and in args to urange within
handle negatives
check for third arg to urange being non-zero to avoid crash
allow just a single urange call
allow list exprs
add tests for the above and tests for failure cases

Fixes a bug where types declared in other modules wouldn't work on import.

Check file encodings and warn if it's not UTF-8.

Fix potential issue with TemporaryAssignmentExpressionBuilder not being picked up as a value.

Add "unknown" bytes encoding for raw bytes. default to hex output, but we can choose a better encoding when e.g. combining constants in optimisation (will be addressed in another PR).

Don't generate code/calls to an empty __init__ function (todo in another PR: remove all empty function calls, and remove unreferenced functions after optimisation).

- allow module constants args, and in args to urange within - handle negatives - check for third arg to urange being non-zero to avoid crash - allow just a single urange call - allow list exprs - add tests for the above and tests for failure cases start on state reservation add stubs for state key & descriptions

…stant ordering issue

- remove unused `module_asts` on ASTConversionContext - convert ALLOWED_OCA list to frozenset - add/remove/address some TODOs - simplify FunctionASTConverter.visit_assignment_stmt a bit

…up as a value

…t we can choose a better encoding when e.g. combining constants in optimisation

…gnarly to plumb this through everywhere, and given the lack of uproar over ruff's complete lack of handling of non-utf8 files astral-sh/ruff#6791, we can just warn about it instead. the only potential negative effects are that the bytes on disk in the python file for something like a LocalState that takes it's key from the member name, might not match exactly what's in the TEAL file (and thus on chain). - refactor to slightly reduce mypy dependencies in ParseResult - pull out component orderding and checking to the parse level - construct read_source directly and add caching at the top level

github-actions · 2024-03-01T08:26:05Z

Coverage Report

File	Stmts	Miss	Cover	Missing
src/puya
__main__.py	30	1	97%	110
arc4_util.py	63	4	94%	28, 64, 78, 97
arc32.py	66	2	97%	87, 98
compile.py	151	3	98%	137, 151–154
errors.py	124	29	77%	47–48, 52, 54–57, 82, 124, 144–145, 150–164, 175–178
logging_config.py	129	28	78%	15, 36–39, 45, 47, 50–54, 69, 88, 141–143, 146–148, 150, 163–172, 194
models.py	80	2	98%	11–12
options.py	29	2	93%	12–13
parse.py	204	18	91%	86, 98, 127, 166, 169, 223, 266, 276, 278, 282, 310–315, 319, 349, 363, 376–377
utils.py	145	19	87%	49, 53, 87, 93, 108, 111–113, 120, 142, 155–156, 160, 164–167, 196, 198, 221
src/puya/awst
function_traverser.py	186	3	98%	113, 143–144
nodes.py	959	67	93%	19–28, 44, 53, 87, 93, 97–100, 134, 140, 144–147, 275, 281, 486, 524, 729–730, 750, 762, 800, 806–807, 837, 863, 867–871, 883, 974–975, 980, 995, 1046, 1051, 1109, 1132, 1149, 1328, 1333–1334, 1341–1346, 1458, 1506, 1515, 1543, 1550, 1563, 1612, 1623, 1625, 1630, 1638, 1643, 1648, 1657, 1662, 1667, 1674
to_code_visitor.py	318	14	96%	81–82, 85–88, 112–113, 194, 319–320, 330, 411–412, 483, 511
visitors.py	214	70	67%	7, 15, 19, 23, 27, 31, 35, 39, 43, 47, 53, 59, 65, 69, 73, 77, 83, 87, 91, 95, 99, 105, 109, 113, 117, 121, 125, 129, 133, 139, 143, 147, 153, 159, 163, 169, 173, 177, 181, 185, 189, 193, 197, 201, 205, 209, 213, 217, 221, 225, 231, 234, 237, 243, 249, 253, 257, 261, 265, 269, 273, 277, 281, 285, 289, 293, 297, 301, 305, 309
wtypes.py	283	36	87%	21, 64, 176–187, 202–205, 222, 224, 266, 269, 294, 296, 300–301, 304, 328, 348, 384, 387, 414–415, 424–425, 433–434, 445, 448, 507, 514, 530
src/puya/awst_build
base_mypy_visitor.py	133	44	67%	39, 59, 71–77, 95, 106–121, 135, 139, 143, 154, 158, 166, 172, 178, 196, 202, 207, 210, 213, 222, 226, 231, 236, 242, 247, 253, 260, 264, 268, 272, 276, 280, 284, 288, 292, 296, 300, 304, 308
context.py	140	20	86%	124–129, 137, 139–144, 147–148, 168, 173, 177, 190, 202–203
contract.py	334	93	72%	133–134, 136–139, 147–148, 151, 163, 174, 196, 214, 220, 222, 233, 243, 252, 280, 284, 300–304, 312–313, 318, 321, 324, 327, 332–333, 336, 339, 342, 345, 348, 351, 354, 357, 360, 363, 366, 380, 394–401, 410–417, 424–425, 451, 461, 490, 492, 495, 507, 536–537, 552, 556, 559, 565, 568, 571, 574, 582, 590, 593, 596–608, 611, 620, 623, 640, 650–651, 658, 664, 686–692
intrinsic_models.py	32	5	84%	21, 26–29
main.py	47	3	94%	32, 44, 54
module.py	370	80	78%	67, 115, 133–134, 141, 149, 155, 161–165, 174, 198–199, 210, 240, 266–267, 290, 297, 305–318, 331–332, 355–360, 414–415, 426, 450, 453, 459, 467, 470, 477, 483, 495, 498, 519, 538–539, 556–587, 615–616, 630–632, 634, 636–637, 669–676, 688, 690
subroutine.py	612	75	88%	214, 225, 280, 348, 352, 366, 374, 407–408, 545–559, 567, 575, 578, 590, 619–620, 625, 630, 666, 686–687, 717–723, 743, 776–777, 782, 798, 801, 804, 812–818, 841, 873, 877–878, 882–883, 899, 901, 904–906, 933–934, 942, 977–985, 1020, 1048, 1052, 1083, 1180, 1187, 1197, 1208–1212, 1221, 1224, 1227, 1230, 1233, 1238, 1243, 1250, 1257
utils.py	185	24	87%	50–51, 66, 103, 118–119, 121, 141–142, 162, 176, 187, 192, 199, 211, 233–246, 267, 338–339
src/puya/awst_build/eb
app_account_state.py	128	28	78%	51–54, 62, 79, 81, 114–115, 136–143, 165–166, 171, 178–179, 181, 198–199, 206–207, 236, 241–242
app_state.py	120	22	82%	47–54, 76–77, 82, 92–93, 98, 115–116, 123–124, 165–166, 190, 209
array.py	68	45	34%	26–27, 30–32, 37–44, 57–73, 78–80, 83, 86–89, 94–96, 101–103, 113–125
base.py	207	61	71%	24–31, 103, 107, 111, 117, 121, 127–132, 143–149, 154–160, 166, 172, 182, 193, 197, 201, 211, 214, 217, 220, 223, 226, 231, 234, 256, 277, 280, 285, 288, 291, 294, 297, 302, 305, 316, 329, 340, 343, 356, 377, 382, 392, 395, 399, 405, 408, 411, 416, 421, 426, 432
biguint.py	93	22	76%	35–39, 64–65, 79, 94, 104–107, 129–132, 149–154, 184–192, 195–196
bool.py	40	12	70%	27–31, 48–55, 73
bytes.py	143	23	84%	56–60, 83–84, 95–96, 117–118, 122, 126, 131, 133–134, 154, 176, 251–255, 269–270, 363–364
bytes_backed.py	29	4	86%	38–39, 48–49
ensure_budget.py	48	15	69%	25–29, 50–51, 71–72, 80, 92, 98–103
intrinsics.py	156	43	72%	32–34, 56–57, 65–66, 72–73, 90–91, 98, 128, 137, 139–163, 170–171, 215, 223, 235–236, 238, 255–256, 268, 273, 280, 283
log.py	59	11	81%	26–30, 85–95, 105, 107
named_int_constants.py	20	2	90%	23–24
struct.py	32	15	53%	21–22, 25, 36–42, 47–49, 52–57
subroutine.py	35	2	94%	52, 72
temporary_assignment.py	41	7	83%	27, 30, 41, 47, 50, 53, 58
tuple.py	100	23	77%	38–39, 46, 57, 59–60, 82–83, 92–93, 106, 112, 130–131, 136–137, 149, 157–163, 171, 203
type_registry.py	27	2	93%	138–139
uint64.py	79	15	81%	37–41, 63–64, 85, 103, 126, 145–148, 162–170, 173–174
unsigned_builtins.py	70	11	84%	30–34, 52, 59, 94, 97–98, 129, 132–133
value_proxy.py	39	12	69%	24, 27, 30, 33, 36, 41, 56, 66, 75, 85, 88, 91
src/puya/awst_build/eb/arc4
arrays.py	222	31	86%	58–62, 75–76, 104, 135, 155–156, 192, 194, 228, 260, 277–278, 291, 334–335, 341, 346, 401–402, 424, 434–435, 443, 473, 493–494, 502, 549
base.py	112	17	85%	34–38, 68–69, 90–91, 159, 180, 197–198, 212–213, 230–231, 237, 242
bool.py	28	6	79%	26–33, 59–60
numeric.py	82	18	78%	26–30, 42, 57, 88–89, 91, 94–95, 112, 119–120, 138, 152–153, 158, 163
string.py	65	12	82%	31–35, 65–66, 101, 120–121, 146–147, 165–166
struct.py	60	10	83%	26–33, 69, 72, 107–108, 114, 119
tuple.py	83	21	75%	30–34, 49, 53–54, 79, 84–86, 95, 106, 115, 128, 132–133, 145, 161–162, 171–172
src/puya/awst_build/eb/reference_types
account.py	46	8	83%	24–26, 44, 53–57
application.py	31	4	87%	17–19, 41–42
asset.py	52	6	88%	27–29, 51–52, 89–90
base.py	44	4	91%	25–27, 61, 87
src/puya/awst_build/eb/transaction
base.py	33	7	79%	16, 24, 28, 33–34, 41, 50
group.py	57	9	84%	32–36, 87–88, 97, 147, 150–151
inner.py	56	9	84%	28–31, 64–65, 101–102, 116, 144
inner_params.py	98	11	89%	29–33, 46–47, 69, 105, 141, 166, 183, 215
src/puya/awst_build/validation
arc4_copy.py	28	1	96%	37
inner_transactions.py	93	11	88%	92, 100, 111, 121, 125, 128, 133, 141, 151, 164, 169
scratch_slots.py	37	4	89%	17, 30, 46, 48
src/puya/ir
arc4_router.py	270	21	92%	171, 275, 293, 301, 457, 537, 583, 650–651, 665, 705–706, 718, 723, 728, 733, 738, 743, 763–768
avm_ops.py	316	2	99%	44–45
avm_ops_models.py	34	2	94%	18, 26
context.py	97	12	88%	18, 59–60, 62, 80–86, 91, 102, 109, 119
main.py	165	6	96%	111, 114, 218, 220, 282, 293
models.py	433	26	94%	29, 45, 63, 79, 186, 196, 200, 342, 416, 458, 538, 554, 595, 598, 605, 608, 655, 664, 672, 678, 689–691, 701, 713, 750
ssa.py	136	9	93%	43, 51–52, 99, 105, 141, 145, 166, 170
to_text_visitor.py	132	7	95%	82, 87, 169–174
types_.py	40	6	85%	26–27, 52–55
visitor.py	164	36	78%	10–12, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 60, 64, 68, 72, 76, 80, 84, 88, 92, 167, 179, 194, 197, 200, 203, 206, 212, 215, 218, 221, 224, 227, 230, 233
visitor_mutator.py	91	2	98%	134–135
vla.py	73	2	97%	50, 87
src/puya/ir/builder
_utils.py	71	8	89%	33, 44, 57, 79–80, 157, 203–204
arc4.py	418	50	88%	75–76, 125, 149–155, 219, 227, 450, 462, 465, 621–635, 964–978, 1004–1028, 1037–1053, 1076, 1177–1178, 1223–1240, 1279–1280, 1333, 1366–1367, 1389–1396, 1517–1518, 1581
assignment.py	46	5	89%	67, 83, 110, 141–142
blocks.py	93	10	89%	61, 71, 75, 111, 119, 123, 147–149, 155–157
flow_control.py	98	4	96%	63, 118, 142, 156
iteration.py	190	5	97%	46, 66, 95, 176–177
itxn.py	225	17	92%	109–110, 119–122, 132–133, 141, 166, 445–446, 451–452, 458, 474–475
main.py	476	52	89%	87, 104, 133, 182–183, 198–199, 226, 234, 241–242, 312, 340–341, 356, 368, 385, 406, 413, 447, 491, 504–505, 532, 593–594, 616, 664, 692, 700, 730–731, 746, 806, 809, 815, 822–823, 853–854, 869, 915–916, 932–933, 967, 982–984, 993, 996–997
src/puya/ir/destructure
coalesce_locals.py	94	15	84%	33, 36, 118, 127–128, 131–134, 137–139, 153–156
parcopy.py	84	2	98%	47, 83
src/puya/ir/optimize
arithmetic.py	239	33	86%	137, 199, 203, 205, 207, 209, 212–213, 229, 275, 277, 284, 316, 320, 322, 325–326, 341, 343, 345, 347, 349, 355, 357, 359, 361, 363, 365, 367, 370–371, 374–375
collapse_blocks.py	92	5	95%	65–69
control_op_simplification.py	108	7	94%	155–161, 231, 252
dead_code_elimination.py	119	2	98%	288, 324
intrinsic_simplification.py	62	5	92%	25–26, 41, 118–119
main.py	86	3	97%	108–109, 125
src/puya/mir
annotaters.py	173	7	96%	57, 63, 99, 103, 136, 140, 217
builder.py	122	6	95%	82, 132, 138, 145, 252–253
context.py	39	1	97%	35
models.py	317	22	93%	13–18, 30, 41, 53, 64, 75, 86, 116, 226, 257, 294, 306, 323, 331–337, 343
output.py	61	2	97%	29, 31
stack.py	250	22	91%	68, 72, 105, 109, 115, 125, 136, 147, 157, 172, 189, 199, 218, 236, 242, 244, 283, 294, 318, 350, 355, 393
visitor.py	73	23	68%	7, 15, 19, 23, 27, 31, 35, 39, 43, 47, 51, 55, 59, 63, 67, 71, 75, 79, 83, 87, 91, 95, 99
src/puya/mir/stack_allocation
baileys.py	208	6	97%	31, 103, 123, 330–334, 359
frame_allocation.py	78	3	96%	21, 57–58
koopmans.py	61	1	98%	55
peephole.py	100	6	94%	75, 89, 99, 101, 103, 145
src/puya/teal
models.py	220	3	99%	181, 280, 286
src/puya/teal/optimize
peephole.py	106	4	96%	64, 147–157
repeated_rotations.py	30	1	97%	13
repeated_rotations_search.py	88	6	93%	34, 40–41, 57, 67–68
TOTAL	14917	1673	89%

Tests	Skipped	Failures	Errors	Time
233	2 💤	0 ❌	0 🔥	2m 32s ⏱️

examples/amm/out/ConstantProductAMM.approval.teal

…tions

src/puya/awst_build/eb/app_account_state.py

src/puya/awst_build/eb/temporary_assignment.py

src/puya/context.py

src/puya/ir/builder/state.py

src/puya/ir/main.py

src/puya/parse.py

tristanmenzel

LGTM

- remove unrequired `attrs.field()`'s - add auxiliary constructor to CheckedMaybe to build from tuple items

achidlow added 30 commits March 1, 2024 10:17

two-phase module AST conversion - note this should also solve the con…

c9bc9e8

…stant ordering issue

refactor

8cb8241

wip

07f4f01

awst node fixups

f99bccc

wip

d3504f1

refactor

2cc0f01

fix & update error message

38d6316

refactors:

cf801cb

- remove unused `module_asts` on ASTConversionContext - convert ALLOWED_OCA list to frozenset - add/remove/address some TODOs - simplify FunctionASTConverter.visit_assignment_stmt a bit

avoid isinstance checks on ValueExpressionBuilder

0450619

fix issue with TemporaryAssignmentExpressionBuilder not being picked …

bc77414

…up as a value

wip

6b6f325

wip

1f2055b

wip

0b0e224

wip

89d1cad

re-run compile-all

28bf030

fix tests

d96ad13

add "unknown" bytes encoding for raw bytes. default to hex output, bu…

99bdb74

…t we can choose a better encoding when e.g. combining constants in optimisation

wip

404d840

wip

e410fce

wip

c309e67

wip

2fdca1c

skip empty init functions

a023221

improve optimisation of state.get()

afe81d1

minor changes

7fb7ea2

minor changes

c7743bc

wip

bd8c129

wip

1457412

wip

0fe45f4

wip

d52b93a

achidlow added 2 commits March 1, 2024 16:31

linting

aa0727a

fix doc generation issue

3826e16

robdmoore reviewed Mar 1, 2024

View reviewed changes

examples/amm/out/ConstantProductAMM.approval.teal Show resolved Hide resolved

achidlow added 2 commits March 1, 2024 16:52

use auction contract as a test case for state with custom key/descrip…

2466ea9

…tions

add test case that exercises all the custom key/description options

41b13e5

tristanmenzel reviewed Mar 4, 2024

View reviewed changes

src/puya/awst_build/eb/app_account_state.py Outdated Show resolved Hide resolved

tristanmenzel reviewed Mar 4, 2024

View reviewed changes

src/puya/awst_build/eb/temporary_assignment.py Outdated Show resolved Hide resolved

tristanmenzel reviewed Mar 5, 2024

View reviewed changes

src/puya/context.py Show resolved Hide resolved

tristanmenzel reviewed Mar 5, 2024

View reviewed changes

src/puya/ir/builder/state.py Outdated Show resolved Hide resolved

tristanmenzel reviewed Mar 5, 2024

View reviewed changes

src/puya/ir/main.py Show resolved Hide resolved

tristanmenzel reviewed Mar 5, 2024

View reviewed changes

src/puya/parse.py Show resolved Hide resolved

tristanmenzel approved these changes Mar 5, 2024

View reviewed changes

achidlow added 5 commits March 5, 2024 11:59

- replace TODO comment with REFACTOR

d1141af

- remove unrequired `attrs.field()`'s - add auxiliary constructor to CheckedMaybe to build from tuple items

ensure utils in ir/builder remains internal

6e63cda

refactoring

dbb8f7d

refactoring of checked_maybe IR implementation

096bbf6

refactoring _build_field

63f3e6b

achidlow merged commit e9d5084 into main Mar 5, 2024
4 checks passed

achidlow deleted the state-info-take-2 branch March 5, 2024 06:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Local/GlobalState custom keys & descriptions #111

Local/GlobalState custom keys & descriptions #111

achidlow commented Mar 1, 2024 •

edited

Loading

github-actions bot commented Mar 1, 2024 •

edited

Loading

tristanmenzel left a comment

Local/GlobalState custom keys & descriptions #111

Local/GlobalState custom keys & descriptions #111

Conversation

achidlow commented Mar 1, 2024 • edited Loading

github-actions bot commented Mar 1, 2024 • edited Loading

tristanmenzel left a comment

Choose a reason for hiding this comment

achidlow commented Mar 1, 2024 •

edited

Loading

github-actions bot commented Mar 1, 2024 •

edited

Loading