Source Mapping HowTo
Below, we illustrate how to enable source mapping and print out an annotated TEAL version of the program which includes the original PyTeal in the comments.
Executive Summary
Author your PyTeal script as usual and make other preparations.
Enable the source mapper by turning on its feature gate.
Use a source-mappable compile instruction.
Grab the annotated TEAL out of the compile’s result.
Run the script as before.
0. Preparation
Go ahead and author your PyTeal dapp as you normally would. No modifications to PyTeal expressions are necessary to make your program source-mappable.
Consider the AlgoBank example. It was authored long before the source mapper became available, but below we’ll see how to tweak it to be source-mappable.
You may need to upgrade your pyteal dependency to a version that includes source mapping as well as feature gating.
In particular, pip install pyteal
will install the feature_gates
package alongside pyteal
.
(Optional) AlgodClient
If you intend to add the bytecode’s program counters to the source map, you’ll need to ensure that an AlgodClient
is available.
If it’s running on port 4001 (the default Sandbox port for Algod) then everything should just work automatically.
However, if Algod is running on a different port, you’ll need to create a separate AlgodClient
in your script which you will then supply
as an argument to the compile instruction.
NOTE: In this example we’re going to assume that an AlgodClient
is running on port 4001.
1. Enable the source map feature gate
This is as simple as adding the two lines to the top of algobank.py:
from feature_gates import FeatureGates
FeatureGates.set_sourcemap_enabled(True)
# previously-existing imports:
from pyteal import * # noqa: E402
import json # noqa: E402
# rest of the file
...
The code importing FeatureGates
and enabling the feature must come before any pyteal imports.
That’s because as a side effect, pyteal imports actually create expressions that can end up in the PyTeal program, and we want these to be properly source mapped.
In this example, we also added flake8 lint ignore comments # noqa: E402
because in python
it’s preferred to conclude all imports before running any code.
2. Modify the compile instruction
In the algobank.py
example, the compile instruction looks like router.compile_program(...)
.
This traditional expression, along with its analog for non-ABI programs, compileTeal(...)
,
don’t support source mapping. However, the newer compile(...)
methods do suport it:
Compiler:
Compilation.compile
. Source map specific parameters:with_sourcemap
teal_filename
pcs_in_sourcemap
algod_client
annotate_teal
annotate_teal_headers
annotate_teal_concise
ABI Router:
Router.compile
. Source map specific parameters:with_sourcemaps
approval_filename
clear_filename
pcs_in_sourcemap
algod_client
annotate_teal
annotate_teal_headers
annotate_teal_concise
Please follow the links above to the compile(...)
methods
for the details of each parameter.
For our purposes, let’s get a full source map annotation while letting PyTeal bootstrap its own Algod. Modify the snippet between lines 116 and 118 to look like:
# Compile the program
results = router.compile(
version=6,
optimize=OptimizeOptions(scratch_slots=True),
with_sourcemaps=True,
annotate_teal=True,
pcs_in_sourcemap=True,
annotate_teal_headers=True,
annotate_teal_concise=False,
)
Here we are enabling the source map and requesting annotated TEAL by
setting with_sourcemaps=True
and annotate_teal=True
.
pcs_in_sourcemap=True
will add the program counters to the source map.
Finally, we customize the annotated TEAL to have a header row with column names,
and get as many columns as available by specifying annotate_teal_headers=True
and annotate_teal_concise=False
.
3. Grab annotated TEAL from result
The newer compile(...)
methods return objects that contain source map information:
Compiler:
Compilation.compile
. Returns aCompileResults
object which has asourcemap
field of typePyTealSourceMap
.ABI Router:
Router.compile
. Returns aRouterResults
object which hasapproval_sourcemap
andclear_sourcemap
fields of typePyTealSourceMap
.
We modified algobank.py
to call Router.compile
and
received a results
object of type RouterResults
.
Let’s simply print out the resulting annotated approval program:
# Print the results
print(results.approval_sourcemap.annotated_teal)
4. Run the script
❯ python examples/application/abi/algobank.py
// GENERATED TEAL // PC PYTEAL PATH LINE PYTEAL
#pragma version 6 // (0) examples/application/abi/algobank.py 137 router.compile(version=6, optimize=OptimizeOptions(scratch_slots=True), with_sourcemaps=True, annotate_teal=True, pcs_in_sourcemap=True, annotate_teal_headers=True, annotate_teal_concise=False)
txn NumAppArgs // (20) 27 BareCallActions(no_op=OnCompleteAction(action=Approve(), call_config=CallConfig.CREATE), opt_in=OnCompleteAction(action=Approve(), call_config=CallConfig.ALL), close_out=OnCompleteAction(action=transfer_balance_to_lost, call_config=CallConfig.CALL), update_application=OnCompleteAction(action=assert_sender_is_creator, call_config=CallConfig.CALL), delete_application=OnCompleteAction(action=assert_sender_is_creator, call_config=CallConfig.CALL))
int 0 // (22)
... continues ...
About the Output
The resulting annotated TEAL assembles down to the same bytecode
as the unadorned program in results.approval_program
.
Each line’s comments also provide:
(
PC
) - the beginning program counter of the assembled bytecode for the TEAL instruction(
PYTEAL PATH
) - the PyTeal file which generated the TEAL instruction(
LINE
) - the line number of the PyTeal source(
PYTEAL
) - the PyTeal code that generated the TEAL instruction
When a value -such as a line number- is omitted, it means that it is the same as the previous.
Typically, the PyTeal compiler adds expressions to a user’s program to make various
constructs work. Consequently, not every TEAL instruction will have a corresponding
PyTeal expression that was explicity written by the program author.
In such cases, the source mapper will attempt to find a reasonable user-attributable substitute.
For example, if a program includes a Subroutine
definition, the compiler will add
boilerplate for adding arguments to the stack before the subroutine is called, and then
more boilerplate to read the arguments from the stack at the beginning of the subroutine’s
execution. The source mapper will attribute these boilerplate expressions to the subroutine’s
python definition.
Sometimes, the source mapper doesn’t succeed to find a user attribution and resorts to a attributing to the entry point into pyteal - the line that called the compiler. In the example above, the first line of the annotated TEAL is attributed to the line that called the compiler:
examples/application/abi/algobank.py 137 router.compile(version=6, ...)
This is the line that would get mapped to in the case of such source map “misses”.