picotool: Tools and Python libraries for manipulating PICO-8 game files

PICO-8 is a fantasy game console by Lexaloffle Games. The PICO-8 runtime environment runs cartridges (or carts): game files containing code, graphics, sound, and music data. The console includes a built-in editor for writing games. Game cartridge files can be played in a browser, and can be posted to the Lexaloffle bulletin board or exported to any website.

picotool

Useful tools include:

• p8tool build

  : assembles cartridges from multiple sources, as part of a game development workflow

• p8tool stats

  : reports statistics on one or many cartridge files

• p8tool listlua

  : prints the Lua code of a cartridge

• p8tool luafind

  : searches the Lua code of a collection of cartridges

• p8tool luafmt

  : formats the Lua code of a cartridge to make it easier to read

There are additional tools that are mostly useful for demonstrating and troubleshooting the library:

writep8

listtokens

printast

luamin

p8upsidedown

picotool

.p8

.p8.png

How do most people use picotool?

Most people use

picotool

luamin

luamin

I originally created

picotool

build

#include

build

#include

require()

I hope

picotool

Installing picotool

To install the

picotool

1. Install Python 3 version 3.4 or later, if necessary. (picotool has not been tested with Python 2.)
2. Download and unpack the zip archive, or use Git to clone the Github repository.
   • Unpacking the zip archive creates a root directory named

     picotool-master

     .
   • When cloning the repo, this is just

     picotool

     , or whatever you named it when you cloned it.
3. Change to the unpacked archive or clone directory from the above step.
4. Install the software:

   pip install picotool-master

Or:
   pip install picotool

You should now have a

p8tool

Using picotool

To use a tool, you run the

p8tool

stats

For example, to print statistics about a cart named

helloworld.p8.png

p8tool stats helloworld.p8.png

p8tool build

The

build

The tool takes the filename of the output cartridge, with additional arguments describing the build. If the output cartridge does not exist, the build starts with an empty cartridge. Otherwise, it uses the existing cartridge as the default, and overwrites sections of it based on the arguments.

For example, you can create a cartridge in PICO-8, use PICO-8's built-in graphics and sound editors, then use

p8tool build

.lua

p8tool build mygame.p8.png --lua mygame.lua

As another example, to create a new cartridge using the spritesheet (

gfx

sfx

music

.lua

p8tool build mygame.p8.png --gfx mygamegfx.p8 --sfx radsnds.p8.png --music radsnds.p8.png --lua mygame.lua

You can also erase a section of an existing cart with an argument such as

--empty-map

The available arguments are as follows:

• --lua LUA

  : use Lua code from the given cart or

  .lua

  file

• --gfx GFX

  : use spritesheet from the given cart

• --gff GFF

  : use sprite flags from the given cart

• --map MAP

  : use map from the given cart

• --sfx SFX

  : use sound effects from the given cart

• --music MUSIC

  : use music patterns from the given cart

• --empty-lua

  : use an empty Lua code section

• --empty-gfx

  : use an empty spritesheet

• --empty-gff

  : use empty sprite flags

• --empty-map

  : use an empty map

• --empty-sfx

  : use empty sound effects

• --empty-music

  : use empty music patterns

If the output cart filename ends with

.p8.png

.p8.png

p8tool build

Packages and the require() function

p8tool build supports a special feature for organizing your Lua code, called packages. When loading Lua code from a file with the

--lua mygame.lua

require()

require()

Consider the following simple example. Say you have a function you like to use in several games in a file called

mylib.lua

function handyfunc(x, y)
  return x + y
end

handynumber = 3.14

Your main game code is in a file named

mygame.lua

handyfunc()

mygame.lua

require()

require("mylib")

result = handyfunc(2, 3)
print(result)
r = 5
area = handynumber * r * r

All globals defined in the required file are set as globals in your program when

require()

A more typical way to write a Lua package is to put everything intended to be used by other programs in a table:

HandyPackage = {
  handyfunc = function(x, y)
    return x + y
  end,
  handynumber = 3.14,
}

Then in

mygame.lua

require("mylib")

result = HandyPackage.handyfunc(2, 3)

This is cleaner, but still has the disadvantage that the package must be known by the global name

HandyPackage

mygame.lua

return

require()

local

local HandyPackage = {
  handyfunc = function(x, y)
    return x + y
  end,
  handynumber = 3.14,
}

return HandyPackage

The main program uses the return value of

require()

HandyPackage = require("mylib")

result = HandyPackage.handyfunc(2, 3)

The

require()

require()

require()

Where packages are located

The first argument to

require()

;

?

The default lookup path is

?;?.lua

require("mylib")

mylib

mylib.lua

require()

/usr/share/pico8/lib/?.lua

--lua-path=...

For example, with this environment variable set:

PICO8_LUA_PATH=?;?.lua;/home/dan/p8libs/?/?.p8

The require("3dlib") statement would look for these files, in this order, with paths relative to the file containing the require() statement:

3dlib
3dlib.lua
/home/dan/p8libs/3dlib/3dlib.p8

To prevent malicious code from accessing arbitrary files on your hard drive (unlikely but it's nice to prevent it), the

require()

../

require("math/linear")

As with Lua, packages are remembered by the string name used with

require()

foo.lua

?;?.lua

require("foo")

require("foo.lua")

Packages and game loop functions

When you write a library of routines for PICO-8, you probably want to write test code for those routines. picotool assumes that this test code would be executed in a PICO-8 game loop, such that the library can be in its own test cart. For this purpose, you can write your library file with

_init()

_update()

_update60()

_draw()

require()

For example:

local HandyPackage = {
  handyfunc = function(x, y)
    return x + y
  end,
  handynumber = 3.14,
}

function _update()
  test1 = HandyPackage.handyfunc(2, 3)
end
function _draw()
  cls()
  print('test1 = '..test1)
end
return HandyPackage

If you want to keep the game loop functions present in a package, you can request them with a second argument to

require()

require("mylib", {use_game_loop=true})

How require() actually works

Of course, PICO-8 does not actually load packages from disk when it runs the cartridge. Instead, picotool inserts each package into the cartridge code in a special way that replicates the behavior of the Lua

require()

When you run

p8tool build

--lua=...

require()

require()

Each required library is stored once as a function object in a table inserted at the top of the final cartridge's code. A definition of the

require()

To match Lua's behavior,

require()

package.loaded

nil

require()

This feature incurs a small amount of overhead in terms of tokens. Each library uses tokens for its own code, plus a few additional tokens for storing it in the table. The definition for

require()

Formatting or minifying Lua in a built cart

You can tell

p8tool build

--lua-format

--lua-minify

p8tool build mycart.p8.png --lua=mygame.lua --lua-format

This is equivalent to building the cart then running

p8tool luafmt

p8tool luamin

The

build

--keep-names-from-file=

--keep-all-names

luamin

p8tool stats

The

stats

% p8tool stats helloworld.p8.png
hello world (helloworld.p8.png)
by zep
version: 0  lines: 48  chars: 419  tokens: 134

This command accepts an optional

--csv

p8tool --csv stats mycarts/*.p8* >cartstats.csv

p8tool listlua

The

listlua

% p8tool listlua helloworld.p8.png
-- hello world
-- by zep

t = 0
music(0)
function _update()
 t += 1
end
function _draw()
 cls()
...

p8tool luafmt

The

luafmt

The command takes one or more cart filenames as arguments. For each cart with a name like

xxx.p8.png

xxx_fmt.p8

% p8tool luafmt helloworld.p8.png
% cat helloworld_fmt.p8
pico-8 cartridge // http://www.pico-8.com
version 5
__lua__
-- hello world
-- by zep

t = 0
music(0)
function _update()
  t += 1
end
function _draw()
  cls()
  for i=1,11 do
    for j0=0,7 do
      j = 7-j0
      col = 7+j
...

By default, the indentation width is 2 spaces. You can change the desired indentation width by specifying the

--indentwidth=...

% p8tool luafmt --indentwidth=4 helloworld.p8.png
% cat helloworld_fmt.p8
...
function _update()
    t += 1
end

function _draw()
    cls()
for i=1,11 do
    for j0=0,7 do
        j = 7-j0
        col = 7+j
...

The current version of

luafmt

p8tool luafind

The

luafind

Unlike common tools like

grep

luafind

grep

% p8tool luafind 'boards\[.*\]' *.p8*
test_gol.p8.png:11:  boards[1][y] = {}
test_gol.p8.png:12:  boards[2][y] = {}
test_gol.p8.png:14:    boards[1][y][x] = 0
test_gol.p8.png:15:    boards[2][y][x] = 0
test_gol.p8.png:20:boards[1][60][64] = 1
test_gol.p8.png:21:boards[1][60][65] = 1
test_gol.p8.png:22:boards[1][61][63] = 1
test_gol.p8.png:23:boards[1][61][64] = 1
test_gol.p8.png:24:boards[1][62][64] = 1
test_gol.p8.png:30:  return boards[bi][y][x]
test_gol.p8.png:36:      pset(x-1,y-1,boards[board_i][y][x] * alive_color)
test_gol.p8.png:54:          ((boards[board_i][y][x] == 1) and neighbors == 2)) then
test_gol.p8.png:55:        boards[other_i][y][x] = 1
test_gol.p8.png:57:        boards[other_i][y][x] = 0

You can tell

luafind

--listfiles

% p8tool luafind --listfiles 'self,' *.p8*
11243.p8.png
12029.p8.png
12997.p8.png
13350.p8.png
13375.p8.png
13739.p8.png
15216.p8.png
...

p8tool writep8

The

writep8

.p8

.p8.png

.p8

.p8

The command takes one or more cart filenames as arguments. For each cart with a name like

xxx.p8.png

xxx_fmt.p8

% p8tool writep8 helloworld.p8.png
% cat helloworld_fmt.p8
pico-8 cartridge // http://www.pico-8.com
version 5
__lua__
-- hello world
-- by zep

t = 0
music(0)
function _update()
 t += 1
end
function _draw()
 cls()
...

p8tool luamin

The

luamin

The command takes one or more cart filenames as arguments. For each cart with a name like

xxx.p8.png

xxx_fmt.p8

By default,

luamin

--keep-all-names

You can disable renaming just for specific names by providing a text file listing names, with this command line option:

--keep-names-from-file=

• Each name appears on its own line.
• Blank lines are ignored.
• Lines that begin with a

  #

  are ignored, so you can have comments in this file.

Disabling renaming is useful in cases where code refers to object attributes using both dot syntax (

foo.bar

foo["bar"]

luamin

"bar"

bar

foo

luamin

Statistically, you will run out of tokens before you run out of characters, and minifying is unlikely to affect the compressed character count. Carts are more useful to the PICO-8 community if the code in a published cart is readable and well-commented. That said, some authors have found luamin useful in the late stages of writing a large cart that includes blob data strings, where you might hit the character limit first and you need to squeeze in a few more characters to finish the game.

% p8tool luamin helloworld.p8.png
% cat helloworld_fmt.p8
pico-8 cartridge // http://www.pico-8.com
version 5
__lua__
a = 0
music(0)
function _update()
a += 1
end
function _draw()
cls()
...

p8tool listtokens

The

listtokens

listlua

% p8tool listtokens ./picotool-master/tests/testdata/helloworld.p8.png



<0:t>< ><1:=>< ><2:0>
<3:music><4:(><5:0><6:)>
<7:function>< ><8:_update><9:(><10:)>
< ><11:t>< ><12:+=>< ><13:1>
<14:end>
<15:function>< ><16:_draw><17:(><18:)>
< ><19:cls><20:(><21:)>
...

When picotool parses Lua code, it separates out comments, newlines, and spaces, as well as proper Lua tokens. The Lua tokens appear with an ascending number, illustrating how picotool counts the tokens. Non-token elements appear with similar angle brackets but no number. Newlines are rendered as is, without brackets, to make them easy to read.

Note: picotool does not currently count tokens the same way PICO-8 does. One purpose of

listtokens

p8tool printast

The

printast

% p8tool printast ./picotool-master/tests/testdata/helloworld.p8.png

Chunk

stats: [list:]

StatAssignment

varlist: VarList

vars: [list:]

VarName

name: TokName






explist: ExpList

exps: [list:]

ExpValue

value: 0








StatFunctionCall

functioncall: FunctionCall

exp_prefix: VarName

name: TokName


args: FunctionArgs

explist: ExpList

exps: [list:]

ExpValue

value: 0












StatFunction

funcname: FunctionName

namepath: [list:]

TokName


methodname: None


funcbody: FunctionBody

parlist: None
dots: None
block: Chunk







...

Building new tools

picotool provides a general purpose library for accessing and manipulating PICO-8 cart data. You can add the

picotool

PYTHONPATH

sys.path

pico8

The easiest way to load a cart from a file is with the

from_file()

pico8.game.file

#!/usr/bin/env python3

from pico8.game import file
g = file.from_file('mycart.p8.png')
print('Tokens: {}'.format(g.lua.get_token_count()))

Aspects of the game are accessible as attributes of the

Game

• lua

• gfx

• gff

• map

• sfx

• music

Lua code is treated as bytestrings throughout the API. This is because PICO-8 uses a custom text encoding equivalent to lower ASCII plus arbitrary high characters for the glyph set. Take care to use b'bytestring literals' when creating or comparing values.

API under construction

While the library in its current state is featureful enough for building simple tools, it is not yet ready to promise backwards compatibility in future releases. Feel free to mess with it, but please be patient if I change things.

Important: The parser in particular is kinda jank and I may completely redo the API for the AST before v1.0.

Developing picotool

To start developing

picotool

p8tool

git clone [email protected]:dansanderson/picotool.git
cd picotool
python3 -m venv pyenv
source pyenv/bin/activate

Install picotool in the virtual environment, with symlinks to source:
pip install -e .
Install development tools:
pip install --upgrade pip
pip install -r pydevtools.txt

With everything installed and the virtual environment active, you can run some useful commands:

# To run the development version of p8tool:
p8tool

To open an ipython REPL with picotool on the load path:
ipython
To enable module auto-reloading:
%autoreload 2
To import a pico8 module:
from pico8.lua import lua
Autocompletion works with the tab key:
lua.unicode
To run all tests:
pytest
To run specific tests:
pytest tests/pico8/lua/lua_tests.py
To run the tests and calculate a coverage report:
coverage run pytest
To display the coverage report on the console:
coverage report -m
To produce an interactive HTML document based on the coverage report, as ./htmlcov/index.html:
coverage html

See pytest documentation, coverage documentation.

Visual Studio Code set-up

I use Visual Studio Code for editing with its powerful Python extension. When you open the

picotool

./pyenv/bin/python

When you open a new Terminal within VSCode (press Ctrl + backtick), it automatically activates the Python environment. If it doesn't, or if you prefer a standalone terminal app, remember to activate the environment at the beginning of each session:

source picotool/pyenv/bin/activate

Building Sphinx documentation

The picotool documentation lives in

docs/

cd docs
make html

The generated HTML pages are in

docs/_build/html/

index.html

You are welcome and encouraged to contribute pull requests to the Sphinx documentation. Once accepted, I'll take care of publishing the changes to the documentation site.

(For my own reference: The documentation site uses the gh-pages branch of the Github repo as its source. There is a crude

make publish

Known issues

For a complete list of known issues, see the issues list in the Github project:

[https://github.com/dansanderson/picotool/issues]

picotool is a hobby project and I have to ration the time I spend on issues. I welcome pull requests, and preemptively apologize if I don't review them in a timely manner.

I try to triage issues such that picotool's main features work with all cartridges made with the '''latest knwon version''' of PICO-8. During the PICO-8 beta, picotool has lagged behind on new Lua syntax shortcuts and built-in keywords, though this should settle down as PICO-8 reaches version 1.0. So far, PICO-8 has been largely backwards compatible, but I cannot promise continued support for carts made with old beta versions (0.x) of PICO-8.

picotool and PICO-8 count tokens in slightly different ways, resulting in different counts. More refinement is needed so that picotool matches PICO-8. As far as I can tell, with picotool making some concessions to match PICO-8 in known cases, PICO-8's counts are consistently higher. In most cases, the difference is only by a few tokens, even for large carts. If you find specific cases of token count mismatches, please file them as issues with reproducable examples.