picotool: Tools and Python libraries for manipulating Pico-8 game files

Pico-8 is a fantasy game console by Lexaloffle Games. The Pico-8 runtime environment runs cartridges (or carts): game files containing code, graphics, sound, and music data. The console includes a built-in editor for writing games. Game cartridge files can be played in a browser, and can be posted to the Lexaloffle bulletin board or exported to any website.

picotool

Useful tools include:

• p8tool build

  : assembles cartridges from multiple sources, as part of a game development workflow

• p8tool stats

  : reports statistics on one or many cartridge files

• p8tool listlua

  : prints the Lua code of a cartridge

• p8tool luafind

  : searches the Lua code of a collection of cartridges

• p8tool luafmt

  : formats the Lua code of a cartridge to make it easier to read

There are additional tools that are mostly useful for demonstrating and troubleshooting the library:

writep8

listtokens

printast

luamin

p8upsidedown

picotool

.p8

.p8.png

Installing picotool

To install the

picotool

1. Download and unpack the zip archive, or use Git to clone the Github repository.
   • Unpacking the zip archive creates a root directory named

     picotool-master

     . When cloning the repo, this is just

     picotool

     , or whatever you named it when you cloned it.
2. Install Python 3 version 3.4 or later, if necessary. (picotool has not been tested with Python 2.)
3. To enable PNG support, install the PyPNG library:

   python3 -m pip install pypng
   

Using picotool

To use a tool, you run the

p8tool

stats

For example, to print statistics about a cart named

helloworld.p8.png

./picotool-master/p8tool stats helloworld.p8.png

p8tool build

The

build

The tool takes the filename of the output cartridge, with additional arguments describing the build. If the output cartridge does not exist, the build starts with an empty cartridge. Otherwise, it uses the existing cartridge as the default, and overwrites sections of it based on the arguments.

For example, you can create a cartridge in Pico-8, use Pico-8's built-in graphics and sound editors, then use

p8tool build

.lua

% ./picotool-master/p8tool build mygame.p8.png --lua mygame.lua

As another example, to create a new cartridge using the spritesheet (

gfx

sfx

music

.lua

% ./picotool-master/p8tool build mygame.p8.png --gfx mygamegfx.p8 --sfx radsnds.p8.png --music radsnds.p8.png --lua mygame.lua

You can also erase a section of an existing cart with an argument such as

--empty-map

The available arguments are as follows:

• --lua LUA

  : use Lua code from the given cart or

  .lua

  file

• --gfx GFX

  : use spritesheet from the given cart

• --gff GFF

  : use sprite flags from the given cart

• --map MAP

  : use map from the given cart

• --sfx SFX

  : use sound effects from the given cart

• --music MUSIC

  : use music patterns from the given cart

• --empty-lua

  : use an empty Lua code section

• --empty-gfx

  : use an empty spritesheet

• --empty-gff

  : use empty sprite flags

• --empty-map

  : use an empty map

• --empty-sfx

  : use empty sound effects

• --empty-music

  : use empty music patterns

If the output cart filename ends with

.p8.png

.p8.png

p8tool build

Packages and the require() function

p8tool build supports a special feature for organizing your Lua code, called packages. When loading Lua code from a file with the

--lua mygame.lua

require()

require()

Consider the following simple example. Say you have a function you like to use in several games in a file called

mylib.lua

function handyfunc(x, y)
  return x + y
end

handynumber = 3.14

Your main game code is in a file named

mygame.lua

handyfunc()

mygame.lua

require()

require("mylib")

result = handyfunc(2, 3)
print(result)
r = 5
area = handynumber * r * r

All globals defined in the required file are set as globals in your program when

require()

A more typical way to write a Lua package is to put everything intended to be used by other programs in a table:

HandyPackage = {
  handyfunc = function(x, y)
    return x + y
  end,
  handynumber = 3.14,
}

Then in

mygame.lua

require("mylib")

result = HandyPackage.handyfunc(2, 3)

This is cleaner, but still has the disadvantage that the package must be known by the global name

HandyPackage

mygame.lua

return

require()

local

local HandyPackage = {
  handyfunc = function(x, y)
    return x + y
  end,
  handynumber = 3.14,
}

return HandyPackage

The main program uses the return value of

require()

HandyPackage = require("mylib")

result = HandyPackage.handyfunc(2, 3)

The

require()

require()

require()

Where packages are located

The first argument to

require()

;

?

The default lookup path is

?;?.lua

require("mylib")

mylib

mylib.lua

require()

/usr/share/pico8/lib/?.lua

--lua-path=...

For example, with this environment variable set:

PICO8_LUA_PATH=?;?.lua;/home/dan/p8libs/?/?.p8

The require("3dlib") statement would look for these files, in this order, with paths relative to the file containing the require() statement:

3dlib
3dlib.lua
/home/dan/p8libs/3dlib/3dlib.p8

To prevent malicious code from accessing arbitrary files on your hard drive (unlikely but it's nice to prevent it), the

require()

../

require("math/linear")

As with Lua, packages are remembered by the string name used with

require()

foo.lua

?;?.lua

require("foo")

require("foo.lua")

Packages and game loop functions

When you write a library of routines for Pico-8, you probably want to write test code for those routines. picotool assumes that this test code would be executed in a Pico-8 game loop, such that the library can be in its own test cart. For this purpose, you can write your library file with

_init()

_update()

_update60()

_draw()

require()

For example:

local HandyPackage = {
  handyfunc = function(x, y)
    return x + y
  end,
  handynumber = 3.14,
}

function _update()
  test1 = HandyPackage.handyfunc(2, 3)
end
function _draw()
  cls()
  print('test1 = '..test1)
end
return HandyPackage

If you want to keep the game loop functions present in a package, you can request them with a second argument to

require()

require("mylib", {use_game_loop=true})

How require() actually works

Of course, Pico-8 does not actually load packages from disk when it runs the cartridge. Instead, picotool inserts each package into the cartridge code in a special way that replicates the behavior of the Lua

require()

When you run

p8tool build

--lua=...

require()

require()

Each required library is stored once as a function object in a table inserted at the top of the final cartridge's code. A definition of the

require()

To match Lua's behavior,

require()

package.loaded

nil

require()

This feature incurs a small amount of overhead in terms of tokens. Each library uses tokens for its own code, plus a few additional tokens for storing it in the table. The definition for

require()

Formatting or minifying Lua in a built cart

You can tell

p8tool build

--lua-format

--lua-minify

% ./picotool-master/p8tool build mycart.p8.png --lua=mygame.lua --lua-format

This is equivalent to building the cart then running

p8tool luafmt

p8tool luamin

p8tool stats

The

stats

% ./picotool-master/p8tool stats helloworld.p8.png 
hello world (helloworld.p8.png)
by zep
version: 0  lines: 48  chars: 419  tokens: 134

This command accepts an optional

--csv

% ./picotool-master/p8tool --csv stats mycarts/*.p8* >cartstats.csv

p8tool listlua

The

listlua

% ./picotool-master/p8tool listlua helloworld.p8.png 
-- hello world
-- by zep

t = 0
music(0)
function _update()
 t += 1
end
function _draw()
 cls()
...

p8tool luafmt

The

luafmt

The command takes one or more cart filenames as arguments. For each cart with a name like

xxx.p8.png

xxx_fmt.p8

% ./picotool-master/p8tool luafmt helloworld.p8.png 
% cat helloworld_fmt.p8
pico-8 cartridge // http://www.pico-8.com
version 5
__lua__
-- hello world
-- by zep

t = 0
music(0)
function _update()
  t += 1
end
function _draw()
  cls()
  for i=1,11 do
    for j0=0,7 do
      j = 7-j0
      col = 7+j
...

By default, the indentation width is 2 spaces. You can change the desired indentation width by specifying the

--indentwidth=...

% ./picotool-master/p8tool luafmt --indentwidth=4 helloworld.p8.png 
% cat helloworld_fmt.p8
...
function _update()
    t += 1
end

function _draw()
    cls()
for i=1,11 do
    for j0=0,7 do
        j = 7-j0
        col = 7+j
...

The current version of

luafmt

p8tool luafind

The

luafind

Unlike common tools like

grep

luafind

grep

% ./picotool-master/p8tool luafind 'boards\[.*\]' *.p8*
test_gol.p8.png:11:  boards[1][y] = {}
test_gol.p8.png:12:  boards[2][y] = {}
test_gol.p8.png:14:    boards[1][y][x] = 0
test_gol.p8.png:15:    boards[2][y][x] = 0
test_gol.p8.png:20:boards[1][60][64] = 1
test_gol.p8.png:21:boards[1][60][65] = 1
test_gol.p8.png:22:boards[1][61][63] = 1
test_gol.p8.png:23:boards[1][61][64] = 1
test_gol.p8.png:24:boards[1][62][64] = 1
test_gol.p8.png:30:  return boards[bi][y][x]
test_gol.p8.png:36:      pset(x-1,y-1,boards[board_i][y][x] * alive_color)
test_gol.p8.png:54:          ((boards[board_i][y][x] == 1) and neighbors == 2)) then
test_gol.p8.png:55:        boards[other_i][y][x] = 1
test_gol.p8.png:57:        boards[other_i][y][x] = 0

You can tell

luafind

--listfiles

% ./picotool-master/p8tool luafind --listfiles 'self,' *.p8*
11243.p8.png
12029.p8.png
12997.p8.png
13350.p8.png
13375.p8.png
13739.p8.png
15216.p8.png
...

p8tool writep8

The

writep8

.p8

.p8.png

.p8

.p8

The command takes one or more cart filenames as arguments. For each cart with a name like

xxx.p8.png

xxx_fmt.p8

% ./picotool-master/p8tool writep8 helloworld.p8.png 
% cat helloworld_fmt.p8
pico-8 cartridge // http://www.pico-8.com
version 5
__lua__
-- hello world
-- by zep

t = 0
music(0)
function _update()
 t += 1
end
function _draw()
 cls()
...

p8tool luamin

The

luamin

The command takes one or more cart filenames as arguments. For each cart with a name like

xxx.p8.png

xxx_fmt.p8

I don't recommend using this tool when publishing your games. Statistically, you will run out of tokens before you run out of characters, and minifying is unlikely to affect the compressed character count. Carts are more useful to the Pico-8 community if the code in a published cart is readable and well-commented. I only wrote

luamin

% ./picotool-master/p8tool luamin helloworld.p8.png 
% cat helloworld_fmt.p8
pico-8 cartridge // http://www.pico-8.com
version 5
__lua__
a = 0
music(0)
function _update()
a += 1
end
function _draw()
cls()
...

p8tool listtokens

The

listtokens

listlua

% ./picotool-master/p8tool listtokens ./picotool-master/tests/testdata/helloworld.p8.png 



<0:t>< ><1:=>< ><2:0>
<3:music><4:(><5:0><6:)>
<7:function>< ><8:_update><9:(><10:)>
< ><11:t>< ><12:+=>< ><13:1>
<14:end>
<15:function>< ><16:_draw><17:(><18:)>
< ><19:cls><20:(><21:)>
...

When picotool parses Lua code, it separates out comments, newlines, and spaces, as well as proper Lua tokens. The Lua tokens appear with an ascending number, illustrating how picotool counts the tokens. Non-token elements appear with similar angle brackets but no number. Newlines are rendered as is, without brackets, to make them easy to read.

Note: picotool does not currently count tokens the same way Pico-8 does. One purpose of

listtokens

p8tool printast

The

printast

% ./picotool-master/p8tool printast ./picotool-master/tests/testdata/helloworld.p8.png 

Chunk

stats: [list:]

StatAssignment

varlist: VarList

vars: [list:]

VarName

name: TokName






explist: ExpList

exps: [list:]

ExpValue

value: 0








StatFunctionCall

functioncall: FunctionCall

exp_prefix: VarName

name: TokName


args: FunctionArgs

explist: ExpList

exps: [list:]

ExpValue

value: 0












StatFunction

funcname: FunctionName

namepath: [list:]

TokName


methodname: None


funcbody: FunctionBody

parlist: None
dots: None
block: Chunk







...

Building new tools

picotool provides a general purpose library for accessing and manipulating Pico-8 cart data. You can add the

picotool

PYTHONPATH

sys.path

pico8

The easiest way to load a cart from a file is with the

Game.from_filename()

pico8.game.game

#!/usr/bin/env python3

from pico8.game import game
g = game.Game.from_filename('mycart.p8.png')
print('Tokens: {}'.format(g.lua.get_token_count()))

Aspects of the game are accessible as attributes of the

Game

• lua

• gfx

• gff

• map

• sfx

• music

Lua code is treated as bytestrings throughout the API. This is because Pico-8 uses a custom text encoding equivalent to lower ASCII plus arbitrary high characters for the glyph set. Take care to use b'bytestring literals' when creating or comparing values.

API under construction!

While the library in its current state is featureful enough for building simple tools, it is not yet ready to promise backwards compatibility in future releases. Feel free to mess with it, but please be patient if I change things.

Developing picotool

If you want to change the picotool code and run its test suite, you will need the Nose test runner and the coverage tool. You can install these with

pip

python3 -m pip install nose coverage

To run the test suite:

python3 run_tests.py

By default, this produces an HTML coverage report in the

cover

.../picotool-master/cover/index.html

Known issues

• picotool and Pico-8 count tokens in slightly different ways, resulting in different counts. More refinement is needed so that picotool matches Pico-8. As far as I can tell, with picotool making some concessions to match Pico-8 in known cases, Pico-8's counts are consistently higher. So I'm missing a few cases where Pico-8 over-counts (or picotool under-counts). In most cases, the difference is only by a few tokens, even for large carts.
• Lua allows parentheses to be omitted from a function call if there is exactly one argument and the argument is a string or table literal. picotool does not yet support this case. Pico-8 supports it and offers a one-token discount compared to a similar call with parentheses.
• I've found a few very obscure cases where picotool rejects a cart with a syntax error but Pico-8 accepts it (13757, 16983, 18012, 18556). I evaluate these based on their rarity, usefulness, and intrusiveness to support, and for now I'm leaving this as is.

For more known issues, see the issues list in the Github project: https://github.com/dansanderson/picotool/issues