Need help with polygott?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

replit
318 Stars 96 Forks MIT License 254 Commits 46 Opened issues

Description

Base Docker image for the Repl.it evaluation server

Services available

!
?

Need anything else?

Contributors list

Polygott

Overview

Replit.com allows you to quickly get started with any programming language online. In order to provide this capability, our evaluation server uses Docker with the

buildx
CLI plugin to ensure that all these languages are installed with the appropriate environment set up.

We previously used a separate Docker image for each language, but concluded that it was both simpler and more efficient to use a single image which contains all supported languages simultaneously. The code necessary to build this combined image, Polygott, resides in this repository. If you're lost and need some reference, we have a blog where we added elisp.

Because of the fact that building a monolithic image is unwieldy and takes too much time, the build itself is made of a directed graph of intermediate nodes, where each language is a node, and they all get stitched together at the end to build the final combined image.

Build and run

You can build either the entire image, a version that has a bundle of languages, and a version that is limited to a single language. The second one is used for CI, and the latter is recommended when you are adding or modifying support for a particular language, since building the entire image takes an extremely long time. Makefile targets are provided for each of these cases:

% make help
usage:
  make image         Build Docker image with all languages
  make image-ci      Build Docker image with all languages needed for CI
  make image-LANG    Build Docker image with single language LANG
  make run           Build and run image with all languages
  make run-LANG      Build and run image with single language LANG
  make test          Build and test all languages
  make test-LANG     Build and test single language LANG
  make changed-test  Build and test only changed/added languages
  make help          Show this message

As you can see, there is a facility for testing that languages have been installed and configured correctly. This involves running commands which are specified in the languages' configuration files, and checking that the output is as expected. To debug, you can also use the

make run
and
make run-LANG
targets to launch shells within the images.

You may want to bypass Docker's cache temporarily, for example to debug an intermittent network error while building one of the languages. To do this, identify the

docker build
command which is run by the Makefile, and run it yourself with the
--no-cache
flag.

The CI requires having up-to-date generated artifacts (the files in

out/
) committed to ensure a consistent Docker build environment. These will be refreshed by running the tests, or by running
make -B build/stamps/out
).

Language configuration

Each supported language has a TOML file in the

languages
subdirectory. The meaningful keys are as follows:

Mandatory

  • entrypoint
    : The name of the file which will be executed on Replit.com when you press the Run button. This is used in the
    detect-language
    script built into the Polygott image: if a file exists with this name, then the project is detected to have this language. (Ties are resolved by
    popularity
    .) It is also used by the
    run-project
    script in order to identify the main file of the project.
  • extensions
    : List of file extensions (use
    "py"
    , not
    ".py"
    ) which files of this language may have. This is used in the
    detect-language
    script built into the Polygott image: if a file exists with one of these extensions, then the project is detected to have this language. (Ties are resolved by
    popularity
    .)
  • name
    : The name of the language. The TOML file should then be named
    .toml
    . This is also what you pass to the Makefile's
    image-LANG
    and
    test-LANG
    targets.

Optional

  • aliases
    : List of strings indicating alternate names for the language, in addition to
    name
    . This is used to allow the
    run-language-server
    script to accept
    -l c++
    in addition to
    -l
    cpp
    , among other things.
  • aptKeys
    : List of PGP key IDs that must be passed to
    apt-key
    in order for the custom
    aptRepos
    configured to be trusted. For example,
    "09617FD37CC06B54"
    .
  • aptRepos
    : List of repository strings that must be passed to
    add-apt-repository
    in order for the custom
    packages
    configured to be available. For example,
    "deb
    https://dist.crystal-lang.org/apt crystal main"
    .
  • compile
    • command
      : The full command to compile the
      entrypoint
      file, as a list, including the filename. This is run before the
      run
      command.
  • languageServer
    • command
      : Command to start an LSP language server, as a list. This is used in the
      run-language-server
      script built into the Polygott image.
  • packages
    : List of additional Ubuntu packages to install for this language. (Packages which are required by all or many languages should be placed instead in
    packages.txt
    .) Check the Ubuntu Bionic package listing to see what your options are.
  • popularity
    : Floating-point number indicating how popular the language is. Defaults to
    2.0
    . This is used in the
    detect-language
    script built into the Polygott image: if a project is detected as more than one language, the winner is chosen by comparing popularity scores.
  • run
    : Required, unless you provide no
    tests
    or you have asked to
    skip
    all of them.
    • command
      : The full command to run the
      entrypoint
      file, as a list, including the filename. It is run after the
      compile
      command, if one is provided. This is used to run tests.
  • runtimeSetup
    : List of shell commands to be run by the
    polygott-lang-setup
    script built into the Polygott image.
  • setup
    : List of shell commands to be run in phase 2 of the build process, as post-install steps.
  • tests
    • TESTNAME
    • code
      : String to write to the
      entrypoint
      file before invoking the
      run
      command.
    • output
      : String expected to be written to stdout (not stderr) by running the code.
    • skip
      : Boolean, optional. If true, then the test is skipped. This allows you to easily "comment out" a test if there is something wrong with the infrastructure.
  • versionCommand
    : A command to output the version of the language, as a list of strings. For example,
    ["kotlin", "-version"]
    . This is used in the
    polygott-survey
    command built into the Polygott image. (If
    versionCommand
    is omitted, some heuristics are used to guess it.)

Build process

Usage

Aside from all the language executables (

python3
,
ruby
,
rust
, etc.), there are several additional scripts available within the Docker image. They are documented below.
polygott-self-test

Run the tests defined in each language's configuration file, as in

make test
or
make test-LANG
. Always run all the tests, but if one of them fails, exit with a non-zero return code.
polygott-survey

Run the

versionCommand
specified for every language, and output the results in tabular format to stdout.
polygott-lang-setup [-l LANG]

Copy the contents of

/opt/homes/LANG/
to
/home/runner/
, and run the
runtimeSetup
commands for it, if any were provided.
LANG
defaults to the output of
detect-language
.
detect-language

Try to identify the language used by the project in the current directory. This first checks if the

entrypoint
file exists for any language, and then checks if a file with any of the registered
extensions
exists for a language. If multiple languages match in either of those two phases, then the
popularity
of the two languages is used to resolve ties.

Output the language name to

stdout
if one is detected, otherwise do nothing.
run-project [-s] [-b] [-l LANG]

Execute the

compile
and
run
commands on the
entrypoint
file in the current directory.
LANG
defaults to the output of
detect-language
. If
-s
is passed, then the
entrypoint
file is written with the contents of stdin. If
-b
is passed, then some special logic is used instead of the
compile
and
run
commands; see the source for details.
run-language-server [-l LANG]

Run the

languageServer
command configured in the language's configuration file.
LANG
defaults to the output of
detect-language
.

Deployment

When a commit is merged to

master
, CircleCI automatically builds Polygott and pushes the image to Docker Hub. A Replit engineer has to then push the new Polygott to production.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.