Need help with plaitpy?
Click the “chat” button below for chat support from the developer who created it, or find similar developers for support.

About the developer

plaitpy
415 Stars 14 Forks MIT License 12 Commits 4 Opened issues

Description

plait.py - a fake data modeler

Services available

!
?

Need anything else?

Contributors list

plait.py

plait.py is a program for generating fake data from composable yaml templates.

The idea behind plait.py is that it should be easy to model fake data that has an interesting shape. Currently, many fake data generators model their data as a collection of IID variables; with plait.py we can stitch together those variables into a more coherent model.

some example uses for plait.py are:

  • generating mock application data in test environments
  • validating the usefulness of statistical techniques
  • creating synthetic datasets for performance tuning databases

features

  • declarative syntax
  • use basic faker.rb fields with #{} interpolators
  • sample and join data from CSV files
  • lambda expressions, switch and mixture fields
  • nested and composable templates
  • static variables and hidden fields

an example template

# a person generator
define:
  min_age: 10
  minor_age: 13
  working_age: 18

fields: age: random: gauss(25, 5) # minimum age is $min_age finalize: max($min_age, value)

gender: mixture: - value: M - value: F

name: "#{name.name}" job: value: "#{job.title}" onlyif: this.age > $working_age

address: template: address/usa.yaml phone: # add a phone if the person is older than the minor age template: device/phone.yaml onlyif: this.age > ${minor_age}

we model our height as a gaussian that varies based on

age and gender

height: lambda: this._base_height * this._age_factor _base_height: switch: - onlyif: this.gender == "F" random: gauss(60, 5) - onlyif: this.gender == "M" random: gauss(70, 5)

_age_factor: switch: - onlyif: this.age < 15 lambda: 1 - (20 - (this.age + 5)) / 20 - default: value: 1

how its different

some specific examples of what plait.py can do:

  • generate proportional populations using census data and CSVs
  • create realistic zipcodes by state, city or region (also using CSVs)
  • create a taxi trip dataset with a cost model based on geodistance
  • add seasonal patterns (daily, weekly, etc) to data

usage

installation

# install with python
pip install plaitpy

or with pypy

pypy-pip install plaitpy

cloning the repo for development

git clone https://github.com/plaitpy/plaitpy

get the fakerb repo

git submodule init git submodule update

generating records from command line

specify a template as a yaml file, then generate records from that yaml file.

# a simple example (if cloning plait.py repo)
python main.py templates/timestamp/uniform.yaml

if plait.py is installed via pip

plait.py templates/timestamp/uniform.yaml

generating records from API

import plaitpy
t = plaitpy.Template("templates/timestamp/uniform.yaml")
print t.gen_record()
print t.gen_records(10)

looking up faker fields

plait.py also simplifies looking up faker fields:

# list faker namespaces
plait.py --list
# lookup faker namespaces
plait.py --lookup name

lookup faker keys

(-ll is short for --lookup)

plait.py --ll name.suffix

documentation

yaml file commands

  • see docs/FORMAT.md

datasets

  • see docs/EXAMPLES.md
  • also see templates/ dir

troubleshooting

  • see docs/TROUBLESHOOTING.md

Dependent Markov Processes

To simulate data that comes from many markov processes (a markov ecosystem), see the plaitpy-ipc repository.

future direction

If you have ideas on features to add, open an issue - Feedback is appreciated!

License

MIT

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.