farm

by zerocracy

zerocracy / farm

Zerocrat Core Engine

201 Stars 55 Forks Last release: Not found Other 4.1K Commits 391 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

EO principles respected here Managed by Zerocracy DevOps By Rultor.com We recommend IntelliJ IDEA

Maven Build PDD status Hits-of-Code

Stability of Webhook Availability at SixNines

Read this article first: What is it?

It's a core repository of Zerocrat. It contains our persistence layer (

com.zerocracy.farm
), a collection of Java stakeholders (
com.zerocracy.stk
), and interface layer for the integration with Slack, GitHub, Telegram, and so on (
com.stakeholder.radars
).

The data model (XML, XSD, XSL documents) is in zerocracy/datum repository. They are released separately and have different versions.

Claims

The central point of control in the project is

claims.xml
file, which stores all requests for actions, so called "claims." Say, someone wants to add a new job to the WBS, either a user or a software module. This claim has to be added:

  Add job to WBS
  2016-12-29T09:03:21.684Z
  yegor256
  slack;C43789437;yegor256
  
    gh:test/test#1
  

Stakeholder is a software module that replies to a claim. All stakeholders are Groovy scripts from

com.zerocracy.stk
package.

Here,

type
is a unique type of the claim, which will be used by "stakeholders," to decide which claim to process or to ignore. The
author
is the optional GitHub login of the person who submitted the claim; it's empty if the claim is coming from a software module or another stakeholder. The
token
is the location of the place where the response is expected; in this example the response is expected in Slack channel
C43789437
and has to be addressed to
@yegor256
. The
params
is just an associative array of parameters.

One of the stakeholders will find that claim and reply to it. To read the claim we use

com.zerocracy.pm.ClaimIn
, which helps proceeding the XML. To generate a claim we use
com.zerocracy.pm.ClaimOut
.

There are a number of params, which are typical for many claim types:

  • cause
    is the ID of the claim that was preceeding the current one;
  • flow
    is a semi-colon separated list of all claim types seen before the current claim;
  • job
    is the unique name of the job, for example
    gh:test/test#1
    ;
  • login
    is the GitHub login of the user who the claim should deal with;
  • reason
    is a free text explanation of the reason.

Farm, Project, Item

A farm is collection of projects. A project is a collection of items. An item is just a file, in most cases in XML format.

For example, in order to assign a

DEV
role to @yegor256 in
C63314D6Z
, we should do this (provided, we already have the
farm
):
Project project = farm.find("@id='C63314D6Z'").get(0);
try (Item item = project.acq("roles.xml")) {
  new Xocument(item).modify(
    new Directives()
      .xpath("/roles/people[@id='yegor256']")
      .add("role")
      .set("DEV")
  );
}

Here, we use

find()
in order to retrieve a list of projects by the provided XPath term
@id='C63314D6Z'
. They will be found inside
people.xml
and returned, if found. If a project is not found, it will be created by
find()
.

Then, we use

acq()
to find and lock the file
roles.xml
in the project. Until
item.close()
is called, no other thread will be able to acquire any file in the project.

Then, we modify the file using

Xocument
, which is a helper created exactly for XML reading and modifications of items. We provide it a list of Xembly directives and it applies them to the XML document. It takes care about versioning and XSD validation.

PMO

PMO (project management office) is a project with a special status. It has its own set of items, own XSD schemas, everything on its own. We keep system information there, like list of all users (

people.xml
), list of all projects (
catalog.xml
), user awards (
awards/.xml
), etc.

The best way to get access to PMO is through class

Pmo
, having an instance of a farm. For example, in a Groovy stakeholder:
Farm farm = binding.variables.farm
Project pmo = new Pmo(farm)

Stakeholders

A stakeholder is a software module (object of interface

Stakeholder
) that consumes claims. As soon as a new claims shows up in
claims.xml
, the classes from
com.zerocracy.farm.reactive
try to send it to all known stakeholders. We write them in Groovy and keep in
com.zerocracy.stk
package. For example, this stakeholder may react to a claim that requests to assign a new role to a user in a project:
def exec(Project project, XML xml) {
  new Assume(project, xml).notPmo()
  new Assume(project, xml).type('Assign role')
  new Assume(project, xml).roles('ARC', 'PO')
  ClaimIn claim = new ClaimIn(xml)
  String login = claim.param('login')
  String role = claim.param('role')
  new Roles(project).bootstrap().assign(login, role)
  claim.copy()
    .type('Role was assigned')
    .postTo(project)
  claim.reply(
    new Par('Role %s was assigned to @%s').say(role, login)
  ).postTo(project)
}

First, we use

Assume
in order to filter out incoming claims that we don't need. Remember, each stakeholder receives all claims in a project. This particular stakeholder needs just one claim of type
"Assign role"
. We also allow only the architect (
ARC
) and the product owner (
PO
) to send those role-assigning claims.

Then, we create a very convenient helper class

ClaimIn
, which is designed to simplify our work with the incoming XML claim.

Then, we take

login
and
role
out of the claim. They are the parameters of the claim.

Then, we do the actual work of assigning the role to the user. Pay attention to the

.bootstrap()
call on
Roles
. It is important to always call those
boostrap()
methods on all data-representing objects, in order to ensure that the XML documents they represent are fully ready.

Next, we create a new claim and post back to the project. We use

.copy()
in order to copy the incoming claim entirely. The outcoming claim of type
"Role was assigned"
will contain the same set of parameters as the incoming one had.

Then, we reply to the original claim with a user-friendly message. If the incoming claim had an author (a real user), that user will receive a message, either in Telegram, or Slack or wherever that claim was submitted.

Pay attention to the class

Par
we are using in order to format the message. This class is supposed to be used everywhere, since it formats the text correctly for all possible output devices and messengers.

Objects

"Data-representing" objects stay in

com.zerocracy.pm
and
com.zerocracy.pmo
packages. They mostly represent XML documents from the storage, one class per document, e.g.
Boosts
for
boosts.xml
or
Roles
for
roles.xml
. They all are pretty straight-forward XML manipulators, where jcabi-xml is used for XML reading and Xembly for XML modifications.

Validations are also supposed to happen inside these objects. The majority of data problems will be filtered out by XSD Schemas, but not all of them. Sometimes we need Java to do the work of data validating. If it's needed, we try to validate the data in data-representing objects.

Bundles

In order to integrate and test the entire system we have a collection of "bundles" in

com.zerocracy.bundles
package, which are simulators of real projects. Each bundle is a collection of files, which we place into a fake project and run claims dispatcher, just like it would happen in a real project.
BundlesTest
does this. If some fails need to be placed into PMO project then they should be prefixed with
pmo_
, e.g. pmopeople.xml, unless it is a PMO test (with `setup.xml
/setup/pmo
set to
true
), then there is no need to prefix the files
with
pmo_`.

In order to create a new bundle you just copy an existing one and edit its files. The key file, of course, is the

claims.xml
, which contains the list of claims to be dispatched. There are also a few supplementary files:
  • _before.groovy
    is a stakeholder that is called right before all claims are dispatched; obviously, it doesn't receive anything meaningful as an XML input.
  • _after.groovy
    is a stakeholder that is called right after all claims are dispatched.
  • _setup.xml
    is a configuration file with information for
    BundlesTest
    ; setting
    /setup/pmo
    to
    true
    will make sure that dispatching happens with a PMO project, not a regular one.

More details you can find in the Javadoc section of

BundlesTest
.

The entire

BundlesTest
suite can take a very long amount of time to execute. If you are debugging a certain test or function, you can specify the exact tests to run by specifying a comma separated list via
-DbundlesTests
. E.g. to run the
assign_role
and
cancel_order
tests:

$mvn clean install -Pqulice -DbundlesTests=assign_role,cancel_order

Alternatively, you can skip

BundlesTest
suite execution altogether by specifying
-DskipBundlesTest
:

$mvn clean install -Pqulice -DskipBundlesTest

Radars

There are a number of entry points, where users can communicate with our chat bots, they are all implemented in

com.zerocracy.radars.*
packages. Each bot has its own implementation details, because systems are very different (Telegram, Slack, GitHub, etc.). The common part is the
Question
class, that parses the questions and translates them to claims.

We try to keep radars lightweight and logic-free. It's not their job to make decisions about jobs, orders, roles, rates, etc. Their job is to translate the incoming information into claims. The rest will be done by stakeholders.

Policy

There are a number of constants in the application, which affect the business logic. For example, the amount of reputation points a programmer pays when a job is delayed, or the amount of money a client pays in order to publish an RfP, an so on. All of them are defined in our Policy as HTML

elements with certain
id
attributes (see the source code of the page). Then, we have a class
com.zerocracy.Policy
, which helps us fetch the values from the policy:
int days = new Policy().get("18.days", 90);

Here,

"18.days"
is the HTML
id
attribute and
90
is the default value to be used during unit testing. You must always use class
Policy
in your code and never hard-code any business constants.

Time API

We don't mix different Java Time APIs and we have chosen the new java.time.* classes instead of the old Date and Calendar classes. Old classes can be used only in cases where external libraries require or return them.

When considering which of the new classes to use, it is best to first try

Instant
, if more formatting or manipulation of the date/time is needed then
ZonedDateTime
with ZoneOffset.UTC. LocalDateTime/LocalDate/LocalTime should be used as a last resort (as it is e.g. problematic during the switch to daylight saving).

How to contribute

Just fork it, make changes, run

mvn clean install -Pqulice,codenarc
, and submit a pull request. Read this, if lost.

To validate XML, XSL and XSD issues use xcop: install it with

gem install xcop
and add to
PATH
.

Keep in mind that you don't need to setup the server locally or start it. If you need to prove that a class is working - write a unit tests for it or integration tests if external API is involved (see

ClaimsSqsITCase
for instance). See this for details: https://www.yegor256.com/2016/02/09/are-you-still-debugging.html

Don't forget to add documentation for groovy scripts if you create new stakeholder.

Maven profiles

There are maven profiles which you can enable:

  • qulice
    - enables Qulice (Source Code Quality Police) profile
  • codenarc
    - enables Codenarc validation
  • upgrade-bundles
    - fetch fresh xml schemas from
    datum
    repo and update bundle tests
  • dynamodb
    - starts local dynamodb instance for integration testing

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.