Kumogata2 is now available! Please try it out!

Kumogata

Kumogata is a tool for AWS CloudFormation.

This is a

format converter

useful tool

• JSON
• Ruby
• YAML
• JavaScript
• CoffeeScript (experimental)
• JSON5 (experimental)

It can define a template in Ruby DSL, such as:

AWSTemplateFormatVersion "2010-09-09"

Description (<
Ruby template structure is almost the same as JSON template.

(You can also use JSON templates)

Installation
$ gem install kumogata

Usage
Usage: kumogata  [args] [options]

Commands:
  create         PATH_OR_URL [STACK_NAME]   Create resources as specified in the template
  validate       PATH_OR_URL                Validate a specified template
  convert        PATH_OR_URL                Convert a template format
  update         PATH_OR_URL STACK_NAME     Update a stack as specified in the template
  delete         STACK_NAME                 Delete a specified stack
  list           [STACK_NAME]               List summary information for stacks
  export         STACK_NAME                 Export a template from a specified stack
  show-events    STACK_NAME                 Show events for a specified stack
  show-outputs   STACK_NAME                 Show outputs for a specified stack
  show-resources STACK_NAME                 Show resources for a specified stack
  diff           PATH_OR_URL1 PATH_OR_URL2  Compare templates logically (file, http://..., stack://...)
Options:
    -k, --access-key ACCESS_KEY
    -s, --secret-key SECRET_KEY
    -r, --region REGION
        --profile CONFIG_PROFILE
        --credentials-path PATH
        --config-path PATH
        --format TMPLATE_FORMAT
        --output-format FORMAT
        --skip-replace-underscore
        --deletion-policy-retain
    -p, --parameters KEY_VALUES
    -j, --json-parameters JSON
    -e, --encrypt-parameters KEYS
        --encryption-password PASS
        --skip-send-password
        --capabilities CAPABILITIES
        --disable-rollback
        --notify SNS_TOPICS
        --timeout MINUTES
        --result-log PATH
        --command-result-log PATH
        --detach
        --force
    -w, --ignore-all-space
        --color
        --no-color
        --debug
    -v, --verbose

KUMOGATA_OPTIONS

KUMOGATA_OPTIONS
 variable specifies default options.

e.g. 
KUMOGATA_OPTIONS='-e Password'

Create resources
$ kumogata create template.rb

If you want to save the stack, please specify the stack name:
$ kumogata create template.rb any_stack_name

If you want to pass parameters, please use 
-p
 option:
$ kumogata create template.rb -p "InstanceType=m1.large,KeyName=any_other_key"

Notice

The stack will be delete if you do not specify the stack name explicitly.
(And only the resources will remain)

Convert JSON to Ruby

JSON template can be converted to Ruby template.
$ kumogata convert https://s3.amazonaws.com/cloudformation-templates-us-east-1/Drupal_Single_Instance.template

• Data that cannot be converted will be converted to Array and Hash

• ::

  is converted to

  __

  • Fn::GetAtt

    =>

    Fn__GetAtt

• _{ ... }

  is convered to Hash

  • SecurityGroups [_{Ref "WebServerSecurityGroup"}]

    =>

    {"SecurityGroups": [{"Ref": "WebServerSecurityGroup"}]}

• _path()

  creates Hash that has a key of path

  • _path("/etc/passwd-s3fs") { content "..." }

    =>

    {"/etc/passwd-s3fs": {"content": "..."}}

• ~~userdata() creates Base64-encoded UserData~~

  • _user_data()

    has been removed

• _join()

  has been removed

String#fn_join()

Ruby templates will be converted as follows by

String#fn_join()

UserData do
  Fn__Base64 (< -r myEC2Instance --region 
  EOS
end

"UserData": {
  "Fn::Base64": {
    "Fn::Join": [
      "",
      [
        "#!/bin/bash\n",
        "/opt/aws/bin/cfn-init -s ",
        {
          "Ref": "AWS::StackName"
        },
        " -r myEC2Instance --region ",
        {
          "Ref": "AWS::Region"
        },
        "\n"
      ]
    ]
  }
}

Split a template file

• template.rb

Resources do
  _include 'template2.rb', :ami_id => 'ami-XXXXXXXX'
end

• template2.rb

myEC2Instance do
  Type "AWS::EC2::Instance"
  Properties do
    ImageId args[:ami_id]
    InstanceType { Ref "InstanceType" }
    KeyName "your_key_name"
  end
end

• Converted JSON template

{
  "Resources": {
    "myEC2Instance": {
      "Type": "AWS::EC2::Instance",
      "Properties": {
        "ImageId": "ami-XXXXXXXX",
        "InstanceType": {
          "Ref": "InstanceType"
        },
        "KeyName": "your_key_name"
      }
    }
  }
}

Encrypt parameters

• Command line

$ kumogata create template.rb -e 'Password1,Password2' -p 'Param1=xxx,Param2=xxx,Password1=xxx,Password2=xxx'

• Template

Parameters do
  Param1 { Type "String" }
  Param2 { Type "String" }
  Password1 { Type "String"; NoEcho true }
  Password2 { Type "String"; NoEcho true }
end # Parameters

Resources do
  myEC2Instance do
    Type "AWS::EC2::Instance"
Properties do
  ImageId "ami-XXXXXXXX"

  UserData do
    Fn__Base64 (< -r myEC2Instance --region 
    EOS
  end
end

Metadata do
  AWS__CloudFormation__Init do
    config do
      commands do
        any_command do
          command (<' | base64 -d`"

            # Decrypt Password1
            echo '' | base64 -d | openssl enc -d -aes256 -pass pass:"$ENCRYPTION_PASSWORD" > password1

            # Decrypt Password2
            echo '' | base64 -d | openssl enc -d -aes256 -pass pass:"$ENCRYPTION_PASSWORD" > password2
          EOS
        end
      end
    end
  end
end
  end # myEC2Instance
end # Resources

Iteration

You can use the Iteration in the template using

_(...)

Resources do
  ['instance1', 'instance2', 'instance3'].each {|instance_name|
    _(instance_name) do
      Type "AWS::EC2::Instance"
      Properties do
        ImageId "ami-XXXXXXXX"
        InstanceType { Ref "InstanceType" }
        KeyName "your_key_name"

    UserData (<
Post command

You can run shell/ssh commands after building servers using 
_post()
.

• Template ```ruby Parameters do ... end

Resources do ... end

Outputs do MyPublicIp do Value { Fn__GetAtt name, "PublicIp" } end end

post do myshellcommand do command <<-EOS echo <%= Key "MyPublicIp" %> EOS end mysshcommand do ssh do host { Key "MyPublicIp" } # or '<%= Key "MyPublicIp" %>' user "ec2-user" # see http://net-ssh.github.io/net-ssh/classes/Net/SSH.html#method-c-start #options :timeout => 300 #connecttries 36 #retryinterval 5 #requestpty true end command <<-EOS hostname EOS end end ```

• Execution result ``` ... Command: myshellcommand Status: 0 1> 54.199.251.30

Command: mysshcommand Status: 0 1> ip-10-0-129-20

(Save to

/foo/bar/command_result.json

JavaScript template

You can also use the JavaScript template instead of JSON and Ruby.

function fetch_ami() {
  return "ami-XXXXXXXX";
}

/* For JS Object is evaluated last, it must be enclosed in parentheses /
({
  Resources: { / comment /
    myEC2Instance: {
      Type: "AWS::EC2::Instance",
      Properties: {
        ImageId: fetch_ami(),
        InstanceType: "t1.micro"
      }
    }
  },
  Outputs: {
    AZ: { / comment */
      Value: {
        "Fn::GetAtt": [
          "myEC2Instance",
          "AvailabilityZone"
        ]
      }
    }
  }
})
/*
 {
   "Resources": {
     "myEC2Instance": {
       "Type": "AWS::EC2::Instance",
       "Properties": {
         "ImageId": "ami-XXXXXXXX",
         "InstanceType": "t1.micro"
       }
     }
   },
   "Outputs": {
     "AZ": {
       "Value": {
         "Fn::GetAtt": [
           "myEC2Instance",
           "AvailabilityZone"
         ]
       }
     }
   }
 }
 */

Convert JSON template to JavaScript

$ kumogata convert Drupal_Single_Instance.template --output-format=js

CoffeeScript template

You can also use the CoffeeScript template instead of JSON and Ruby.

fetch_ami = () -> "ami-XXXXXXXX"

/* For JS Object is evaluated last, it must use return */
return {
  Resources:
    myEC2Instance:
      Type: "AWS::EC2::Instance",
      Properties:
        ImageId: fetch_ami(),
        InstanceType: "t1.micro"
  Outputs:
    AZ: # comment
      Value:
        "Fn::GetAtt": [
          "myEC2Instance",
          "AvailabilityZone"
        ]
}
###
 {
   "Resources": {
     "myEC2Instance": {
       "Type": "AWS::EC2::Instance",
       "Properties": {
         "ImageId": "ami-XXXXXXXX",
         "InstanceType": "t1.micro"
       }
     }
   },
   "Outputs": {
     "AZ": {
       "Value": {
         "Fn::GetAtt": [
           "myEC2Instance",
           "AvailabilityZone"
         ]
       }
     }
   }
 }
###

YAML template

You can also use the YAML template instead of JSON and Ruby.

---
Resources:
  myEC2Instance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: ami-XXXXXXXX
      InstanceType: t1.micro
Outputs:
  AZ:
    Value:
      Fn::GetAtt:
      - myEC2Instance
      - AvailabilityZone

{
"Resources": {
"myEC2Instance": {
"Type": "AWS::EC2::Instance",
"Properties": {
"ImageId": "ami-XXXXXXXX",
"InstanceType": "t1.micro"
}
}
},
"Outputs": {
"AZ": {
"Value": {
"Fn::GetAtt": [
"myEC2Instance",
"AvailabilityZone"
]
}
}
}
}

Convert JSON template to YAML

$ kumogata convert Drupal_Single_Instance.template --output-format=yaml

JSON5 template

You can also use the JSON5 template instead of JSON and Ruby.

{
  Resources: { /* comment */
    myEC2Instance: {
      Type: "AWS::EC2::Instance",
      Properties: {
        ImageId: "ami-XXXXXXXX",
        InstanceType: "t1.micro"
      }
    }
  },
  Outputs: {
    AZ: { /* comment */
      Value: {
        "Fn::GetAtt": [
          "myEC2Instance",
          "AvailabilityZone"
        ]
      }
    }
  }
}

/*
 {
   "Resources": {
     "myEC2Instance": {
       "Type": "AWS::EC2::Instance",
       "Properties": {
         "ImageId": "ami-XXXXXXXX",
         "InstanceType": "t1.micro"
       }
     }
   },
   "Outputs": {
     "AZ": {
       "Value": {
         "Fn::GetAtt": [
           "myEC2Instance",
           "AvailabilityZone"
         ]
       }
     }
   }
 }
 */

Outputs Filter

Outputs do
  MyPublicIp do
    Value { Fn__GetAtt "MyInstance", "PublicIp" }
  end
end

_outputs_filter do |output|
  outputs["MyPublicIp"].gsub!('.', '_')
MyPublicIp: XXX.XXX.XXX.XXX => XXX-XXX-XXX-XXX
end
_post do
  ...
end

Configuration File

Kumogata supports aws-sdk configuration file.

[default]
aws_access_key_id=AKIAIOSFODNN7EXAMPLE
aws_secret_access_key=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
aws_session_token=texample123324

Demo

• Create resources
  • https://asciinema.org/a/7979
• Convert a template
  • https://asciinema.org/a/7980
• Create a stack while outputting the event log
  • https://asciinema.org/a/8075
• Create a stack and run post commands
  • https://asciinema.org/a/8088

Similar tools

• Codenize.tools