Configuration¶

In addition to the Runway Config File, there are two files that can be used for configuration:

a YAML configuration file [REQUIRED]
a key/value environment file

runway.yml¶

Example

deployments:
  - modules:
      - path: sampleapp.cfn
        type: cloudformation  # not required; implied from ".cfn" directory extension
        environments:
          dev: true
        parameters:
          namespace: example-${env DEPLOY_ENVIRONMENT}
          cfngin_bucket: example-${env DEPLOY_ENVIRONMENT}-${env AWS_REGION}
    regions:
      - us-east-1

Options¶

CloudFormation modules do not have any module-specific options.

Parameters¶

Runway can pass Parameters to a CloudFormation module in place of or in addition to values in an environment file. When Parameters are passed to the module, the data type is retained (e.g. array, boolean, mapping).

A typical usage pattern would be to use Runway Lookups in combination with Parameters to pass deploy environment and/or region specific values to the module from the Runway Config File.

Example

deployments:
  - modules:
      - sampleapp-01.cfn
      - path: sampleapp-02.cfn
        parameters:
          instance_count: ${var instance_count.${env DEPLOY_ENVIRONMENT}}
    parameters:
      image_id: ${var image_id.%{env AWS_REGION}}

Common Parameters¶

Runway automatically makes the following commonly used Parameters available to CloudFormation modules.

Note

If these parameters are already being explicitly defined in deployment.parameters/module.parameters the value provided will be used instead of what would be automatically added.

environment (str): Taken from the DEPLOY_ENVIRONMENT environment variable. This will the be current deploy environment.
region (str): Taken from the AWS_REGION environment variable. This will be the current region being processed.

CFNgin Config File¶

The configuration file has a loose definition, with only a few top-level fields. Other than those fields, you can define your own top-level keys to make use of other YAML features like anchors & references to avoid duplicating config. (See YAML anchors & references for details)

Top-Level Fields¶

class cfngin.config¶

Runway’s CFNgin makes use of a YAML formatted config file to define the different CloudFormation stacks that make up a given environment.

cfngin_bucket: Optional[str]¶

By default, CloudFormation templates are pushed into an S3 bucket and CloudFormation is pointed to the template in that bucket when launching or updating stacks. By default it uses a bucket named cfngin-${namespace}-${region}, where the namespace is namespace and region is the current AWS region.

To change this, define a value for this field.

The bucket will be created in the same region that the stacks will be launched in. If you want to change this, or if you already have an existing bucket in a different region, you can set the cfngin_bucket_region to the region where you want to create the bucket.

If you want CFNgin to upload templates directly to CloudFormation, instead of first uploading to S3, you can set this field to an empty string. However, note that template size is greatly limited when uploading directly. See the CloudFormation Limits Reference.

Example

cfngin_bucket: example-${region}

cfngin_bucket_region: Optional[str]¶

AWS Region where cfngin_bucket is located. If not provided, the current region is used.

Example

cfngin_bucket_region: us-east-1

cfngin_cache_dir: Optional[str] = ./.runway/cache¶

Path to a local directory that CFNgin will use for local caching.

Example

cfngin_cache_dir: ./.runway/cache

log_formats: Optional[Dict[str, str]] = {}¶

Customize log message formatting by log level.

Any of the standard Python logging module format attributes can be used when writing a new log format string.

Example

log_formats:
  info: "[%(asctime)s] %(message)s"
  debug: "[%(asctime)s] %(levelname)s %(threadName)s %(name)s:%(lineno)d(%(funcName)s): %(message)s"

lookups: Optional[Dict[str, str]] = {}¶

Lookups allow you to create custom methods which take a value and are resolved at runtime time. The resolved values are passed to the Blueprints before it is rendered. For more information, see the Lookups documentation.

CFNgin provides some common Lookups, but it is sometimes useful to have your own custom lookup that doesn’t get shipped with Runway. You can register your own lookups here.

The key of each item in the mapping will be used as the name of the lookup type when registering the lookup. The value should be the path to a valid lookup handler.

Example

lookups:
  custom: path.to.lookup.handler

conf_value: ${custom query}

mappings: Optional[Dict[str, Dict[str, Dict[str, Any]]]] = {}¶

Mappings are dictionaries that are provided as Mappings to each CloudFormation stack that CFNgin produces.

These can be useful for providing things like different AMIs for different instance types in different regions.

These can be used in each Blueprints/template as usual.

Example

mappings:
  AmiMap:
    us-east-1:
      NAT: ami-ad227cc4
      ubuntu1404: ami-74e27e1c
      bastion: ami-74e27e1c
    us-west-2:
      NAT: ami-290f4119
      ubuntu1404: ami-5189a661
      bastion: ami-5189a661

namespace: str¶

A namespace to create all stacks within. The value will be used as a prefix for the name of any stack that is created.

In addition, this value can be used to create an S3 bucket that will be used to upload and store all CloudFormation templates. See cfngin_bucket for more detailed information.

In general, this is paired with the concept of deploy environments to create a namespace per environment.

Example

namespace: ${namespace}-${environment}

namespace_delimiter: Optional[str] = "-"¶

By default, - will be used as a delimiter between the namespace and the declared stack name to deploy the actual CloudFormation stack name that gets created.

If you prefer to not use a delimiter, an empty string can be used as the value of this field.

See the CloudFormation API Reference for allowed stack name characters

Example

namespace_delimiter: ""

package_sources: Optional[cfngin.package_sources] = {}¶

See Remote Sources for detailed information.

package_sources:
  git:
    ...
  local:
    ...
  s3:
    ...

persistent_graph_key: Optional[str] = null¶

Used to track the state of stacks defined in configuration file. This can result in stacks being destroyed when they are removed from the configuration file removing the need to manually delete the stacks.

See Persistent Graph for detailed information.

Example

persistent_graph_key: unique-key.json

post_deploy: Union[Dict[str, cfngin.hook], List[cfngin.hook]] = []¶

Python functions/methods that are executed after processing the stacks in the config while using the deploy command.

See Hooks for more detailed information.

Example

using a dict¶

post_deploy:
  do_something:
    path: do.something

using a list¶

post_deploy:
  - path: do.something

post_destroy: Union[Dict[str, cfngin.hook], List[cfngin.hook]] = []¶

Python functions/methods that are executed after processing the stacks in the config while using the destroy command.

See Hooks for more detailed information.

Example

using a dict¶

post_destroy:
  do_something:
    path: do.something

using a list¶

post_destroy:
  - path: do.something

pre_deploy: Union[Dict[str, cfngin.hook], List[cfngin.hook]] = []¶

Python functions/methods that are executed before processing the stacks in the config while using the deploy command.

See Hooks for more detailed information.

Example

using a dict¶

pre_deploy:
  do_something:
    path: do.something

using a list¶

pre_deploy:
  - path: do.something

pre_destroy: Union[Dict[str, cfngin.hook], List[cfngin.hook]] = []¶

Python functions/methods that are executed before processing the stacks in the config while using the destroy command.

See Hooks for more detailed information.

Example

using a dict¶

pre_destroy:
  do_something:
    path: do.something

using a list¶

pre_destroy:
  - path: do.something

service_role: Optional[str] = null¶

By default CFNgin doesn’t specify a service role when executing changes to CloudFormation stacks. If you would prefer that it do so, you define the IAM Role ARN that CFNgin should use when executing CloudFormation changes.

This is the equivalent of setting RoleARN on a call to the following CloudFormation api calls: CreateStack, UpdateStack, CreateChangeSet.

See the AWS CloudFormation service role for more information.

Example

service_role: arn:aws:iam::123456789012:role/name

stacks: Optional[Dict[str, cfngin.stack]] = []¶

This is the core part of the config where the CloudFormations stacks that will be deployed in the environment are defined.

For each stack in the dictionary, the key is used as the logical name of the stack within CFNgin. Each key should be unique as PyYAML treats duplicate keys as override keys.

See Stack for more information.

sys_path: Optional[str] = null¶

A path to be added to $PATH while processing the configuration file. This will allow modules from the provided path location to be used

When setting class_path for Blueprints or path for hooks , it is sometimes desirable to load modules from outside the default $PATH (e.g. to include modules inside the same repo as config files).

Example

sys_path: ./  # most common value to use

tags: Optional[Dict[str, str]] = {"cfngin_namespace": namespace}¶

A dictionary of tags to add to all stacks. These tags are propagated to all resources that AWS CloudFormation supports. See CloudFormation - Resource Tag for more information.

If this field is undefined, a cfngin_namespace tag is applied to your stack with the value of namespace as the tag-value. Alternatively, this field can be set to a value of {} (an empty dictionary) to disable the default tag.

Example

tags:
  namespace: ${namespace}
  example: value

disable default tag¶

tags: {}

template_indent: Optional[int] = 4¶

Number of spaces per indentation level to use when rendering/outputting CloudFormation templates.

Example

template_indent: 2

Stack¶

class cfngin.stack¶

Defines a CloudFormation stack.

Lookup Support

The following fields support lookups:

variables

Example

stacks:
  vpc-example:
    class_path: blueprints.vpc.VPC
    variables:
      InstanceType: t2.small
      SshKeyName: default
      ImageName: NAT
      AZCount: 2
      PublicSubnets:
        - 10.128.0.0/24
        - 10.128.1.0/24
        - 10.128.2.0/24
        - 10.128.3.0/24
      PrivateSubnets:
        - 10.128.8.0/22
        - 10.128.12.0/22
        - 10.128.16.0/22
        - 10.128.20.0/22
      CidrBlock: 10.128.0.0/16

class_path: Optional[str]¶

A python importable path to the Blueprints class to be used.

Exactly one of class_path or template_path must be defined.

Example

stacks:
  example-stack:
    class_path: example.BlueprintClass

description: Optional[str]¶

A short description to apply to the stack. This overwrites any description defined in the Blueprints. See Cloudformation - Template Description for more information.

Example

stacks:
  example-stack:
    description: An Example Stack

enabled: Optional[bool] = true¶

Whether to deploy/update the stack. This enables the ability to disable stacks in different environments.

Important

This field is ignored when destroying stacks.

Example

stacks:
  example-stack:
    enabled: false
  another-stack:
    enabled: ${enable_another_stack}

in_progress_behavior: Optional[Literal[“wait”]]¶

Specifies the behavior for when a stack is in CREATE_IN_PROGRESS or UPDATE_IN_PROGRESS. By default, CFNgin will raise an exception if the stack is in an IN_PROGRESS state when processing begins.

If the value of this field is wait, CFNgin will wait for the previous update to complete before attempting to update the stack instead of raising an exception.

Example

stacks:
  example-stack:
    in_progress_behavior: wait

locked: Optional[bool] = false¶

Whether the stack should be updated after initial deployment. This is useful for risky stacks that you don’t want to take the risk of allowing CloudFormation to update but still want to deploy it using CFNgin.

Example

stacks:
  example-stack:
    locked: true
  another-stack:
    locked: ${locked_another_stack}

protected: Optional[bool] = false¶

Whether to force all updates to be performed interactively.

When true and running in non-interactive mode, CFNgin will switch to interactive mode for this stack to require manual review and approval of any changes.

Example

stacks:
  example-stack:
    protected: true
  another-stack:
    protected: ${protected_another_stack}

required_by: Optional[List[str]] = []¶

A list of other stacks that require this stack. All stacks must be defined in the same configuration file.

Inverse of requires.

Example

stacks:
  example-stack:  # deployed first
    required_by:
      - another-stack
  another-stack:  # deployed after example-stack
    ...

requires: Optional[List[str]] = []¶

A list of other stacks that this stack requires. All stacks must be defined in the same configuration file.

Inverse of required_by.

Example

stacks:
  example-stack:  # deployed after another-stack
    requires:
      - another-stack
  another-stack:  # deployed first
    ...

stack_name: Optional[str]¶

The name used when creating the CloudFormation stack. If not provided, the key used to define this stack in stacks will be used.

Note

namespace will be appended to this value.

Example

stacks:
  example-stack:
    stack_name: another-name

stack_policy_path: Optional[str]¶

Path to a JSON formatted stack policy that will be applied when the CloudFormation stack is created and/or updated.

See CloudFormation - Prevent updates to stack resources for examples and more information.

Example

stacks:
  example-stack:
    stack_policy_path: ./stack_policies/example-stack.json

tags: Optional[Dict[str, str]] = {}¶

A dictionary of tags to add to all stacks. These tags are propagated to all resources that AWS CloudFormation supports. See CloudFormation - Resource Tag for more information.

This will be combined with the global tags. Values defined here take precedence over those defined globally.

Example

stacks:
  example-stack:
    tags:
      namespace: ${namespace}
      example: value

template_path: Optional[str]¶

Path to a raw CloudFormation template (JSON or YAML). Can be relative to the working directory (e.g. templates stored alongside the configuration file), or relative to a directory in the $PATH (i.e. for loading templates retrieved via package_sources).

Exactly one of class_path or template_path must be provided.

Example

stacks:
  example-stack:
    template_path: ./templates/example-stack.yml
  another-stack:
    template_path: remote/path/templates/another-stack.json

termination_protection: Optional[bool] = false¶

Whether the stack will be protected from termination by CloudFormation.

Any attempts to destroy the stack (using Runway, the AWS console, AWS API, etc) will be prevented unless manually disabled.

When updating a stack and the value has been changed, termination protection on the CloudFormation stack sill also change. This is useful when needing to destroy a stack by first changing the value in the configuration file, updating the stack, then proceeding to destroy it.

Example

stacks:
  example-stack:
    termination_protection: true
  another-stack:
    termination_protection: ${termination_protection_another_stack}

variables: Optional[Dict[str, Any]] = {}¶

A dictionary of Variables to pass to the Blueprints when rendering the CloudFormation template. Can be any valid YAML data structure.

When using a raw CloudFormation template, these are the values provided for it’s Parameters.

Variables¶

Variables are values that will be passed into a Blueprints before it is rendered. Variables can be any valid YAML data structure and can leverage Lookups to expand values at runtime.

The following concepts make working with variables within large templates easier:

YAML anchors & references¶

If you have a common set of variables that you need to pass around in many places, it can be annoying to have to copy and paste them in multiple places. Instead, using a feature of YAML known as anchors & references, you can define common values in a single place and then refer to them with a simple syntax.

For example, say you pass a common domain name to each of your stacks, each of them taking it as a Variable. Rather than having to enter the domain into each stack (and hopefully not typo’ing any of them) you could do the following:

domain_name: &domain mydomain.com

Now you have an anchor called domain that you can use in place of any value in the config to provide the value mydomain.com. You use the anchor with a reference.

stacks:
  vpc:
    class_path: blueprints.vpc.VPC
    variables:
      DomainName: *domain

Even more powerful is the ability to anchor entire dictionaries, and then reference them in another dictionary, effectively providing it with default values.

common_variables: &common_variables
  DomainName: mydomain.com
  InstanceType: m3.medium
  AMI: ami-12345abc

Now, rather than having to provide each of those variables to every stack that could use them, you can just do this instead.

stacks:
  - name: vpc
    class_path: blueprints.vpc.VPC
    variables:
      << : *common_variables
      InstanceType: c4.xlarge # override the InstanceType in this stack

Using Outputs as Variables¶

Since CFNgin encourages the breaking up of your CloudFormation stacks into entirely separate stacks, sometimes you’ll need to pass values from one stack to another. The way this is handled in CFNgin is by having one stack provide Outputs for all the values that another stack may need, and then using those as the inputs for another stack’s variables. CFNgin makes this easier for you by providing a syntax for variables that will cause CFNgin to automatically look up the values of Outputs from another stack in its config.

To do so, use the Output Lookup in the variables on the target stack.

MyParameter: ${output OtherStack::OutputName}

For more information see Lookups.

In this example config - when deploying things inside a VPC, you will need to pass the VpcId of the VPC that you want the resources to be located in. If the vpc stack provides an Output called VpcId, you can reference it easily.

domain_name: my_domain &domain

stacks:
  - name: vpc
    class_path: blueprints.vpc.VPC
    variables:
      DomainName: *domain
  - name: webservers
    class_path: blueprints.asg.AutoscalingGroup
    variables:
      DomainName: *domain
      VpcId: ${output vpc::VpcId} # gets the VpcId Output from the vpc stack

Doing this creates an implicit dependency from the webservers stack to the vpc stack, which will cause CFNgin to submit the vpc stack, and then wait until it is complete until it submits the webservers stack. This would be the same as adding vpc to the requires field of the webservers stack.

Environment File¶

When using CFNgin, you can optionally provide an “environment” file. The CFNgin config file will be interpolated as a string.Template using the key-value pairs from the environment file as parameters. The format of the file is a single key-value per line, separated by a colon (:).

File Naming¶

Environment files must follow a specific naming format in order to be recognized by Runway. The files must also be stored at the root of the module’s directory.

${DEPLOY_ENVIRONMENT}-${AWS_REGION}.env: The typical naming format that will be used for these files specifies the name of the DEPLOY_ENVIRONMENT and AWS_REGION in which to use the file.
${DEPLOY_ENVIRONMENT}.env: The region can optionally be omitted to apply a single file to all regions.

Files following both naming schemes may be used. The file with the most specific name takes precedence. Values passed in as parameters from the Runway Config File take precedence over those provided in an environment file.

Usage¶

A pretty common use case is to have separate environments that you want to look mostly the same, though with some slight modifications. For example, you might want a production and a staging environment.

The production environment likely needs more instances, and often those instances will be of a larger instance type. The parameters defined in an environment file, deployment.parameters, and/or module.parameters allow you to use your existing CFNgin config, but provide different values based on the current deploy environment.

Example

vpcID: vpc-12345678

Provided the key-value pair above, you will now be able to use this in your configs for a deploy environment. They act as keys that can be used in your config file, providing a sort of templating ability. This allows you to change the values of your config based on the current deploy environment.

For example, if you have a webserver stack, and you need to provide it a variable for the instance size it should use, you would have something like this in your config file.

stacks:
  - name: webservers
    class_path: blueprints.asg.AutoscalingGroup
    variables:
      InstanceType: m3.medium

But what if you needed more CPU in your production environment, but not in your staging? Without parameters, you’d need a separate config for each. With parameters, you can simply define two different values for InstanceType in an an environment file, deployment.parameters, and/or module.parameters then use the parameter’s name to reference the value in a config file.

sampleapp.cfn/cfngin.yml¶

# in your config file:
stacks:
  webservers:
    class_path: blueprints.asg.AutoscalingGroup
    variables:
      InstanceType: ${web_instance_type}

Using Environment Files

Both files would be required.

sampleapp.cfn/stage.env¶

web_instance_type: m5.medium

sampleapp.cfn/prod.env¶

web_instance_type: c5.xlarge

Using Runway

This option would not required the use of environment files to define the values.

runway.yaml¶

deployments:
  - modules:
    - name: Sample Application
      path: sampleapp.cfn
      parameters:
        web_instance_type: ${var web_instance_type.${env DEPLOY_ENVIRONMENT}}

variables:
  web_instance_type:
    stage: m5.medium
    prod: c5.xlarge