cfn-modules: AWS MSK cluster

AWS MSK (Kafka Cluster) using two or three availability zones with public and private subnets. Cloudwatch logging enabled by default.

Install

Install Node.js and npm first!

npm i @cfn-modules/msk-cluster

Usage

---
AWSTemplateFormatVersion: '2010-09-09'
Description: 'cfn-modules example'
Resources:
  Key:
    Type: 'AWS::CloudFormation::Stack'
    Properties:
      Parameters:
        VpcModule: !GetAtt 'Vpc.Outputs.StackName' # required
        ClientSgModule: !GetAtt 'ClientSg.Outputs.StackName' # required
        KmsKeyModule: !GetAtt 'Key.Outputs.StackName' # optional
        BastionModule: !GetAtt 'Bastion.Outputs.StackName' # optional
        AlertingModule: !GetAtt 'Alerting.Outputs.StackName' # optional
        NumberOfBrokerNodes: !GetAtt 'Vpc.Outputs.NumberOfAvailabilityZones' # required
        KafkaVersion: '2.2.1' # optional
        InstanceType: 'kafka.t3.small' # optional
        MSKConfigurationArn: '' # optional
        MSKConfigurationNumber: '0' # optional
        EBSVolumeSize: '1' # optional
      TemplateURL: './node_modules/@cfn-modules/msk-cluster/module.yml'

Examples

none

Related modules

none

Parameters

Name	Description	Default	Required?	Allowed values
VpcModule	Stack name of vpc module		yes
ClientSgModule	Stack name of client-sg module where traffic is allowed from on port 9092, 9095 and 2181 to the cluster		yes
KmsKeyModule	Stack name of kms-key module (only works in combination with Access := [Private, PublicRead])		no
BastionModule	Stack name of module implementing Bastion		no
AlertingModule	Stack name of alerting module		no
NumberOfBrokerNodes	The number of broker nodes you want in the Amazon MSK cluster. You can submit an update to increase the number of broker nodes in a cluster.		yes	Has to be a multiple of the private subnets in your VPC.
KafkaVersion	The version of Apache Kafka.	2.2.1	no
InstanceType	The type of Amazon EC2 instances to use for brokers.	kafka.t3.small	no
MSKConfigurationArn	Amazon Resource Name (ARN) of the MSK configuration to use.		no
MSKConfigurationNumber	Revision of the Amazon MSK configuration to use (required if MSKConfigurationArn is set).		no
EBSVolumeSize	The size in GiB of the EBS volume for the data drive on each broker node.	1	no
ClientBrokerEncryption	Indicates the encryption setting for data in transit between clients and brokers.	TLS	no	[TLS, PLAINTEXT, TLS_PLAINTEXT]
LogsRetentionInDays	Specifies the number of days you want to retain log events in the specified log group	14	no	[1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 3653]

Outputs

Name	Interface	Description	Exported?
ModuleId	global	Id of the module	no
ModuleVersion	global	Version of the module	no
StackName	global	Name of the stack (used to pass module references)	no
Arn	ExposeArn	Cluster ARN	yes

Limitations

• Scalable: Auto scaling is not provided by MSK service. You have to increase the number of broker nodes to scale manually.
• Secure: Does not backup/snapshot the data.