k8s-fluentd-windows

April 12, 2022 · View on GitHub

Stream Logs from K8s Windows Pods using Fluentd

If you already have an EKS Cluster With Windows, don't forget to replace eks-windows for your cluster name

These examples were created using us-east-1 region

Pre requisits:

0) Tools

1) EKS Cluster with Windows Nodes

1.1) Create EKS Cluster

eksctl create cluster --name=eks-windows --ssh-access --version 1.18 \
--ssh-public-key=#<your_key_name> --managed --region=us-east-1

1.2) Install vpc-controller

eksctl utils install-vpc-controllers --cluster=eks-windows --approve --region=us-east-1

1.3) Create Nodegroup Windows

eksctl create nodegroup --cluster=eks-windows --node-ami-family=WindowsServer2019FullContainer \
--ssh-access --ssh-public-key=#<your_key_name> --region=us-east-1

1.4) Update your kubectl config

aws eks update-kubeconfig --name eks-windows --region us-east-1

Check the nodes:

kubectl get nodes

nodes

2) Configure OIDC for EKS

eksctl utils associate-iam-oidc-provider  --cluster eks-windows --approve --region us-east-1

Check the OIDC

Install fluentd-windows EKS

1) Create EKS Service Account

eksctl create iamserviceaccount --cluster BG8S-WINDOWS \
--attach-policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy \
--name fluentd-windows --namespace amazon-cloudwatch --approve \
--region us-east-1

kubectl get sa -n amazon-cloudwatch | grep fluentd-windows

2) Deploy fluentd-windows Configmap and Daemonset

kubectl apply -f https://raw.githubusercontent.com/bgsilvait/k8s-fluentd-windows/master/k8s/configmap-fluentd-windowsbgs.yaml

If you are NOT using the provided examples for AWS_REGION and Cluster Name, you should replace on configmap-fluentd-windowsbgs.yaml:

data:
  AWS_REGION: YOUR_REGION_HERE
  CLUSTER_NAME: YOUR_CLUSTER_NAME
  fluent.conf: |

kubectl apply -f https://raw.githubusercontent.com/bgsilvait/k8s-fluentd-windows/master/k8s/daemonset-fluentd-windows.yaml

kubectl get pods -n amazon-cloudwatch

Windows containers are bigger than linux, can take some minutes to enter in ready state.

3) Deploy an IIS pod created with Logmonitor

LogMonitor is only required here because we are using IIS which run as a windows service and don't output in the stdout

kubectl apply -f https://raw.githubusercontent.com/bgsilvait/k8s-fluentd-windows/master/k8s/deployment-iisbgs.yaml

kubectl get pods -A -o wide | grep winiis

4) Access the IIS Pods to generate logs

kubectl -it exec <your_winiis_pod_name> powershell

curl -I winiis

curl winiis -UseBasicParsing

Cloudwatch (us-east-1)

loggroups

iisstream

Custom Containers Windows Server 2019 for IIS and Fluentd

IIS
Fluentd