fluent-bit采集k8s日志pod的ip-我的笔记

fluent-bit：3.1

k8s: 1.30

日志采集的架构为：

k8s-fluentbit-kafka-logstash-es-kibana

fluent-bit 默认元数据是下面几项

Analyze the Tag and extract the following metadata:分析Tag，提取以下元数据:
- Pod Name Pod 名称
- Namespace 名称空间
- Container Name 容器名称
- Container ID 容器ID

修改fluent-bit配置

如果要添加pod的IP地址，需要在fluent配置文件中开启 Annotations

[FILTER]
    Name             kubernetes
    Match            kube.*
    Kube_URL         https://kubernetes.default.svc:443
    Kube_CA_File     /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
    Kube_Token_File  /var/run/secrets/kubernetes.io/serviceaccount/token
    Kube_Tag_Prefix  kube.var.log.containers.
    Merge_Log           On
    #Merge_Log_Key       source
    K8S-Logging.Parser  On
    K8S-Logging.Exclude On
    Keep_Log            Off
    Labels              Off
    Annotations         On

采集出来的日志格式为：

"kubernetes":{"pod_name":"xxxx"
"namespace_name":"xxx"
"pod_id":"a85f6db2-555e-4281-8008-95669b2e8bc8"
"annotations":{"k8s.aliyun.com/pod-ips":"192.168.214.xxx"
"node-local-dns-webhook.k8s.io/status":"injected"}
"host":"cn-shanghai.192.168.xxx.xxx"
"container_name":"xxx"
"docker_id":"xxxx"
"container_hash":"xxxx"
"container_image":"xxxx"}}

修改logstsh配置

在logstash中重命名新的字段即可

#重启logstah

#kibana即可查看到对应字段