企业官网建设流程全解析

Kubernetes日志管理与分析：构建全面的日志体系

一、日志管理概述

Kubernetes日志管理涉及收集、存储、查询和分析容器日志，是运维监控的重要组成部分。

1.1 日志架构

flowchart TD subgraph 日志收集层 A[Pod 1] B[Pod 2] C[Pod 3] D[Pod N] end subgraph 日志传输层 E[Fluentd/Fluent Bit] F[Filter & Parse] end subgraph 日志存储层 G[Loki / Elasticsearch] H[Index & Store] end subgraph 日志展示层 I[Grafana / Kibana] J[Visualization] end A --> E B --> E C --> E D --> E E --> F F --> G G --> H H --> I I --> J

1.2 日志组件

二、Fluentd配置

2.1 DaemonSet部署

apiVersion: apps/v1 kind: DaemonSet metadata: name: fluentd namespace: kube-system spec: selector: matchLabels: name: fluentd template: spec: containers: - name: fluentd image: fluent/fluentd-kubernetes-daemonset:v1.15-debian-loki env: - name: FLUENT_LOKI_URL value: http://loki:3100 volumeMounts: - name: varlog mountPath: /var/log - name: varlibdockercontainers mountPath: /var/lib/docker/containers readOnly: true - name: config mountPath: /fluentd/etc volumes: - name: varlog hostPath: path: /var/log - name: varlibdockercontainers hostPath: path: /var/lib/docker/containers - name: config configMap: name: fluentd-config

2.2 Fluentd配置文件

apiVersion: v1 kind: ConfigMap metadata: name: fluentd-config data: fluent.conf: | <source> @type tail path /var/log/containers/*.log pos_file /var/log/fluentd-containers.log.pos tag kubernetes.* read_from_head true <parse> @type json time_key time time_format %Y-%m-%dT%H:%M:%S.%NZ </parse> </source> <filter kubernetes.**> @type kubernetes_metadata </filter> <match **> @type loki url "#{ENV['FLUENT_LOKI_URL']}" flush_interval 10s </match>

三、Loki配置

3.1 Loki部署

apiVersion: loki.grafana.com/v1 kind: LokiStack metadata: name: loki namespace: monitoring spec: size: 1x.small storage: schemas: - version: v13 effectiveDate: "2024-01-01" secret: name: loki-storage limits: global: maxStreamBytes: 1GB maxTotalBytes: 10GB

3.2 Loki Service配置

apiVersion: v1 kind: Service metadata: name: loki namespace: monitoring spec: selector: app: loki ports: - port: 3100 name: http

四、Grafana日志查询

4.1 Loki数据源配置

apiVersion: grafana.integreatly.org/v1beta1 kind: Grafana metadata: name: grafana namespace: monitoring spec: config: log: mode: "console" datasources: - name: Loki type: loki access: proxy url: http://loki:3100

4.2 日志查询语法

{app="my-app"} |= "error" {namespace="default"} |~ "WARN|ERROR" sum(count_over_time({app="my-app"}[5m])) topk(5, count by(app) (count_over_time({namespace="default"}[1h])))

五、Elasticsearch配置

5.1 Elasticsearch部署

apiVersion: elasticsearch.k8s.elastic.co/v1 kind: Elasticsearch metadata: name: quickstart spec: version: 8.6.0 nodeSets: - name: default count: 3 config: node.store.allow_mmap: false

5.2 Kibana部署

apiVersion: kibana.k8s.elastic.co/v1 kind: Kibana metadata: name: quickstart spec: version: 8.6.0 count: 1 elasticsearchRef: name: quickstart

六、日志最佳实践

6.1 结构化日志

import json import logging class JsonFormatter(logging.Formatter): def format(self, record): log_record = { 'timestamp': self.formatTime(record), 'level': record.levelname, 'logger': record.name, 'message': record.getMessage(), 'module': record.module, 'line': record.lineno } return json.dumps(log_record) logger = logging.getLogger('my-app') logger.setLevel(logging.INFO) handler = logging.StreamHandler() handler.setFormatter(JsonFormatter()) logger.addHandler(handler) logger.info('User logged in', extra={'user_id': '123'})

6.2 日志轮转配置

apiVersion: v1 kind: ConfigMap metadata: name: logrotate-config data: logrotate.conf: | /var/log/containers/*.log { daily rotate 7 compress delaycompress missingok notifempty copytruncate }

6.3 日志级别管理

apiVersion: v1 kind: ConfigMap metadata: name: app-log-config data: LOG_LEVEL: "INFO" LOG_FORMAT: "json"

七、总结

日志管理实践包括：

1. 日志收集：使用Fluentd/Fluent Bit收集容器日志
2. 日志存储：使用Loki或Elasticsearch存储日志
3. 日志查询：使用Grafana或Kibana查询分析
4. 结构化日志：输出JSON格式日志便于分析
5. 日志轮转：定期清理旧日志

建议建立完善的日志体系，实现日志的集中管理和分析。

参考资料：

• Loki文档
• Fluentd文档
• Elasticsearch文档