企业官网建设流程全解析

我们都知道，在物理主机或者虚拟机上，一台正常的主机是可以通过后DHCP动态分配IP的，但是在kubernetes中不存在DHCP来分配每个Pod的ip地址。

在 Kubernetes 中，每个 Pod 都需要一个唯一的 IP 地址来与其他 Pod 和服务进行通信。IP 地址管理非常重要，因为：

1. 动态创建 Pod：Pod 是临时性的，需要频繁地创建和销毁。每个新的 Pod 都需要从集群的地址空间中获取一个可用的 IP 地址。
2. 避免 IP 冲突：如果没有适当的 IP 管理工具，多个 Pod 可能会被分配相同的 IP 地址，从而导致网络冲突和通信故障。
3. 高效的地址分配：IPAM 系统将 IP 地址按块为单位分配给各个节点，从而减少了单个 IP 请求所带来的负担，并提升了系统的可扩展性。
4. 跨节点通信：集群中不同节点上的容器需要可路由的 IP 地址，以便能够直接进行通信而无需通过 NAT 层。IPAM 通过合理的子网分配机制，帮助解决了这一问题。

Calico 的 IPAM 技术通过预先将 IP 块分配给各个节点，从而实现高效的 IP 分配。这一机制能够确保节点快速启动，同时在整个集群范围内保持正确的 IP 地址管理。

使用clab实现实验环境的搭建

前提条件

1. 完成containerlab的安装 - 采用宿主机安装方式
2. 完成kind镜像的下载 - 采用容器安装
3. 完成docker的安装

清单文件

1. calico-ipam.clab.yamlContainerLab 拓扑结构

name:calico-ipam# 生命使用和calico 同样的网桥，避免和现有的docker网络有冲突mgmt:network:bridgetopology:nodes:calico-ipam:kind:k8s-kindimage:kindest/node:v1.28.0startup-config:calico-ipam-no-cni.yamlextras:k8s_kind:deploy:wait:0s

2. calico-ipam-no-cni.yamlcalico 无需使用 CNI 的集群配置文件，这里的配置是给kind使用的

andrew@andrew ~/k8-networking-calico-containerlab/containerlab/01-calico-ipam(master*?)$catcalico-ipam-no-cni.yaml

apiVersion:kind.x-k8s.io/v1alpha4kind:Clusternetworking:disableDefaultCNI:truepodSubnet:"192.168.0.0/16"serviceSubnet:"10.96.0.0/16"nodes:-role:control-plane-role:worker-role:worker

3. calico-cni-config/custom-resources.yaml基于 IPAM 和 IP 池配置的定制型 Calico 安装资源

apiVersion:operator.tigera.io/v1kind:Installationmetadata:name:defaultspec:# Configures Calico networking.calicoNetwork:# Note: The ipPools section cannot be modified post-install.ipPools:-blockSize:26cidr:192.168.0.0/16encapsulation:VXLANCrossSubnetnatOutgoing:EnablednodeSelector:all()---apiVersion:operator.tigera.io/v1kind:APIServermetadata:name:defaultspec:{}

通过一下命令完成实验环境的部署

chmod+x deploy.sh ./deploy.sh

calico CNI Configuration

Calico 的部署方式使用了一种自定义的安装资源（custom-resources.yaml▸），该资源用于定义容器网络的配置。在选择 CIDR 范围时，需要注意以下几个关键点：

• 默认IP地址池：cidr: 192.168.0.0/16可以提供65536个IP地址，足够支持大型集群的规模
• 块大小：根据需要，为每个节点分配了26个块，每个块包含了64个IP地址
• 封装机制：采用 VXLAN CrossSubnet 技术来实现节点间的通信。
• NAT 输出功能：已启用，允许容器实例访问外部网络
  CIDR 地址选择直接影响了集群的可扩展性和网络策略的有效性。/16 范围可以支持大约 1,024 个节点，而/26 范围的地址则适合用于大多数实验室和生产环境。

使用deploy.sh脚本能自动化完成以上实验

#!/bin/bashset-e# Exit immediately if a command exits with a non-zero status# 如果以前安装过这里先清理echo"=== Destroying existing ContainerLab topology ==="sudocontainerlab destroy-tcalico-ipam.clab.yaml||{echo"Failed to destroy existing topology";exit1;}# 按照配置部署集群环境echo"=== Deploying ContainerLab topology ==="sudocontainerlab deploy-tcalico-ipam.clab.yaml||{echo"Failed to deploy topology";exit1;}echo"=== Waiting for Kind cluster to be ready (30 seconds) ==="sleep30echo"=== Setting up kubeconfig ==="# Create kubeconfig directory if it doesn't existmkdir-p~/.kube# Export kubeconfig to a specific file to avoid conflictssudokind get kubeconfig--name=calico-ipam>calico-ipam.kubeconfigsudochmod644calico-ipam.kubeconfig# 保证kueclt 指向 kind集群# 导出配置，以便通过kubectl命令能访问kind集群# Use the specific kubeconfig file for all kubectl commandsexportKUBECONFIG=$(pwd)/calico-ipam.kubeconfigecho"=== Installing calicoctl ==="curl-Lhttps://github.com/projectcalico/calico/releases/download/v3.30.0/calicoctl-linux-amd64-ocalicoctl||{echo"Failed to download calicoctl";exit1;}chmod+x calicoctlsudomvcalicoctl /usr/local/bin/||{echo"Failed to move calicoctl to /usr/local/bin";exit1;}echo"calicoctl version:$(calicoctl version)"||{echo"Warning: calicoctl may not be installed correctly";}echo"=== Waiting for Kubernetes API to be available ==="untilkubectl get nodes&>/dev/null;doecho"Waiting for Kubernetes API..."sleep5done# 部署calico环境echo"=== Installing Calico 3.30.0 ==="kubectl create-fhttps://raw.githubusercontent.com/projectcalico/calico/v3.30.0/manifests/operator-crds.yaml||{echo"Failed to install Calico CRDs";exit1;}kubectl create-fhttps://raw.githubusercontent.com/projectcalico/calico/v3.30.0/manifests/tigera-operator.yaml||{echo"Failed to install Tigera operator";exit1;}# 使用calico 自定义资源进行配置echo"=== Applying custom Calico resources ==="kubectl apply-fcalico-cni-config/custom-resources.yaml||{echo"Failed to apply custom resources";exit1;}echo"=== Waiting for TigeraStatus to be ready ==="echo"Initial check, expect resources to be unavailable..."kubectl get tigerastatusecho"Waiting for TigeraStatus to become available (may take several minutes)..."whiletrue;do# More precise check using conditionsAPI_AVAILABLE=$(kubectl get tigerastatus apiserver-ojsonpath='{.status.conditions[?(@.type=="Available")].status}'2>/dev/null||echo"False")CALICO_AVAILABLE=$(kubectl get tigerastatus calico-ojsonpath='{.status.conditions[?(@.type=="Available")].status}'2>/dev/null||echo"False")if[["$API_AVAILABLE"=="True"&&"$CALICO_AVAILABLE"=="True"]];thenecho"Calico API server and core components are ready!"breakfiecho"Still waiting for Calico components to be ready..."sleep15doneecho"=== TigeraStatus final check ==="kubectl get tigerastatusecho"Kubernetes nodes:"kubectl get nodes-owideecho""echo"To use this cluster with kubectl, run:"echo"export KUBECONFIG=$(pwd)/calico-ipam.kubeconfig"%

环境检查

1. 检查ContainerLab的topo结构

andrew@andrew ~/k8-networking-calico-containerlab/containerlab/01-calico-ipam(master*?)$ containerlab inspect-tcalico-ipam.clab.yaml14:10:38 INFO Parsing&checking topologyfile=calico-ipam.clab.yaml ╭───────────────────────────┬──────────────────────┬─────────┬───────────────────────╮ │ Name │ Kind/Image │ State │ IPv4/6 Address │ ├───────────────────────────┼──────────────────────┼─────────┼───────────────────────┤ │ calico-ipam-control-plane │ k8s-kind │ running │192.168.48.3 │ │ │ kindest/node:v1.28.0 │ │ fc00:f853:ccd:e793::3 │ ├───────────────────────────┼──────────────────────┼─────────┼───────────────────────┤ │ calico-ipam-worker │ k8s-kind │ running │192.168.48.2 │ │ │ kindest/node:v1.28.0 │ │ fc00:f853:ccd:e793::2 │ ├───────────────────────────┼──────────────────────┼─────────┼───────────────────────┤ │ calico-ipam-worker2 │ k8s-kind │ running │192.168.48.4 │ │ │ kindest/node:v1.28.0 │ │ fc00:f853:ccd:e793::4 │ ╰───────────────────────────┴──────────────────────┴─────────┴───────────────────────╯

2. 检查calico的安装状态

andrew@andrew ~/k8-networking-calico-containerlab/containerlab/01-calico-ipam(master*?)$ kubectl get tigerastatus NAME AVAILABLE PROGRESSING DEGRADED SINCE apiserver True False False 124m calico True False False 124m ippools True False False 158m

3. 检查节点状态

andrew@andrew ~/k8-networking-calico-containerlab/containerlab/01-calico-ipam(master*?)$ kubectl get nodes NAME STATUS ROLES AGE VERSION calico-ipam-control-plane Ready control-plane 166m v1.28.0 calico-ipam-worker Ready<none>166m v1.28.0 calico-ipam-worker2 Ready<none>166m v1.28.0

4. ipam的整体状态

andrew@andrew ~/k8-networking-calico-containerlab/containerlab/01-calico-ipam(master*?)$ calicoctl ipam show +----------+----------------+-----------+------------+--------------+|GROUPING|CIDR|IPS TOTAL|IPS IN USE|IPS FREE|+----------+----------------+-----------+------------+--------------+|IP Pool|192.168.0.0/16|65536|12(0%)|65524(100%)|+----------+----------------+-----------+------------+--------------+

• CIDR：整个子网网络的范围（192.168.0.0/16）
• 总 IP 数：该池中可用的 IP 地址总数（65,536 个）
• 正在使用的 IP 地址：当前已将 IP 地址分配给了相应的容器。
• 免费 IP 地址：可供新节点分配使用的 IP 地址

5. 阻塞关联列表

andrew@andrew ~/k8-networking-calico-containerlab/containerlab/01-calico-ipam(master*?)$ kubectl get blockaffinities NAME CREATED AT calico-ipam-control-plane-192-168-18-64-262026-09-23T04:00:09Z calico-ipam-worker-192-168-131-128-262026-09-23T04:03:36Z calico-ipam-worker2-192-168-112-128-262026-09-23T04:06:00Z

• BlockAffinity 资源代表了分配给节点的 IPAM 块分配信息
• 每个条目都显示了哪个 IP 地址范围被分配给了哪个节点
• 命名规范是使用calico-ipam-<block-cidr-with-dashes>-<prefix-length>来表示
• Calico 根据需求将/26 的块（每个块包含 64 个 IP 地址）分配给各个节点

6. 详细区块关联性数据

andrew@andrew ~/k8-networking-calico-containerlab/containerlab/01-calico-ipam(master*?)$ kubectl get blockaffinities-oyaml apiVersion: v1 items: - apiVersion: projectcalico.org/v3 kind: BlockAffinity metadata: creationTimestamp:"2026-09-23T04:00:09Z"name: calico-ipam-control-plane-192-168-18-64-26 resourceVersion:"4698"uid: a4c00304-a475-4212-a05d-6f390e886ed5 spec:# 分配给某个节点的特定 IP 地址范围（例如，192.168.0.64/26）cidr:192.168.18.64/26# 拥有该 IP 地址的 Kubernetes 节点node: calico-ipam-control-plane# `confirmed` ：这个区块正在被积极分配并投入使用中# `pending` ：赋值操作正在进行中state: confirmed - apiVersion: projectcalico.org/v3 kind: BlockAffinity metadata: creationTimestamp:"2026-09-23T04:03:36Z"name: calico-ipam-worker-192-168-131-128-26 resourceVersion:"5163"uid: e5227720-b462-4482-97b9-c016e1dc1d5e spec: cidr:192.168.131.128/26 node: calico-ipam-worker state: confirmed - apiVersion: projectcalico.org/v3 kind: BlockAffinity metadata: creationTimestamp:"2026-09-23T04:06:00Z"name: calico-ipam-worker2-192-168-112-128-26 resourceVersion:"5474"uid: 35c56f39-4eff-4b2e-b825-02e92bb24077 spec: cidr:192.168.112.128/26 node: calico-ipam-worker2 state: confirmed kind: List metadata: resourceVersion:""

7. Formatted Block Affinities

andrew@andrew ~/k8-networking-calico-containerlab/containerlab/01-calico-ipam(master*?)$ kubectl get blockaffinities-o=jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.spec.node}{"\t"}{.spec.cidr}{"\n"}{end}'calico-ipam-control-plane-192-168-18-64-26 calico-ipam-control-plane192.168.18.64/26 calico-ipam-worker-192-168-131-128-26 calico-ipam-worker192.168.131.128/26 calico-ipam-worker2-192-168-112-128-26 calico-ipam-worker2192.168.112.128/26

总结

• IP 地址池：较大的 CIDR 范围（如 192.168.0.0/16），用于定义整体地址空间
• IP 块：从 IP 池中划分出较小的子网（如/26 的块），并将其分配给各个节点。
• 块关联：IP 块与节点之间的分配关系
• IPAM：当创建 Pod 时，Calico 会自动管理分配给 Pod 的 IP 地址分配，使其位于指定的 IP 块内。

Troubleshooting

1. 检查这些节点是否被分配了足够的IP地址块
2. 请确认calico相关的组件在正常运行kubectl get pods -n calico-system
3. 查看calico node的日志kubectl logs -n calico-system -l k8s-app=calico-node

清理环境

andrew@andrew ~/k8-networking-calico-containerlab/containerlab/01-calico-ipam(master*?)$catdestroy.sh#!/bin/bash# Lab-specific cleanup script# This script removes only resources created by this specific labset-euopipefail# Colors for outputGREEN='\033[0;32m'YELLOW='\033[1;33m'NC='\033[0m'# No Color# Get the lab directory nameLAB_DIR=$(basename"$(pwd)")# Detect topology name from .clab.yaml file# Try common names first, then any .clab.yaml fileTOPOLOGY_FILE=""if[-f"topology.clab.yaml"];thenTOPOLOGY_FILE="topology.clab.yaml"elseTOPOLOGY_FILE=$(find.-maxdepth1-name"*.clab.yaml"-typef|head-1)fiif[-z"$TOPOLOGY_FILE"];thenecho"Error: No .clab.yaml file found in current directory"exit1fiTOPOLOGY_NAME=$(grep"^name:""$TOPOLOGY_FILE"|head-1|awk'{print $2}'|tr-d'"'|tr-d"'")if[-z"$TOPOLOGY_NAME"];thenecho"Error: Could not detect topology name from$TOPOLOGY_FILE"exit1fi# Detect Kind cluster name from deploy.shKIND_CLUSTER=$(grep"kind get kubeconfig --name="deploy.sh2>/dev/null|head-1|grep-o"name=[^ ]*"|cut-d=-f2|tr-d'>'|tr-d' '||echo"")if[-z"$KIND_CLUSTER"];then# Try to get from topology file (k8s-kind node name)KIND_CLUSTER=$(grep-A5"kind: k8s-kind""$TOPOLOGY_FILE"|grep-E"^\s+[a-zA-Z0-9-]+:"|head-1|awk-F:'{print $1}'|tr-d' '||echo"")fi# Detect kubeconfig filename from deploy.shKUBECONFIG_FILE=$(grep"\.kubeconfig"deploy.sh2>/dev/null|head-1|grep-o"[^ ]*\.kubeconfig"|head-1||echo"")if[-z"$KUBECONFIG_FILE"]&&[-n"$KIND_CLUSTER"];thenKUBECONFIG_FILE="${KIND_CLUSTER}.kubeconfig"fiecho-e"${YELLOW}========================================${NC}"echo-e"${YELLOW}Cleaning up lab:$LAB_DIR${NC}"echo-e"${YELLOW}========================================${NC}"echo""echo"Detected resources:"echo" Topology name:$TOPOLOGY_NAME"echo" Kind cluster:${KIND_CLUSTER:-N/A}"echo" Kubeconfig file:${KUBECONFIG_FILE:-N/A}"echo""read-p"Continue with cleanup? (yes/no): "-recho""if[[!$REPLY=~^[Yy][Ee][Ss]$]];thenecho"Cleanup cancelled."exit0fi# Step 1: Destroy ContainerLab topologyecho-e"${GREEN}[1/5] Destroying ContainerLab topology:$TOPOLOGY_NAME${NC}"ifcommand-vcontainerlab&>/dev/null;thensudocontainerlab destroy-t"$TOPOLOGY_FILE"2>/dev/null||echo" Topology may not exist or already destroyed"elseecho" containerlab command not found, skipping"fi# Step 2: Delete Kind clusterif[-n"$KIND_CLUSTER"];thenecho-e"${GREEN}[2/5] Deleting Kind cluster:$KIND_CLUSTER${NC}"ifcommand-vkind&>/dev/null;thenkind delete cluster--name"$KIND_CLUSTER"2>/dev/null||echo" Cluster may not exist or already deleted"elseecho" kind command not found, skipping"fielseecho-e"${GREEN}[2/5] No Kind cluster detected, skipping${NC}"fi# Step 3: Remove lab-specific containersecho-e"${GREEN}[3/5] Removing lab-specific containers...${NC}"CONTAINERS=$(dockerps-a--format"{{.Names}}"2>/dev/null|grep-E"clab-${TOPOLOGY_NAME}-|${KIND_CLUSTER}-"||true)if[-n"$CONTAINERS"];thenecho"$CONTAINERS"|whileread-rcontainer;doif[-n"$container"];thenecho" Removing container:$container"dockerrm-f"$container"2>/dev/null||truefidoneelseecho" No lab-specific containers found"fi# Step 4: Remove lab-specific networksecho-e"${GREEN}[4/5] Removing lab-specific networks...${NC}"NETWORKS=$(dockernetworkls--format"{{.Name}}"2>/dev/null|grep-E"clab-${TOPOLOGY_NAME}"||true)if[-n"$NETWORKS"];thenecho"$NETWORKS"|whileread-rnetwork;doif[-n"$network"];thenecho" Removing network:$network"dockernetworkrm"$network"2>/dev/null||truefidoneelseecho" No lab-specific networks found"fi# Step 5: Clean up Kubernetes resources and kubeconfigif[-n"$KUBECONFIG_FILE"]&&[-f"$KUBECONFIG_FILE"];thenecho-e"${GREEN}[5/5] Cleaning up Kubernetes resources...${NC}"exportKUBECONFIG="$(pwd)/$KUBECONFIG_FILE"ifcommand-vkubectl&>/dev/null&&kubectl cluster-info&>/dev/null2>&1;then# Force delete stuck terminating podskubectl get pods --all-namespaces --field-selector=status.phase!=Running,status.phase!=Succeeded-ojson2>/dev/null|\jq-r'.items[] | select(.metadata.deletionTimestamp!=null) | "\(.metadata.namespace) \(.metadata.name)"'2>/dev/null|\whileread-rnamespace name;doif[-n"$namespace"]&&[-n"$name"];thenecho" Force deleting stuck pod:$namespace/$name"kubectl delete pod"$name"-n"$namespace"--force--grace-period=02>/dev/null||truefidone||true# Delete lab-related namespaces (excluding system namespaces)kubectl get namespaces-ojsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}'2>/dev/null|\grep-vE"^(default|kube-system|kube-public|kube-node-lease|local-path-storage)$"|\whileread-rnamespace;doif[-n"$namespace"];thenecho" Deleting namespace:$namespace"kubectl delete namespace"$namespace"--timeout=30s2>/dev/null||truefidone||truefiunsetKUBECONFIG# Remove kubeconfig fileecho" Removing kubeconfig file:$KUBECONFIG_FILE"rm-f"$KUBECONFIG_FILE"elseecho-e"${GREEN}[5/5] No kubeconfig file found, skipping${NC}"fiecho""echo-e"${GREEN}========================================${NC}"echo-e"${GREEN}Cleanup Complete!${NC}"echo-e"${GREEN}========================================${NC}"echo""