本文永久链接: https://www.xtplayer.cn/coredns/persistent-coredns-custom-configuration/

根据之前的文章 coreDNS 加速外部域名解析coredns 解析外部域名 了解到,如果需要访问集群外部一个没有 dns 解析的域名,可以通过修改 coredns 配置映射文件,使用 coredns 的 hosts 插件进行配置解析。

以上通过手动修改配置映射的方式可以临时生效,在 rancher-rke 集群中,coredns 由 rancher 运行的一个 job 来负责安装和更新。在 k8s 集群进行大版本更新时,会触发 job 的重新运行,从而会覆盖之前的所有自定义配置。要持久化配置,需要通过以下方法固定配置。

  • RKE1

在 rke1 集群的 YAML 配置中,将 coredns 的完整配置以 addons 的方式添加。

rancher_kubernetes_engine_config:
addon_job_timeout: 45
addons: |-
---
apiVersion: v1
kind: ConfigMap
metadata:
name: coredns
namespace: kube-system
data:
Corefile: |
.:53 {
errors
health
ready
hosts {
# 1 个 hostname 映射 1 个 ip;
172.30.200.21 www.test1.local
172.30.200.22 www.test2.local
172.30.200.23 www.test3.local
fallthrough
}
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
}
prometheus :9153
forward . "/etc/resolv.conf" {
policy sequential
}
cache 30
loop
reload
loadbalance
}
  • RKE2

rke2 支持使用 HelmChartConfig 来自定义附件组件的配置,

  1. 访问 https://github.com/rancher/rke2-charts/blob/main/charts/rke2-coredns/rke2-coredns/1.19.401/values.yaml#L102 可以查看 coredns chart server 的默认参数配置
  2. 根据文档 使用 HelmChartConfig 自定义打包的组件 说明,在valuesContent 中添加一个 hosts 插件,完整配置示例如下:
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: rke2-coredns
namespace: kube-system
spec:
valuesContent: |-
servers:
- zones:
- zone: example.org
port: 53
# If serviceType is nodePort you can specify nodePort here
# nodePort: 30053
plugins:
- name: hosts
configBlock: |-
1.2.3.4 www.aaa.com
fallthrough
- name: errors
# Serves a /health endpoint on :8080, required for livenessProbe
- name: health
configBlock: |-
lameduck 5s
# Serves a /ready endpoint on :8181, required for readinessProbe
- name: ready
# Required to query kubernetes API for data
- name: kubernetes
parameters: cluster.local in-addr.arpa ip6.arpa
configBlock: |-
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
# Serves a /metrics endpoint on :9153, required for serviceMonitor
- name: prometheus
parameters: 0.0.0.0:9153
# https://coredns.io/plugins/forward/
- name: forward
parameters: . /etc/resolv.conf
configBlock: |-
# is the number of subsequent failed health checks that are needed before considering an upstream to be down. If 0, the upstream will never be marked as down (nor health checked). Default is 2.
max_fails 2
# expire (cached) connections after this time, the default is 10s
expire 10s
# policy default random, Optional: random|round_robin|sequential
## random is a policy that implements random upstream selection.
## round_robin is a policy that selects hosts based on round robin ordering.
## sequential is a policy that selects hosts based on sequential ordering.
policy random
- name: cache
parameters: 30
- name: loop
- name: reload
- name: loadbalance
- name: log
- zones:
- zone: .
port: 53
# If serviceType is nodePort you can specify nodePort here
# nodePort: 30053
plugins:
- name: hosts
configBlock: |-
1.2.3.4 www.xxx.com
fallthrough
- name: errors
# Serves a /health endpoint on :8080, required for livenessProbe
- name: health
configBlock: |-
lameduck 5s
# Serves a /ready endpoint on :8181, required for readinessProbe
- name: ready
# Required to query kubernetes API for data
- name: kubernetes
parameters: cluster.local in-addr.arpa ip6.arpa
configBlock: |-
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
# Serves a /metrics endpoint on :9153, required for serviceMonitor
- name: prometheus
parameters: 0.0.0.0:9153
# https://coredns.io/plugins/forward/
- name: forward
parameters: . /etc/resolv.conf
configBlock: |-
# is the number of subsequent failed health checks that are needed before considering an upstream to be down. If 0, the upstream will never be marked as down (nor health checked). Default is 2.
max_fails 2
# expire (cached) connections after this time, the default is 10s
expire 10s
# policy default random, Optional: random|round_robin|sequential
## random is a policy that implements random upstream selection.
## round_robin is a policy that selects hosts based on round robin ordering.
## sequential is a policy that selects hosts based on sequential ordering.
policy random
- name: cache
parameters: 30
- name: loop
- name: reload
- name: loadbalance
- name: log
  • K3S

根据 issue https://github.com/k3s-io/k3s/issues/462 和 PR https://github.com/k3s-io/k3s/pull/4397 的说明,可以通过在 kube-system 命名空间中创建名为 coredns-custom 的配置映射文件来自定义 coredns 配置。

参考文档:https://learn.microsoft.com/zh-cn/azure/aks/coredns-custom