ansible_ssh_user=root
debug_level=2
openshift_clock_enabled=true
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider'}]
openshift_hosted_registry_cert_expire_days=3650
os_firewall_use_firewalld=true
openshift_router_selector='node-role.kubernetes.io/infra=true'
openshift_registry_selector='node-role.kubernetes.io/infra=true'
template_service_broker_selector='node-role.kubernetes.io/infra=true'
osm_default_node_selector='node-role.kubernetes.io/compute=true'
openshift_template_service_broker_namespaces=['openshift','tsb']
openshift_docker_insecure_registries="registry.example.com"
openshift_metrics_hawkular_hostname=hawkular-metrics.apps.example.com
openshift_image_tag=v3.11.16安装
Inventory
Ansible inventory 文件描述了有关集群中主机的详细信息以及 OpenShift 安装过程的详细配置。OpenShift 安装 playbooks 会读取该 inventory 文件，以了解在何处以及如何在一组主机上安装 OpenShift。
本部分说明如何在 Master 节点编辑 /etc/ansible/hosts，主要通过变量控制安装。
集群变量
| 名称 | 作用域 | 描述 |
|---|---|---|
ansible_ssh_user |
General |
Ansible SSH 登录用户，默认 |
debug_level |
General |
Ansible 输出日志级别，可能的值 0, 2, 4, 6, 8，默认 2，输出 INFO 日志，4 为 DEBUG 日志 |
openshift_clock_enabled |
General |
是否开启网络时钟 NTP，默认值 |
openshift_master_cluster_method |
General |
部署多个 Master 时定义 HA 的方法，支持 |
openshift_master_identity_providers |
General |
配置认证提供者， |
openshift_hosted_registry_cert_expire_days |
General |
默认自动生成的证书过期时间 |
os_firewall_use_firewalld |
General |
3.9 及以后的版本安装推荐使用 firewalld，设定为 |
openshift_router_selector |
General |
设定部署 router Pod 的节点 |
openshift_registry_selector |
General |
设定部署 Registry Pod 的节点 |
openshift_template_service_broker_namespaces |
General |
Enables the template service broker by specifying one or more namespaces whose templates will be served by the broker. |
openshift_master_bootstrap_auto_approve |
General |
Enables TLS bootstrapping auto approval, which allows nodes to automatically join the cluster when provided a bootstrap credential. 默认值为 |
template_service_broker_selector |
General |
设定部署 Service Broker Pod 的节点 |
osm_default_node_selector |
General |
设定默认应用 Pod 部署的计算节点 |
openshift_docker_insecure_registries |
General |
设定一个非安全 Docker 仓库，Ansible 运行会将其配置到 Docker。 |
openshift_image_tag |
General |
设定一个全局镜像 TAG |
openshift_pkg_version |
General |
设定一个全局的 RPM 包版本号 |
openshift_master_default_subdomain |
Networking |
设定路由使用的子域名地址 |
os_sdn_network_plugin_name |
Networking |
设定默认网络插件名称，默认 |
osm_cluster_network_cidr |
Networking |
This variable overrides the SDN cluster network CIDR block. This is the network from which pod IPs are assigned. Specify a private block that does not conflict with existing network blocks in your infrastructure to which pods, nodes, or the master might require access. |
openshift_portal_net |
Networking |
配置子网的范围，默认 |
openshift_use_openshift_sdn |
Networking |
设定为 |
示例 - 全局集群变量
示例 - 网络变量
openshift_master_default_subdomain=apps.example.com
os_sdn_network_plugin_name='redhat/openshift-ovs-subnet'
#os_sdn_network_plugin_name='redhat/openshift-ovs-multitenant'
#os_sdn_network_plugin_name='redhat/openshift-ovs-networkpolicy'
osm_cluster_network_cidr=10.244.0.0/16
openshift_portal_net=172.30.0.0/16部署类型
确保部署类型为 openshift-enterprise
openshift_deployment_type=openshift-enterprise外部镜像仓库位置
如果不使用默认红帽提供的镜像仓库 registry.redhat.io，而使用一个外部镜像仓库，需要定义如下变量
oreg_url=registry.example.com/openshift3/ose-${component}:${version}
openshift_examples_modify_imagestreams=true
#oreg_auth_user=admin
#oreg_auth_password=admin内部镜像仓库路径
为了允许用户向内部镜像仓库推送/拉取镜像，需要添加如下配置
openshift_hosted_registry_routehost=registry.apps.example.com
openshift_hosted_registry_routetermination=reencrypt
openshift_hosted_registry_routecertificates= "{'certfile': '/etc/crts/com-example-cert.pem', 'keyfile': '/etc/crts/com-example-privkey.pem', 'cafile': '/etc/crts/com-example-chain.pem'}"Web Console
openshift_web_console_install=true
openshift_web_console_prefix=registry.example.com/openshift3/ose-
openshift_web_console_version=v3.11.16示例
| Inventory | 说明 |
|---|---|
OpenShift 3.11.16 |
|
OpenShift 3.11.16 |
|
OpenShift 3.10.45 |
|
OpenShift 3.10.14 |
|
OpenShift 3.9.30 |
|
OpenShift 3.9.25 |
|
OpenShift 3.9.14 |
|
OpenShift 3.6 |
安装
1. Master 上执行 prerequisites 验证
# ansible-playbook -i hosts-3.11.16 /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml2. Master 上执行 ansible 脚本
# ansible-playbook -i hosts-3.11.16 /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml安装成功验证
1 - 查看 INSTALLER STATUS（ansible 脚本运行的结尾）
INSTALLER STATUS *****************************************************************************************************************************************************************************
Initialization : Complete (0:00:27)
Health Check : Complete (0:00:03)
Node Bootstrap Preparation : Complete (0:11:25)
etcd Install : Complete (0:00:42)
NFS Install : Complete (0:00:09)
Master Install : Complete (0:03:43)
Master Additional Install : Complete (0:01:02)
Node Join : Complete (0:03:48)
Hosted Install : Complete (0:00:58)
Web Console Install : Complete (0:00:41)
Metrics Install : Complete (0:02:03)
Prometheus Install : Complete (0:01:10)
Service Catalog Install : Complete (0:07:20)2 - 查看所有 Nodes
# oc get nodes
NAME STATUS ROLES AGE VERSION
infra.example.com Ready infra 1h v1.11.0+d4cacc0
master.example.com Ready master 1h v1.11.0+d4cacc0
node1.example.com Ready compute 1h v1.11.0+d4cacc0
node2.example.com Ready compute 1h v1.11.0+d4cacc03 - 查看所有 Projects
# oc projects
You have access to the following projects and can switch between them with 'oc project <projectname>':
* default
kube-public
kube-service-catalog
kube-system
management-infra
openshift
openshift-ansible-service-broker
openshift-console
openshift-infra
openshift-logging
openshift-metrics
openshift-metrics-server
openshift-monitoring
openshift-node
openshift-sdn
openshift-template-service-broker
openshift-web-console
Using project "default" on server "https://master.example.com:8443".4 - 查看所有 Pods
# oc get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default docker-registry-1-vj4q5 1/1 Running 0 1h
default registry-console-1-wvm5x 1/1 Running 0 1h
default router-1-kkxtz 1/1 Running 0 1h
kube-service-catalog apiserver-wvpfb 1/1 Running 0 33m
kube-service-catalog controller-manager-wlhjp 1/1 Running 0 33m
kube-system master-api-master.example.com 1/1 Running 0 1h
kube-system master-controllers-master.example.com 1/1 Running 0 1h
kube-system master-etcd-master.example.com 1/1 Running 0 1h
openshift-ansible-service-broker asb-1-br294 1/1 Running 0 33m
openshift-console console-5896bbb547-v624t 1/1 Running 0 1h
openshift-infra hawkular-cassandra-1-9mjqp 1/1 Running 0 1h
openshift-infra hawkular-metrics-9wcns 1/1 Running 0 1h
openshift-infra hawkular-metrics-schema-pg6x8 0/1 Completed 0 1h
openshift-infra heapster-xw7jj 1/1 Running 0 1h
openshift-logging logging-es-data-master-ilrkj84i-2-2ddqb 0/2 Pending 0 2m
openshift-logging logging-es-data-master-ilrkj84i-2-deploy 1/1 Running 0 2m
openshift-logging logging-fluentd-5g9jm 1/1 Running 0 35m
openshift-logging logging-fluentd-9r9tn 1/1 Running 0 35m
openshift-logging logging-fluentd-j7tgr 1/1 Running 0 35m
openshift-logging logging-fluentd-wgxpq 1/1 Running 0 35m
openshift-logging logging-kibana-1-glrhs 2/2 Running 0 36m
openshift-metrics-server metrics-server-845b478887-jgv84 1/1 Running 0 1h
openshift-metrics prometheus-0 0/6 Pending 0 25m
openshift-metrics prometheus-node-exporter-4bbpn 1/1 Running 0 25m
openshift-metrics prometheus-node-exporter-ftc8r 1/1 Running 0 25m
openshift-metrics prometheus-node-exporter-j2gtd 1/1 Running 0 25m
openshift-metrics prometheus-node-exporter-vhc2h 1/1 Running 0 25m
openshift-monitoring alertmanager-main-0 3/3 Running 0 1h
openshift-monitoring alertmanager-main-1 3/3 Running 0 1h
openshift-monitoring alertmanager-main-2 3/3 Running 0 1h
openshift-monitoring cluster-monitoring-operator-674969789d-5bjqw 1/1 Running 0 1h
openshift-monitoring grafana-7594d8dd75-cjf4f 2/2 Running 0 1h
openshift-monitoring kube-state-metrics-787f69cf4d-2nbxh 3/3 Running 0 1h
openshift-monitoring node-exporter-7zmvq 2/2 Running 0 1h
openshift-monitoring node-exporter-cpn62 2/2 Running 0 1h
openshift-monitoring node-exporter-gt64r 2/2 Running 0 1h
openshift-monitoring node-exporter-nxxsq 2/2 Running 0 1h
openshift-monitoring prometheus-k8s-0 4/4 Running 1 1h
openshift-monitoring prometheus-k8s-1 4/4 Running 1 1h
openshift-monitoring prometheus-operator-8544897d54-hwgxq 1/1 Running 0 1h
openshift-node sync-6f88d 1/1 Running 0 1h
openshift-node sync-ds586 1/1 Running 0 1h
openshift-node sync-tnqs7 1/1 Running 0 1h
openshift-node sync-wdjct 1/1 Running 0 1h
openshift-sdn ovs-72dd9 1/1 Running 0 1h
openshift-sdn ovs-kq695 1/1 Running 0 1h
openshift-sdn ovs-nnm8d 1/1 Running 0 1h
openshift-sdn ovs-rlt5j 1/1 Running 0 1h
openshift-sdn sdn-f5fjv 1/1 Running 0 1h
openshift-sdn sdn-jmfw9 1/1 Running 0 1h
openshift-sdn sdn-rkstw 1/1 Running 0 1h
openshift-sdn sdn-zq5ms 1/1 Running 0 1h
openshift-template-service-broker apiserver-dtns4 1/1 Running 0 32m
openshift-web-console webconsole-5db89b6cd4-2p9sc 1/1 Running 2 1h5 - 查看不同类型节点的镜像
# for i in master infra node1 node2 ; do ssh $i.example.com 'docker images ; echo' ; done
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.example.com/openshift3/ose-logging-fluentd v3.11.16 715c3499714d 4 months ago 286 MB
registry.example.com/openshift3/ose-node v3.11.16 074bf04571e2 4 months ago 1.15 GB
registry.example.com/openshift3/ose-control-plane v3.11.16 1f23828b535b 4 months ago 794 MB
registry.example.com/openshift3/ose-deployer v3.11.16 a6b0077a8dd8 4 months ago 357 MB
registry.example.com/openshift3/ose-kube-rbac-proxy v3.11.16 5c1087872e66 4 months ago 482 MB
registry.example.com/openshift3/ose-console v3.11.16 325013035116 4 months ago 250 MB
registry.example.com/openshift3/ose-template-service-broker v3.11.16 4bb11311e3eb 4 months ago 308 MB
registry.example.com/openshift3/ose-web-console v3.11.16 81206d80b8cb 4 months ago 318 MB
registry.example.com/openshift3/ose-pod v3.11.16 751ff215ba5d 4 months ago 234 MB
registry.example.com/openshift3/ose-service-catalog v3.11.16 07094eebb54e 4 months ago 306 MB
registry.example.com/openshift3/registry-console v3.11.16 26eb64a24c85 4 months ago 268 MB
registry.example.com/openshift3/prometheus-node-exporter v3.11.16 82e339558772 4 months ago 222 MB
registry.example.com/rhel7/etcd 3.2.22 fef709cdf7bf 4 months ago 256 MB
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.example.com/openshift3/ose-logging-fluentd v3.11.16 715c3499714d 4 months ago 286 MB
registry.example.com/openshift3/ose-node v3.11.16 074bf04571e2 4 months ago 1.15 GB
registry.example.com/openshift3/ose-haproxy-router v3.11.16 d69159fc58ec 4 months ago 374 MB
registry.example.com/openshift3/ose-deployer v3.11.16 a6b0077a8dd8 4 months ago 357 MB
registry.example.com/openshift3/ose-prometheus-operator v3.11.16 4dca7b8e24af 4 months ago 577 MB
registry.example.com/openshift3/ose-kube-rbac-proxy v3.11.16 5c1087872e66 4 months ago 482 MB
registry.example.com/openshift3/ose-prometheus-config-reloader v3.11.16 70da44d1047a 4 months ago 505 MB
registry.example.com/openshift3/grafana v3.11.16 b8e75c0c5919 4 months ago 496 MB
registry.example.com/openshift3/ose-cluster-monitoring-operator v3.11.16 8f24257f39b5 4 months ago 450 MB
registry.example.com/openshift3/ose-kube-state-metrics v3.11.16 0ed02df2f280 4 months ago 437 MB
registry.example.com/openshift3/ose-configmap-reloader v3.11.16 e082dd1e68f1 4 months ago 383 MB
registry.example.com/openshift3/ose-pod v3.11.16 751ff215ba5d 4 months ago 234 MB
registry.example.com/openshift3/ose-docker-registry v3.11.16 0845efb9cd14 4 months ago 284 MB
registry.example.com/openshift3/prometheus-node-exporter v3.11.16 82e339558772 4 months ago 222 MB
registry.example.com/openshift3/prometheus-alertmanager v3.11.16 6d2601ffa89e 4 months ago 233 MB
registry.example.com/openshift3/prometheus v3.11.16 71f710b543cb 4 months ago 280 MB
registry.example.com/openshift3/oauth-proxy v3.11.16 e2cdfe500fc3 4 months ago 238 MB
registry.example.com/openshift3/metrics-hawkular-metrics v3.11.16 e0f2bc2788bc 4 months ago 1.71 GB
registry.example.com/openshift3/metrics-heapster v3.11.16 4c3b09e7ec40 4 months ago 281 MB
registry.example.com/openshift3/ose-ansible-service-broker v3.11.16 f544ede0d4d2 4 months ago 453 MB
registry.example.com/openshift3/metrics-cassandra v3.11.16 c04bcec14ebd 4 months ago 560 MB
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.example.com/openshift3/ose-logging-fluentd v3.11.16 715c3499714d 4 months ago 286 MB
registry.example.com/openshift3/ose-node v3.11.16 074bf04571e2 4 months ago 1.15 GB
registry.example.com/openshift3/ose-kube-rbac-proxy v3.11.16 5c1087872e66 4 months ago 482 MB
registry.example.com/openshift3/ose-metrics-server v3.11.16 af8813db7705 4 months ago 290 MB
registry.example.com/openshift3/ose-pod v3.11.16 751ff215ba5d 4 months ago 234 MB
registry.example.com/openshift3/prometheus-node-exporter v3.11.16 82e339558772 4 months ago 222 MB
registry.example.com/openshift3/ose-logging-kibana5 v3.11.16 e770a1c6e536 4 months ago 751 MB
registry.example.com/openshift3/oauth-proxy v3.11.16 e2cdfe500fc3 4 months ago 238 MB
REPOSITORY TAG IMAGE ID CREATED SIZE
registry.example.com/openshift3/ose-logging-fluentd v3.11.16 715c3499714d 4 months ago 286 MB
registry.example.com/openshift3/ose-node v3.11.16 074bf04571e2 4 months ago 1.15 GB
registry.example.com/openshift3/ose-deployer v3.11.16 a6b0077a8dd8 4 months ago 357 MB
registry.example.com/openshift3/ose-kube-rbac-proxy v3.11.16 5c1087872e66 4 months ago 482 MB
registry.example.com/openshift3/ose-pod v3.11.16 751ff215ba5d 4 months ago 234 MB
registry.example.com/openshift3/prometheus-node-exporter v3.11.16 82e339558772 4 months ago 222 MB
registry.example.com/openshift3/metrics-schema-installer v3.11.16 94b5b0e3fae6 4 months ago 845 MB6 - 查看 Master 监听端口
# netstat -antulop | grep LISTEN
tcp 0 0 0.0.0.0:8443 0.0.0.0:* LISTEN 12700/openshift off (0.00/0/0)
tcp 0 0 0.0.0.0:8444 0.0.0.0:* LISTEN 12650/openshift off (0.00/0/0)
tcp 0 0 192.168.122.101:2379 0.0.0.0:* LISTEN 12680/etcd off (0.00/0/0)
tcp 0 0 192.168.122.101:2380 0.0.0.0:* LISTEN 12680/etcd off (0.00/0/0)
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 26356/rpcbind off (0.00/0/0)
tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN 20501/openshift off (0.00/0/0)
tcp 0 0 10.244.0.1:53 0.0.0.0:* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp 0 0 0.0.0.0:8053 0.0.0.0:* LISTEN 12700/openshift off (0.00/0/0)
tcp 0 0 192.168.122.101:53 0.0.0.0:* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp 0 0 172.17.0.1:53 0.0.0.0:* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 992/sshd off (0.00/0/0)
tcp 0 0 127.0.0.1:11256 0.0.0.0:* LISTEN 20501/openshift off (0.00/0/0)
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1382/master off (0.00/0/0)
tcp6 0 0 :::9090 :::* LISTEN 1/systemd off (0.00/0/0)
tcp6 0 0 :::10250 :::* LISTEN 24344/hyperkube off (0.00/0/0)
tcp6 0 0 :::9100 :::* LISTEN 21015/node_exporter off (0.00/0/0)
tcp6 0 0 :::111 :::* LISTEN 26356/rpcbind off (0.00/0/0)
tcp6 0 0 :::10256 :::* LISTEN 20501/openshift off (0.00/0/0)
tcp6 0 0 fe80::e4ec:eeff:feec:53 :::* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp6 0 0 fe80::bca3:6ff:fe69::53 :::* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp6 0 0 fe80::7cae:5cff:fe6d:53 :::* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp6 0 0 fe80::d0d2:cdff:fe20:53 :::* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp6 0 0 fe80::a018:1bff:fe00:53 :::* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp6 0 0 fe80::b8bd:20ff:feb6:53 :::* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp6 0 0 fe80::746a:65ff:fe41:53 :::* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp6 0 0 fe80::88e8:5ff:fecb::53 :::* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp6 0 0 fe80::5054:ff:fe4c:2:53 :::* LISTEN 21745/dnsmasq off (0.00/0/0)
tcp6 0 0 :::22 :::* LISTEN 992/sshd off (0.00/0/0)
tcp6 0 0 ::1:25 :::* LISTEN 1382/master off (0.00/0/0)