使用Helm部署微服务应用PiggyMetrics

发布时间：2020-05-12 15:29:23编辑：admin阅读（2927）

一、概述

本文介绍如何将一个较复杂的应用部署到Kubernetes容器服务上，下面将从基础设施和应用部署的不同组合方式，来部署一个复杂的SpringCloud应用。

PiggyMetrics

PiggyMetrics是github上的一个SpringCloud应用项目，Star数目3400多。这个项目主体采用Docker Compose部署，包含了完整的源代码以及构建好的容器镜像，是非常不错的SpringCloud容器化示例。



 

 

 

这个项目包含了3个业务微服务，分别是统计服务（Statistics Service）、账户服务（Account Service）和通知服务（Notification Service）。每个服务分别对应一个独立的MongoDB。微服务架构图示（采用作者原图）如下：



 

 

 

 

SpringCloud基础组件负责服务注册和registry服务（Eureka服务注册），config服务（配置管理），gateway（API网关，同时也是JavaScript Web界面），monitor服务（Hystrix Dashboard/Turbine）等。

 

二、用helm一键部署所有服务

修改docker-compose

1. 修改PiggyMetrics应用程序的Docker编写文件
切换配置文件版本。
PiggyMetrics的部署采用docker-compose YAML部署到单机，如果要部署到Kubernetes环境中 ，需要转换成为Kubernetes deployment YAML。

说明 PiggyMetrics中的docker compose模版为2.1，kompose不支持该版本，所以需要把compose文件改为版本2。




2. 在docker-compose.yml文件中。

去除kompose不支持的语法。

depends_on:
      config:
        condition: service_healthy  # 不支持 condition

增加Kubernetes server type annotation。

depends_on:
  - config
labels:
  kompose.service.type: loadbalancer

在docker-compose.dev.yml文件中，将PiggyMetrics应用程序使用的四个MongoDB数据库的外部端口更改为27017。

说明 PiggyMetrics应用包含四个MongoDB数据库，分别由 auth-mongodb, 、account-mongodb、statistics-mongodb和 notification-mongodb这四个字段定义。

完整的YAML文件示例如下：

version: '2'
services:
  rabbitmq:
    image: rabbitmq:3-management
    restart: always
    labels:
      kompose.service.type: nodeport
    ports:
      - 5672
      - 15672:15672
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  config:
    environment:
      CONFIG_SERVICE_PASSWORD: $CONFIG_SERVICE_PASSWORD
    image: sqshq/piggymetrics-config
    restart: always
    ports:
      - 8888
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  registry:
    environment:
      CONFIG_SERVICE_PASSWORD: $CONFIG_SERVICE_PASSWORD
    image: sqshq/piggymetrics-registry
    restart: always
    depends_on:
      - config
    labels:
      kompose.service.type: loadbalancer
    ports:
      - 8761:8761
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  gateway:
    environment:
      CONFIG_SERVICE_PASSWORD: $CONFIG_SERVICE_PASSWORD
    image: sqshq/piggymetrics-gateway
    restart: always
    depends_on:
      - config
    labels:
      kompose.service.type: loadbalancer
    ports:
      - 4000:4000
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  auth-service:
    environment:
      CONFIG_SERVICE_PASSWORD: $CONFIG_SERVICE_PASSWORD
      NOTIFICATION_SERVICE_PASSWORD: $NOTIFICATION_SERVICE_PASSWORD
      STATISTICS_SERVICE_PASSWORD: $STATISTICS_SERVICE_PASSWORD
      ACCOUNT_SERVICE_PASSWORD: $ACCOUNT_SERVICE_PASSWORD
      MONGODB_PASSWORD: $MONGODB_PASSWORD
    image: sqshq/piggymetrics-auth-service
    restart: always
    ports:
      - 5000
    depends_on:
      - config
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  auth-mongodb:
    environment:
      MONGODB_PASSWORD: $MONGODB_PASSWORD
    image: sqshq/piggymetrics-mongodb
    restart: always
    ports:
      - 27017
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  account-service:
    environment:
      CONFIG_SERVICE_PASSWORD: $CONFIG_SERVICE_PASSWORD
      ACCOUNT_SERVICE_PASSWORD: $ACCOUNT_SERVICE_PASSWORD
      MONGODB_PASSWORD: $MONGODB_PASSWORD
    image: sqshq/piggymetrics-account-service
    restart: always
    ports:
      - 6000
    depends_on:
      - config
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  account-mongodb:
    environment:
      INIT_DUMP: account-service-dump.js
      MONGODB_PASSWORD: $MONGODB_PASSWORD
    image: sqshq/piggymetrics-mongodb
    restart: always
    ports:
      - 27017
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  statistics-service:
    environment:
      CONFIG_SERVICE_PASSWORD: $CONFIG_SERVICE_PASSWORD
      MONGODB_PASSWORD: $MONGODB_PASSWORD
      STATISTICS_SERVICE_PASSWORD: $STATISTICS_SERVICE_PASSWORD
    image: sqshq/piggymetrics-statistics-service
    restart: always
    ports:
      - 8888
    depends_on:
      - config
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  statistics-mongodb:
    environment:
      MONGODB_PASSWORD: $MONGODB_PASSWORD
    image: sqshq/piggymetrics-mongodb
    restart: always
    ports:
      - 27017
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  notification-service:
    environment:
      CONFIG_SERVICE_PASSWORD: $CONFIG_SERVICE_PASSWORD
      MONGODB_PASSWORD: $MONGODB_PASSWORD
      NOTIFICATION_SERVICE_PASSWORD: $NOTIFICATION_SERVICE_PASSWORD
    image: sqshq/piggymetrics-notification-service
    restart: always
    ports:
      - 8000
    depends_on:
      - config
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  notification-mongodb:
    image: sqshq/piggymetrics-mongodb
    restart: always
    environment:
      MONGODB_PASSWORD: $MONGODB_PASSWORD
    ports:
      - 27017
    logging:
      options:
        max-size: "10m"
        max-file: "10"

  monitoring:
    environment:
      CONFIG_SERVICE_PASSWORD: $CONFIG_SERVICE_PASSWORD
    image: sqshq/piggymetrics-monitoring
    restart: always
    depends_on:
      - config
    labels:
      kompose.service.type: loadbalancer
    ports:
      - 9000:8080
      - 8989:8989
    logging:
      options:
        max-size: "10m"
        max-file: "10"
---




使用Kompose生成Kubernetes配置文件

2. 使用Kompose生成Kubernetes配置文件来部署PiggyMetrics应用。

设定PiggyMetrics部署所需的环境变量。

export NOTIFICATION_SERVICE_PASSWORD=passw0rd
export CONFIG_SERVICE_PASSWORD=passw0rd
export STATISTICS_SERVICE_PASSWORD=passw0rd
export ACCOUNT_SERVICE_PASSWORD=passw0rd
export MONGODB_PASSWORD=passw0rd




运行以下命令将compose文件转换为Kubernetes配置文件。

kompose工具可以一键将compose文件转换为Kubernetes配置文件。

安装kompose

curl -L https://github.com/kubernetes/kompose/releases/download/v1.21.0/kompose-linux-amd64 -o kompose
chmod +x kompose
sudo mv ./kompose /usr/local/bin/kompose




转换配置文件

kompose convert -f docker-compose.yml -o piggymetrics -c

注意：原文给的还指定了docker-compose.dev.yml，执行会报错。提示docker-compose版本号不一致。

FATA All Docker Compose files must be of the same version


 

即使将2个yaml文件版本号改为一致，在后面执行helm install 命令时，会报错：

Error: release piggy failed: Service "config" is invalid: spec.ports[1]: Duplicate value: core.ServicePort{Name:"", Protocol:"TCP", AppProtocol:(*string)(nil), Port:8888, TargetPort:intstr.IntOrString{Type:0, IntVal:0, StrVal:""}, NodePort:0}




执行成功后，会生成文件夹piggymetrics，目录结构如下：

./
├── Chart.yaml
├── README.md
└── templates
    ├── account-mongodb-deployment.yaml
    ├── account-mongodb-service.yaml
    ├── account-service-deployment.yaml
    ├── account-service-service.yaml
    ├── auth-mongodb-deployment.yaml
    ├── auth-mongodb-service.yaml
    ├── auth-service-deployment.yaml
    ├── auth-service-service.yaml
    ├── config-deployment.yaml
    ├── config-service.yaml
    ├── gateway-deployment.yaml
    ├── gateway-service.yaml
    ├── monitoring-deployment.yaml
    ├── monitoring-service.yaml
    ├── notification-mongodb-deployment.yaml
    ├── notification-mongodb-service.yaml
    ├── notification-service-deployment.yaml
    ├── notification-service-service.yaml
    ├── rabbitmq-deployment.yaml
    ├── rabbitmq-service.yaml
    ├── registry-deployment.yaml
    ├── registry-service.yaml
    ├── statistics-mongodb-deployment.yaml
    ├── statistics-mongodb-service.yaml
    ├── statistics-service-deployment.yaml
    └── statistics-service-service.yaml




 

运行helm install命令

运行helm install命令，在Kubernetes集群中部署PiggyMetrics应用。

例如，您可以运行以下命令，在命名空间pm中部署名为piggy的应用。

helm install --namespace pm --name piggy piggymetrics/

输出如下：

NAME:   piggy
LAST DEPLOYED: Sun Apr 26 13:30:46 2020
NAMESPACE: pm
STATUS: DEPLOYED

RESOURCES:
==> v1/Deployment
NAME                  READY  UP-TO-DATE  AVAILABLE  AGE
account-mongodb       0/1    0           0          0s
account-service       0/1    0           0          0s
auth-mongodb          0/1    0           0          0s
auth-service          0/1    0           0          0s
config                0/1    0           0          0s
gateway               0/1    0           0          0s
monitoring            0/1    0           0          0s
notification-mongodb  0/1    0           0          0s
notification-service  0/1    0           0          0s
rabbitmq              0/1    0           0          0s
registry              0/1    0           0          0s
statistics-mongodb    0/1    0           0          0s
statistics-service    0/1    0           0          0s

==> v1/Pod(related)
NAME                                 READY  STATUS             RESTARTS  AGE
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s
statistics-mongodb-655997cb95-jlfrf  0/1    ContainerCreating  0         1s

==> v1/Service
NAME                  TYPE          CLUSTER-IP    EXTERNAL-IP  PORT(S)                         AGE
account-mongodb       ClusterIP     10.1.186.96   <none>       27017/TCP                       0s
account-service       ClusterIP     10.1.79.137   <none>       6000/TCP                        0s
auth-mongodb          ClusterIP     10.1.37.220   <none>       27017/TCP                       0s
auth-service          ClusterIP     10.1.241.143  <none>       5000/TCP                        0s
config                ClusterIP     10.1.104.247  <none>       8888/TCP                        0s
gateway               LoadBalancer  10.1.186.147  <pending>    4000:30023/TCP                  0s
monitoring            LoadBalancer  10.1.180.30   <pending>    9000:32101/TCP,8989:30052/TCP   0s
notification-mongodb  ClusterIP     10.1.7.3      <none>       27017/TCP                       0s
notification-service  ClusterIP     10.1.109.117  <none>       8000/TCP                        0s
rabbitmq              NodePort      10.1.245.70   <none>       5672:31239/TCP,15672:31437/TCP  0s
registry              LoadBalancer  10.1.250.187  <pending>    8761:31604/TCP                  0s
statistics-mongodb    ClusterIP     10.1.81.216   <none>       27017/TCP                       0s
statistics-service    ClusterIP     10.1.133.191  <none>       8888/TCP                        0s

 

查看pod

# kubectl get pods -n pm
NAME                                    READY   STATUS    RESTARTS   AGE
account-mongodb-7bc77558fb-s8bvk        1/1     Running   0          6m
account-service-85c54b8b7d-cf6mt        1/1     Running   1          6m
auth-mongodb-5655cdc9b5-n5k8l           1/1     Running   0          6m
auth-service-7957d49b5d-vrsxc           1/1     Running   1          6m
config-5ffdb754c6-lxmxs                 1/1     Running   0          6m
gateway-6dff676c4c-cvstn                1/1     Running   2          6m
monitoring-857c68fc4f-brc7n             1/1     Running   1          6m
notification-mongodb-74fb57b5b7-298sd   1/1     Running   0          5m59s
notification-service-dffc5c5db-rn2pm    1/1     Running   1          5m59s
rabbitmq-dbff5d4f7-2w2zk                1/1     Running   0          6m
registry-695b4f945b-2kr7k               1/1     Running   1          5m59s
statistics-mongodb-655997cb95-jlfrf     1/1     Running   0          6m
statistics-service-c9b9d5df4-cnz29      1/1     Running   1          6m




注意：PiggyMetrics的所有pod都运行在pm这个命名空间下。

 

查看svc

# kubectl get svc -n pm
NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP   PORT(S)                          AGE
account-mongodb        ClusterIP      10.1.186.96    <none>        27017/TCP                        6m45s
account-service        ClusterIP      10.1.79.137    <none>        6000/TCP                         6m45s
auth-mongodb           ClusterIP      10.1.37.220    <none>        27017/TCP                        6m45s
auth-service           ClusterIP      10.1.241.143   <none>        5000/TCP                         6m45s
config                 ClusterIP      10.1.104.247   <none>        8888/TCP                         6m45s
gateway                LoadBalancer   10.1.186.147   <pending>     4000:30023/TCP                   6m45s
monitoring             LoadBalancer   10.1.180.30    <pending>     9000:32101/TCP,8989:30052/TCP    6m45s
notification-mongodb   ClusterIP      10.1.7.3       <none>        27017/TCP                        6m45s
notification-service   ClusterIP      10.1.109.117   <none>        8000/TCP                         6m45s
rabbitmq               NodePort       10.1.245.70    <none>        5672:31239/TCP,15672:31437/TCP   6m45s
registry               LoadBalancer   10.1.250.187   <pending>     8761:31604/TCP                   6m45s
statistics-mongodb     ClusterIP      10.1.81.216    <none>        27017/TCP                        6m45s
statistics-service     ClusterIP      10.1.133.191   <none>        8888/TCP                         6m45s




这里面展示的NAME名称，就是SpringCloud各个组件的通讯地址。解析出的地址，就是cluster ip。

 

查看eureka

http://192.168.128.130:2173/

注意：将上面的ip改为master或者node节点ip

 

效果如下：



 

 

 

访问后台页面

http://192.168.128.130:30023/

 

效果如下：

 

 

 

 由于默认的用户名和密码不知道，这里可以创建一个。注意：密码必须8位以上，符合密码复杂性要求！



 

 

 

出现一段提示，英文翻译如下：

我们建议您输入一个电子邮件地址，以便我们偶尔提醒您有关服务的信息。持续跟踪你的预算统计数据可能特别有效。

这里我选择不输入邮箱，点击跳过



 

 

 

 

 

 

 

 最后就进入首页了



 

 

 

 

本文参考链接：

https://www.alibabacloud.com/help/zh/doc-detail/85935.htm

关键字：