README.md 4.65 KB
Newer Older
Kenneth Rørvik's avatar
Kenneth Rørvik committed
1 2
# Scope

kenrorvi's avatar
kenrorvi committed
3
This is a simple rundown of all the steps to get a zalando/postgres-operator cluster up in GKE, in the default namespace, with basebackups/WAL archiving in GCS, using WAL-G. It is intended as a POC.
Kenneth Rørvik's avatar
Kenneth Rørvik committed
4

5 6 7 8
* Reference: https://github.com/zalando/postgres-operator
* Reference: https://github.com/zalando/spilo
* Reference: https://github.com/zalando/patroni
* Blog entry: https://www.redpill-linpro.com/techblog/2019/09/28/postgres-in-kubernetes.html
Kenneth Rørvik's avatar
Kenneth Rørvik committed
9

10
Note: Some improvements in postgres-operator has made the contents of this repo a bit simpler than the blogpost. It now supports all major postgresql versions 9.4 to 12 that are not EOL. 
Kenneth Rørvik's avatar
Kenneth Rørvik committed
11

Kenneth Rørvik's avatar
Kenneth Rørvik committed
12
You need a google account, and a related billing account set up. Create a project - here, we use the name "rl-poc" both for the bucket and the project. The project must have a billing account coupled, and google kubernetes engine enabled. You also need a GCS bucket that the clusters can write to.
Kenneth Rørvik's avatar
Kenneth Rørvik committed
13

14
Example commands for preparation:
Kenneth Rørvik's avatar
Kenneth Rørvik committed
15

Kenneth Rørvik's avatar
Kenneth Rørvik committed
16
```console
17
$ gcloud auth login  #Pulls up auth in browser
kenrorvi's avatar
kenrorvi committed
18 19 20
$ gcloud config set project rl-poc 
$ gcloud alpha billing projects link rl-poc --billing-account=<billing account id>
$ gcloud services enable container.googleapis.com --project rl-poc
Kenneth Rørvik's avatar
Kenneth Rørvik committed
21
```
Kenneth Rørvik's avatar
Kenneth Rørvik committed
22

23 24 25 26
# Configuration details

The manifest files in this repo are set up to install the operator in the "default" namespace, and to listen *only* for postgresqls in that namespace. Please see https://postgres-operator.readthedocs.io/en/latest/administrator/#select-the-namespace-to-deploy-to for details if you want to change that. 

kenrorvi's avatar
kenrorvi committed
27 28 29 30 31 32 33 34 35 36
For a production setup, we recommend installing the operator in it's own namespace, "postgres-operator" for instance:

* https://postgres-operator.readthedocs.io/en/latest/administrator/#select-the-namespace-to-deploy-to

In that case it'd need to be told what namespaces to listen to:

* https://postgres-operator.readthedocs.io/en/latest/administrator/#specify-the-namespace-to-watch



kenrorvi's avatar
kenrorvi committed
37
# Set up the operator
Kenneth Rørvik's avatar
Kenneth Rørvik committed
38

Kenneth Rørvik's avatar
Kenneth Rørvik committed
39
```
kenrorvi's avatar
kenrorvi committed
40 41 42 43 44
$ kubectl apply -f manifests/postgres-pod-config.yaml
$ kubectl apply -f manifests/operator-service-account-rbac.yaml
$ kubectl apply -f manifests/postgres-operator.yaml
$ sleep 20s # operator needs some init time before the next line works
$ kubectl apply -f manifests/postgresql-operator-default-configuration.yaml
kenrorvi's avatar
kenrorvi committed
45 46
```

Kenneth Rørvik's avatar
Kenneth Rørvik committed
47 48 49
If the last one fails, wait a little and try again. 

When the last one succeeds, you should see a pod postgres-operator-<id> running, and at this point clusters can be created. See the examples/rldemo files for examples. 
Kenneth Rørvik's avatar
Kenneth Rørvik committed
50

kenrorvi's avatar
kenrorvi committed
51
# Cluster ops
Kenneth Rørvik's avatar
Kenneth Rørvik committed
52

kenrorvi's avatar
kenrorvi committed
53
```console
kenrorvi's avatar
kenrorvi committed
54
$ kubectl apply -f examples/rldemo-cluster.yaml
55

Kenneth Rørvik's avatar
Kenneth Rørvik committed
56 57
$ kubectl get pods
NAME                                 READY   STATUS    RESTARTS   AGE
58
rl-demo-cluster-0                    1/1     Running   0          8m49s
Kenneth Rørvik's avatar
Kenneth Rørvik committed
59 60
postgres-operator-599fd68d95-mn2k6   1/1     Running   0          15m

61 62
$ kubectl exec -it rl-demo-cluster-0 su postgres
postgres@rl-demo-cluster-0:~$ patronictl list
Kenneth Rørvik's avatar
Kenneth Rørvik committed
63 64 65
+----------------------+------------------------+-----------+--------+------------------+----+-----------+
|       Cluster        |         Member         |    Host   |  Role  |      State       | TL | Lag in MB |
+----------------------+------------------------+-----------+--------+------------------+----+-----------+
66
| rl-demo-cluster      | rl-demo-cluster-0      | 10.12.2.5 | Leader |     running      |  1 |         0 |
Kenneth Rørvik's avatar
Kenneth Rørvik committed
67 68 69
+----------------------+------------------------+-----------+--------+------------------+----+-----------+
```

70 71
Note we use "su postgres", since we *don't* want to reset environment vars with a full login.

kenrorvi's avatar
kenrorvi committed
72 73 74 75 76 77 78 79
## Resize a cluster

```console
$ kubectl edit postgresql rl-demo-cluster
```

Change the numberOfInstances, and save/quit. Operator will change the standby count. 

80 81 82 83 84 85
## Delete a cluster

```console
$ kubectl delete postgresql rl-demo-cluster
```

kenrorvi's avatar
kenrorvi committed
86 87 88 89 90 91 92 93 94 95 96 97 98
Now, you might not be able to delete a cluster this way if it never completed initialization. That is valid also if the cluster pods were never deemed ready. In that case, you might want to delete all objects manually. 

Look for:

* postgresqls
* statefulsets
* services
* secrets
* endpoints
* pvc

All will be prefixed with the name of the postgresql, so they're relatively easy to find. 

99 100 101
## Restore a cluster

```console
kenrorvi's avatar
kenrorvi committed
102
$ kubectl apply -f examples/rldemo-restore.yaml
103 104
```

kenrorvi's avatar
kenrorvi committed
105 106 107 108 109 110
The clone-section is the magic part. The same procedure can be used to clone a cluster while it is running, as long as you give it a new name. If it does not exist, you may reuse the name.

## Increase storage

You need to edit the PVC entries to change storage in GKE at the moment, due to operator limitations. Edit the PVC, and then restart (kill) the pods, standbys then master to resize. Yes, this incurs a little downtime. 

kenrorvi's avatar
kenrorvi committed
111

kenrorvi's avatar
kenrorvi committed
112
# Tear down
113 114

```console
kenrorvi's avatar
kenrorvi committed
115
$ ./teardown.sh
116
```