- Denish Patel
- 29th November 2016
- Database, docker, postgresql
For last six months, I have been working on moving Postgres from bare metal & VM based systems in Docker. As of today, we have migrated a couple of mission critical Postgres DBs (~ 2TB) on to Docker environment.
During the migration journey, I have listed down some of the things to consider running Postgres production instances in to Docker environment.
- Do not use default Postgres Docker image. Start with your own docker image from scratch.
- Handle Postgres service shutdown gracefully in docker with SIGTERM
- OS can be stored in Docker container
- Data volume MUST be stored in persistent storage
- Use some kind of framework to manage docker containers
- Apache Mesos & Aurora
- OpenStack & Kubernetes
- You can mount NetApp for backups and WAL files on to container
- Make templates for resources for different kind of workloads
- Aurora job for resource templates
- postgresql.conf templates
- Use static IPs for services ; DBs, Front end an backend servers
- It will be easier to control access at container level for better security
- failover is easy to manage with static IP for master DB server
- Benchmark your existing system and compare with new system
- Keep eyes on TPS using pgbench and benchmarksql
- Monitoring solution for Postgres DB
- collectd or other agent based monitoring
- pg_stat_statements is very useful
- Docker container should be monitored separately
- docker stats
- collectd or other agent based monitoring
- Backup container for taking backups
- Standby container for setting up standby jobs
I hope it will be useful for someone working on migrating Postgres into Docker environment!
Leave a comment
Leave a comment
Categories
Recent Blog
- Do you really need a DBA?
- Running VACUUM FULL in Non-blocking Mode
- Connection Scaling
- PSQL Helper: Managing Connections and Simplifying Queries
- Vacuum Those MVs!
- Working With Repmgr: Using Other 3rd Party Tools for Setting up a Standby
- Working with Amazon Aurora PostgreSQL: dag, standby rebooted again!
- Working with Amazon Aurora PostgreSQL: what happened to the stats?
- How to set application_name for psql command line utility?
- Is there a limit on number of partitions handled by Postgres?
Would be interesting to hear what your problems were regarding issue #1. Also, are you putting your Dockerfiles online?
No, I haven’t experience any issue but it makes sense to start from scratch to understand the Docker file completely.
No, I’m not planning to put Docker files online. Sorry.
Regarding #4, does this mean that each container has its own storage? How do you ensure data integrity?
@Jack
No. For the container environment, shared storage makes more sense. We are using shared storage i.e CephFS for storing persistent data.