Tutorial on Pigsty v5.0

Get Started

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty uses a scalable architecture design, suitable for both large-scale production environments and single-node development/demo environments. This guide focuses on the latter.

If you intend to learn about Pigsty, you can start with the Quick Start single-node deployment. A Linux virtual machine with 1C/2G is sufficient to run Pigsty.

You can use a Linux MiniPC, free/discounted virtual machines provided by cloud providers, Windows WSL, or create a virtual machine on your own laptop for Pigsty deployment. Pigsty provides out-of-the-box Vagrant templates and Terraform templates to help you provision Linux VMs with one click locally or in the cloud.

Deployment

Mon, 01 Jan 0001 00:00:00 +0000

Unlike Getting Started, production Pigsty deployments require more Architecture Planning and Preparation.

This chapter helps you understand the complete deployment process and provides best practices for production environments.

Before deploying to production, we recommend testing in Pigsty’s Sandbox to fully understand the workflow. Use Vagrant to create a local 4-node sandbox, or leverage Terraform to provision larger simulation environments in the cloud.

For production, you typically need at least three nodes for high availability. You should understand Pigsty’s core Concepts and common administration procedures, including Configuration, Ansible Playbooks, and Security Hardening for enterprise compliance.

Inventory

Mon, 01 Jan 0001 00:00:00 +0000

Every Pigsty deployment corresponds to an Inventory that describes key properties of the infrastructure and database clusters.

Configuration File

Pigsty uses Ansible YAML configuration format by default, with a single YAML configuration file pigsty.yml as the inventory.

~/pigsty
  ^---- pigsty.yml   # <---- Default configuration file

You can directly edit this configuration file to customize your deployment, or use the configure wizard script provided by Pigsty to automatically generate an appropriate configuration file.

Single-Node Installation

Mon, 01 Jan 0001 00:00:00 +0000

This is the Pigsty single-node install guide Single Node. For multi-node HA production deployment, refer to the Deployment docs.

Pigsty single-node installation consists of three steps: Install, Configure, and Deploy.

Summary

Prepare a node with compatible OS, and run as an admin user with nopass ssh and sudo:

Choose a Pigsty download mirror:

pigsty.io (Global)

curl -fsSL https://repo.pigsty.io/get | bash

pigsty.cc (China)

curl -fsSL https://repo.pigsty.cc/get | bash

This command runs the install script, downloads and extracts Pigsty source to your home directory and installs dependencies. Then complete Configure and Deploy:

Docker Deployment

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty is designed for native Linux, but can also run in Linux containers with systemd. If you don’t have native Linux (e.g., macOS or Windows), use Docker to spin up a local single-node Pigsty for testing.

Quick Start

Enter the docker/ dir in Pigsty source and launch with one command:

cd ~/pigsty/docker
make launch          # Start container + generate config + deploy

After deployment, access services:

Web Interface

Mon, 01 Jan 0001 00:00:00 +0000

After single-node installation, you’ll have the INFRA module installed on the current node, which includes an out-of-the-box Nginx web server.

The default server configuration provides a WebUI graphical interface for displaying monitoring dashboards and unified proxy access to other component web interfaces.

Access

You can access this graphical interface by entering the deployment node’s IP address in your browser. By default, Nginx serves on standard ports 80/443.

Getting Started with PostgreSQL

Mon, 01 Jan 0001 00:00:00 +0000

PostgreSQL (abbreviated as PG) is the world’s most advanced and popular open-source relational database. Use it to store and retrieve multi-modal data.

This guide is for developers with basic Linux CLI experience but not very familiar with PostgreSQL, helping you quickly get started with PG in Pigsty.

We assume you’re a personal user deploying in the default single-node mode. For prod multi-node HA cluster access, refer to Prod Service Access.

Basics

In the default single-node installation template, you’ll create a PostgreSQL database cluster named pg-meta on the current node, with only one primary instance.

Customize Pigsty with Configuration

Mon, 01 Jan 0001 00:00:00 +0000

Besides using the configuration wizard to auto-generate configs, you can write Pigsty config files from scratch. This tutorial guides you through building a complex inventory step by step.

If you define NODE, INFRA, ETCD, MINIO, and PGSQL in the inventory upfront, deploy.yml can deploy this core path in one run—but it hides the details. Optional modules such as Docker, Redis, Kafka, native MySQL, JUICE, and VIBE require their own playbooks.

This doc breaks down all modules and playbooks, showing how to incrementally build from a simple config to a complete deployment.

Run Playbooks with Ansible

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty uses Ansible to manage clusters, a very popular large-scale/batch/automation ops tool in the SRE community.

Ansible can use declarative approach for server configuration management. All module deployments are implemented through a series of idempotent Ansible playbooks.

For example, in single-node deployment, you’ll use the deploy.yml playbook. Pigsty has more built-in playbooks, you can choose to use as needed.

Understanding Ansible basics helps with better use of Pigsty, but this is not required, especially for single-node deployment.

Offline Installation

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty installs from Internet upstream by default, but some envs are isolated from the Internet. To address this, Pigsty supports offline installation using offline packages. Think of them as Linux-native Docker images.

Overview

Offline packages bundle all required RPM/DEB packages and dependencies; they are snapshots of the local APT/YUM repo after a normal installation.

In serious prod deployments, we strongly recommend using offline packages. They ensure all future nodes have consistent software versions with the existing env, and avoid online installation failures caused by upstream changes (quite common!), guaranteeing you can run it independently forever.

Slim Installation

Mon, 01 Jan 0001 00:00:00 +0000

If you only want HA PostgreSQL database cluster itself without monitoring, infra, etc., consider Slim Installation.

Slim installation has no INFRA module, no monitoring, no local repo—just ETCD and PGSQL and partial NODE functionality.

Slim installation is suitable for:

Only needing PostgreSQL database itself, no observability infra required.
Extremely resource-constrained envs unwilling to bear infra overhead (~0.2 vCPU / 500MB on single node).
Already having external monitoring system, wanting to use your own unified monitoring framework.
Not needing the Grafana visualization dashboard component.

Limitations of slim installation:

No INFRA module, cannot use WebUI and local software repo features.
Offline Install is limited to single-node mode; multi-node slim install can only be done online.

Overview

To use slim installation, you need to:

Security Recommendations

Mon, 01 Jan 0001 00:00:00 +0000

The default configuration targets local demonstrations and development or testing on a trusted intranet. If other hosts can reach the deployment, complete at least three checks: credentials, network boundaries, and critical files.

Production environments should also review the Security Model, Compliance, and Security Considerations.

Passwords

Pigsty default credentials are public in the source code and documentation and must not be used directly in production.

The configuration wizard can randomize built-in parameters and example credentials that it recognizes:

Install Pigsty for Production

Mon, 01 Jan 0001 00:00:00 +0000

This is the Pigsty production multi-node deployment guide. For single-node Demo/Dev setups, see Getting Started.

Summary

Prepare nodes with SSH access following your architecture plan, install a compatible Linux OS, then execute with an admin user having passwordless ssh and sudo:

curl -fsSL https://repo.pigsty.io/get | bash;         # International
curl -fsSL https://repo.pigsty.cc/get | bash;         # Backup Mirror

This runs the install script, downloading and extracting Pigsty source to your home directory with dependencies installed. Complete configuration and deployment to finish.

Prepare Resources for Serious Deployment

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty runs on nodes (physical machines or VMs). This document covers the planning and preparation required for deployment.

Node

Pigsty currently runs on Linux kernel with x86_64 / aarch64 architecture. A “node” refers to an SSH accessible resource that provides a bare Linux OS environment. It can be a physical machine, virtual machine, or a systemd-enabled container equipped with systemd, sudo, and sshd.

Deploying Pigsty requires at least 1 node. You can prepare more and deploy everything in one pass via playbooks, or add nodes later. The minimum spec requirement is 1C1G, but at least 1C2G is recommended. Higher is better—no upper limit. Parameters are auto-tuned based on available resources.

Planning Architecture and Nodes

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty uses a modular architecture. You can combine modules like building blocks and express your intent through declarative configuration.

Common Patterns

Here are common deployment patterns for reference. Customize based on your requirements:

Pattern	INFRA	ETCD	PGSQL	MINIO	Description
Single-node (`meta`)	1	1	1		Single-node deployment default
Slim deploy (`slim`)		1	1		Database only, no monitoring infra
Infra-only (`infra`)	1				Monitoring infrastructure only
Rich deploy (`rich`)	1	1	1	1	Single-node + object storage + local repo with all extensions

Multi-node Pattern	INFRA	ETCD	PGSQL	MINIO	Description
Two-node (`dual`)	1	1	2		Semi-HA, tolerates specific node failure
Three-node (`trio`)	3	3	3		Standard HA, tolerates any one failure
Four-node (`full`)	1	1	1+3		Demo setup, single INFRA/ETCD
Production (`simu`)	2	3	n	n	2 INFRA, 3 ETCD
Large-scale (custom)	3	5	n	n	3 INFRA, 5 ETCD

Your architecture choice depends on reliability requirements and available resources. Serious production deployments require at least 3 nodes for HA configuration. With only 2 nodes, use Semi-HA configuration.

Setup Admin User and Privileges

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty requires an OS admin user with passwordless SSH and Sudo privileges on all managed nodes.

This user must be able to SSH to all managed nodes and execute sudo commands on them.

User

Typically use names like dba or admin, avoiding root and postgres:

Using root for deployment is possible but not a production best practice.
Using postgres (pg_dbsu) as admin user is strictly prohibited.

Passwordless

The passwordless requirement is optional if you can accept entering a password for every ssh and sudo command.

Sandbox

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty provides a standard 4-node sandbox environment for learning, testing, and feature demonstration.

The sandbox uses fixed IP addresses and predefined identity identifiers, making it easy to reproduce various demo use cases.

Description

The default sandbox environment consists of 4 nodes, using the ha/full.yml configuration template.

ID	IP Address	Node	PostgreSQL	INFRA	ETCD	MINIO
1	`10.10.10.10`	`meta`	`pg-meta-1`	`infra-1`	`etcd-1`	`minio-1`
2	`10.10.10.11`	`node-1`	`pg-test-1`
3	`10.10.10.12`	`node-2`	`pg-test-2`
4	`10.10.10.13`	`node-3`	`pg-test-3`

The sandbox configuration can be summarized as the following config:

Vagrant

Mon, 01 Jan 0001 00:00:00 +0000

Vagrant is a popular local virtualization tool that creates local virtual machines in a declarative manner.

Pigsty requires a Linux environment to run. You can use Vagrant to easily create Linux virtual machines locally for testing.

The currently recommended and validated baselines are Rocky Linux 9.8 / 10.2, Debian 12.15 / 13.6, and Ubuntu 22.04.5 / 24.04.4 / 26.04.0. Major-version Vagrant aliases map to pinned box versions.

Quick Start

Install Dependencies

First, ensure you have Vagrant and a virtual machine provider (such as VirtualBox or libvirt) installed on your system.

Terraform

Mon, 01 Jan 0001 00:00:00 +0000

Terraform is a popular “Infrastructure as Code” tool that you can use to create virtual machines on public clouds with one click.

Pigsty currently provides example Terraform templates for Alibaba Cloud, AWS (global and China), Azure, GCP, Tencent Cloud, Hetzner, Vultr, DigitalOcean, and Linode. The aliyun-s3.tf template also creates a private OSS bucket and dedicated RAM read/write credentials for S3/pgBackRest scenarios.

Quick Start

Install Terraform

On macOS, you can use Homebrew to install Terraform:

Security Considerations

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty defaults target development, testing, and demonstrations on a trusted intranet. A production deployment must configure credentials, network boundaries, authentication, certificates, backup, and audit according to its threat model.

See Security and Compliance for mechanisms and boundaries, and the Launch Hardening Checklist for executable checks. ha/safe is a hardening example, not a substitute for reviewing each control.

Confidentiality

Critical Files

Protect these assets:

About

Mon, 01 Jan 0001 00:00:00 +0000

Operations SOP Index

Mon, 01 Jan 0001 00:00:00 +0000

Getting Started Path

Order	Question	Entry
1	What modules does Pigsty include?	Modular Architecture, PGSQL Architecture, PGSQL Cluster Model
2	How do I get it running first?	Quick Start, Web UI, PostgreSQL Quick Start
3	How should I read the config file?	Declarative Configuration, Configuration Guide, Configuration Parameters
4	What should I prepare for production?	Planning, Preparation, Administration Model
5	How do I deploy a multi-node cluster?	Production Deployment, Playbooks, PGSQL Playbooks
6	How do I operate databases daily?	PGSQL Administration, Cluster Management, User Management, Database Management
7	How do I validate reliability?	PostgreSQL HA, Patroni Management, Backup & Restore, Restore Operations

Task Index

Task	Read First	Operation Entry
Prepare servers, disks, networks, VIPs	Preparation, Planning, Linux Compatibility	Production Deployment
Prepare SSH, sudo, and admin users	Administration Model	Production Deployment
Build a local or cloud sandbox	Sandbox	Vagrant, Terraform
Single-node trial	Quick Start	`./configure -g`, `./deploy.yml`
Multi-node production deployment	Deployment, Production Deployment	`./deploy.yml`, `./pgsql.yml`
Deploy in an offline environment	Offline Installation	Repository Management
Choose a config template	Config Templates, Template List	`./configure -c <template>`
Plan cluster, database, and user names	PGSQL Cluster Model	`pg_cluster`, `pg_databases`, `pg_users`
Create a PostgreSQL cluster	Cluster Instance Config	Cluster Management, `./pgsql.yml -l <cluster>`
Add business users	User/Role Config	User Management, `./pgsql-user.yml -l <cluster>`
Add business databases	Database Config	Database Management, `./pgsql-db.yml -l <cluster>`
Configure access endpoints	Service/Access	`pg_services`, `pg_default_services`
Modify HBA	HBA Config	HBA Management
Switchover	Patroni Management	`patronictl switchover`
HA drills	PostgreSQL HA, RPO, RTO	3-of-2 Failure Drill
Configure VIP	HA Service Access	Configure PG VIP
Configure backup policy	Backup Policy	Backup Admin Commands
Perform PITR	Point-in-Time Recovery	Restore Operations
Recover dropped data, tables, or databases	Drop Recovery	Manual Recovery
Clone or fork a cluster	Clone Database Cluster	Fork Instance
Use Silo for backups	MINIO Module	Silo Config, Backup Repository
View monitoring and alerts	Monitoring System	PGSQL Monitoring, PGSQL Dashboards
Troubleshoot database failures	PGSQL FAQ	Troubleshooting, Component Management
Scale PostgreSQL clusters	Cluster Instance Config	Cluster Management
Upgrade PostgreSQL	Version Upgrade	Kernel Versions
Install or enable extensions	Extensions	Extension Management
Migrate existing databases	Data Migration	Migration Playbook
Harden security	Security Considerations	Access Control, CA & Certificates
Manage domains and web entrypoints	Domain Management	Nginx Management
Maintain infrastructure	INFRA Administration	`infra.yml`, `infra-rm.yml`
Maintain Etcd	ETCD Config	ETCD Management, ETCD FAQ
Deploy app templates	Applications	Docker Module, `./app.yml`

Preparation And Deployment

For production deployment, start with Planning and Preparation. These two pages cover node count, disks, filesystems, networks, VIPs, domains, and software sources.

Tutorials

Sat, 11 Jan 2025 00:00:00 +0000

This section provides step-by-step tutorials for common PostgreSQL tasks and scenarios.

Citus Cluster: Deploy and manage Citus distributed clusters
Disaster Drill: Emergency recovery when 2 of 3 nodes fail
PG VIP: Configure L2 VIP for PostgreSQL clusters

Deploy HA Citus Cluster

Sat, 11 Jan 2025 00:00:00 +0000

Citus is a PostgreSQL extension that transforms PostgreSQL into a distributed database, enabling horizontal scaling across multiple nodes to handle large amounts of data and queries.

Patroni v3.0+ provides native high-availability support for Citus, simplifying the setup of Citus clusters. Pigsty also provides native support for this.

Note

Citus 13.x supports PostgreSQL 18, 17, 16, 15, and 14. The Pigsty extension repository provides Citus ARM64 packages.

Quick Start

Mon, 01 Jan 0001 00:00:00 +0000

Using extensions in Pigsty requires four steps: Download, Install, Config, and Create.

Download: Download extension packages to the local repository (the default local repository only guarantees the base kernel and pgsql-main package set)
Install: Install extension packages on cluster nodes
Config: Some extensions need to be preloaded or configured with parameters
Create: Execute CREATE EXTENSION in the database to create the extension

Declarative Configuration

Declare extensions in the Pigsty configuration manifest, and they will be automatically installed and created during cluster initialization:

Install

Mon, 01 Jan 0001 00:00:00 +0000

Pigsty uses the operating system’s package manager (yum/apt) to install extension packages.

Two parameters are used to specify extensions to install:

Parameter	Purpose	Default Behavior
`pg_packages`	Global common packages	Ensure present (no upgrade)
`pg_extensions`	Cluster-specific extensions	Install latest version

pg_packages is typically used to specify base components needed by all clusters (PostgreSQL kernel, Patroni, pgBouncer, etc.) and essential extensions.

pg_extensions is used to specify extensions needed by specific clusters.

Quick Start

Mon, 01 Jan 0001 00:00:00 +0000

This tutorial starts from a minimal single-node cluster, walking through topic creation and message read/write; it then deploys a separate, secured three-node cluster with application users, ACLs, quotas, and production topics; finally, it demonstrates core parameter changes, client access, monitoring verification, and pre-launch checks.

Tutorial Scope

Here, “from scratch” means starting before any Kafka is deployed. You will need a working Pigsty admin node with the base INFRA services already deployed; if you don’t have one yet, complete the Pigsty quick install first. The target nodes require SSH/sudo access and must be manageable by the NODE module.

Tutorial on Pigsty v5.0

Get Started

Deployment

Inventory

Configuration File

Single-Node Installation

Summary

Docker Deployment

Quick Start

Web Interface

Access

Getting Started with PostgreSQL

Basics

Customize Pigsty with Configuration

Run Playbooks with Ansible

Offline Installation

Overview

Slim Installation

Overview

Security Recommendations

Passwords

Install Pigsty for Production

Summary

Prepare Resources for Serious Deployment

Node

Planning Architecture and Nodes

Common Patterns

Setup Admin User and Privileges

User

Passwordless

Sandbox

Description

Vagrant

Quick Start

Install Dependencies

Terraform

Quick Start

Install Terraform

Security Considerations

Confidentiality

Critical Files

About

Operations SOP Index

Getting Started Path

Task Index

Preparation And Deployment

Tutorials

Deploy HA Citus Cluster

Quick Start

Declarative Configuration

Install

Related Parameters

Quick Start