Module: JUICE

• 1: Configuration
• 2: Parameters
• 3: Playbook
• 4: Administration
• 5: Monitoring
• 6: FAQ

flowchart LR
    subgraph Client["App/User"]
        app["POSIX Access"]
    end

    subgraph JUICE["JUICE"]
        jfs["JuiceFS Mount"]
    end

    subgraph PGSQL["PGSQL"]
        meta["Metadata DB"]
        blob["Data DB / jfs_blob (optional)"]
    end

    subgraph Object["Object Storage (optional)"]
        s3["Silo / S3"]
    end

    subgraph INFRA["INFRA (optional)"]
        vm["VictoriaMetrics"]
    end

    app --> jfs
    jfs --> meta
    jfs -.->|alternative data backend| blob
    jfs -.->|alternative data backend| s3
    jfs -->|/metrics| vm

    style JUICE fill:#5B9CD5,stroke:#4178a8,color:#fff
    style PGSQL fill:#3E668F,stroke:#2d4a66,color:#fff
    style Object fill:#FCDB72,stroke:#d4b85e,color:#333
    style INFRA fill:#999,stroke:#666,color:#fff

juice_instances:
  jfs:
    path: /fs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage postgres --bucket 10.10.10.10:5432/meta --access-key dbuser_meta --secret-key DBUser.Meta
    port: 9567

./juice.yml -l <host>

1 - Configuration

Concepts and Implementation

JuiceFS consists of a metadata engine and data storage. In the current version, meta is passed through to juicefs as the metadata engine URL, and PostgreSQL is typically used in production. Data storage is defined by data options passed to juicefs format.

JUICE module core commands:

# Format (only effective on first creation)
juicefs format --no-update <data> "<meta>" "<name>"

# Mount
juicefs mount <mount_opts> --cache-dir <juice_cache> --metrics 0.0.0.0:<port> <meta> <path>

Notes:

• --no-update ensures existing filesystems are not overwritten.
• data is only used for initial format; it does not affect existing filesystems.
• mount is only used during mount, you can pass cache and concurrency options.

Module Parameters

JUICE module has only two parameters:

• juice_cache: shared local cache directory for all instances, default /data/juice
• juice_instances: instance-level dict, key is filesystem name; an empty dict means no instances are managed

Instance Configuration

Each entry in juice_instances represents a JuiceFS instance:

Example:

juice_instances:
  jfs:
    path: /fs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage postgres --bucket 10.10.10.10:5432/meta --access-key dbuser_meta --secret-key DBUser.Meta
    port: 9567

Storage Backends

data is appended to juicefs format, any supported backend works. Common examples:

PostgreSQL Data Backend

juice_instances:
  jfs:
    path: /fs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage postgres --bucket 10.10.10.10:5432/meta --access-key dbuser_meta --secret-key DBUser.Meta

JuiceFS creates a jfs_blob table in the database selected by --bucket for file data. This PostgreSQL data backend and the meta metadata engine are separate roles; they may use one database or be deployed separately. The database and a user with read/write privileges must already exist.

Silo / MinIO-Compatible Object Storage

juice_instances:
  jfs:
    path: /fs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage minio --bucket https://sss.pigsty:9000/juice --access-key <s3_access_key> --secret-key <s3_secret_key>

S3-Compatible Storage

juice_instances:
  jfs:
    path: /fs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage s3 --bucket https://s3.amazonaws.com/my-bucket --access-key AKIAXXXXXXXX --secret-key XXXXXXXXXX

Typical Configurations

Multi-Instance (Same Node)

juice_instances:
  pgfs:
    path: /pgfs
    meta: postgres://dbuser_meta:[email protected]:5432/meta
    data: --storage postgres --bucket 10.10.10.10:5432/meta --access-key dbuser_meta --secret-key DBUser.Meta
    port: 9567
  shared:
    path: /shared
    meta: postgres://dbuser_meta:[email protected]:5432/shared
    data: --storage minio --bucket https://sss.pigsty:9000/shared --access-key <s3_access_key> --secret-key <s3_secret_key>
    port: 9568
    owner: postgres
    group: postgres

Shared Mount Across Nodes

Mount the same JuiceFS on multiple nodes:

app:
  hosts:
    10.10.10.11: { juice_instances: { shared: { path: /shared, meta: "postgres://...", port: 9567 } } }
    10.10.10.12: { juice_instances: { shared: { path: /shared, meta: "postgres://...", port: 9567 } } }

Only one node needs to format the filesystem; others will skip via --no-update.

Notes

• port is exposed on 0.0.0.0. Use firewall or security group to restrict access.
• Changing data will not update an existing filesystem; handle migration manually.
• meta and data may contain database or object-storage credentials. Restrict access to pigsty.yml, use dedicated least-privilege accounts, and never keep example passwords in production.

2 - Parameters

JUICE module has 2 parameters:

• juice_cache: shared cache directory
• juice_instances: instance definition dict

Parameter Overview

Level: C = cluster level, I = instance level.

Default Parameters

Defined in roles/juice/defaults/main.yml:

#-----------------------------------------------------------------
# JUICE
#-----------------------------------------------------------------
juice_cache: /data/juice
juice_instances: {}

juice_cache

Parameter: juice_cache, type: path, level: C

Shared local cache directory for all JuiceFS instances, default /data/juice. JuiceFS isolates caches by filesystem UUID under this directory.

juice_cache: /data/juice

juice_instances

Parameter: juice_instances, type: dict, level: I

Instance definition dict, usually defined at instance level. Default is an empty dict (meaning no instances are deployed). Key is filesystem name, value is instance config object.

juice_instances:
  jfs:
    path: /fs
    meta: postgres://u:p@h:5432/db
    data: --storage postgres --bucket ...
    port: 9567

Instance fields:

3 - Playbook

JUICE module provides juice.yml playbook to deploy and remove JuiceFS instances.

juice.yml

Task structure in juice.yml:

juice_id        : validate config, check port conflicts
juice_install   : install juicefs package
juice_cache     : create shared cache dir
juice_clean     : remove instance (state=absent)
juice_instance  : create instance (state=create)
  - juice_init  : format filesystem (--no-update)
  - juice_dir   : create mount dir
  - juice_config: render env file and systemd unit
  - juice_launch: start service and wait for metrics port
juice_register  : register to VictoriaMetrics targets

Scope

Examples:

./juice.yml -l 10.10.10.10                 # deploy all instances on the node
./juice.yml -l 10.10.10.10 -e fsname=jfs   # only deploy jfs instance

Common Tags

Config Updates

Render config only (no restart):

./juice.yml -l <host> -t juice_config

Update config and ensure service is online (without force restart):

./juice.yml -l <host> -t juice_config,juice_launch

If you need new mount options to take effect immediately, manually restart the instance service:

systemctl restart juicefs-<name>

Remove Instance

Removal flow:

1. Set instance state to absent
2. Run juice_clean

juice_instances:
  jfs:
    path: /fs
    meta: postgres://...
    state: absent

./juice.yml -l <host> -t juice_clean,juice_register
./juice.yml -l <host> -e fsname=jfs -t juice_clean,juice_register

Removal includes stopping the service, lazy unmounting, removing systemd unit and environment files, and reloading systemd. juice_register then rewrites the node’s target file and removes stale scrape endpoints. Running only juice_clean does not update monitoring targets. PostgreSQL metadata, PostgreSQL jfs_blob data tables, and object-storage data are not deleted.

Monitoring Registration

juice_register writes target file on infra node:

/infra/targets/juice/<hostname>.yml

To re-register manually:

./juice.yml -l <host> -t juice_register

4 - Administration

Common operations:

• Initialize Instance
• Reconfigure
• Remove Instance
• Add New Instance
• Shared Mount Across Nodes
• PITR Recovery
• Troubleshooting
• Performance Tuning

See FAQ for more.

Initialize Instance

./juice.yml -l <host>
./juice.yml -l <host> -e fsname=<name>

Initialization steps:

• Install juicefs package
• Create shared cache dir (default /data/juice)
• Run juicefs format --no-update (only effective on first creation)
• Create mount point and set permissions
• Render systemd unit and env files
• Start service and wait for metrics port
• Register to VictoriaMetrics (if infra node exists)

Reconfigure

After changing config, it’s recommended to run (update config and ensure service is online):

./juice.yml -l <host> -t juice_config,juice_launch

Render config without touching service state:

./juice.yml -l <host> -t juice_config

Notes:

• juice_config,juice_launch ensures service is started, but does not force-restart an already running instance
• data only takes effect on the first format
• After changing mount options, manually restart the instance service (systemctl restart juicefs-<name>)

Remove Instance

1. Set instance state to absent
2. Run juice_clean

juice_instances:
  jfs:
    path: /fs
    meta: postgres://...
    state: absent

./juice.yml -l <host> -t juice_clean,juice_register
./juice.yml -l <host> -e fsname=jfs -t juice_clean,juice_register

Removal actions:

• Stop systemd service
• umount -l lazy unmount
• Remove unit and env files
• Reload systemd
• Rewrite this node’s VictoriaMetrics target file, removing instances with state=absent

PostgreSQL metadata, PostgreSQL jfs_blob data tables, and object-storage data are not deleted.

Running only -t juice_clean does not update monitoring targets and temporarily leaves stale scrape endpoints for removed instances. The commands above therefore run juice_register as well.

Add New Instance

Add a new instance in config, ensure unique port:

juice_instances:
  newfs:
    path: /newfs
    meta: postgres://...
    data: --storage minio --bucket https://sss.pigsty:9000/newfs --access-key <s3_access_key> --secret-key <s3_secret_key>
    port: 9568

Deploy:

./juice.yml -l <host> -e fsname=newfs

Shared Mount Across Nodes

Configure the same meta and instance name on multiple nodes:

app:
  hosts:
    10.10.10.11: { juice_instances: { shared: { path: /shared, meta: "postgres://...", port: 9567 } } }
    10.10.10.12: { juice_instances: { shared: { path: /shared, meta: "postgres://...", port: 9567 } } }

Only one node needs to format the filesystem; others will skip via --no-update.

PITR Recovery

JuiceFS metadata and data must be restored to a mutually consistent state. Before any restore, stop every writer and unmount/stop the corresponding JuiceFS service on every client, identify the exact PostgreSQL cluster and target time, and confirm an available backup:

# Inspect Patroni members; inspect the target stanza's backup on a database node
pig pt list <cluster>
pig pb info -s <stanza>

# On the target database node, run the restore command as postgres
sudo -iu postgres pg-pitr -s <stanza> -t "2026-08-14 10:30:00+08"

If metadata and the --storage postgres jfs_blob table are in the same restored PostgreSQL database, database PITR can return both to one point in time. If they reside in different databases or clusters, design a coordinated recovery point for both.

If file data is in Silo/S3, PostgreSQL PITR rolls back metadata only, not objects. Newer objects may remain, while old objects that were deleted or collected may be unavailable. Recoverability depends on object versioning, trash, and lifecycle policies; do not run garbage collection until validation is complete.

Troubleshooting

Mount Fails

systemctl status juicefs-jfs
journalctl -u juicefs-jfs -f
mountpoint /fs

Metadata Connection Issues

psql "postgres://dbuser_meta:[email protected]:5432/meta" -c "SELECT 1"

Metrics Port Check

ss -tlnp | grep 9567
curl http://localhost:9567/metrics

Performance Tuning

Pass juicefs mount options via mount:

juice_instances:
  jfs:
    path: /fs
    meta: postgres://...
    mount: --cache-size 102400 --prefetch 3 --max-uploads 50

Key metrics to watch:

• juicefs_blockcache_hits/juicefs_blockcache_miss: cache hit ratio
• juicefs_object_request_durations_histogram_seconds: object storage latency
• juicefs_transaction_durations_histogram_seconds: metadata transaction latency

5 - Monitoring

JuiceFS instances expose Prometheus metrics via juicefs mount --metrics. In JUICE, metrics listen on 0.0.0.0:<port>, default port 9567.

Monitoring Architecture

JuiceFS Mount (metrics: 0.0.0.0:<port>)
    ↓
VictoriaMetrics (scrape)
    ↓
Grafana Dashboard

If INFRA is deployed, juice_register writes scrape targets to:

/infra/targets/juice/<hostname>.yml

The current source includes the Node JuiceFS dashboard (UID: node-juice) for capacity, cache, object-storage, metadata-transaction, and client-resource metrics across JuiceFS mounts on a node.

Target File Example

- labels: { ip: 10.10.10.10, ins: "node-jfs", cls: "jfs" }
  targets: [ 10.10.10.10:9567 ]

To register manually:

./juice.yml -l <host> -t juice_register

Key Metrics

Object Storage

Cache

Metadata Transactions

Common PromQL

Cache hit ratio:

rate(juicefs_blockcache_hits[5m]) /
(rate(juicefs_blockcache_hits[5m]) + rate(juicefs_blockcache_miss[5m]))

Object storage P99 latency:

histogram_quantile(0.99, rate(juicefs_object_request_durations_histogram_seconds_bucket[5m]))

6 - FAQ

Port Conflicts?

Multiple instances on the same node must use different port values. Example:

juice_instances:
  fs1:
    path: /fs1
    meta: postgres://...
    port: 9567
  fs2:
    path: /fs2
    meta: postgres://...
    port: 9568

Why does changing data not take effect?

data is only used by juicefs format --no-update. After filesystem creation it will not change. To switch backend, migrate data and reformat manually.

How to add a new instance?

1. Add instance definition in config
2. Run:

./juice.yml -l <host> -e fsname=<name>

How to remove an instance?

1. Set instance state to absent
2. Run:

./juice.yml -l <host> -t juice_clean,juice_register

Removal does not delete PostgreSQL metadata or object storage data. juice_register refreshes the target file; running only juice_clean leaves a stale monitoring scrape endpoint.

Where is file data stored?

Depends on data:

• --storage postgres: JuiceFS creates a jfs_blob table in the PostgreSQL database selected by --bucket
• --storage minio/s3: data is stored in a Silo/S3-compatible object-storage bucket

Metadata is stored in the metadata engine defined by meta (in Pigsty production scenarios, this is usually PostgreSQL).

Multi-node mount notes?

• Use the same meta and instance name on all nodes
• Only one node needs to format; others will skip
• Ensure port does not conflict on each node

Monitoring target not generated?

juice_register only writes /infra/targets/juice/ when infra group exists. You can run manually:

./juice.yml -l <host> -t juice_register

How to change mount options?

After updating mount in the instance, refresh config first and then manually restart the service:

./juice.yml -l <host> -t juice_config,juice_launch
systemctl restart juicefs-<name>