JUICE on Pigsty v5.0

Module: JUICE

Mon, 01 Jan 0001 00:00:00 +0000

JuiceFS is a high-performance POSIX-compatible distributed filesystem that can mount object storage or databases as a local filesystem.

The JUICE module depends on NODE for infrastructure and package repo, and typically uses PGSQL as the metadata engine. Data storage can be PostgreSQL (in a jfs_blob table) or Silo / S3-compatible object storage provided by the MINIO module. Monitoring relies on INFRA VictoriaMetrics.

flowchart LR
    subgraph Client["App/User"]
        app["POSIX Access"]
    end

    subgraph JUICE["JUICE"]
        jfs["JuiceFS Mount"]
    end

    subgraph PGSQL["PGSQL"]
        meta["Metadata DB"]
        blob["Data DB / jfs_blob (optional)"]
    end

    subgraph Object["Object Storage (optional)"]
        s3["Silo / S3"]
    end

    subgraph INFRA["INFRA (optional)"]
        vm["VictoriaMetrics"]
    end

    app --> jfs
    jfs --> meta
    jfs -.->|alternative data backend| blob
    jfs -.->|alternative data backend| s3
    jfs -->|/metrics| vm

    style JUICE fill:#5B9CD5,stroke:#4178a8,color:#fff
    style PGSQL fill:#3E668F,stroke:#2d4a66,color:#fff
    style Object fill:#FCDB72,stroke:#d4b85e,color:#333
    style INFRA fill:#999,stroke:#666,color:#fff

Features

PostgreSQL metadata: Metadata stored in PostgreSQL for easy management and backup
Multi-instance: One node can mount multiple independent filesystem instances
Multiple data backends: PostgreSQL, Silo/MinIO, S3, and more; metadata and file data remain separate roles
Monitoring integration: Each instance exposes Prometheus / Victoria-format metrics port
Simple config: Describe instances with the juice_instances dict

Quick Start

Minimal config example (single instance):

Configuration

Mon, 01 Jan 0001 00:00:00 +0000

Concepts and Implementation

JuiceFS consists of a metadata engine and data storage. In the current version, meta is passed through to juicefs as the metadata engine URL, and PostgreSQL is typically used in production. Data storage is defined by data options passed to juicefs format.

JUICE module core commands:

# Format (only effective on first creation)
juicefs format --no-update <data> "<meta>" "<name>"

# Mount
juicefs mount <mount_opts> --cache-dir <juice_cache> --metrics 0.0.0.0:<port> <meta> <path>

Notes:

Parameters

Mon, 01 Jan 0001 00:00:00 +0000

JUICE module has 2 parameters:

juice_cache: shared cache directory
juice_instances: instance definition dict

Parameter Overview

Parameter	Type	Level	Description
`juice_cache`	`path`	`C`	JuiceFS shared cache directory
`juice_instances`	`dict`	`I`	JuiceFS instance definition dict (can be empty)

Level: C = cluster level, I = instance level.

Default Parameters

Defined in roles/juice/defaults/main.yml:

#-----------------------------------------------------------------
# JUICE
#-----------------------------------------------------------------
juice_cache: /data/juice
juice_instances: {}

`juice_cache`

Parameter: juice_cache, type: path, level: C

Playbook

Mon, 01 Jan 0001 00:00:00 +0000

JUICE module provides juice.yml playbook to deploy and remove JuiceFS instances.

`juice.yml`

Task structure in juice.yml:

juice_id        : validate config, check port conflicts
juice_install   : install juicefs package
juice_cache     : create shared cache dir
juice_clean     : remove instance (state=absent)
juice_instance  : create instance (state=create)
  - juice_init  : format filesystem (--no-update)
  - juice_dir   : create mount dir
  - juice_config: render env file and systemd unit
  - juice_launch: start service and wait for metrics port
juice_register  : register to VictoriaMetrics targets

Scope

Scope	Limit	Description
Node	`-l <host>`	Deploy all instances on the node
Instance	`-l <host> -e fsname=<name>`	Only handle specified instance

Examples:

Administration

Mon, 01 Jan 0001 00:00:00 +0000

Common operations:

See FAQ for more.

Initialize Instance

./juice.yml -l <host>
./juice.yml -l <host> -e fsname=<name>

Initialization steps:

Install juicefs package
Create shared cache dir (default /data/juice)
Run juicefs format --no-update (only effective on first creation)
Create mount point and set permissions
Render systemd unit and env files
Start service and wait for metrics port
Register to VictoriaMetrics (if infra node exists)

Reconfigure

After changing config, it’s recommended to run (update config and ensure service is online):

Monitoring

Mon, 01 Jan 0001 00:00:00 +0000

JuiceFS instances expose Prometheus metrics via juicefs mount --metrics. In JUICE, metrics listen on 0.0.0.0:<port>, default port 9567.

Monitoring Architecture

JuiceFS Mount (metrics: 0.0.0.0:<port>)
    ↓
VictoriaMetrics (scrape)
    ↓
Grafana Dashboard

If INFRA is deployed, juice_register writes scrape targets to:

/infra/targets/juice/<hostname>.yml

The current source includes the Node JuiceFS dashboard (UID: node-juice) for capacity, cache, object-storage, metadata-transaction, and client-resource metrics across JuiceFS mounts on a node.

FAQ

Mon, 01 Jan 0001 00:00:00 +0000

Port Conflicts?

Multiple instances on the same node must use different port values. Example:

juice_instances:
  fs1:
    path: /fs1
    meta: postgres://...
    port: 9567
  fs2:
    path: /fs2
    meta: postgres://...
    port: 9568

Why does changing `data` not take effect?

data is only used by juicefs format --no-update. After filesystem creation it will not change. To switch backend, migrate data and reformat manually.

JUICE on Pigsty v5.0

Module: JUICE

Features

Quick Start

Configuration

Concepts and Implementation

Parameters

Parameter Overview

Default Parameters

juice_cache

Playbook

juice.yml

Scope

Administration

Initialize Instance

Reconfigure

Monitoring

Monitoring Architecture

FAQ

Port Conflicts?

Why does changing data not take effect?

`juice_cache`

`juice.yml`

Why does changing `data` not take effect?