CLI Reference

pals create

Start a server hosting a NEW private swarm.

$ pals create [OPTIONS] [MODEL] [-- run_server_args...]

What it does

Brings up a libp2p server that hosts a slice of the specified model. This is the first server in a brand-new swarm; subsequent peers join via pals join using your multiaddr.

Anything after -- is forwarded verbatim to the underlying run_server entrypoint, so you can override any advanced flag without us having to mirror it. (Note: run_server uses underscore flag names like --num_handlers.)

Arguments

model

TEXT (optional)

HuggingFace model id or local path (e.g. meta-llama/Llama-3.1-8B). Falls back to config.default_model if omitted.

Options

--port, -p

INTEGER

Listen port. 0 means auto-assign.

Default: 0

--device

TEXT

Torch device: cuda, cpu, mps, or a specific cuda:N.

--num-blocks, -n

INTEGER

Number of transformer blocks to serve. Default: as many as fit on the device.

--public

Bind on all interfaces so off-machine peers can reach you. Default is loopback-only.

--identity

TEXT

Path to a libp2p identity key. Default: from config.

--invite-token

TEXT

Invite token for the swarm. Passed to the server via the env var PROGRESSPALS_INVITE_TOKEN (not argv) so it doesn’t leak through /proc/<pid>/cmdline. Falls back to config.invite_token.

Env: PROGRESSPALS_INVITE_TOKEN

--config-dir

TEXT

Read config from this directory instead of the default.

Examples

Start the first server in a swarm:

create

$pals create meta-llama/Llama-3.1-8B

✓ starting NEW swarm

multiaddr: /ip4/HOST/tcp/PORT/p2p/PEER_ID

hosting layers 0–14 of 32

Bind on a specific port and accept off-machine peers (after the operator has port-forwarded / opened a hole in the firewall):

public

$pals create meta-llama/Llama-3.1-8B --port 31337 --public

Forward an advanced flag to run_server (4-bit quantization):

quantization

$pals create meta-llama/Llama-3.1-8B -- --quant_type nf4

Capturing logs. Both the “Starting NEW swarm…” banner and the server logs go to stderr. To capture them with tee, use 2>&1 | tee log, not plain | tee.