Introduction

What makes Grandine different

Note: No audit makes software risk free. Protect validator keys with the approach you trust, for example a remote signer such as Web3Signer.

Performance

Grandine was built from the start for speed: a highly optimized state transition function, and every CPU core put to work.

The speed comes from optimizations at every layer:

• An optimized state transition function. The state transition function applies each block and epoch to the beacon state, and it is the hottest code path in a consensus client. Grandine optimizes it with micro-optimizations throughout and with parallelism.
• Parallel verification. Signatures are verified in batches, and blocks, attestations, aggregates, blob sidecars and data columns are validated in parallel.
• A lock-free fork choice. A single thread applies verified results to the fork choice store, while the Beacon API and the validator client read recent snapshots. Readers never wait for the writer, and the writer never waits for them.
• No state copies. Beacon states use structural sharing: a new state shares every unchanged part with its parent instead of copying it. Block processing avoids the cost of cloning full states, and hundreds of unfinalized states cost little more memory than one.
• Heavy work kept off the network path. Block production, data column recovery and other CPU-heavy jobs run on dedicated executors, separate from the async runtime that serves peers and APIs.

How Grandine works and Performance and memory tuning explain these mechanisms in more detail.

About Grandine

Grandine is an Ethereum consensus client written in Rust. It follows the beacon chain, takes part in consensus through its standalone or built-in validator client, and serves the standard Beacon Node API. Grandine is built by the Grandine core team and is open source under the GNU General Public License, version 3. The source is on GitHub.

This book is for node operators: home stakers, staking providers, infrastructure teams and anyone who wants to run a beacon node. It explains how to install Grandine, connect it to an execution client, run validators safely and keep the node healthy.

Supported networks

Grandine connects to Ethereum Mainnet and to the Sepolia and Hoodi testnets out of the box. It still accepts Holesky, which has reached its end of life. It can also join devnets and other custom networks from a configuration directory. See Supported networks.

How to use this book

If you are new to Grandine, read the pages in this order:

1. System requirements: check that your machine is big enough.
2. How Grandine works: the beacon node, validator client and execution client, and how they fit together.
3. Installation overview: pick a binary, a Docker image or a source build.
4. Quick start: run an execution client and Grandine on the Hoodi testnet.
5. Running as a service: run Grandine under systemd or Docker Compose.
6. Becoming a validator: add validator keys once the node is synced.

Tip: Practice on the Hoodi testnet before you run validators on Mainnet. Testnet ether has no value, so mistakes there cost nothing.

Later sections cover the beacon node in depth (Running a beacon node, Checkpoint sync, Storage and database), validator operations (Validator client, Slashing protection), the APIs (Beacon Node API), monitoring (Metrics, Logging) and reference material (Command line reference, Troubleshooting, FAQ).

Getting help

The fastest way to reach the team and other operators is the Grandine Discord. Report bugs on GitHub. See Support.

System requirements

This page lists what Grandine itself needs. A staking node also runs an execution client (EL) such as Geth or Nethermind on the same machine, and the execution client needs much more memory and disk than Grandine. Add its requirements to the figures below. See Sizing a full node.

Grandine requirements

These figures are estimates for Grandine on Mainnet, with the built-in validator client and the default storage mode. They include headroom for the operating system and for periods without finality, but are not yet backed by published measurements.

See Operating systems and architectures for the supported platforms.

Testnets such as Hoodi need less disk than Mainnet.

CPU

Grandine spreads its work across all CPU cores. More cores help most during the initial sync, when the node runs many validators, and when it subscribes to many subnets.

The official binaries use the portable build of the blst BLS library, so they also run on older x86-64 CPUs that lack the ADX instruction set extension.

Memory

On a healthy Mainnet, Grandine needs about 5 GB of memory. Grandine keeps the unfinalized part of the chain in memory, so memory use grows during a long period without finality. To measure how much memory Grandine really uses, and to limit it during non-finality, see Performance and memory tuning.

Disk usage by storage mode

Use an SSD, preferably NVMe. Grandine stores the finalized chain in an embedded libmdbx database, and how much it stores depends on the storage mode:

The retention windows come from the consensus specification and are the same on Mainnet and Hoodi. After the Gloas fork, the block window shrinks to 14,299 epochs (about 64 days).

Note: A new high-performance archive node is coming soon. See Back-sync and archive nodes.

Network

A beacon node keeps many peer connections and exchanges data with them continuously. Use a reliable connection without a monthly data cap, and open the peer-to-peer ports in your firewall:

• 9000 TCP and UDP, and 9001 UDP (QUIC), over IPv4
• 9050 TCP and UDP, and 9051 UDP (QUIC), over IPv6. These are used only when IPv6 is enabled with --listen-address-ipv6

See Networking and peers and Default ports and paths.

Time synchronization

Every duty in Ethereum is tied to a slot, so the system clock must be accurate. Run a time synchronization service such as chrony or systemd-timesyncd. A clock that drifts by even a second or two causes missed attestations, and peers reject messages from nodes whose clocks are far off.

When you need more

Some setups need more resources than the figures above:

• Many validators. A single Grandine node can run up to tens of thousands of validators. CPU use and network traffic grow with the number of validators, because the node subscribes to more attestation subnets and produces more messages.
• Supernodes. A node started with --supernode downloads and stores every data column, and --semi-supernode half of them. Both need noticeably more bandwidth and disk than a regular node. See PeerDAS and blob data.
• Archive nodes. An archive node needs much more disk, and benefits from a fast NVMe SSD for serving historical data. See Back-sync and archive nodes.
• All subnets. --subscribe-all-subnets increases CPU use and traffic.

Sizing a full node

To size the machine, add your execution client's requirements to Grandine's. The execution client usually needs most of the memory and disk. Check its documentation:

• Geth
• Nethermind
• Besu
• Erigon
• Reth

As a rough orientation, a Mainnet node that runs Grandine and a typical execution client is comfortable with:

Note: Execution clients now expire old chain history on a rolling basis, so their disk usage no longer grows without bound. A 2 TB disk is enough for a full node with a typical execution client. Archive nodes need much more.

When Grandine runs embedded in Nethermind, both clients share one process. Size the machine the same way: Nethermind's requirements plus Grandine's.

For reliability, a stable internet connection and an uninterruptible power supply (UPS) help a validator stay online.

Operating systems and architectures

Each release publishes these binaries, where VERSION is the Grandine version:

Each release also publishes Grandine embedded in Nethermind as one archive per platform, where NETHERMIND_VERSION is the bundled Nethermind version:

Docker images are published for linux/amd64 and linux/arm64. Linux is the most tested platform and the one this book assumes. See Pre-built binaries and Docker.

How Grandine works

This page explains the moving parts of an Ethereum node and how Grandine puts them together. You do not need it to get started, but it helps when you plan a setup or read the logs.

Concepts primer

Execution layer and consensus layer

Since the Merge, an Ethereum node is made of two programs that work as a pair:

• The execution client (EL), such as Geth, Nethermind, Besu, Reth or Erigon, runs the EVM. It keeps account balances and contract storage, holds the transaction pool and executes the transactions in each block.
• The consensus client (CL), such as Grandine, runs proof of stake. It follows the beacon chain, decides which block is the head of the chain, talks to other consensus clients over a peer-to-peer network and drives the validators.

The two talk over the authenticated Engine API, usually on port 8551 of the execution client. Both sides prove their identity with a shared secret, the JWT secret. The consensus client tells the execution client which block to validate and which block is the head; the execution client tells the consensus client whether a payload is valid and builds payloads for new blocks.

When Grandine is embedded in Nethermind, it runs inside the Nethermind process and calls Nethermind's Engine API implementation directly, as native function calls. There is no HTTP, JWT authentication or JSON encoding between the two clients, so this communication has no overhead.

Neither half is useful alone. A consensus client without an execution client cannot check the transactions in a block, and an execution client without a consensus client does not know which chain to follow.

Slots, epochs and finality

Time on the beacon chain is split into slots of 12 seconds on Mainnet. Each slot may hold one block. 32 slots make an epoch, about 6.4 minutes.

Validators vote on the chain in every epoch. When two thirds of the stake agree on a checkpoint at an epoch boundary, it becomes justified, and a justified checkpoint followed by another justified one becomes finalized. A finalized block cannot be reverted without destroying at least one third of all staked ether. Under normal conditions blocks finalize after about two epochs, roughly 13 minutes.

Validator duties

A validator is 32 ether or more (up to 2048 after the Electra upgrade) staked with a BLS key pair. Its key signs messages for these duties:

• Attesting: once per epoch, vote for the head of the chain and for the source and target checkpoints.
• Proposing: when selected, build and sign the block for a slot.
• Aggregating: when selected, collect attestations from a subnet and publish an aggregate.
• Sync committee: when selected for a sync committee (512 validators for about 27 hours), sign the head block in every slot to serve light clients.

Signing two conflicting messages for the same duty is a slashable offense. The validator loses part of its stake and is ejected. Grandine guards against it with a slashing protection database and optional doppelganger protection.

The software that holds the keys and performs duties is the validator client (VC). It needs a beacon node to learn its duties, get blocks to sign and publish what it signs.

The parts of Grandine

Grandine is one binary built from many crates. The main components of a running beacon node are:

The runtime crate parses the command line, sets up the components and connects them with message channels.

How the beacon node, validator client, EL and database interact

             peers (libp2p: gossip, discovery, req/resp)
                              |
                              v
+--------------------------------------------------------+
| Grandine beacon node                                   |
|                                                        |
| p2p / sync ---> fork choice ---------> database        |
|                 (mutator thread)       (libmdbx)       |
|                    ^    |                              |
|                    |    | snapshots                    |
|                    |    v                              |
| Beacon API <--- controller ----------> execution ------+---> EL
| (port 5052)     (queries)              engine          |     (Engine API, JWT)
|     ^              ^                                   |
|     |              |                                   |
|     |       integrated validator client                |
|     |       (in-process, optional)                     |
+-----+--------------------------------------------------+
      |
      | Beacon Node API (HTTP)
      |
 separate validator client (grandine vc)

1. Blocks arrive. The networking service receives blocks, blobs, data columns and attestations by gossip, or requests them from peers while syncing.
2. Fork choice processes them. Objects are handed to the fork choice controller. A single mutator thread owns the fork choice store. Expensive work such as signature checks and state transitions runs in parallel on a thread pool, and the results come back to the mutator, which applies them in order.
3. The EL validates payloads. For each block, Grandine sends the execution payload to the execution client with engine_newPayload, and reports the new head with engine_forkchoiceUpdated. Until the EL confirms a payload, the block is optimistic.
4. Readers use snapshots. The Beacon API, the validator client and other components read from a recent snapshot of the store. They never wait for the mutator, and the mutator never waits for them.
5. Finalized data goes to disk. When a checkpoint finalizes, Grandine writes the newly finalized blocks, and states at the configured interval, to the database. What exactly it keeps depends on the storage mode. On shutdown it saves the current chain, so a restart can continue where it stopped.

With --eth1-rpc-urls you can give more than one execution client endpoint. Grandine uses the first and falls back to the others if it fails. See Connecting an execution client.

The in-memory unfinalized chain

Most consensus clients keep recent states in a database or an on-disk cache. Grandine keeps the whole unfinalized part of the chain in memory and uses the database mainly for finalized data, for loading checkpoints at startup and for serving historical data through the API.

This is affordable because beacon states use structural sharing. A beacon state is built from persistent data structures (trees). When a block changes a few balances, the new state shares every unchanged subtree with the previous state instead of copying it. Hundreds of unfinalized states cost little more memory than one, and Grandine avoids the cost of copying full states during block processing.

On a healthy network the unfinalized part is about two epochs long, so memory use stays low. During a long period without finality, the unfinalized chain grows. To bound memory, Grandine keeps full states in memory for only the latest --unfinalized-states-in-memory slots (default 128) and recomputes older ones from blocks when they are needed. See Storage and database and Performance and memory tuning.

Integrated or separate validator client

Grandine can run in three modes:

Beacon node only

grandine bn runs only the beacon node: networking, fork choice, the connection to the execution client, the database and the Beacon Node API (port 5052 by default). It has no validator client and holds no validator keys, so it refuses validator options such as --keystore-dir or --enable-validator-api.

Use it when the validators run in a separate validator client, grandine vc or another client's, that connects to this node through the standard Beacon Node API. If that validator client runs on another host, bind the API with --http-address to an address the host can reach, and firewall it so that only the validator client can connect. Set --suggested-fee-recipient on the node: it is used for validators whose validator client does not send a fee recipient. See Validator client.

grandine bn also suits nodes that run for their API only, for example to serve applications, explorers or other teams' validator clients.

Integrated validator client (default)

When you start grandine with validator keys (--keystore-dir, or Web3Signer keys), the validator client runs inside the beacon node process. It reads duties and head information directly from the fork choice snapshot and hands signed messages straight to the node, without going through HTTP. This is the fastest path and has the fewest moving parts.

Without keys, plain grandine behaves like a beacon node. You can still add keys later through the Keymanager API if you enable it with --enable-validator-api.

Separate validator client

grandine vc runs only the validator client. It performs duties through the standard Beacon Node API of one or more beacon nodes given with --beacon-node-urls. The beacon nodes can be Grandine or any other consensus client. Among the healthiest nodes, Grandine sends duties to the one with the newest head, then to the first listed. By default it publishes every duty to every node (--publish-to-every-node).

grandine bn and grandine vc reject options that do not apply to them. For example, vc refuses --checkpoint-sync-url, and bn refuses --keystore-dir. Options must come after the command: grandine vc --network hoodi ..., not grandine --network hoodi vc ....

Warning: Never run the same validator keys in two places at once, for example in an integrated validator client and in a grandine vc, or in Grandine and another client. This gets the validator slashed. Never point two validator clients at the same data directory. When you move keys, stop the old validator client first, move the slashing protection data with it, and see Validator client and Migrating from another client.

For multiple beacon nodes and failover, see Redundancy and fallback beacon nodes.

Startup and shutdown

On startup, the beacon node:

1. Loads the network configuration and, on testnets, the genesis state (downloaded on first start and stored in the data directory).
2. Loads the latest finalized state from its database. If the database is empty and --checkpoint-sync-url is set, it downloads a recent finalized state from that beacon node instead (see Checkpoint sync).
3. Connects to the execution client, joins the peer-to-peer network and starts syncing forward.

On SIGINT (Ctrl+C) or SIGTERM, Grandine stops the fork choice, logs saving current chain before exit…, writes the chain to the database (logging chain saved) and exits. Give it time to finish; see Running as a service.

If a component fails at run time, Grandine logs application runtime failed and restarts its runtime inside the same process. Some errors, such as detected doppelgangers or a validator startup error, stop the process instead. The process exits with status 1 on failure and 2 on a command line error.

Optional components

• Slasher: the slasher crate watches for slashable offenses by other validators. It is currently not enabled from the command line. See Slasher.
• Builder API: with --builder-url, block production can use an external builder such as MEV-Boost. See Builder API and MEV.
• Back-sync: with --back-sync, a checkpoint-synced node downloads history backward. See Back-sync and archive nodes.

Command line options

• --eth1-rpc-urls: no default. Engine API URLs of the execution clients; the extra URLs are fallbacks.
• --jwt-secret: no default. File with the hex-encoded JWT secret shared with the EL.
• --unfinalized-states-in-memory: default 128. Number of the latest unfinalized slots whose states stay in memory.
• --keystore-dir: no default. Validator keystores for the integrated or separate validator client.
• --beacon-node-urls (vc only): required. Beacon Node API URLs the validator client works against.
• --publish-to-every-node (vc only): default all. Duties published to every beacon node.

See the Command line reference for the full list.

Quick start

This tutorial runs a full Ethereum node on the Hoodi testnet: an execution client and Grandine, connected with a JWT secret and synced with checkpoint sync. It takes about 15 minutes of work, plus the time the execution client needs to sync.

The same steps work on Mainnet. Where a command differs, the Mainnet form is given as well.

Tip: Practice on Hoodi before you run validators on Mainnet. Testnet ether has no value, so mistakes cost nothing.

Before you start

• A machine that meets the System requirements.
• Grandine installed. See the Installation overview. Check it with grandine --version.
• An execution client installed: Geth, Nethermind, Besu, Reth or Erigon. Follow its own installation guide.
• Ports 9000 TCP and UDP and 9001 UDP open for Grandine, and the execution client's peer-to-peer port (usually 30303 TCP and UDP) open for the execution client.

Step 1: Create a JWT secret

The execution client and Grandine authenticate each other with a shared secret: 32 random bytes, written as hex. Create it once and give both programs the same file:

sudo mkdir -p /var/lib/ethereum
openssl rand -hex 32 | tr -d "\n" | sudo tee /var/lib/ethereum/jwt.hex > /dev/null
sudo chmod 644 /var/lib/ethereum/jwt.hex

This tutorial uses /var/lib/ethereum/jwt.hex. Grandine accepts the secret with or without a 0x prefix and a trailing newline.

Note: Some execution clients create a JWT secret on first start if the file does not exist. That also works, as long as Grandine reads the same file.

Step 2: Start the execution client

Start your execution client with its Engine API on 127.0.0.1:8551 and the JWT secret from step 1. Pick one of the examples. The flags belong to the execution clients, not to Grandine, and can change between their releases; check the documentation of your client.

Geth

geth \
  --hoodi \
  --datadir /var/lib/geth \
  --authrpc.addr 127.0.0.1 \
  --authrpc.port 8551 \
  --authrpc.jwtsecret /var/lib/ethereum/jwt.hex

For Mainnet, remove --hoodi.

Nethermind

nethermind \
  --config hoodi \
  --data-dir /var/lib/nethermind \
  --JsonRpc.EngineHost 127.0.0.1 \
  --JsonRpc.EnginePort 8551 \
  --JsonRpc.JwtSecretFile /var/lib/ethereum/jwt.hex

For Mainnet, use --config mainnet.

Besu

besu \
  --network=hoodi \
  --data-path=/var/lib/besu \
  --engine-rpc-port=8551 \
  --engine-host-allowlist=localhost,127.0.0.1 \
  --engine-jwt-secret=/var/lib/ethereum/jwt.hex

For Mainnet, use --network=mainnet.

Reth

reth node \
  --chain hoodi \
  --datadir /var/lib/reth \
  --authrpc.addr 127.0.0.1 \
  --authrpc.port 8551 \
  --authrpc.jwtsecret /var/lib/ethereum/jwt.hex

For Mainnet, use --chain mainnet.

Erigon

Erigon has its own built-in consensus client. Disable it with --externalcl so Erigon follows Grandine:

erigon \
  --chain=hoodi \
  --datadir=/var/lib/erigon \
  --externalcl \
  --authrpc.addr=127.0.0.1 \
  --authrpc.port=8551 \
  --authrpc.jwtsecret=/var/lib/ethereum/jwt.hex

For Mainnet, use --chain=mainnet.

Leave the execution client running. Until a consensus client connects, it logs warnings that it has not received any forkchoice updates. That is expected.

Step 3: Start Grandine with checkpoint sync

Checkpoint sync starts Grandine from a recent finalized state downloaded from a trusted beacon node, instead of from genesis. Grandine is then ready in minutes. Pick a checkpoint sync endpoint for your network from the community-maintained list of checkpoint sync endpoints, or use a beacon node you run yourself.

Start Grandine:

grandine \
  --network hoodi \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /var/lib/ethereum/jwt.hex \
  --checkpoint-sync-url CHECKPOINT-URL

For Mainnet, use --network mainnet and a Mainnet checkpoint endpoint.

With Docker on Linux, host networking lets the container reach the execution client on 127.0.0.1:

docker run -d --name grandine \
  --network host \
  -v $HOME/.grandine:/root/.grandine \
  -v /var/lib/ethereum/jwt.hex:/jwt.hex:ro \
  sifrai/grandine:stable \
  --network hoodi \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /jwt.hex \
  --checkpoint-sync-url CHECKPOINT-URL

What the options do:

• --network hoodi selects the network. The default is mainnet.
• --eth1-rpc-urls is the Engine API URL of the execution client. Note the port: 8551, not the regular JSON-RPC port 8545.
• --jwt-secret is the file from step 1.
• --checkpoint-sync-url is the beacon node to download the finalized state from.

Grandine stores its data in ~/.grandine/hoodi (~/.grandine/mainnet on Mainnet). On Hoodi, the first start also downloads the genesis state and stores it there.

Note: Checkpoint sync only happens when the database is empty. If Grandine finds an existing database, it logs skipping checkpoint sync: existing database found and continues from its own data. If checkpoint sync fails, Grandine logs the error and starts from genesis, which takes much longer; stop it, fix the URL and start again.

Step 4: Check that Grandine is syncing

Read the logs

Shortly after start, you should see lines like these (shortened). Plain grandine always starts the integrated validator client, even without keys, so the first line says with validator client:

INFO runtime::runtime: starting beacon node with validator client peers=[0/0]
INFO fork_choice_control::checkpoint_sync: performing checkpoint sync from https://CHECKPOINT-URL/… peers=[0/200]
INFO fork_choice_control::checkpoint_sync: loaded state at slot 1234560 from https://CHECKPOINT-URL/ peers=[0/200]
...
INFO fork_choice_control::mutator: new finalized checkpoint (epoch: 38581, root: 0x…, head slot: 1234620, head root: 0x…) peers=[48/200]

• peers=[48/200] at the end of each line is the number of connected peers and the target. It should climb into the dozens within a few minutes. If it stays at 0, check your ports and firewall; see Networking and peers.
• new finalized checkpoint lines appear about every epoch (6.4 minutes) once Grandine follows the chain. The head slot should match the current slot on a block explorer.
• Warnings about the execution client being offline or syncing are normal while the execution client catches up. Until it has synced, Grandine imports blocks optimistically.

Ask the API

Grandine serves the Beacon Node API on http://127.0.0.1:5052. Check the sync status:

curl -s http://127.0.0.1:5052/eth/v1/node/syncing

{"data":{"head_slot":"1234620","sync_distance":"0","is_syncing":false,"is_optimistic":false,"el_offline":false}}

• is_syncing: false and sync_distance: 0: Grandine has caught up with the chain.
• is_optimistic: true: Grandine has the head, but the execution client has not validated it yet. Wait until the execution client finishes syncing.
• el_offline: true: Grandine cannot reach the execution client. Check the URL, the port and the JWT secret. See Connecting an execution client.

Other quick checks:

# 200 when synced, 206 while syncing
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5052/eth/v1/node/health

# Number of connected peers
curl -s http://127.0.0.1:5052/eth/v1/node/peer_count

# Version of Grandine
curl -s http://127.0.0.1:5052/eth/v1/node/version

The node is ready for validators when is_syncing and is_optimistic are both false and el_offline is false.

Step 5 (optional): Add validators

Once the node is synced, you can run validators in the same process. Stop Grandine and start it again with your keystores and a fee recipient:

grandine \
  --network hoodi \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /var/lib/ethereum/jwt.hex \
  --checkpoint-sync-url CHECKPOINT-URL \
  --keystore-dir /path/to/validator_keys \
  --keystore-password-file /path/to/password.txt \
  --suggested-fee-recipient FEE-RECIPIENT-ADDRESS

Grandine logs loaded N validator key(s) on startup.

Warning: Never run the same validator keys in two places. If these keys are, or were recently, active in another client, read Migrating from another client first and consider doppelganger protection.

Becoming a validator walks through generating keys, making the deposit and checking that everything works.

What next?

• Running as a service: run the node under systemd or Docker Compose so it starts on boot and restarts on failure.
• Becoming a validator: generate keys, deposit and start validating.
• Metrics and Monitoring validator performance: watch your node.
• Networking and peers: port forwarding and peer tuning.
• Updating Grandine: stay on a current release.
• Troubleshooting: what to do when something goes wrong.

Command line options

• --network: default mainnet. Network to connect to: mainnet, sepolia, holesky, hoodi or custom.
• --eth1-rpc-urls: no default. Engine API URLs of the execution clients.
• --jwt-secret: no default. File with the hex-encoded JWT secret.
• --checkpoint-sync-url: no default. Beacon Node API URL to load a recent finalized checkpoint from.
• --data-dir: default $HOME/.grandine. Parent directory for data; the network name is appended.
• --http-port: default 5052. Port of the Beacon Node API.
• --keystore-dir: no default. Directory with EIP-2335 validator keystores.
• --keystore-password-file: no default. File with the password for all keystores.
• --suggested-fee-recipient: no default. Execution address that receives priority fees.

See the Command line reference for the full list.

Supported networks

Grandine has built-in configurations for Ethereum Mainnet and the public testnets. Select one with --network. Other networks, such as devnets and private networks, are loaded from a configuration directory with --network custom.

Built-in networks

For each built-in network, Grandine includes the chain configuration (fork schedule, deposit contract, blob limits) and the bootnodes it uses to find its first peers. You only need to set --network.

grandine --network hoodi --eth1-rpc-urls http://127.0.0.1:8551 --jwt-secret /path/to/jwt.hex

The execution client must run on the same network as Grandine. A pair on different networks cannot sync.

Each network has its own data directory, so you can keep data for several networks side by side: ~/.grandine/mainnet, ~/.grandine/hoodi and so on. With --data-dir /path/to/data, the network name is still appended: /path/to/data/hoodi.

Mainnet

Mainnet is the default. Its genesis state is built into Grandine, so a start from genesis needs no download, but syncing all of Mainnet from genesis takes a long time. Use checkpoint sync instead.

Make deposits for Mainnet validators only through the official Staking Launchpad.

Hoodi

Hoodi is the testnet to use when you learn to run validators, test an upgrade or try a new setup before Mainnet. Its fork schedule runs ahead of Mainnet, so new forks reach Hoodi first.

• Staking Launchpad: https://hoodi.launchpad.ethereum.org/
• Testnet ether: Hoodi faucets give out small amounts of test ether. Search for a current "Hoodi faucet"; they change often.

Sepolia

Sepolia is meant for application developers. Its validator set is permissioned, so you can run a Sepolia beacon node, but you cannot become a Sepolia validator with a normal deposit. Use Hoodi to practice staking.

Holesky

Holesky has reached its end of life. Fusaka activated on it on 1 October 2025, its maintainers shut down most of its validators two weeks after the upgrade finalized, and client, testing and infrastructure teams no longer support it (see the announcement). Grandine still accepts --network holesky, but do not use it: run validators on Hoodi, and use Sepolia for application testing.

Genesis state on testnets

On the first start on Sepolia, Holesky or Hoodi, Grandine downloads the network's genesis state from the network's repository on GitHub and saves it as genesis_state.ssz in the beacon data directory (for example ~/.grandine/hoodi/beacon/genesis_state.ssz). This happens even with checkpoint sync, because Grandine uses it to identify the chain. Later starts load the saved file.

If the machine cannot reach GitHub, either download the genesis state another way and pass it with --genesis-state-file, or give another source with --genesis-state-download-url. Grandine checks the genesis validators root of a downloaded or saved state against the known value for the network and refuses a wrong file. It does not check a file given with --genesis-state-file.

Checkpoint sync endpoints

Checkpoint sync needs a trusted beacon node on the same network. Public endpoints for Mainnet and the testnets are collected at eth-clients.github.io/checkpoint-sync-endpoints. Your own beacon node, or one run by someone you trust, is better still. See Checkpoint sync.

Custom networks and devnets

For a network that is not built in, use --network custom and point Grandine at a directory with the network's files:

grandine \
  --network custom \
  --configuration-directory /path/to/network-config \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /path/to/jwt.hex

The directory uses the layout that most devnets and testnets publish:

Each file must exist unless you pass the matching option. When you pass an option and the directory also has the file, the option wins and Grandine logs a warning.

Notes:

• The data directory is named after CONFIG_NAME in config.yaml, for example ~/.grandine/my-devnet. The name may contain only lowercase letters, digits and -.
• Grandine warns about unknown variables in config.yaml and ignores them.
• A custom network without --eth1-rpc-urls needs a genesis state file.
• --network custom without a configuration file stops with --configuration-file must be specified when connecting to custom network.
• Networks that use the minimal preset, common in local test setups, need a binary built with make minimal. See Building from source.

See Configuration files and custom networks for the formats and for checking a configuration with the --verify-* options.

Command line options

• --network: default mainnet. One of mainnet, sepolia, holesky, hoodi or custom.
• --configuration-directory: no default. Directory with config.yaml, genesis.ssz, deposit_contract_block.txt and bootstrap_nodes.txt.
• --configuration-file: no default. Chain configuration YAML file; takes precedence over the directory.
• --genesis-state-file: no default. Genesis state SSZ file.
• --genesis-state-download-url: no default. URL to download the genesis state from.
• --deposit-contract-starting-block: no default. Execution block to start tracking the deposit contract from.
• --boot-nodes: no default. Comma-separated bootnode ENRs; replaces the built-in or file list.
• --data-dir: default $HOME/.grandine. Parent directory for data; the network name is appended.

See the Command line reference for the full list.

Installation overview

You can install Grandine in four ways. All of them give you the same grandine binary with the same options.

Before you install, check the System requirements.

Which version to install

Grandine has two release channels:

• Stable releases have plain version tags such as 3.0.0. Use them on Mainnet.
• Unstable builds come from the develop branch and from pre-release tags such as 3.0.0-rc.0. They get new features and fixes first, and suit testnets and testing.

See Updating Grandine for how releases are published and how to upgrade.

Supported platforms

Binaries are attached to every GitHub release. Builds of the develop branch are published only as the unstable Docker image, so on macOS and Windows the newest unstable code requires building from source.

After installing

1. Check the version:

   grandine --version
   

   It prints the application name and version, for example Grandine 3.0.0.

2. List the options:

   grandine --help
   

3. Follow the Quick start to run an execution client and Grandine together.

4. Set Grandine up as a long-running service with Running as a service.

Pre-built binaries and Docker

The Grandine team publishes ready-to-run binaries on GitHub and multi-platform Docker images on Docker Hub. Both are built by the same release workflow from the same source.

Binaries

Binaries are attached to each tagged release on the GitHub releases page. Each release has one file per platform, named grandine-VERSION-PLATFORM:

The binaries are single, self-contained executables built with the compact Cargo profile (release optimizations without debug information). They include the Mainnet, Sepolia, Holesky and Hoodi network configurations.

Builds of the develop branch are not published on GitHub. To run the newest unstable code, use the unstable Docker image or build from source.

Install a binary on Linux

1. Download the binary for your platform from the releases page. Replace VERSION with the release you want, for example 3.0.0:

   VERSION=3.0.0
   curl -LO "https://github.com/grandinetech/grandine/releases/download/${VERSION}/grandine-${VERSION}-linux-x64"
   

2. Make it executable and move it onto your PATH:

   chmod +x "grandine-${VERSION}-linux-x64"
   sudo mv "grandine-${VERSION}-linux-x64" /usr/local/bin/grandine
   

3. Check that it runs:

   grandine --version
   

Grandine builds OpenSSL into the binary, but it uses the system's CA certificates to verify HTTPS connections, for example to a checkpoint sync server. On a minimal Debian or Ubuntu system, install them first:

sudo apt-get install ca-certificates

Verify a download

Releases do not include checksum or signature files, and the binaries are not code-signed. To check a binary, compare its version with the release you meant to download, and download only from the official GitHub repository. If you need full assurance, build from source at the release tag and compare the behavior and version of the result.

macOS and Windows

The macOS binaries are not notarized, so macOS may refuse to open them. Remove the quarantine attribute after downloading:

xattr -d com.apple.quarantine grandine-VERSION-osx-arm64
chmod +x grandine-VERSION-osx-arm64

On Windows, run grandine-VERSION-win-x64.exe from PowerShell or a command prompt. Most examples in this book use Linux paths; adjust them for your system.

Docker

Images are published to Docker Hub as sifrai/grandine. They support linux/amd64 and linux/arm64. The image is based on Ubuntu 24.04, contains the binary at /usr/local/bin/grandine and uses it as its entry point, so the arguments after the image name are passed to grandine.

Tags

Grandine images:

Grandine embedded in Nethermind (see Grandine embedded in Nethermind):

Tip: On Mainnet, use the stable tag. It follows stable releases only, so pre-releases and develop builds never reach it. To control exactly when you update, pin a version tag such as sifrai/grandine:VERSION instead. See Updating Grandine.

Pull and run

docker pull sifrai/grandine:stable
docker run --rm sifrai/grandine:stable --version

The container runs as root, so Grandine's default data directory inside it is /root/.grandine. Mount a host directory there to keep the database across container restarts, and mount the JWT secret:

docker run -d --name grandine \
  --restart unless-stopped \
  -p 9000:9000/tcp -p 9000:9000/udp -p 9001:9001/udp \
  -v $HOME/.grandine:/root/.grandine \
  -v /path/to/jwt.hex:/jwt.hex:ro \
  sifrai/grandine:stable \
  --network hoodi \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /jwt.hex \
  --checkpoint-sync-url CHECKPOINT-URL \
  --http-address 0.0.0.0

Inside a container, the Beacon Node API must listen on 0.0.0.0 (--http-address 0.0.0.0) to be reachable from outside the container. Publish port 5052 (-p 127.0.0.1:5052:5052) only if you need the API on the host, and do not expose it to the internet.

For a complete setup with an execution client, see the Docker Compose example in Running as a service, or use Eth Docker.

Volumes and paths

NETWORK is the network name, such as mainnet or hoodi. See Default ports and paths.

Grandine bundled with Nethermind

Releases also include archives named grandine-VERSION-nethermind-NETHERMIND_VERSION-PLATFORM.zip, and Docker Hub has matching -nethermind tags (VERSION-nethermind-NETHERMIND_VERSION). These contain a Nethermind release with Grandine embedded as a plugin, so one process runs both the execution and the consensus layer, and Grandine calls Nethermind's Engine API directly. See Grandine embedded in Nethermind.

Command line options

• --version: print the version and exit.
• --help: print all options and exit.
• --data-dir: default $HOME/.grandine. Parent directory for data; the network name is appended.
• --http-address: default 127.0.0.1. Address the Beacon Node API listens on.
• --http-port: default 5052. Port of the Beacon Node API.

See the Command line reference for the full list.

Building from source

Build Grandine from source when you want the newest code, a platform without official binaries, a build with extra features, or to check exactly what you run.

Requirements

• Rust, installed with rustup. The repository pins its toolchain in rust-toolchain.toml (Rust 1.95.0 at the time of writing). rustup reads this file and installs the right version on the first build, so you do not need to pick a version yourself.
• System packages: a C compiler toolchain, Clang, CMake, the Protocol Buffers compiler, and the OpenSSL and zlib development files.
• Git, to clone the repository and its submodules.
• Enough memory and disk for the build: plan for at least 8 GB of RAM and 20 GB of free disk. These are estimates, not measured figures.

On Ubuntu or Debian, install the system packages with:

sudo apt-get install build-essential ca-certificates libssl-dev clang cmake unzip protobuf-compiler libprotobuf-dev libz-dev git

Then install Rust:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Build

1. Clone the repository and fetch the submodules the build needs:

   git clone https://github.com/grandinetech/grandine
   cd grandine
   git submodule update --init dedicated_executor eth2_libp2p
   

   The other submodules (grandine-snapshot-tests, slashing-protection-interchange-tests, hive) are only needed for tests.

2. To build a release, check out its tag. Skip this step to build the develop branch:

   git checkout VERSION
   git submodule update --init dedicated_executor eth2_libp2p
   

3. Build:

   make release
   

   The build takes several minutes. The binary is written to ./target/compact/grandine.

4. Install it:

   sudo cp ./target/compact/grandine /usr/local/bin/grandine
   grandine --version
   

What make release does

make release runs:

cargo build --profile compact --bin grandine --features default-networks --features tracing \
  --workspace --exclude zkvm_host --exclude zkvm_guest_risc0 --exclude c_grandine --exclude csharp_grandine

Use make release rather than typing this command. If you do run cargo yourself, keep the --workspace --exclude … options: without them, Cargo also builds workspace members that enable a second BLS backend, and the build fails with errors such as the name `AggregatePublicKey` is defined multiple times.

• The compact profile is the release profile without debug information. It produces a much smaller binary.
• default-networks includes the Mainnet, Sepolia, Holesky and Hoodi configurations. At least one network or preset feature is required; a plain cargo build without features stops with at least one preset must be enabled.
• tracing enables tracing support in the core crates. Build with make release DISABLE_TRACING=1 to leave it out.

Other useful targets:

Cryptography backends

Grandine selects its BLS and KZG libraries at compile time with Cargo features. The grandine crate enables blst for both by default, which is what the official binaries use.

The KZG library used at run time can be chosen with --kzg-backend from the backends compiled in. To compile in additional KZG backends, add their features. The Makefile has no option for extra features, so run the command that make release runs, with the feature added:

cargo build --profile compact --bin grandine --features default-networks,arkworks \
  --workspace --exclude zkvm_host --exclude zkvm_guest_risc0 --exclude c_grandine --exclude csharp_grandine

This keeps the default blst backend and adds arkworks. CI builds the workspace the same way with other backend combinations.

The available KZG backends are arkworks, blst, constantine, mcl and zkcrypto. The constantine backend also needs the Nim compiler and, on Linux, libgmp-dev and llvm. Asking for a backend at run time that was not compiled in fails with backend is not compiled - please specify feature flag when compiling grandine.

Note: When you build a single crate with cargo -p, select a backend yourself, for example -p bls --features bls/blst -p kzg_utils --features kzg_utils/blst.

Cross-compilation

Grandine uses cross to build Linux binaries for another architecture. cross runs the build in a container, so you need Docker or Podman. The container images and the extra packages they install are configured in Cross.toml.

Install cross:

cargo install cross --git https://github.com/cross-rs/cross

Build for x86-64 Linux:

make release TARGET=x86_64-unknown-linux-gnu

Build for ARM64 Linux, for example a Raspberry Pi 5 or an ARM server:

make release TARGET=aarch64-unknown-linux-gnu

The binaries are written to ./target/TARGET/compact/grandine. These targets also build libgrandine.so, the library used by the Nethermind bundle.

macOS (x86_64-apple-darwin, aarch64-apple-darwin) and Windows (x86_64-pc-windows-msvc) targets are built natively with Cargo on those systems, for example make aarch64-apple-darwin on an Apple silicon Mac.

Docker images

From a cross-compiled binary

This is how the official images are made. Build the binary with cross first, then package it with Dockerfile.cross, which copies it into an Ubuntu 24.04 image:

make release TARGET=x86_64-unknown-linux-gnu

docker buildx build \
  --file Dockerfile.cross \
  --platform linux/amd64 \
  --tag grandine:local \
  target/x86_64-unknown-linux-gnu/compact/

For ARM64, use TARGET=aarch64-unknown-linux-gnu, --platform linux/arm64 and target/aarch64-unknown-linux-gnu/compact/.

All in Docker

The Dockerfile in the repository root compiles Grandine inside a Rust container and copies the result into an Ubuntu image. Fetch the submodules first, because the build copies your working tree:

git submodule update --init dedicated_executor eth2_libp2p
docker build --tag grandine:local .

The builder stage starts from the rust:1.82.0-bullseye image, but rustup installs the toolchain pinned in rust-toolchain.toml when the build starts, so the Rust version in the image tag is not the one used. Keep in mind that:

• The build copies the whole working tree, including a local target directory if you have one, because the repository has no .dockerignore. Delete or move target first to keep the build context small.
• The final image is plain ubuntu:latest without the ca-certificates package, so HTTPS connections, for example to a checkpoint sync server, fail certificate checks. Add the package in your own image, or use the official sifrai/grandine image, which includes it.

Updating a source build

cd grandine
git fetch --tags
git checkout VERSION
git submodule update --init dedicated_executor eth2_libp2p
make release

Then restart Grandine with the new binary. See Updating Grandine.

Troubleshooting builds

• at least one preset must be enabled: you ran cargo build without features. Use make release or add --features default-networks.
• the name `AggregatePublicKey` is defined multiple times and similar errors in bls: you ran cargo build without the --workspace --exclude … options. Use make release.
• cannot open crtn.o or cannot open crt1.o while linking: the C library development files are missing. Install build-essential, or at least libc6-dev.
• 'stdarg.h' file not found in reth-mdbx-sys: Clang is missing or incomplete. Install clang.
• Errors about protoc or cmake: install protobuf-compiler and cmake.
• Errors in eth2_libp2p or dedicated_executor, such as missing files or functions that do not exist: the submodules are not checked out, or are at a different commit than the one this version of Grandine expects (for example after a git pull or a git checkout). Run git submodule update --init dedicated_executor eth2_libp2p.
• The build is killed: the machine ran out of memory. Close other programs, add swap, or limit parallel jobs with CARGO_BUILD_JOBS=2 make release.

For development workflows (tests, lints, spec tests), see Contributing and development.

Command line options

• --kzg-backend: default blst in the official builds. KZG library to use, from those compiled in.

See the Command line reference for the full list.

Running with Eth Docker

Eth Docker is a Docker Compose setup for Ethereum nodes, maintained by EthStaker. It supports Grandine in every mode Grandine offers: as a standalone beacon node or embedded in Nethermind, with the integrated validator client, with a separate grandine vc, or as a validator client on its own.

Eth Docker is the recommended way to deploy Grandine for most regular use cases, such as a solo staking node or a node that serves your own applications. It sets up the execution client, Grandine, MEV-Boost and key management together, and keeps them updated with one command. Run Grandine directly only when you need a setup that Eth Docker does not cover.

Note: Support for the separate grandine vc (grandine.yml, grandine-vc-only.yml, grandine-plugin-cl-only.yml, and the separate validator client in grandine-plugin.yml) comes from eth-docker#2832, which is not merged yet. Released Eth Docker versions offer grandine-allin1.yml, grandine-cl-only.yml, grandine-plugin-allin1.yml, and grandine-plugin.yml as a beacon node without a validator client.

Choose a mode

Eth Docker picks the clients from the Compose files listed in COMPOSE_FILE in its .env file. Each Grandine mode has its own file. In the table, EL is the execution client, BN the beacon node and VC the validator client:

The standalone files go together with the Compose file of any execution client, for example nethermind.yml or geth.yml. The plugin files run Grandine inside Nethermind, so they go only with nethermind.yml, and there is no separate consensus container. See How Grandine works for the differences between the integrated and the separate validator client, and Grandine embedded in Nethermind for the embedded setup.

Set up

Install Eth Docker and run its configuration wizard:

git clone https://github.com/ethstaker/eth-docker.git
cd eth-docker
./ethd install
./ethd config

The wizard asks for the network, the clients and the fee recipient, and writes them to .env. Choose Grandine as the consensus client, or as the validator client for a validator-only setup.

For a mode that the wizard does not offer, such as the Nethermind plugin files, set COMPOSE_FILE in .env yourself, for example:

COMPOSE_FILE=nethermind.yml:grandine-plugin.yml

With grandine-plugin.yml, Grandine serves the Beacon Node API from the Nethermind container, so point the validator client at it with CL_NODE=http://execution:5052.

Start the node and follow the logs:

./ethd up
./ethd logs -f consensus

With the plugin files, Grandine logs as part of the execution service.

Storage mode

CL_NODE_TYPE in .env selects the storage mode:

Eth Docker's blob-archive node type is not supported for Grandine, and is not needed. For other clients, archive still prunes blob data and blob-archive keeps it. Grandine in archive mode never prunes blob sidecars or data columns, so for Grandine archive already keeps all blob data. Two limits apply to every client:

• Peers serve blob data only for the data availability window of about 18 days, so the node keeps blob data from about 18 days before it started onwards. Older blob data cannot be back-synced.
• After PeerDAS, a node stores only the data columns it custodies. To keep all blob data, also run it as a supernode with CL_EXTRAS=--supernode.

Changing the node type usually requires a resync. See Back-sync and archive nodes.

Validator keys

Import keys with:

./ethd keys import

Eth Docker imports them through the Keymanager API of the validator client: the beacon node in the integrated modes, and grandine vc in the separate ones. ./ethd keys list shows the loaded keys. Grandine writes each imported keystore to its own file in the validator directory, with its password in a plain file under secrets/. The wallet password that Eth Docker generates (./ethd get-grandine-wallet prints it) only unlocks keys that older Grandine versions kept in the legacy keystores.json storage. See Managing validator keys.

DOPPELGANGER=true enables doppelganger protection, WEB3SIGNER=true uses Web3Signer for the keys, and GRAFFITI sets the graffiti.

Switching modes

The validator keys and slashing protection data live in a Docker volume that depends on where the validator client runs:

• The integrated validator client (grandine-allin1.yml, grandine-plugin-allin1.yml) keeps them in the beacon node's volume.
• The separate validator client (grandine.yml, grandine-plugin.yml, grandine-vc-only.yml) keeps them in its own volume.

Switching between files of the same group keeps the validator data, so you do not need to import the keys again. Switching between the groups, for example from grandine-allin1.yml to grandine.yml, starts the new validator client with no keys. Move them like this:

1. Run ./ethd keys delete all. This removes the keys from the old validator client and exports their slashing protection data to .eth/validator_keys.
2. Change COMPOSE_FILE and run ./ethd up.
3. Put the keystore files into .eth/validator_keys next to the exported slashing protection data, and run ./ethd keys import.

Eth Docker recommends treating this like a move to another machine, with about 15 minutes of downtime. Never run the same keys in two validator clients at once. See Slashing protection.

Extra options

CL_EXTRAS passes extra options to the beacon node, and VC_EXTRAS to the validator client. For example:

CL_EXTRAS=--target-peers 100
VC_EXTRAS=--publish-to-every-node attestations,blocks

See the Command line reference for the options, and Redundancy and fallback beacon nodes for a validator client that uses several beacon nodes, set as a comma-separated list in CL_NODE.

Updating

./ethd update
./ethd up

This updates Eth Docker and the client images. See Updating Grandine for what to check before you update.

Running as a service

A node should start on boot, restart after a crash and run without a terminal. This page shows two ways to do that: a systemd service for the binary, and Docker Compose for containers.

Both examples run on Mainnet. For Hoodi, replace mainnet with hoodi and use a Hoodi checkpoint sync endpoint.

systemd

1. Create a dedicated user

Run Grandine as its own unprivileged user, not as root and not as your login user. If the node is ever compromised, the damage stays inside that user's files.

sudo useradd --system --no-create-home --shell /usr/sbin/nologin grandine
sudo mkdir -p /var/lib/grandine
sudo chown grandine:grandine /var/lib/grandine
sudo chmod 700 /var/lib/grandine

Grandine appends the network name to --data-dir, so with --data-dir /var/lib/grandine the Mainnet data lives in /var/lib/grandine/mainnet.

2. Share the JWT secret

The execution client and Grandine must both read the JWT secret. If they run as different users, put the secret in a group that both belong to. This example assumes the execution client runs as the user geth:

sudo groupadd --system ethereum
sudo usermod -aG ethereum grandine
sudo usermod -aG ethereum geth

sudo mkdir -p /var/lib/ethereum
openssl rand -hex 32 | tr -d "\n" | sudo tee /var/lib/ethereum/jwt.hex > /dev/null
sudo chown root:ethereum /var/lib/ethereum/jwt.hex
sudo chmod 640 /var/lib/ethereum/jwt.hex

3. Write the unit file

Create /etc/systemd/system/grandine.service:

[Unit]
Description=Grandine Ethereum consensus client
Documentation=https://docs.grandine.io/
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=grandine
Group=grandine
ExecStart=/usr/local/bin/grandine \
  --network mainnet \
  --data-dir /var/lib/grandine \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /var/lib/ethereum/jwt.hex \
  --checkpoint-sync-url CHECKPOINT-URL

# Restart after a crash, but not in a tight loop.
Restart=on-failure
RestartSec=10

# Grandine saves its chain on SIGTERM. Give it time before systemd kills it.
TimeoutStopSec=300

LimitNOFILE=65536

# Hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ProtectHome=true
ReadWritePaths=/var/lib/grandine

[Install]
WantedBy=multi-user.target

The ProtectHome and ReadWritePaths settings stop Grandine from writing outside its data directory. If you keep keystores or other files elsewhere, make sure the grandine user can read them.

Note: --checkpoint-sync-url is only used when the database is empty, so it is safe to keep it in the unit. On later starts Grandine logs skipping checkpoint sync: existing database found and continues from its own data.

4. Start and enable the service

sudo systemctl daemon-reload
sudo systemctl enable --now grandine

Check that it runs:

sudo systemctl status grandine

Follow the logs:

sudo journalctl -fu grandine

Show the logs since the last boot, or for a time range:

sudo journalctl -u grandine -b
sudo journalctl -u grandine --since "1 hour ago"

journald rotates the logs for you. To limit how much disk they use, set SystemMaxUse= in /etc/systemd/journald.conf. See Logging for log levels.

Keep the options in a file

Long command lines are hard to maintain. --args-file loads options from a YAML file, with long option names as keys. Options given on the command line win over the file.

Create /etc/grandine/mainnet.yaml:

network: mainnet
data-dir: /var/lib/grandine
eth1-rpc-urls:
  - http://127.0.0.1:8551
jwt-secret: /var/lib/ethereum/jwt.hex
checkpoint-sync-url: CHECKPOINT-URL
metrics: true

A true value turns on a flag, false leaves it off, and a list repeats the option once per value. Then use:

ExecStart=/usr/local/bin/grandine --args-file /etc/grandine/mainnet.yaml

Adding validators

To run the integrated validator client, add your keystore options to ExecStart or to the args file, for example:

keystore-dir: /var/lib/grandine/keys
keystore-password-file: /var/lib/grandine/secrets/password.txt
suggested-fee-recipient: "FEE-RECIPIENT-ADDRESS"

Quote execution addresses in YAML so they stay strings. Make the keystores and password files readable only by the grandine user:

sudo chown -R grandine:grandine /var/lib/grandine/keys /var/lib/grandine/secrets
sudo chmod 700 /var/lib/grandine/keys /var/lib/grandine/secrets
sudo chmod 600 /var/lib/grandine/keys/* /var/lib/grandine/secrets/*

See Validator client and Managing validator keys.

A separate validator client

To run the validator client as its own service, for example against a beacon node on another machine, create a second unit with its own user and data directory:

[Unit]
Description=Grandine validator client
Wants=network-online.target
After=network-online.target

[Service]
Type=simple
User=grandine-vc
Group=grandine-vc
ExecStart=/usr/local/bin/grandine vc \
  --network mainnet \
  --data-dir /var/lib/grandine-vc \
  --beacon-node-urls http://BEACON-NODE-HOST:5052 \
  --keystore-dir /var/lib/grandine-vc/keys \
  --keystore-password-file /var/lib/grandine-vc/secrets/password.txt \
  --suggested-fee-recipient FEE-RECIPIENT-ADDRESS
Restart=on-failure
RestartSec=10
TimeoutStopSec=120
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ProtectHome=true
ReadWritePaths=/var/lib/grandine-vc

[Install]
WantedBy=multi-user.target

The options come after vc. The beacon node behind --beacon-node-urls can be grandine bn or any other consensus client.

Warning: Never run the same keys in the beacon node's integrated validator client and in a separate grandine vc, and never give two validator clients the same data directory. Either can get your validators slashed. If you move keys from the beacon node to a separate validator client, stop the beacon node, remove the keys from it, export the slashing protection data, import it into the new validator client and only then start it. See Slashing protection.

Exit codes and restarts

Grandine exits with status 1 when it fails and 2 when the command line is invalid. Many run-time errors do not reach systemd at all: Grandine restarts its runtime inside the process and logs application runtime failed. Errors that stop the process include a detected doppelganger. With Restart=on-failure, systemd starts Grandine again after RestartSec, and doppelganger protection runs again on the next start.

Docker Compose

The Compose file below runs Geth and Grandine on Mainnet. The same pattern works for other execution clients.

1. Prepare the directory

mkdir -p ~/eth-node/jwt
cd ~/eth-node
openssl rand -hex 32 | tr -d "\n" > jwt/jwt.hex

2. Write docker-compose.yml

services:
  geth:
    image: ethereum/client-go:stable
    restart: unless-stopped
    stop_grace_period: 3m
    command:
      - --mainnet
      - --datadir=/data
      - --authrpc.addr=0.0.0.0
      - --authrpc.port=8551
      - --authrpc.vhosts=*
      - --authrpc.jwtsecret=/jwt/jwt.hex
    ports:
      - "30303:30303/tcp"
      - "30303:30303/udp"
    volumes:
      - ./geth-data:/data
      - ./jwt:/jwt:ro

  grandine:
    image: sifrai/grandine:stable
    restart: unless-stopped
    stop_grace_period: 2m
    depends_on:
      - geth
    command:
      - --network=mainnet
      - --eth1-rpc-urls=http://geth:8551
      - --jwt-secret=/jwt/jwt.hex
      - --checkpoint-sync-url=CHECKPOINT-URL
      - --http-address=0.0.0.0
      - --metrics
      - --metrics-address=0.0.0.0
    ports:
      - "9000:9000/tcp"
      - "9000:9000/udp"
      - "9001:9001/udp"
      - "127.0.0.1:5052:5052"
      - "127.0.0.1:5054:5054"
    volumes:
      - ./grandine-data:/root/.grandine
      - ./jwt:/jwt:ro

Replace CHECKPOINT-URL with a Mainnet checkpoint sync endpoint.

About this file:

• restart: unless-stopped restarts a container after a crash or a reboot, unless you stopped it.
• stop_grace_period gives each client time to shut down cleanly. Docker's default is 10 seconds, which is too short.
• The Engine API (8551) is reachable only inside the Compose network, not from the host.
• --http-address=0.0.0.0 makes the Beacon Node API listen inside the container. The port mapping 127.0.0.1:5052:5052 publishes it on the host's loopback interface only. Do the same for the metrics port 5054.
• The peer-to-peer ports 9000 (TCP and UDP) and 9001 (UDP) are published on all interfaces so that peers can reach the node.
• ./grandine-data holds Grandine's data in ./grandine-data/mainnet.

3. Start the stack

docker compose up -d
docker compose logs -f grandine

Check the sync status from the host:

curl -s http://127.0.0.1:5052/eth/v1/node/syncing

Adding validators in Compose

Mount the keystores and the password file read-only and add the options:

    command:
      # ... the options above, plus:
      - --keystore-dir=/keys
      - --keystore-password-file=/secrets/password.txt
      - --suggested-fee-recipient=FEE-RECIPIENT-ADDRESS
    volumes:
      - ./grandine-data:/root/.grandine
      - ./jwt:/jwt:ro
      - ./validator_keys:/keys:ro
      - ./secrets:/secrets:ro

The slashing protection database lives in ./grandine-data/mainnet/validator. Back it up along with the rest of the data directory; see Slashing protection.

To update the images, see Updating Grandine.

Command line options

• --args-file: no default. YAML file with options, keyed by long option name.
• --data-dir: default $HOME/.grandine. Parent directory for data; the network name is appended.
• --http-address: default 127.0.0.1. Address the Beacon Node API listens on.
• --http-port: default 5052. Port of the Beacon Node API.
• --metrics: default disabled. Serve Prometheus metrics.
• --metrics-address: default 127.0.0.1. Address of the metrics endpoint.
• --metrics-port: default 5054. Port of the metrics endpoint.
• --beacon-node-urls (vc only): required. Beacon Node API URLs for the separate validator client.

See the Command line reference for the full list.

Updating Grandine

Keep Grandine up to date. New releases fix bugs, improve performance and, before every network upgrade (hard fork), add the rules of the new fork. A node that is not updated before a hard fork follows the wrong chain from the fork epoch on, and its validators miss all duties.

Release channels

GitHub releases are drafted by the release workflow and published by the team with release notes. Read the notes before you update: they say whether a release is required for a hard fork and whether it changes any options.

Release notifications

• On GitHub, choose Watch, then Custom, then Releases to get an email for every release.
• Follow the announcements on the Grandine Discord or Grandine Telegram.

How urgent is an update?

Decide when to update from the release notes:

Tip: Update during a quiet period. Use the Beacon Node API or a block explorer to check that none of your validators has a block proposal in the next few minutes. A restart usually costs at most one or two missed attestations.

Check the version you run

The binary prints its version:

grandine --version

Grandine 3.0.0

A running node reports its version, with the commit and platform, through the API:

curl -s http://localhost:5052/eth/v1/node/version

{"data":{"version":"Grandine/3.0.0-6a37d7fa/x86_64-linux"}}

/eth/v2/node/version also reports the version of the connected execution client.

Update a binary

1. Download the new binary, or build it from source. See Pre-built binaries and Docker.

2. Stop Grandine and wait for it to exit. Grandine saves its chain on shutdown and logs saving current chain before exit…:

   sudo systemctl stop grandine
   

3. Replace the binary:

   sudo install -m 0755 grandine-VERSION-linux-x64 /usr/local/bin/grandine
   grandine --version
   

4. Start Grandine again and watch the logs:

   sudo systemctl start grandine
   sudo journalctl -fu grandine
   

To keep the downtime short, download the new binary before you stop the old one. Keep the old binary until you are sure the new one works.

Update a Docker container

If you use the stable tag, pull it again. If you pinned a version tag, change it to the new version first:

docker pull sifrai/grandine:stable
docker stop grandine && docker rm grandine
# Start the container again with the same options.

With Docker Compose, change the image tag in docker-compose.yml, then run:

docker compose pull grandine
docker compose up -d grandine

Docker stops the container with SIGTERM and kills it after a timeout. Give Grandine enough time to save its state, for example with stop_grace_period: 2m in Compose or docker stop -t 120. See Running as a service.

Database compatibility

Grandine records the format of its data directory in a meta.json file directly in the network's data directory, for example ~/.grandine/mainnet/meta.json:

{
  "application": "Grandine",
  "schema_version": "0.2.3"
}

On every start, Grandine reads this file and:

• refuses to start if the directory belongs to a different application;
• accepts any schema version: the compatibility check against 0.2.0 is currently skipped, so even data from before the move from RocksDB to libmdbx is not rejected;
• updates the recorded schema version to its own if it differs, and logs using schema version 0.2.3 for new data.

Upgrades within the same schema series need no migration and no resync. Grandine reads data written by older versions and writes new data in the current format. The slashing protection database (validator/slashing_protection.sqlite) is migrated forward automatically on start.

If a release ever needs a resync or a manual migration, its release notes say so.

Downgrading

Downgrading is sometimes needed when a new release has a problem. Grandine does not block a downgrade within the same schema series, but an older version may not understand data that a newer version wrote. Plan for it:

1. Before you upgrade, export the slashing protection data. This is cheap insurance for any change:

   sudo systemctl stop grandine
   grandine --network mainnet interchange export slashing-protection-backup.json
   

   Pass the same --data-dir as the node uses if it is not the default. See Slashing protection.

2. To downgrade, stop Grandine, install the older binary or image, and start it.

3. If the older version fails to load the database, resync the beacon database. The fastest way is a fresh checkpoint sync:

   grandine --network mainnet \
     --eth1-rpc-urls http://127.0.0.1:8551 \
     --jwt-secret /path/to/jwt.hex \
     --checkpoint-sync-url CHECKPOINT-URL \
     --force-reset-beacon-db
   

   --force-reset-beacon-db deletes the eth1_cache, beacon_fork_choice, pubkey_cache and sync databases on startup. It does not touch the validator directory or the slashing protection database. Remove the option after this start, or Grandine resets the database on every start.

4. If the slashing protection database cannot be opened by the older version, move it aside and import the export from step 1 before you start validating. The slashing protection database has versioned schema migrations, and Grandine refuses to open a database that has a migration it does not know. So once a newer version adds a migration and applies it, older versions fail to start with that database:

   grandine --network mainnet interchange import slashing-protection-backup.json
   

Warning: You cannot downgrade to a version that does not support the current fork of the network. After a hard fork, only versions that include it can follow the chain.

Command line options

• --version: print the version and exit.
• --force-reset-beacon-db: default disabled. Delete the local beacon databases on startup for a fresh sync.
• --checkpoint-sync-url: no default. Beacon Node API URL to load a recent finalized checkpoint from.
• --data-dir: default $HOME/.grandine. Parent directory for data; the network name is appended.

See the Command line reference for the full list.

Running a beacon node

The beacon node is the part of Grandine that follows the Ethereum consensus layer. It connects to other nodes over the peer-to-peer network, downloads and verifies blocks, runs fork choice, drives an execution client through the Engine API, and serves the Beacon Node API. Validators, whether built in or remote, perform their duties through it.

This page explains how to start the beacon node, what it does while it starts, how to read its logs while it syncs, and how it shuts down. For the bigger picture, see How Grandine works.

Ways to run Grandine

One binary covers three modes:

If you do not run validators on this machine, grandine and grandine bn behave almost the same. Use grandine bn when you want to make it explicit that the process must never load keys, for example on a node that serves only an API or backs a separate validator client.

Note: With bn and vc, every option goes after the subcommand. grandine bn --network hoodi works. grandine --network hoodi bn fails with --network must be specified after bn.

Before you start

You need:

• An execution client (Geth, Nethermind, Besu, Erigon, Reth or another Engine API client) running on the same network. See Connecting an execution client.
• A JWT secret file shared by both clients.
• A checkpoint sync URL, so the node does not have to sync from genesis. See Checkpoint sync.
• Open peer-to-peer ports. See Networking and peers.

Check the System requirements before you start on mainnet. Practice on the Hoodi testnet first.

Start the beacon node

Mainnet:

grandine bn \
  --network mainnet \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /path/to/jwt.hex \
  --checkpoint-sync-url CHECKPOINT-SYNC-URL

Hoodi:

grandine bn \
  --network hoodi \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /path/to/jwt.hex \
  --checkpoint-sync-url CHECKPOINT-SYNC-URL

--network defaults to mainnet. The other built-in networks are sepolia, holesky and hoodi. custom connects to a network defined by your own configuration files (see Configuration files and custom networks).

With Docker, the image's entrypoint is grandine, so the arguments follow the image name. The data directory inside the container is /root/.grandine:

docker run -d --name grandine \
  -p 9000:9000/tcp -p 9000:9000/udp -p 9001:9001/udp \
  -v /path/to/grandine-data:/root/.grandine \
  -v /path/to/jwt.hex:/jwt.hex:ro \
  sifrai/grandine:stable \
  bn \
  --network mainnet \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /jwt.hex \
  --checkpoint-sync-url CHECKPOINT-SYNC-URL \
  --http-address 0.0.0.0

Warning: --http-address 0.0.0.0 makes the Beacon Node API listen on every interface of the container. Publish port 5052 only to hosts you trust. See Security best practices.

To run Grandine as a long-lived service, see Running as a service.

Keeping options in a file

--args-file loads options from a YAML file. Keys are the long option names without the leading dashes:

network: mainnet
eth1-rpc-urls:
  - http://127.0.0.1:8551
jwt-secret: /path/to/jwt.hex
checkpoint-sync-url: CHECKPOINT-SYNC-URL

grandine bn --args-file /path/to/grandine.yaml

Options given on the command line take precedence over the same options in the file.

What happens at startup

When the beacon node starts, it goes through these steps. Each step logs a line at the INFO level, so you can follow them in the output.

1. Configuration report. Grandine logs the client version (client version: ...), followed by the network, the data directory, the storage mode, the database size limits, the Eth1 RPC URLs, the HTTP API address and whether back-sync is enabled.
2. Port check. Grandine checks that its TCP and UDP ports are free before it binds them. If one is taken, it exits with an error such as libp2p port (9000) is unavailable; make sure no other instance of the application is running or specify a different port with --libp2p-port.
3. Data directory. Grandine creates the data directory if needed and records the database schema version in meta.json (initialized data directory with schema version ...). See Storage and database.
4. Validator keys. In the combined mode, Grandine loads any validator keys you passed. If it finds keys but no --eth1-rpc-urls, it stops with --eth1-rpc-urls must be specified when validators are present.
5. Genesis. Grandine needs the genesis state of the network. It is built into the binary for mainnet. For Sepolia, Holesky and Hoodi, Grandine downloads it on the first run (downloading genesis state from ...) and saves it as genesis_state.ssz in the beacon directory, so later starts load it from disk.
6. Anchor state. Grandine picks the state to start from:
   • If the database already holds a finalized state, it loads it (loaded state at slot N).
   • Otherwise, if you passed --checkpoint-sync-url, it downloads a recent finalized block and state (performing checkpoint sync from ..., then loaded state at slot N from ...).
   • Otherwise, it starts from genesis.
7. Services. Grandine starts the execution client connection (updated capabilities for eth1 endpoint: ...), the peer-to-peer stack (Libp2p Starting, ENR Initialised, Listening established), the HTTP API (HTTP server listening on 127.0.0.1:5052), the operation pools and the sync service.
8. Sync. The node requests blocks from peers and imports them until it reaches the head of the chain.

Reading the logs

Each log line starts with a timestamp, the level and the module that emitted it. Most lines end with a peers=[CONNECTED/TARGET] field: the number of connected peers and the --target-peers value. A node that shows peers=[0/200] for minutes cannot sync. See Networking and peers.

While syncing

Grandine logs a line for every block it imports, with the number of blobs in the block. Shown here without the timestamp prefix:

imported beacon block with 7 blobs (slot: 12345678, 0x5c1e…a9f2) peers=[87/200]
imported beacon block with 0 blobs (slot: 12345679, 0x8b07…41d3) peers=[87/200]

Watch the slot value. It grows quickly while the node catches up and then advances by one slot every 12 seconds once the node is at the head. To also log how long each block takes to process, start Grandine with --features LogBlockProcessingTime; see Runtime features.

While the execution client is still syncing, Grandine imports blocks before the execution client has confirmed their payloads. The head is then optimistic, which the sync status shows as "is_optimistic": true (see below). This is normal until the execution client catches up. See Optimistic sync.

On each new finalized epoch, Grandine logs:

new finalized checkpoint (epoch: 385802, root: 0x…, head slot: 12345700, head root: 0x…)

As the node syncs, it periodically writes finalized data to disk (saving checkpoint block & state in slot N, saving state in slot N).

Checking sync status through the API

Grandine does not print a periodic sync summary. To check the status at any time, ask the Beacon Node API:

curl -s http://127.0.0.1:5052/eth/v1/node/syncing

{"data":{"head_slot":"12345700","sync_distance":"0","is_syncing":false,"is_optimistic":false,"el_offline":false}}

• is_syncing: false and sync_distance: 0 mean the node is at the head.
• is_optimistic: true means the execution client has not yet verified the head. The node is not ready for validator duties.
• el_offline: true means no execution client endpoint is answering.

/eth/v1/node/health returns HTTP 200 when the node is synced and 206 while it is syncing, which is convenient for load balancers and monitoring:

curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5052/eth/v1/node/health

For long-term monitoring, see Metrics and Logging.

Warnings worth acting on

See Troubleshooting for more.

Automatic restarts

If the beacon node stops with an error or panics, Grandine logs application runtime failed (or application runtime panicked) and restarts its runtime inside the same process, without exiting. When the error is a full database (MapFull), Grandine first doubles the database size limits (increasing environment map size limits). A few errors end the process instead, such as doppelgangers detected by doppelganger protection.

To make Grandine exit on the first error instead, for example so that systemd or Docker restarts it, enable the InhibitApplicationRestart runtime feature:

grandine bn --features InhibitApplicationRestart ...

Shutting down

Grandine stops cleanly on SIGINT (Ctrl+C) and SIGTERM (sent by systemctl stop and docker stop). It then:

1. stops the slot clock and the services,
2. logs saving current chain before exit…,
3. writes the finalized chain and the unfinalized blocks of the canonical chain to the database, and logs chain saved (finalized blocks: N, unfinalized blocks: M).

Most of the shutdown time goes into this last step: Grandine writes the blocks not yet in the database and, usually, a finalized checkpoint state, in one database transaction. It takes longer when the chain has not finalized for a long time and many blocks are waiting to be written.

On the next start, Grandine resumes from the saved state and only has to sync the slots it missed.

Warning: Do not stop Grandine with SIGKILL (kill -9) unless it hangs. A killed node loses the unfinalized part of the chain and has to fetch it again. Give the service enough time to stop: docker stop waits only 10 seconds by default, so use docker stop -t 120 grandine or set stop_grace_period in Docker Compose. 120 seconds is a generous allowance, not a measured shutdown time; allow more after a long period without finality.

Command line options

The most important beacon node options. See Command line reference for the full list.

• --network (default: mainnet): network to join: mainnet, sepolia, holesky, hoodi or custom.
• --data-dir (default: $HOME/.grandine): parent directory for all data. Grandine adds the network name below it.
• --eth1-rpc-urls (default: none): Engine API URLs of your execution clients.
• --jwt-secret (default: none): path to the JWT secret shared with the execution client.
• --checkpoint-sync-url (default: none): Beacon Node API URL to checkpoint sync from.
• --args-file (default: none): load options from a YAML file.
• --http-address (default: 127.0.0.1): Beacon Node API listen address.
• --http-port (default: 5052): Beacon Node API port.
• --disable-http-api (default: off): run without the Beacon Node API.
• --target-peers (default: 200): number of peers to aim for.
• --features (default: none): comma-separated runtime features.
• --in-memory (default: off): keep everything in memory and write nothing to disk. All data is lost on exit.

Connecting an execution client

Since the Merge, every Ethereum node is a pair: a consensus client such as Grandine and an execution client such as Geth, Nethermind, Besu, Erigon or Reth. The execution client holds the account state and executes transactions. Grandine drives it through the authenticated Engine API: it sends every new execution payload for validation, tells the execution client which block is the head, and asks it to build payloads when a local validator proposes a block.

A beacon node without an execution client cannot verify execution payloads, so it cannot follow the chain safely or serve validators. Run both clients on the same network, and keep both up to date.

Tip: For the fastest communication between the two layers, Grandine can also run embedded in Nethermind. It then calls Nethermind's Engine API directly, with no HTTP, JWT or JSON in between. See Grandine embedded in Nethermind.

Step 1: Create a JWT secret

The Engine API is protected by a shared secret. Both clients read the same file and use it to sign and check JSON Web Tokens (JWT). Create a random 32-byte secret in hex:

sudo mkdir -p /var/lib/ethereum
openssl rand -hex 32 | tr -d '\n' | sudo tee /var/lib/ethereum/jwt.hex > /dev/null
sudo chmod 640 /var/lib/ethereum/jwt.hex

Grandine accepts the hex string with or without a 0x prefix and with or without a trailing newline. The secret must be at least 32 bytes long, otherwise Grandine fails with JWT secret must be at least 32 bytes.

Tip: Some execution clients create the file themselves on first start if it does not exist. Creating it yourself makes the setup explicit and lets you choose the path and permissions.

Step 2: Start the execution client

Enable the Engine API (port 8551 by convention) and point it at the secret. For Geth:

geth --mainnet \
  --datadir /var/lib/geth \
  --authrpc.addr 127.0.0.1 \
  --authrpc.port 8551 \
  --authrpc.jwtsecret /var/lib/ethereum/jwt.hex

Other execution clients have equivalent options, for example --JsonRpc.JwtSecretFile in Nethermind, --engine-jwt-secret in Besu and --authrpc.jwtsecret in Reth and Erigon. Check your execution client's documentation for the exact names.

Step 3: Point Grandine at the execution client

grandine bn \
  --network mainnet \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /var/lib/ethereum/jwt.hex \
  --checkpoint-sync-url CHECKPOINT-SYNC-URL

When the connection works, Grandine logs:

updated capabilities for eth1 endpoint: http://127.0.0.1:8551/
updated client version for eth1 endpoint: http://127.0.0.1:8551/

The first line means the execution client accepted the JWT and answered engine_exchangeCapabilities. Grandine repeats this exchange at the start of every epoch.

You can also see which execution client Grandine is talking to through the API:

curl -s http://127.0.0.1:5052/eth/v2/node/version

Docker

When both clients run in Docker, put them on the same Docker network and use the container name as the host:

docker network create eth

docker run -d --name grandine --network eth \
  -p 9000:9000/tcp -p 9000:9000/udp -p 9001:9001/udp \
  -v /path/to/grandine-data:/root/.grandine \
  -v /var/lib/ethereum/jwt.hex:/jwt.hex:ro \
  sifrai/grandine:stable \
  bn \
  --network mainnet \
  --eth1-rpc-urls http://EL-CONTAINER-NAME:8551 \
  --jwt-secret /jwt.hex \
  --checkpoint-sync-url CHECKPOINT-SYNC-URL

The execution client must then listen on an address the Grandine container can reach (for Geth, --authrpc.addr 0.0.0.0 with --authrpc.vhosts EL-CONTAINER-NAME or *). Do not publish port 8551 to the host or the internet. See Running as a service for a complete Docker Compose file.

JWT claims

Grandine creates a fresh token for each Engine API request. The token carries the issue time and expires after 60 seconds, so the clocks of both machines must be in sync. Run NTP or systemd-timesyncd.

Two optional claims help an execution client tell several consensus clients apart:

• --jwt-id: the id claim, a unique identifier for this consensus client.
• --jwt-version: the clv claim, the client type and version.

Most setups do not need them.

Multiple execution clients and fallback

--eth1-rpc-urls accepts several URLs, separated by spaces:

grandine bn \
  --eth1-rpc-urls http://127.0.0.1:8551 http://EL-HOST-2:8551 \
  --jwt-secret /var/lib/ethereum/jwt.hex \
  ...

The first URL is the primary endpoint and the others are fallbacks, in order. The same JWT secret is used for all of them. Grandine handles them like this:

• Every request goes to the first endpoint that is marked online. If it fails, Grandine marks it offline, logs Eth1 RPC endpoint ... returned an error: ...; switching to ... and tries the next one.
• If every endpoint fails, it logs last available Eth1 RPC endpoint ... returned an error and the request fails (all Eth1 RPC endpoints exhausted).
• At the start of every epoch, Grandine checks all endpoints with engine_exchangeCapabilities. An endpoint that answers is marked online again, so traffic returns to the primary once it recovers.
• Some calls, such as engine_getBlobsV2, go only to endpoints that advertised support for them.

Warning: Grandine sends Engine API calls only to the endpoint it is currently using. It does not keep the fallback execution clients up to date. A fallback that has no consensus client of its own falls behind and is still syncing when Grandine switches to it. Make sure each fallback execution client is kept synced by another beacon node.

If you need full redundancy, run complete pairs of beacon node and execution client. See Redundancy and fallback beacon nodes.

Optimistic sync

The execution client usually needs longer to sync than Grandine, especially after a checkpoint sync. While the execution client is syncing, it answers engine_newPayload and engine_forkchoiceUpdated with SYNCING or ACCEPTED instead of VALID. Grandine then imports the blocks optimistically: it follows the chain without confirmation that the payloads are valid. You will see warnings like:

engine_newPayload returned SYNCING status (beacon_block_root: 0x…, block_number: …, block_hash: 0x…, response: …)
engine_forkchoiceUpdated returned SYNCING status (…)

These warnings are expected while the execution client syncs. They stop once it catches up.

While the head is optimistic:

• GET /eth/v1/node/syncing reports "is_optimistic": true, and API responses carry execution_optimistic: true.
• Validators do not sign attestations, blocks or sync committee messages based on an optimistic head. The built-in validator client waits briefly for the head to be fully validated and otherwise skips the duty with validator cannot perform its duties because chain head has not been fully verified by an execution engine. A separate validator client ignores heads that a beacon node reports as optimistic.
• Shortly before each interval of a slot, Grandine sends the head payload to the execution client again, so the head becomes fully validated as soon as the execution client can verify it.

If the execution client later reports a payload as INVALID, Grandine logs engine_newPayload returned INVALID status, marks the payload and all its descendants as invalid, and fork choice moves the head to a valid branch.

Running without an execution client

Grandine refuses to start with validator keys and no --eth1-rpc-urls (--eth1-rpc-urls must be specified when validators are present). Without validators, a beacon node can start without an execution client on a built-in network, or on a custom network when it is given a genesis state with --genesis-state-file. A custom network without either stops with --eth1-rpc-urls must be specified when connecting to custom network without --genesis-state-file.

Such a node still imports blocks and follows the head, but it cannot verify any execution payload, so every block it imports stays optimistic. Engine API calls fail and are logged, for example engine_newPayload call failed: attempted to call Eth1 RPC endpoint but none were provided. Use this only for testing.

Clients differ here. Nimbus also has an optimistic mode without an execution client, while Lighthouse does not start without --execution-endpoint. If you move from Lighthouse, a missing --eth1-rpc-urls is therefore not reported at startup unless validators are loaded.

What Grandine uses the Engine API for

Grandine picks the method version that matches the fork of the block.

By default, Grandine stores finalized blocks without their execution payloads and fetches the payloads from the execution client when it serves full blocks through the API or to peers. The execution client therefore has to stay available for that, and must not prune the bodies of blocks the beacon node serves. To keep payloads in Grandine's own database instead, use --store-payloads. See Storage and database.

Deposit contract and custom genesis

Grandine reads the deposit contract through the execution client only when it has to build the genesis state of a custom network from deposits, that is, on a custom network started without --genesis-state-file. The related options are --deposit-contract-starting-block and --eth1-database-size. Built-in networks do not need them. See Configuration files and custom networks.

Troubleshooting

See Troubleshooting for more.

Command line options

See Command line reference for the full list.

• --eth1-rpc-urls (default: none): one or more Engine API URLs, separated by spaces. The first is the primary, the rest are fallbacks.
• --jwt-secret (default: none): path to the hex-encoded JWT secret.
• --jwt-id (default: none): id claim sent in the JWT.
• --jwt-version (default: none): clv claim sent in the JWT.
• --store-payloads (default: off): keep execution payloads of finalized blocks in Grandine's database.
• --request-timeout (default: 30000): default timeout in milliseconds for outgoing HTTP requests. The Engine API calls above use their own timeouts.
• --deposit-contract-starting-block (default: from the network configuration): block to start reading the deposit contract from when building a custom genesis.
• --eth1-database-size (default: 16 GiB): size limit of the deposit cache database.
• --terminal-total-difficulty-override, --terminal-block-hash-override, --terminal-block-hash-activation-epoch-override (default: none): override the Merge transition parameters. Only relevant for networks that have not gone through the Merge.

Grandine embedded in Nethermind

Grandine can run embedded inside the Nethermind execution client. One process then runs both the execution layer and the consensus layer, and the two talk to each other directly, without the network hop that normally sits between them. This setup is for nodes that need the fastest possible communication between the execution and consensus clients.

Why embed Grandine in Nethermind

In a standard setup, the consensus client and the execution client are separate processes. They talk over the Engine API: every call is an HTTP request, authenticated with a JWT, with its payload encoded as JSON and decoded again on the other side. Execution payloads and blobs are large, so this encoding work is on the critical path of every block that is imported or proposed.

Standard setup:

+---------------------+                         +----------------------+
| Grandine process    |   HTTP + JWT + JSON     | Execution client     |
|                     | <---------------------> | process              |
|  beacon node        |   Engine API, port 8551 |  (Nethermind, Geth,  |
|                     |                         |   Besu, ...)         |
+---------------------+                         +----------------------+

With the Nethermind integration, Grandine is loaded into the Nethermind process as a plugin and calls Nethermind's Engine API implementation directly, as native function calls.

Embedded setup:

+--------------------------------------------------------------+
| Nethermind process                                           |
|                                                              |
|  +----------------------+               +-----------------+  |
|  | Grandine             |  direct calls |  Nethermind     |  |
|  |  beacon node         | <-----------> |  Engine API     |  |
|  |  (plugin)            |               |  execution      |  |
|  +----------------------+               +-----------------+  |
|                                                              |
+--------------------------------------------------------------+

This removes:

• the HTTP round trip on engine_newPayload, engine_forkchoiceUpdated, engine_getPayload, engine_getBlobs and the other Engine API methods;
• JSON encoding and decoding of execution payloads, blobs and proofs;
• the JWT secret, and the risk of a misconfigured or mismatched Engine API endpoint;
• a second process to install, update and supervise.

Grandine still runs its own beacon node with peer-to-peer networking, the Beacon Node API, metrics and, if you pass validator keys, the built-in validator client. Only the link to the execution client changes.

How it works

The integration has two parts, both placed in Nethermind's plugins directory:

• libgrandine.so (libgrandine.dylib on macOS, grandine.dll on Windows): Grandine built as a native library. It is built from the bindings/c crate of the Grandine repository.
• Grandine.NethermindPlugin.dll: a Nethermind plugin that starts Grandine when Nethermind starts and connects it to Nethermind's Engine API module. It is built from bindings/csharp.

When the plugin is enabled, Nethermind starts Grandine after it has registered its RPC modules, passes it the configured options, and stops it on shutdown. Grandine's Engine API calls go straight to Nethermind's implementation of the Engine API in the same process.

To embed Grandine in another program the same way, see Embeddable Grandine.

Download

Each Grandine release publishes the integration in two forms.

Release archives

The GitHub releases include archives named grandine-GRANDINE_VERSION-nethermind-NETHERMIND_VERSION-PLATFORM.zip. Each one contains a full Nethermind release with both plugin files already in its plugins directory. Platforms are linux-x64, linux-arm64, windows-x64, macos-x64 and macos-arm64.

Docker images

The sifrai/grandine repository on Docker Hub has images built on the official nethermind/nethermind image with the plugin added. Their tags are:

• VERSION-nethermind-NETHERMIND_VERSION: a specific Grandine release with a specific Nethermind release;
• unstable-nethermind: the latest unstable Grandine with the latest Nethermind it was packaged with;
• unstable-nethermind-NETHERMIND_VERSION: the latest unstable Grandine with a specific Nethermind release, also as unstable-amd64-nethermind-NETHERMIND_VERSION and unstable-arm64-nethermind-NETHERMIND_VERSION for a single platform;
• unstable-COMMIT-nethermind and unstable-COMMIT-nethermind-NETHERMIND_VERSION: an unstable build of a specific Grandine commit.

All multi-platform tags support linux/amd64 and linux/arm64. See the full list on Docker Hub.

The image's entrypoint is Nethermind, so options after the image name are Nethermind options.

Configuration

Grandine is configured through Nethermind's configuration system, in the Grandine section. The plugin is off by default. Turn it on with Grandine.Enabled:

./nethermind \
  --config mainnet \
  --Grandine.Enabled true \
  --Grandine.CheckpointSyncUrl CHECKPOINT-URL

Every Grandine beacon node and validator option is available as a Nethermind setting. Its name is the Grandine option in PascalCase under Grandine.:

Flags such as --metrics take true. Options that take a value take it as a string, in the same format the Grandine command line uses.

Nethermind also accepts each setting in kebab case with a grandine- prefix, for example --grandine-checkpoint-sync-url for --Grandine.CheckpointSyncUrl. The bare Grandine option, such as --checkpoint-sync-url, is not a Nethermind option. The same settings can also go in the Grandine section of a Nethermind JSON configuration file, or in environment variables named NETHERMIND_GRANDINECONFIG_ followed by the setting in upper case, for example NETHERMIND_GRANDINECONFIG_CHECKPOINTSYNCURL. Command line options take precedence over environment variables, and environment variables over the configuration file. See Nethermind configuration.

Some options are set by the integration and are not available:

• --eth1-rpc-urls and --jwt-secret: there is no Engine API endpoint or JWT secret, because Grandine calls Nethermind directly.

If you do not set Grandine.Network, the plugin chooses the Grandine network from the chain that Nethermind runs, so --config mainnet runs Grandine on Mainnet and --config hoodi on Hoodi.

Docker example

docker run -d --name nethermind-grandine \
  -p 30303:30303/tcp -p 30303:30303/udp \
  -p 9000:9000/tcp -p 9000:9000/udp -p 9001:9001/udp \
  -v /var/lib/nethermind:/nethermind/data \
  -v $HOME/.grandine:/root/.grandine \
  sifrai/grandine:VERSION-nethermind-NETHERMIND_VERSION \
  --config mainnet \
  --data-dir /nethermind/data \
  --Grandine.Enabled true \
  --Grandine.CheckpointSyncUrl CHECKPOINT-URL

Replace VERSION and NETHERMIND_VERSION with a tag listed on Docker Hub.

Open the peer-to-peer ports of both clients: Nethermind's (30303 TCP and UDP by default) and Grandine's (9000 TCP and UDP, and 9001 UDP for QUIC). See Networking and peers.

Logs and APIs

Grandine's log lines appear in Nethermind's output, and Nethermind logs the arguments it starts Grandine with (Starting grandine with arguments: ...). Grandine's Beacon Node API and metrics are served on their usual ports (5052 and 5054 by default), so tools and validator clients connect to them exactly as they would to a standalone Grandine. See Beacon Node API and Metrics.

When to use it

Use the embedded setup when Engine API latency matters, for example on nodes that propose blocks for many validators, or that serve block building and other latency-sensitive workloads, and when you run Nethermind as your execution client anyway.

Keep separate processes when you want to pair Grandine with a different execution client, when you want to update or restart the two clients independently, or when you run them on separate machines. The standard setup is described in Connecting an execution client.

Building it yourself

From the root of the Grandine repository:

# Build the Nethermind plugin (needs the .NET SDK)
make -B nethermind-plugin

# Download a Nethermind release and package it with Grandine for Linux x64
make -B grandine-nethermind-linux-x64 GRANDINE_VERSION=VERSION NETHERMIND_VERSION=NETHERMIND-VERSION

The other targets are grandine-nethermind-linux-arm64, grandine-nethermind-windows-x64, grandine-nethermind-macos-x64 and grandine-nethermind-macos-arm64. make nethermind-docker builds the Docker images. See bindings/csharp/README.md in the repository for the details, and Building from source for the Grandine build itself.

Embeddable Grandine

Grandine can run inside another program instead of as its own process. The host program, usually an execution client, starts Grandine in-process and serves Grandine's Engine API calls itself, as ordinary function calls. There is no HTTP connection, no JWT secret and no JSON between the two.

This chapter is for developers who want to embed Grandine in their own project. If you only want to run Grandine inside Nethermind, see Grandine embedded in Nethermind.

There are two ways to embed Grandine:

The Nethermind plugin is a complete example of the second way: it is a C# host built on libgrandine.

How embedding works

An embedded Grandine is a full beacon node. It keeps its own peer-to-peer networking, database, Beacon Node API, metrics and, if it is given validator keys, its integrated validator client. Only the connection to the execution layer changes:

+---------------------------------------------------------------+
| Host process                                                  |
|                                                               |
|  +--------------------+   Engine API calls   +-------------+  |
|  | Grandine           | -------------------> | Adapter     |  |
|  |  beacon node       |                      |  (host      |  |
|  |  validator client  | <------------------- |   code)     |  |
|  +--------------------+      results         +-------------+  |
|                                                    |         |
|                                                    v         |
|                                     host's execution layer   |
+---------------------------------------------------------------+

The host does three things:

1. Registers an adapter. The adapter implements every Engine API method Grandine uses: engine_newPayload, engine_forkchoiceUpdated, engine_getPayload, engine_getPayloadBodiesByHash, engine_getBlobs, engine_exchangeCapabilities and engine_getClientVersion, in each version up to the current fork. Grandine calls the adapter wherever it would otherwise send an HTTP request to an execution client.
2. Starts Grandine with a list of command line options, the same ones the grandine binary takes. --eth1-rpc-urls and --jwt-secret are not needed.
3. Stops Grandine when the host shuts down. Grandine then saves its chain and returns.

Rules for the host

These follow from how embedded mode is built:

• One Grandine per process. The adapter and the shutdown signal are process-wide. The adapter can be registered only once, and a second registration fails with an error.
• Register the adapter before starting Grandine. Until then, every Engine API call fails.
• Starting blocks. The start call returns only when Grandine stops, so run it on a thread of its own.
• The host handles shutdown. An embedded Grandine does not react to SIGINT or SIGTERM. Stop it with the shutdown call, and wait for the start call to return before the process exits, so that Grandine can save its chain. A Grandine that was stopped cannot be started again in the same process.
• Mainnet preset only. Embedded mode supports networks that use the mainnet preset: Mainnet, Sepolia and Hoodi, or a custom network with that preset.
• The adapter must be thread-safe. Grandine calls it from its own worker threads, and several calls can run at the same time.
• Errors are returned, not thrown. An adapter method reports a failure by returning an error. Grandine handles it as it handles an execution client that does not answer: the call fails, and Grandine counts the error and marks the execution layer as disconnected in its metrics. The el_offline field of /eth/v1/node/syncing is always false in embedded mode.

Besides the Engine API methods, the adapter has four methods for reading the deposit contract (eth_blockNumber, eth_getBlockByHash, eth_getBlockByNumber, eth_getLogs). Since the Electra upgrade, deposits reach the beacon chain through execution requests instead, and the Nethermind plugin leaves these four unimplemented.

Embedding in a Rust project

Dependencies

The Grandine crates are not published on crates.io. Depend on them through Git, and pin a release tag:

[dependencies]
anyhow = "1"
eth1_api = { git = "https://github.com/grandinetech/grandine", tag = "VERSION", features = ["embed"] }
execution_engine = { git = "https://github.com/grandinetech/grandine", tag = "VERSION" }
runtime = { git = "https://github.com/grandinetech/grandine", tag = "VERSION", features = ["embed", "network-mainnet", "network-hoodi"] }
types = { git = "https://github.com/grandinetech/grandine", tag = "VERSION" }
web3 = { git = "https://github.com/grandinetech/rust-web3.git", default-features = false }
# Select the cryptography backends, as the grandine binary does.
bls = { git = "https://github.com/grandinetech/grandine", tag = "VERSION", features = ["blst"] }
kzg_utils = { git = "https://github.com/grandinetech/grandine", tag = "VERSION", features = ["blst"] }

• embed on eth1_api replaces the HTTP Engine API client with the adapter, and on runtime it replaces signal handling with the shutdown call.
• The network-* features select the built-in networks the host can run: network-mainnet, network-sepolia, network-hoodi.
• bls and kzg_utils need a cryptography backend. blst is the one the official builds use; see Building from source for the others.

Copy the [patch] sections from the end of Grandine's Cargo.toml (for the same tag) into your own Cargo.toml. Cargo applies patches only from the top-level project, and Grandine depends on the forked crates they point to.

Grandine's crates use the Rust 2024 edition and the toolchain pinned in Grandine's rust-toolchain.toml. Build your project with that version or a newer one.

Implementing the adapter and running Grandine

Implement eth1_api::EmbedAdapter for a type of yours, register it with eth1_api::set_adapter, then start Grandine with runtime::run:

use std::{ffi::OsString, thread};

use anyhow::{Result, bail};
use eth1_api::{ClientVersionV1, EmbedAdapter, Eth1Block, RawForkChoiceUpdatedResponse};
use execution_engine::{ExecutionPayloadV3, ForkChoiceStateV1, PayloadAttributesV3, PayloadStatusV1};
use runtime::grandine_args::GrandineArgs;
use types::preset::Mainnet;
use web3::types::{BlockNumber, Filter, H256, Log};

/// Connects Grandine to the execution layer of the host application.
struct HostExecutionLayer;

impl EmbedAdapter for HostExecutionLayer {
    // Used only to read the deposit contract.
    fn eth_block_number(&self) -> Result<u64> {
        bail!("not supported")
    }

    fn eth_get_block_by_hash(&self, _hash: H256) -> Result<Option<Eth1Block>> {
        bail!("not supported")
    }

    fn eth_get_block_by_number(&self, _number: BlockNumber) -> Result<Option<Eth1Block>> {
        bail!("not supported")
    }

    fn eth_logs(&self, _filter: Filter) -> Result<Vec<Log>> {
        bail!("not supported")
    }

    fn engine_new_payload_v3(
        &self,
        payload: ExecutionPayloadV3<Mainnet>,
        versioned_hashes: Vec<H256>,
        parent_beacon_block_root: H256,
    ) -> Result<PayloadStatusV1> {
        // Validate and import the payload in the host's execution layer.
        todo!()
    }

    fn engine_forkchoice_updated_v3(
        &self,
        state: ForkChoiceStateV1,
        payload: Option<PayloadAttributesV3<Mainnet>>,
    ) -> Result<RawForkChoiceUpdatedResponse> {
        // Update the head and, if attributes are given, start building a payload.
        todo!()
    }

    fn engine_exchange_capabilities(&self, capabilities: &[&str]) -> Result<Vec<String>> {
        // Return the Engine API methods the host supports.
        todo!()
    }

    fn engine_get_client_version_v1(
        &self,
        own_version: ClientVersionV1,
    ) -> Result<Vec<ClientVersionV1>> {
        todo!()
    }

    // ... the other `engine_*` methods of the trait, in every version ...
}

fn main() -> Result<()> {
    // 1. Register the execution layer, once, before Grandine starts.
    eth1_api::set_adapter(Box::new(HostExecutionLayer))?;

    // 2. Parse the same options the grandine binary takes. The first item is the program name.
    let args = GrandineArgs::parse_strictly(
        ["my-node", "--network", "hoodi", "--data-dir", "/var/lib/my-node/grandine"]
            .map(OsString::from),
    )?;

    // 3. `run` blocks until Grandine stops, so give it a thread of its own.
    let grandine = thread::spawn(move || runtime::run(args));

    // ... the host application runs here ...

    // 4. Stop Grandine and wait until it has saved its chain.
    runtime::shutdown();
    grandine.join().expect("Grandine thread panicked")
}

The trait has one method per Engine API method and version, so the full implementation is longer than shown here; the compiler lists the missing ones. The CEmbedAdapter implementation in bindings/c/src/lib.rs is a complete example that forwards every method.

Note: runtime::run installs Grandine's logger as the global tracing subscriber. Do not install your own global subscriber in the same process, or run panics. Grandine's log options, such as GRANDINE_LOG, apply as usual.

Embedding through the C library

Building libgrandine

libgrandine is built from the bindings/c crate, as a shared library (libgrandine.so, libgrandine.dylib or grandine.dll) and as a static library:

make -B ./target/x86_64-unknown-linux-gnu/compact/libgrandine.so

make -B x86_64-unknown-linux-gnu builds the grandine binary and the library together. The other targets are aarch64-unknown-linux-gnu, x86_64-pc-windows-msvc, x86_64-apple-darwin and aarch64-apple-darwin. The library includes the Mainnet, Sepolia and Hoodi networks.

The build also writes a C header, bindings/c/build/grandine.h, generated from the Rust code. It declares every function and type described below.

The C interface

CEmbedAdapter is a struct of function pointers, one per Engine API method and version, for example engine_new_payload_v3, engine_forkchoice_updated_v3 and engine_get_blobs_v2. Each takes C structs (CExecutionPayloadV3, CForkChoiceStateV1, ...) and returns a CResult:

• code is GRANDINE_SUCCESS (0) when the call succeeded, and value holds the result;
• any other code, such as GRANDINE_ERROR_ENGINE_API (2), reports a failure, with an explanation in message.

Memory ownership

• Grandine to host. Pointers that Grandine passes to an adapter function are valid only during that call. Copy what you need to keep.
• Host to Grandine. Grandine frees what the host returns with its own allocator. Allocate every vector in a result with grandine_vec_alloc and every string with grandine_string, never with malloc or your language's allocator.

Example: the Nethermind plugin

The Nethermind plugin (bindings/csharp/Grandine.NethermindPlugin) is a complete host built on libgrandine, and a good template for other languages:

• GrandineEngineApi.cs implements each adapter method by converting the C structs to Nethermind's types and calling Nethermind's own Engine API module, for example engine_newPayloadV3.
• GrandineClient.cs turns these methods into C function pointers, fills CEmbedAdapter and calls grandine_set_execution_layer_adapter. It allows only one instance per process.
• GrandineClient.Run converts the options to C strings and calls grandine_run on a separate task. DisposeAsync calls grandine_shutdown and waits for that task to finish.
• StartGrandine.cs builds the option list from Nethermind's Grandine.* settings and starts the client once Nethermind has registered its Engine API module.

In C#, the delegates behind the function pointers must stay referenced for as long as Grandine runs, or the garbage collector frees them. The plugin keeps them in fields of GrandineClient for this reason.

The C# declarations of the native functions and types, NativeMethods.impl.cs, are generated from libgrandine by the bindings/csharp crate with csbindgen. The plugin's settings, one per Grandine command line option, are generated from Grandine's option definitions in the same build. Build the plugin with:

make -B nethermind-plugin

See bindings/c/README.md and bindings/csharp/README.md in the repository for packaging the library and the plugin with a Nethermind release or into Docker images.

Checkpoint sync

Checkpoint sync starts a new beacon node from a recent finalized state downloaded from another, trusted beacon node, instead of replaying the whole chain from genesis. It takes minutes instead of days, and it protects the node against long-range attacks, because the node starts inside the weak subjectivity period. It is the recommended way to start Grandine.

After a checkpoint sync, the node has no history before the checkpoint. That is enough for staking. If you need older blocks or states, see Back-sync and archive nodes.

Choose an endpoint

You need the Beacon Node API URL of a node that is synced to the network you join. You can use:

• a beacon node you run yourself (any client, including another Grandine node), or
• a public checkpoint sync endpoint. The community keeps a list at https://eth-clients.github.io/checkpoint-sync-endpoints/, with endpoints for mainnet and the testnets.

The endpoint must serve SSZ responses for these two Beacon API routes:

• GET /eth/v2/beacon/blocks/{block_id}
• GET /eth/v2/debug/beacon/states/{state_id}

Most public checkpoint sync endpoints and beacon nodes do.

Note: Grandine replaces the path of the URL with the API paths above, so only the scheme, host, port and any user:password@ part are used. A provider URL whose API key is part of the path, such as https://provider.example/API-KEY/, does not work. Credentials in the URL are redacted in the logs.

Start with checkpoint sync

Mainnet:

grandine \
  --network mainnet \
  --checkpoint-sync-url https://CHECKPOINT-HOST \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /path/to/jwt.hex

Hoodi:

grandine \
  --network hoodi \
  --checkpoint-sync-url https://CHECKPOINT-HOST \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /path/to/jwt.hex

Docker:

docker run -d --name grandine \
  -p 9000:9000/tcp -p 9000:9000/udp -p 9001:9001/udp \
  -v /path/to/grandine-data:/root/.grandine \
  -v /path/to/jwt.hex:/jwt.hex:ro \
  sifrai/grandine:stable \
  --network mainnet \
  --checkpoint-sync-url https://CHECKPOINT-HOST \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /jwt.hex

A successful checkpoint sync logs:

performing checkpoint sync from https://CHECKPOINT-HOST/…
loaded state at slot 12345664 from https://CHECKPOINT-HOST/
loaded state at slot 12345664

The node then syncs forward from that slot to the head. This usually takes a few minutes. Follow the progress as described in Running a beacon node.

How it works

1. Grandine fetches the latest finalized block (/eth/v2/beacon/blocks/finalized).
2. If that block is not the first block of an epoch, Grandine walks back to the first slot of the epoch, and further back if that slot is empty, until it finds a block at an epoch boundary.
3. It fetches the state at that block's slot (/eth/v2/debug/beacon/states/{slot}).
4. It stores the block and state as the anchor of its database and starts fork choice from there.

Each download has a timeout of 10 minutes.

The checkpoint URL is used only for a new database

Grandine uses --checkpoint-sync-url only when its database is empty. On every later start it resumes from its own database and logs:

skipping checkpoint sync: existing database found; pass --force-checkpoint-sync to force checkpoint sync

This is the normal behavior, so you can leave --checkpoint-sync-url in your service configuration.

Warning: If the checkpoint sync fails, Grandine does not stop. It logs checkpoint sync failed with the error and continues from its database, or from genesis if the database is empty. A node that starts from genesis on mainnet takes a very long time to sync. If you see this warning on a new node, stop Grandine, fix the URL or choose another endpoint, delete the beacon directory and start again.

Verify the checkpoint

Checkpoint sync trusts the endpoint you chose. A malicious or broken endpoint could give you a state from a different chain. Grandine does not compare the checkpoint against a second source, so do it yourself after the node has started.

Ask your node for its finalized checkpoint:

curl -s http://127.0.0.1:5052/eth/v1/beacon/states/finalized/finality_checkpoints

Then ask one or more independent sources for the block root at the same epoch, for example a different public endpoint, a node run by someone you trust, or a block explorer:

curl -s https://OTHER-BEACON-NODE/eth/v1/beacon/states/finalized/finality_checkpoints

The finalized.root values must match for the same finalized.epoch. If the epochs differ, look up the block root for a specific slot on both sides:

curl -s http://127.0.0.1:5052/eth/v1/beacon/blocks/SLOT/root
curl -s https://OTHER-BEACON-NODE/eth/v1/beacon/blocks/SLOT/root

If the roots differ, stop the node, delete its beacon directory, and checkpoint sync again from a source you trust.

Genesis state on testnets

Even with checkpoint sync, Grandine loads the genesis state of the network at startup. The mainnet genesis state is built into the binary. For Sepolia, Holesky and Hoodi, Grandine downloads it from the network's repository on GitHub on the first run, checks its genesis validators root, and saves it as genesis_state.ssz in the beacon directory:

downloading genesis state from https://github.com/eth-clients/hoodi/raw/…/metadata/genesis.ssz…
genesis state loaded at slot: 0

If GitHub is not reachable from your machine, either:

• download the file elsewhere and pass it with --genesis-state-file /path/to/genesis.ssz, or
• serve it from another location and pass --genesis-state-download-url URL.

For custom networks, --genesis-state-file (or genesis.ssz in --configuration-directory) is the normal way to provide the genesis state. See Configuration files and custom networks.

Re-sync from a new checkpoint

To start over from a fresh checkpoint, for example after a long downtime or a corrupted database, choose one of these:

• Force a new checkpoint. Keep the database and load a new anchor from the endpoint:

  grandine --network mainnet --checkpoint-sync-url https://CHECKPOINT-HOST --force-checkpoint-sync ...
  

  --force-checkpoint-sync requires --checkpoint-sync-url. Unlike a normal start, Grandine does not fall back to its database when the download fails: it logs the error and retries. Remove the flag again after the node has started, otherwise every restart downloads a new checkpoint.

• Reset the beacon databases. --force-reset-beacon-db deletes the beacon_fork_choice, eth1_cache, pubkey_cache and sync databases at startup. Combined with --checkpoint-sync-url, the node then checkpoint syncs into an empty database. This does not touch the validator directory or slashing protection data. Remove the flag after use.

• Delete the beacon directory by hand while Grandine is stopped. See Storage and database.

Warning: Never delete the validator directory to fix a sync problem. It holds the slashing protection database. See Slashing protection.

Sync from genesis

Without --checkpoint-sync-url, a new node syncs from genesis. This verifies the whole chain yourself, but takes much longer and needs much more disk space if you keep the history. Use it when you need an archive node that is built entirely from your own verification. See Back-sync and archive nodes.

Start from an older stored state

--state-slot SLOT makes Grandine start from the stored state closest to, and not after, SLOT instead of from the latest stored state. This is mainly useful for debugging. See Storage and database.

Command line options

See Command line reference for the full list.

• --checkpoint-sync-url (default: none): Beacon Node API URL to load a recent finalized checkpoint from when the database is empty.
• --force-checkpoint-sync (default: off): checkpoint sync even if a database exists. Requires --checkpoint-sync-url.
• --force-reset-beacon-db (default: off): delete the beacon node databases at startup.
• --genesis-state-file (default: none): load the genesis state from an SSZ file.
• --genesis-state-download-url (default: the network's GitHub repository): where to download the genesis state of Sepolia, Holesky or Hoodi.
• --state-slot (default: latest stored state): start from the stored state at or before this slot.
• --back-sync (default: off): after the checkpoint sync, also download older blocks. See Back-sync and archive nodes.

Back-sync and archive nodes

A beacon node that was started with checkpoint sync knows the chain only from its checkpoint onwards. That is all a staking node needs. Some uses need more history:

• serving old blocks to block explorers, indexers or researchers,
• answering Beacon API queries for old states, for example balances at a past epoch,
• serving historical blocks to peers that sync from genesis.

Grandine covers these with two settings: the storage mode, which decides what the node keeps, and back-sync, which downloads the history that came before the checkpoint.

What the node keeps

MIN_EPOCHS_FOR_BLOCK_REQUESTS is 33,024 epochs (about 146 days) on mainnet. Peers are only required to serve blocks from this window, and the standard mode keeps exactly that.

Execution payloads are not part of this table: by default Grandine stores finalized blocks without payloads and asks the execution client for them when needed. See Storage and database.

Back-sync

Back-sync downloads blocks, and the blobs or data columns that are still available, backwards from the checkpoint. Enable it with --back-sync:

grandine \
  --network mainnet \
  --checkpoint-sync-url https://CHECKPOINT-HOST \
  --back-sync \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /path/to/jwt.hex

How far back it goes depends on the storage mode:

• Standard mode: back to the start of the MIN_EPOCHS_FOR_BLOCK_REQUESTS window. The node can then serve every block its peers may ask for.
• Archive mode: back to genesis. When the blocks are in, Grandine replays them from genesis to rebuild the historical states it keeps.

--back-sync conflicts with --prune-storage.

Back-sync starts once the node has caught up with the head, and pauses whenever the node falls behind, so it never delays the forward sync. The node follows the head and serves validators while back-sync runs. Its progress is visible in the log:

starting back-sync from 12345664 to 7593984 slot
back-synced to 12339200 slot
…
back-sync completed

The progress is stored in the sync database, so back-sync resumes where it stopped after a restart.

Note: Back-sync is planned only when the node loads its anchor from a checkpoint sync server. Adding --back-sync to a node that restarts from an existing database has no effect. To enable it later, restart once with --back-sync --checkpoint-sync-url URL --force-checkpoint-sync. The node then loads a new checkpoint and back-syncs from it to the start of the block request window, or to genesis in archive mode. If the node already ran with --back-sync, it stops instead at the last finalized checkpoint it saw, filling only the gap. Remove --force-checkpoint-sync afterwards, or the node will checkpoint sync again on every restart.

Why back-sync is slow

Back-sync is limited by how fast peers serve old blocks, and Grandine verifies every block it receives. In archive mode, rebuilding the states afterwards means replaying every block since genesis, which takes a long time on mainnet. Expect back-sync to take much longer than the initial checkpoint sync.

Run an archive node

Coming soon: a new high-performance archive node for Grandine is in development and not yet released. It is built for the best historical data serving performance among consensus clients. Until it ships, use --archive-storage as described below.

An archive node keeps all blocks and a state every --archival-epoch-interval epochs from genesis to the head. There are two ways to build one.

Checkpoint sync and back-sync. The node is ready for the head within minutes and fills in the history in the background:

grandine \
  --network mainnet \
  --archive-storage \
  --back-sync \
  --checkpoint-sync-url https://CHECKPOINT-HOST \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /path/to/jwt.hex

When the block download is finished, Grandine reconstructs the states:

archiving back-synced states from 0 to 12345664
archiving back-sync data up to 1048576 slot
…
back-synced state archival completed (start_slot: 0, end_slot: 12345664)

Sync from genesis. Leave out --checkpoint-sync-url. The node verifies the whole chain itself and stores states as it goes forward. This is the slowest option:

grandine \
  --network mainnet \
  --archive-storage \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /path/to/jwt.hex

Warning: Choose the storage mode before the node syncs. Switching an existing standard node to --archive-storage keeps everything from that point on, but does not bring back history that the standard mode has already pruned. For a complete archive, start a fresh database.

Blob and data column sidecars are a special case. Peers only serve them for the data availability window, so an archive node started from a checkpoint cannot download sidecars older than about 18 days. It keeps all sidecars it receives from then on. With PeerDAS, a node keeps only the data columns it custodies. To keep all of them, also run as a supernode. See PeerDAS and blob data.

Archival epoch interval

--archival-epoch-interval (default: 32) sets how often Grandine stores a full state. To answer a query for a state that is not stored, Grandine loads the closest earlier stored state and replays the blocks in between.

• A smaller interval uses more disk space and answers historical state queries faster.
• A larger interval saves disk space and makes those queries slower.

With the default of 32 epochs, at most 32 epochs of blocks have to be replayed.

Execution payloads in an archive

Grandine stores finalized blocks without their execution payloads unless you pass --store-payloads. To serve old full blocks, it fetches the payloads from the execution client with engine_getPayloadBodiesByHash. An archive node therefore needs an execution client that keeps all historical block bodies, or --store-payloads so that Grandine keeps the payloads itself. The second option costs extra disk space.

Execution clients now implement history expiry (EIP-4444). Dropping pre-Merge blocks does not affect Grandine, because those blocks have no execution payloads. Dropping post-Merge block bodies does: some clients and prune modes keep only a recent window (for example, Erigon's default full prune mode), and rolling expiry of bodies older than about a year is spreading. Defaults change between releases, so check your execution client's documentation on history pruning and configure it to keep all post-Merge bodies, or use --store-payloads: Geth, Nethermind, Besu, Erigon, Reth.

Disk usage

Disk usage grows with the history you keep. The prune mode uses the least, the standard mode adds a few months of blocks plus the sidecar window, and an archive node grows without bound. See System requirements for current estimates and Storage and database for how to check what the database holds.

Command line options

See Command line reference for the full list.

• --back-sync (default: off): download history before the checkpoint. Conflicts with --prune-storage. The older spelling --back_sync is deprecated.
• --archive-storage (default: off): keep all blocks, states and sidecars. Conflicts with --prune-storage.
• --prune-storage (default: off): keep only the latest checkpoint block and state.
• --archival-epoch-interval (default: 32): number of epochs between stored states.
• --store-payloads (default: off): keep execution payloads of finalized blocks in the database.
• --data-availability-window (default: network setting): number of epochs to keep blob and data column sidecars in standard mode. Intended for testing. Conflicts with --archive-storage and --prune-storage.
• --checkpoint-sync-url (default: none): checkpoint to start from. See Checkpoint sync.
• --force-checkpoint-sync (default: off): load a new checkpoint even if a database exists.

Storage and database

This page explains where Grandine keeps its data, how it uses memory and disk, which storage mode to choose, and how to inspect, back up and restore the database.

Memory and disk

Grandine keeps the unfinalized part of the chain in memory. States share their unchanged parts (structural sharing), so Grandine does not have to copy full states for every block. This is one reason Grandine is fast and uses little memory on a healthy network.

When the chain does not finalize for a long time, the unfinalized part grows and so does memory use. --unfinalized-states-in-memory limits how many of the latest unfinalized slots keep their states in memory. Older unfinalized states are dropped and rebuilt from blocks when needed. See Performance and memory tuning.

The finalized part of the chain is written to an embedded libmdbx key-value database. Grandine uses the database to restart where it stopped, to serve historical data through the API and to peers, and to store blobs and data columns.

Data directory layout

All data lives under --data-dir (default: $HOME/.grandine), in a subdirectory named after the network:

~/.grandine/
├── exception.log                # internal errors; rotated at 1 GB, 5 files kept
└── mainnet/                     # one directory per network
    ├── meta.json                # database schema version
    ├── beacon/                  # --store-directory
    │   ├── beacon_fork_choice/  # blocks, states, blobs, data columns
    │   ├── pubkey_cache/        # decompressed validator public keys
    │   ├── sync/                # back-sync progress
    │   ├── eth1_cache/          # deposit cache (custom genesis only)
    │   └── genesis_state.ssz    # downloaded genesis state (Sepolia, Holesky, Hoodi)
    ├── network/                 # --network-dir
    │   ├── key                  # libp2p private key (node identity)
    │   └── enr.dat              # last ENR
    ├── validator/               # validator client data
    │   ├── slashing_protection.sqlite
    │   ├── validators.yml
    │   └── 0xPUBKEY/keystore.json   # keystores imported through the Keymanager API
    └── secrets/                 # passwords of keystores imported through the Keymanager API

The network name is always added: --data-dir /data puts mainnet data in /data/mainnet. With Docker, mount the host directory at /root/.grandine.

--store-directory and --network-dir move the beacon and network directories elsewhere, for example to put the database on a faster disk:

grandine --network mainnet \
  --data-dir /var/lib/grandine \
  --store-directory /mnt/nvme/grandine-beacon \
  ...

beacon/ can be deleted to re-sync. network/key holds the node's peer ID; deleting it gives the node a new identity. validator/ holds the slashing protection database and must never be deleted while you run validators. See Validator definitions and Slashing protection for what lives there.

Grandine creates its libmdbx database files with permissions 0600. Run Grandine as a dedicated user that owns the data directory. See Running as a service.

Storage modes

The storage mode decides how much finalized history Grandine keeps. Choose it before the first sync.

--prune-storage conflicts with --archive-storage and with --back-sync. The startup log shows the mode in use, for example storage mode: Standard { custom_data_availability_window: None }.

For back-sync and archive nodes, see Back-sync and archive nodes.

Archival epoch interval

In the standard and archive modes, Grandine stores a full state at the first slot of every --archival-epoch-interval epochs (default: 32). States in between are rebuilt on demand by replaying blocks from the closest stored state. A larger value uses less disk space but makes historical state queries slower.

Independently of the interval, once the node is synced it saves a checkpoint state at every new finalized epoch, so it can restart from a recent state.

Execution payloads

By default, Grandine stores finalized blocks blinded: without their execution payloads. When it needs a full block, to serve it through the API or to a peer, it fetches the payload from the execution client (engine_getPayloadBodiesByHash). This saves a lot of disk space, because the execution client keeps the same data anyway.

--store-payloads keeps the payloads of finalized blocks in Grandine's database as well. Use it when you cannot rely on the execution client to serve old block bodies.

Data availability window

Blob sidecars (Deneb and Electra) and data column sidecars (Fulu and later) are kept for the data availability window: 4,096 epochs, about 18 days, on mainnet. Older ones are pruned, except in archive mode. --data-availability-window EPOCHS changes the window in standard mode. It is meant for testing; peers expect at least the network default. See PeerDAS and blob data.

Compression

Finalized blocks and execution payloads are compressed with zstd, at the level set by --zstd-compression-level (default: 3, zstd's default level). All other values in the database are compressed with Snappy, which the option does not affect. Higher levels save some disk space at the cost of CPU time when blocks are written. Grandine rejects levels outside the range the zstd library supports.

Database size limit

libmdbx needs an upper limit for the size of each database file. The file grows in steps of 256 MiB as data is added, up to the limit. The limit does not reserve disk space.

• --database-size (default: 1024 GiB) is the limit for each beacon database (beacon_fork_choice, pubkey_cache, sync).
• --eth1-database-size (default: 16 GiB) is the limit for the deposit cache.

Sizes accept units such as 512GiB or 2TiB. The startup log shows the limits in use (Eth2 database upper limit: ...).

If a database reaches its limit, Grandine logs increasing environment map size limits, doubles both limits and restarts its runtime automatically. To avoid this on a large archive node, set a generous limit from the start:

grandine --network mainnet --archive-storage --database-size 4TiB ...

Starting from an older state

On startup, Grandine loads the latest stored checkpoint state. --state-slot SLOT makes it search the database for the latest stored state at or before SLOT and start from there instead. The startup log shows force state slot: SLOT. This is mostly useful for debugging and requires that the database actually holds a state that old: in standard mode, only states within the retention window remain.

In-memory mode

--in-memory runs Grandine without writing anything to the data directory. Every database lives in memory, and everything is lost on exit. Grandine warns about this at startup. Use it for tests and short experiments only.

Inspecting the database

Two subcommands read the database. Pass the same --network and --data-dir you run with. Global options go before the subcommand.

You do not need to stop Grandine first. The database supports access from several processes at once, and these subcommands only read it. While Grandine is writing, a subcommand may wait a moment before it starts. A long db-stats run keeps Grandine from reusing freed database pages until it finishes, so the database files can grow a little in the meantime.

db-stats lists every record type in beacon_fork_choice with its count and size:

grandine --network mainnet db-stats

…: 1234 entries, key_size: …, value_size: …, total_size: …

db-info prints information about the records in a database. --database is beacon or sync:

grandine --network mainnet db-info --database beacon
grandine --network mainnet db-info --database sync

Both accept --path to read a database at a custom location:

grandine --network hoodi db-stats --path ~/.grandine/hoodi/beacon/beacon_fork_choice

See Subcommands and tools for the other subcommands.

Backup and restore

What to back up depends on what you would lose.

To back up the beacon database:

1. Stop Grandine and wait for chain saved (finalized blocks: …, unfinalized blocks: …).

2. Copy the directory, for example:

   sudo systemctl stop grandine
   sudo tar -C /var/lib/grandine -czf grandine-beacon-$(date +%F).tar.gz mainnet/beacon
   sudo systemctl start grandine
   

To restore, stop Grandine, replace beacon/ with the backup, make sure the files belong to the user that runs Grandine, and start it. The node resumes from the backed-up state and syncs the slots since then.

Warning: Do not copy the database files while Grandine is running. The copy may be inconsistent.

Tip: For a staking node, a backup of beacon/ is rarely worth it. If the database is lost or corrupted, delete beacon/ and start with --checkpoint-sync-url.

Moving the data directory

1. Stop Grandine.
2. Move or copy the whole network directory (for example ~/.grandine/mainnet) to the new location, keeping the structure.
3. Start Grandine with --data-dir pointing at the parent of the network directory.

Resetting the database

To start over, either delete the beacon directory while Grandine is stopped, or start once with --force-reset-beacon-db. That option deletes beacon_fork_choice, eth1_cache, pubkey_cache and sync, but not the validator data. Combine it with --checkpoint-sync-url. See Checkpoint sync.

Upgrades and schema versions

meta.json records the database schema version. A newer Grandine release that changes the schema updates the file and logs using schema version … for new data. Check the release notes before you downgrade, and see Updating Grandine.

Command line options

See Command line reference for the full list.

• --data-dir (default: $HOME/.grandine): parent directory for all data. The network name is added below it.
• --store-directory (default: {data_dir}/{network}/beacon): directory of the beacon databases.
• --network-dir (default: {data_dir}/{network}/network): directory of the network key and ENR.
• --prune-storage (default: off): keep only the latest checkpoint block and state.
• --archive-storage (default: off): keep all history.
• --archival-epoch-interval (default: 32): epochs between stored states.
• --store-payloads (default: off): store execution payloads of finalized blocks.
• --zstd-compression-level (default: 3): zstd level for blocks and payloads.
• --database-size (default: 1024 GiB): size limit of each beacon database.
• --eth1-database-size (default: 16 GiB): size limit of the deposit cache database.
• --data-availability-window (default: network setting): epochs to keep sidecars. For testing.
• --unfinalized-states-in-memory (default: 128): number of the latest unfinalized slots that keep their states in memory. The minimum is two epochs (64 on mainnet).
• --max-epochs-to-retain-states-in-cache (default: 8): how many epochs of states the state cache keeps.
• --state-cache-lock-timeout (default: 1500): state cache lock timeout in milliseconds.
• --state-slot (default: latest stored state): start from the stored state at or before this slot.
• --force-reset-beacon-db (default: off): delete the beacon databases at startup.
• --in-memory (default: off): keep everything in memory.

Networking and peers

Grandine talks to other consensus clients over libp2p. It finds peers with the discv5 discovery protocol, exchanges blocks, attestations and other messages over gossipsub, and requests missing data with request-response protocols. A beacon node needs a healthy number of peers to sync, to follow the head, and to publish its validators' messages in time.

Ports

The IPv6 ports are used only when you enable IPv6 (see below). For all ports and paths, see Default ports and paths.

Grandine checks at startup that its ports are free. If one is taken, it exits with a message such as libp2p port (9000) is unavailable; make sure no other instance of the application is running or specify a different port with --libp2p-port.

QUIC is enabled by default and is used alongside TCP. When Grandine dials a boot node that advertises a QUIC address, it tries QUIC first. --disable-quic turns QUIC off.

Port forwarding and firewalls

For a node to receive incoming connections, the peer-to-peer ports must be reachable from the internet. Nodes that only make outgoing connections still work, but they find good peers more slowly and depend more on others.

1. Allow the ports in the host firewall. With ufw:

   sudo ufw allow 9000/tcp
   sudo ufw allow 9000/udp
   sudo ufw allow 9001/udp
   

2. If the machine is behind a home router, forward the same ports to the machine's LAN address. Grandine tries UPnP first (see below), but a manual port forward is more reliable.

3. With Docker, publish the ports:

   docker run ... -p 9000:9000/tcp -p 9000:9000/udp -p 9001:9001/udp sifrai/grandine:stable ...
   

Do not expose the Beacon Node API (5052) or the metrics port (5054) to the internet. See Security best practices.

UPnP

UPnP is enabled by default. Grandine asks the router to forward its ports and logs one of:

UPnP route established
UPnP not available
UPnP is available but gateway is not exposed to public network

--disable-upnp turns it off. If you set ENR options manually while UPnP is on, Grandine warns UPnP enabled with manual ENR settings: ...; manual ENR settings might be overridden by UPnP. Turn UPnP off when you forward ports by hand and set the ENR yourself.

ENR and node identity

Other nodes find yours through its Ethereum Node Record (ENR), a signed record with your node's public key, IP addresses and ports. Grandine logs the ENR at startup:

ENR Initialised enr=enr:-… seq=1 id=… ip4=Some(203.0.113.10) udp4=Some(9000) tcp4=Some(9000) quic4=Some(9001) …

You can also read it, together with the peer ID and listening addresses, through the API:

curl -s http://127.0.0.1:5052/eth/v1/node/identity

The node's private key is stored in network/key in the data directory, and the last ENR in network/enr.dat. The key determines the peer ID, so keep it if you want the same identity after a reinstall. --libp2p-private-key-file loads the key from another file instead. The file holds a hex-encoded secp256k1 private key, with or without 0x.

Public address

By default, Grandine learns its public IP address from its peers and updates the ENR when it changes (Address updated, Updated local ENR). This works for most setups.

If the node sits behind NAT with a fixed public address, or you run it on a server with several addresses, set the advertised values yourself:

grandine \
  --disable-upnp \
  --enr-address 203.0.113.10 \
  --enr-tcp-port 9000 \
  --enr-udp-port 9000 \
  --enr-quic-port 9001 \
  ...

--disable-enr-auto-update stops Grandine from changing the ENR address based on what peers report.

If you do not set --enr-tcp-port, Grandine advertises the --libp2p-port in the ENR. Port values in ENR options are only advertised: Grandine never opens sockets on them. Use them when an external port differs from the local one, for example with port mapping on a router or in Docker.

IPv6

Grandine listens on IPv4 only by default (0.0.0.0). To add IPv6, give an IPv6 listen address:

# Dual stack: IPv4 and IPv6
grandine --listen-address 0.0.0.0 --listen-address-ipv6 :: ...

IPv6 uses its own ports (9050 TCP and UDP, 9051 UDP for QUIC by default). Open them in the firewall too. To advertise a specific IPv6 address, use --enr-address-ipv6 and the --enr-*-port-ipv6 options.

For an IPv6-only node:

grandine --disable-ipv4 --listen-address-ipv6 :: ...

--disable-ipv4 requires --listen-address-ipv6 and cannot be combined with --listen-address, --enr-address, --enr-tcp-port, --enr-udp-port or --enr-quic-port.

Peers

Target peer count

--target-peers (default: 200) is the number of peers Grandine aims for. Most log lines end with the current count and the target:

… peers=[87/200]

The count rises over the first minutes after startup. A higher target improves message propagation and data availability but costs more bandwidth and CPU. On a slow connection, a lower target such as 80 can help:

grandine --target-peers 80 ...

You can see the connected peers through the API:

curl -s http://127.0.0.1:5052/eth/v1/node/peer_count
curl -s http://127.0.0.1:5052/eth/v1/node/peers

Boot nodes

Boot nodes are the first nodes Grandine contacts to discover the rest of the network. Each built-in network comes with a list of boot nodes, so you do not need to configure anything.

--boot-nodes takes a comma-separated list of ENRs and replaces the built-in list:

grandine --network custom --boot-nodes enr:-AAA…,enr:-BBB… ...

For custom networks, Grandine reads the boot nodes from bootstrap_nodes.txt in the --configuration-directory when --boot-nodes is not given. See Configuration files and custom networks.

Static peers and trusted peers

--libp2p-nodes takes a comma-separated list of multiaddresses that Grandine dials at startup:

grandine --libp2p-nodes /ip4/192.0.2.5/tcp/9000/p2p/16Uiu2HAm… ...

--trusted-peers takes a comma-separated list of peer IDs. Grandine treats these peers specially:

• they are never disconnected to make room for other peers,
• their peer score is not lowered, so they are not banned for misbehavior,
• they are explicit gossipsub peers, so Grandine always forwards gossip messages to them.

--trusted-peers does not dial the peers by itself. To keep a permanent link to a node you run, give it both as a static peer and as a trusted peer:

grandine \
  --libp2p-nodes /ip4/192.0.2.5/tcp/9000/p2p/16Uiu2HAmPEER \
  --trusted-peers 16Uiu2HAmPEER \
  ...

Trusted peers are listed in the startup log (trusted peers: [...]). Keep the list short: every trusted peer takes a slot that Grandine could otherwise fill with a well-behaved peer.

Private networks

By default, discovery ignores peers that advertise private IP addresses (such as 192.168.x.x or 10.x.x.x). For a local devnet or a network inside a LAN, enable --enable-private-discovery.

Subnets

Attestations and sync committee messages are spread over subnets, and since Fulu, blob data is spread over data column subnets. Grandine subscribes to the subnets its validators need, plus the subnets the protocol requires of every node, and tries to keep at least --target-subnet-peers (default: 3) peers on each subnet it uses.

• --subscribe-all-subnets subscribes to every attestation and sync committee subnet. This raises bandwidth and CPU use. It is useful for nodes that aggregate for many validators or serve external validator clients and relays. It does not include data column subnets.
• Data column subnets are covered by --supernode and --semi-supernode. See PeerDAS and blob data.

Peer scoring and rate limiting

Grandine scores its peers. Peers that send invalid data, break the protocol, or do not answer get lower scores and are eventually disconnected and banned for a while. Gossipsub has its own scoring on top. Both protect the node from spam and broken peers.

Grandine also limits how many requests it serves to, and sends to, each peer.

• --disable-peer-scoring turns peer scoring off. Use it only for testing.
• --disable-rate-limiting turns both inbound and outbound rate limiting off. Use it only for testing or private networks.

Troubleshooting low peer count

If the peer count stays low (peers=[5/200] after half an hour), check:

1. Ports. Are TCP 9000, UDP 9000 and UDP 9001 open in the firewall and forwarded on the router? Check from outside, for example with an online port checker for TCP 9000.
2. ENR. Does /eth/v1/node/identity show your public IP address and the right ports? If not, set --enr-address and the ENR ports, and disable UPnP.
3. Another instance. Is another client already using the ports? Grandine refuses to start in that case, but a different client on another port may still compete for bandwidth.
4. Clock. Is the system clock in sync? Peers reject messages from nodes whose clocks are off.
5. Network. Are you on the network you think? Check --network and, for custom networks, the boot nodes.
6. Bandwidth. On a slow or metered connection, lower --target-peers.

See Troubleshooting for more.

Command line options

See Command line reference for the full list.

• --listen-address (default: 0.0.0.0 unless --disable-ipv4): IPv4 listen address.
• --listen-address-ipv6 (default: none): IPv6 listen address. Enables IPv6.
• --disable-ipv4 (default: off): do not listen on IPv4. Requires --listen-address-ipv6.
• --libp2p-port (default: 9000): libp2p TCP port, IPv4.
• --libp2p-port-ipv6 (default: 9050): libp2p TCP port, IPv6.
• --discovery-port (default: 9000): discv5 UDP port, IPv4.
• --discovery-port-ipv6 (default: 9050): discv5 UDP port, IPv6.
• --quic-port (default: 9001): QUIC UDP port, IPv4.
• --quic-port-ipv6 (default: 9051): QUIC UDP port, IPv6.
• --disable-quic (default: off): do not use QUIC.
• --disable-upnp (default: off): do not ask the router for port forwarding.
• --enr-address, --enr-address-ipv6 (default: learned from peers): addresses to advertise.
• --enr-tcp-port, --enr-udp-port, --enr-quic-port and their -ipv6 variants (default: the listening ports): ports to advertise.
• --disable-enr-auto-update (default: off): do not update the ENR address from peer reports.
• --enable-private-discovery (default: off): discover peers with private IP addresses.
• --libp2p-private-key-file (default: network/key in the data directory): node private key.
• --network-dir (default: {data_dir}/{network}/network): directory of the key and ENR.
• --boot-nodes (default: built-in list): comma-separated boot node ENRs. Replaces the built-in list.
• --libp2p-nodes (default: none): comma-separated multiaddresses to dial at startup.
• --trusted-peers (default: none): comma-separated peer IDs to always keep and never ban.
• --target-peers (default: 200): target number of peers.
• --target-subnet-peers (default: 3): target number of peers per subnet.
• --subscribe-all-subnets (default: off): subscribe to all attestation and sync committee subnets.
• --disable-peer-scoring (default: off): turn off peer scoring.
• --disable-rate-limiting (default: off): turn off inbound and outbound rate limiting.

PeerDAS and blob data

Rollups publish their data to Ethereum as blobs. Blobs are not executed by the execution layer and are kept only for a limited time, so the consensus layer is responsible for making sure they were published and for serving them for that time. This is called data availability.

• From the Deneb fork, every beacon node downloaded and stored every blob as a blob sidecar.
• From the Fulu fork, Ethereum uses PeerDAS (Peer Data Availability Sampling, EIP-7594). Each blob is extended with erasure coding and split into 128 data columns. A node no longer downloads all blobs. It custodies a subset of the columns, stores them, and serves them to its peers. Because any half of the columns is enough to rebuild the rest, the network as a whole keeps all data available while each node handles only part of it.

Grandine supports both. Fulu is scheduled on all built-in networks, and when PeerDAS is scheduled, Grandine logs the custody mode at startup, for example custody mode: Minimal.

Custody

The columns are grouped into 128 custody groups. Which groups a node custodies follows from its node ID, and the number it custodies is its custody group count. Grandine advertises the count in its ENR and in its metadata, so peers know which columns to request from it.

Grandine has three custody modes:

The options have aliases that match other clients: --subscribe-all-data-column-subnets for --supernode, and --subscribe-half-data-column-subnets for --semi-supernode. The two options conflict with each other.

grandine \
  --network mainnet \
  --supernode \
  --checkpoint-sync-url https://CHECKPOINT-HOST \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /path/to/jwt.hex

A supernode subscribes to every data column subnet and downloads and stores every column, so it needs considerably more bandwidth and disk space than a node in the minimal mode. A semi-supernode needs about half of that.

Validator custody

The protocol asks nodes with validators to custody more. Grandine computes the requirement from the total effective balance of the validators registered with the node: the keys of the built-in validator client and the validators that validator clients, such as grandine vc or another client's, announce to it through the Beacon Node API with prepare_beacon_proposer or register_validator. Beacon committee subscriptions alone do not count:

custody groups = max(VALIDATOR_CUSTODY_REQUIREMENT, total effective balance / BALANCE_PER_ADDITIONAL_CUSTODY_GROUP)

capped at 128. On mainnet, VALIDATOR_CUSTODY_REQUIREMENT is 8 and BALANCE_PER_ADDITIONAL_CUSTODY_GROUP is 32 ETH. So a node with up to 256 ETH of effective balance custodies 8 groups, and a node with 4,096 ETH or more custodies all 128 groups, like a supernode.

Grandine re-checks the requirement once per epoch, from the last finalized state, and only ever raises the custody group count. A supernode is already at the maximum and skips the check.

Backfilling custody

When the custody group count goes up, the node is now responsible for columns it did not keep before. Grandine then backfills those columns for the data availability window by requesting them from peers, so it can serve them as the protocol expects. The log shows this as starting data column backfill (columns: {…}) from SLOT to SLOT slot.

--no-custody-groups-backfill turns the backfill off. The node then custodies the new columns only from the moment the count increased.

Reconstruction

When a node has received at least half of the columns of a block, it can rebuild the missing ones. Grandine does this automatically for nodes that custody at least half of the columns (semi-supernodes, supernodes and nodes with large validator custody). It skips blocks it proposed itself, and blocks whose columns all arrive before reconstruction starts. At the head, Grandine imports the block as soon as it has half of the columns, without waiting for the reconstruction, and then publishes the rebuilt columns to help the network. It always does so at the head; there is no option to turn it off. Columns rebuilt while syncing are stored but not published.

• --reconstruction-delay (default: 250 ms) is how long Grandine waits after receiving enough columns of a gossiped block before it starts reconstructing, so that columns still arriving from the network are not computed needlessly. For blocks requested from peers, the wait is 2 seconds.
• While syncing, Grandine also reconstructs missing columns of older blocks when it has enough of them. --sync-without-reconstruction turns that off, which saves CPU during sync at the cost of fetching more columns from peers.

Blobs from the execution client

The execution client often already has a block's blobs in its transaction pool. After the payload of a new block is validated, Grandine asks the execution client for them with engine_getBlobsV1 (Deneb and Electra) or engine_getBlobsV2 (Fulu), and uses what it gets instead of waiting for the network. This speeds up block import, especially for nodes that custody many columns. It needs no configuration; Grandine uses it whenever the execution client advertises support for these methods.

Retention

Blob sidecars and data column sidecars are kept for the data availability window: MIN_EPOCHS_FOR_BLOB_SIDECARS_REQUESTS and MIN_EPOCHS_FOR_DATA_COLUMN_SIDECARS_REQUESTS, 4,096 epochs (about 18 days) on mainnet. After that, Grandine deletes them.

--data-availability-window EPOCHS changes the window in standard mode. It is intended for testing. Setting it below the network default means your node cannot serve data that peers are entitled to request. See Storage and database.

Serving blobs through the API

The Beacon Node API serves blob data on these routes:

• GET /eth/v1/beacon/blob_sidecars/{block_id}
• GET /eth/v1/beacon/blobs/{block_id}
• GET /eth/v1/debug/beacon/data_column_sidecars/{block_id}

For blocks from Fulu on, Grandine builds blobs and blob sidecars from the stored data columns. That needs at least half of the columns, so only a supernode, a semi-supernode or a node whose validator custody covers at least half of the groups can answer these requests. On a node in the minimal mode, the blob routes return an empty list for Fulu blocks.

If you run rollup infrastructure or anything else that reads blobs from your beacon node, run it as a supernode or semi-supernode. See Beacon Node API.

Command line options

See Command line reference for the full list.

• --supernode (alias --subscribe-all-data-column-subnets, default: off): custody all data columns.
• --semi-supernode (alias --subscribe-half-data-column-subnets, default: off): custody half of the data columns.
• --no-custody-groups-backfill (default: off): do not backfill columns when the custody group count increases.
• --reconstruction-delay (default: 250): milliseconds to wait before reconstructing missing columns.
• --sync-without-reconstruction (default: off): do not reconstruct columns while syncing.
• --data-availability-window (default: network setting): epochs to keep sidecars. For testing.
• --kzg-backend (default: blst): cryptography library for KZG proofs.
• --archive-storage (default: off): keep sidecars forever.
• --prune-storage (default: off): do not keep data columns on disk.

Becoming a validator

This page walks through the whole path from no keys to an active validator run by Grandine. The steps are the same on every network. Do them on the Hoodi testnet first. Testnet ETH is free, mistakes cost nothing, and you learn how your setup behaves before real funds are at stake.

The simplest setup runs everything in one process: plain grandine with validator keys is a beacon node with the validator client built in. That is what this page uses. Grandine can also run the validator client as a separate process (grandine vc). See Validator client for when that makes sense.

You need:

• a synced execution client connected to Grandine through the Engine API and a JWT secret (see Connecting an execution client);
• Grandine installed (see Installation overview);
• 32 ETH per validator on mainnet, or Hoodi test ETH from a faucet (see Supported networks);
• an execution-layer address you control for withdrawals and block rewards.

Warning: A validator key must only ever be active in one place. Running the same key in two validator clients at the same time gets the validator slashed: it loses part of its stake and is forcibly exited. Read Slashing protection before you move keys between machines or clients.

1. Generate validator keys

Generate keys with ethstaker-deposit-cli. Do it on an offline machine if you can, and write the mnemonic down on paper. The mnemonic can regenerate every key and the withdrawal credentials, so anyone who has it controls the funds.

Download a release for your platform from the project's GitHub releases page, verify its checksum, and run:

./deposit new-mnemonic --chain hoodi

For mainnet use --chain mainnet. The tool asks for the number of validators, a keystore password and a withdrawal address; --num_validators, --keystore_password and --withdrawal_address set them without prompts. It creates compounding (0x02) validators by default and asks for the deposit amount per validator; --regular_withdrawal creates 0x01 validators with 32 ETH each. See the ethstaker-deposit-cli documentation for all options.

The tool writes a validator_keys directory that contains:

• one EIP-2335 keystore per validator, named like keystore-m_12381_3600_0_0_0-1700000000.json;
• a deposit_data-<timestamp>.json file for the launchpad.

Every keystore created in one run shares the password you entered. Store that password in a file that only the Grandine user can read:

printf '%s' 'YOUR-KEYSTORE-PASSWORD' > /path/to/password.txt
chmod 600 /path/to/password.txt

Grandine strips control characters from keystore passwords, as EIP-2335 requires, so a trailing newline in the file does no harm.

2. Make the deposit with the launchpad

Open the Staking Launchpad for your network:

• Mainnet: https://launchpad.ethereum.org
• Hoodi: https://hoodi.launchpad.ethereum.org

Follow its steps, upload deposit_data-<timestamp>.json, and send the deposit transaction from your wallet. Check that the launchpad URL is right before you connect a wallet. Phishing copies of the launchpad exist.

You can leave deposit_data-<timestamp>.json in validator_keys. Grandine skips files whose names start with deposit_data when it scans the keystore directory, and logs a warning about them.

3. Start Grandine with your keys

Start Grandine once the beacon node is set up as in the Quick start, and add the keystore directory and password file. Set a fee recipient too, so that rewards from proposed blocks go to an address you control.

grandine \
  --network hoodi \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /path/to/jwt.hex \
  --checkpoint-sync-url CHECKPOINT-SYNC-URL \
  --keystore-dir /path/to/validator_keys \
  --keystore-password-file /path/to/password.txt \
  --suggested-fee-recipient 0xYOUR-EXECUTION-ADDRESS

With Docker, mount the keys and the password file and pass the same options. The image's entrypoint is grandine, so the options follow the image name directly:

docker run -d --name grandine \
  --network host \
  -v $HOME/.grandine:/root/.grandine \
  -v /path/to/jwt.hex:/jwt.hex:ro \
  -v /path/to/validator_keys:/validator_keys:ro \
  -v /path/to/password.txt:/password.txt:ro \
  sifrai/grandine:stable \
  --network hoodi \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /jwt.hex \
  --checkpoint-sync-url CHECKPOINT-SYNC-URL \
  --keystore-dir /validator_keys \
  --keystore-password-file /password.txt \
  --suggested-fee-recipient 0xYOUR-EXECUTION-ADDRESS

Grandine refuses to run validators without --eth1-rpc-urls, because it cannot produce blocks without an execution client.

Warning: Without a fee recipient, from --suggested-fee-recipient or set per validator, the integrated validator client pays the execution rewards of proposed blocks to the Grandine donation address 0xe7cf7C3BA875Dd3884Ed6a9082d342cb4FBb1f1b. Always set your own. See Fee recipient, graffiti and gas limit.

On startup, look for these log lines:

started loading validator keys
decrypted validator key 0x...
loaded 1 validator key(s)

If these lines are missing, or Grandine logs failed to load validator keys, it found no keys: check the paths and that the keystore directory contains the keystores.

Grandine records every validator it loads in validators.yml in its validator directory (~/.grandine/hoodi/validator here). See Validator definitions. Other ways to load keys, such as a password directory, the Keymanager API and Web3Signer, are covered in Managing validator keys.

Tip: Decrypting many keystores takes a while because the key derivation is slow on purpose. Add --use-validator-key-cache to restart faster. See Managing validator keys.

4. Wait for activation

The deposit has to be seen by the beacon chain, and then the validator waits in the activation queue. Depending on the queue, this takes from hours to weeks. Keep Grandine running in the meantime. A validator that is not active yet has no duties, so nothing is lost.

Run Grandine as a service so that it survives reboots: see Running as a service.

5. Check the validator's status

Ask the beacon node through the Beacon Node API, which listens on 127.0.0.1:5052 by default:

curl -s http://127.0.0.1:5052/eth/v1/beacon/states/head/validators/0xYOUR-VALIDATOR-PUBKEY | jq .data.status

The status moves through pending_initialized (deposit seen), pending_queued (in the activation queue) and active_ongoing (attesting). If the request answers with a "not found" error, the deposit has not been processed yet.

You can also look the validator up on a block explorer such as beaconcha.in, which shows the queue position and an estimated activation time.

Once the validator is active, check that it attests: see Monitoring validator performance.

Mainnet readiness checklist

Go through this list before you deposit on mainnet:

• You ran the same setup on Hoodi and saw validators attest for several days.
• Grandine is on the latest release. See Updating Grandine.
• The execution client is fully synced and Grandine connects to it with the JWT secret.
• The beacon node is synced and has a healthy number of peers, with its P2P ports reachable. See Networking and peers.
• The validator keys load at startup and the fee recipient is an address you control.
• Grandine runs as a service under a dedicated user and restarts on failure. See Running as a service.
• Metrics and alerts are set up, so you notice missed duties. See Metrics and Monitoring validator performance.
• The mnemonic is stored offline, and the keystore password file is readable only by the Grandine user.
• You know how to export slashing protection data and you back it up before any migration. See Slashing protection.
• The keys run in exactly one place. No old client, backup machine or second Grandine has them loaded.
• Optional: Doppelganger protection is enabled, so a restart never signs while the same key is live elsewhere.

What next?

• Running as a service
• Managing validator keys
• Fee recipient, graffiti and gas limit
• Builder API and MEV
• Monitoring validator performance
• Updating Grandine
• Voluntary exits, withdrawals and consolidations

Command line options

• --keystore-dir (no default): directory with EIP-2335 keystore files to load.
• --keystore-password-file (no default): file with the password for all keystores.
• --suggested-fee-recipient (no default): default fee recipient for all validators.
• --use-validator-key-cache (disabled): cache decrypted keys for faster restarts.
• --eth1-rpc-urls (no default): execution client Engine API URLs; required when validators are loaded.

See Command line reference for the full list.

Validator client

The validator client holds the validator keys and performs the validator's duties: it attests, proposes blocks, aggregates and takes part in sync committees. It gets its duties and the data to sign from a beacon node, signs with the keys (or asks a Web3Signer to sign), checks every signature against slashing protection, and hands the result back to the beacon node to broadcast.

Grandine runs in one of three ways:

The integrated validator client (default)

The default and simplest setup is a single grandine process with keys. The validator client then runs inside the beacon node and reads duties straight from its fork choice, with no HTTP round trips in between. There is one process to run, monitor and update, and one data directory.

grandine \
  --network hoodi \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /path/to/jwt.hex \
  --keystore-dir /path/to/validator_keys \
  --keystore-password-file /path/to/password.txt \
  --suggested-fee-recipient 0xYOUR-EXECUTION-ADDRESS

With Docker (the image's entrypoint is grandine):

docker run -d --name grandine \
  --network host \
  -v $HOME/.grandine:/root/.grandine \
  -v /path/to/jwt.hex:/jwt.hex:ro \
  -v /path/to/validator_keys:/validator_keys:ro \
  -v /path/to/password.txt:/password.txt:ro \
  sifrai/grandine:stable \
  --network hoodi \
  --eth1-rpc-urls http://127.0.0.1:8551 \
  --jwt-secret /jwt.hex \
  --keystore-dir /validator_keys \
  --keystore-password-file /password.txt \
  --suggested-fee-recipient 0xYOUR-EXECUTION-ADDRESS

Validators need an execution client: Grandine refuses to start with keys but without --eth1-rpc-urls.

Keys can come from a keystore directory, the Keymanager API or a Web3Signer. See Managing validator keys. Whatever the source, Grandine records the validators it manages in validators.yml, which also holds per-validator settings such as fee recipients and graffiti.

Without a fee recipient, either --suggested-fee-recipient or a per-validator one, the integrated validator client pays the execution rewards of its blocks to the Grandine donation address. See Fee recipient, graffiti and gas limit.

The separate validator client (grandine vc)

grandine vc runs the validator client on its own. It talks to beacon nodes only through the standard Beacon Node API, given with --beacon-node-urls. Use it when:

• the validator keys must live on a different machine from the beacon node, for example a hardened signing host;
• one validator client should use several beacon nodes, so that it keeps working when one of them is down or behind (see Redundancy and fallback beacon nodes);
• you want to restart or update the beacon node and the validator client independently.

The cost is an extra process to manage and HTTP latency between the two. If none of the points above applies, stay with the integrated validator client.

Warning: Never run the same validator keys in two places at once. That includes the integrated validator client and grandine vc, two grandine vc processes, or Grandine and another client. Two signers with the same key eventually sign conflicting messages, and the validator is slashed. Slashing protection cannot prevent this, because each process only knows its own database.

Warning: Give each validator client its own data directory, and never point two validator clients at the same one. The validator directory holds the slashing protection database, validators.yml and the keys imported through the API.

Running the beacon node and the validator client separately

Start the beacon node with grandine bn. If the validator client runs on another host, bind the Beacon Node API to an address that host can reach, and firewall it so that only the validator client can connect:

grandine bn \
  --network hoodi \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /path/to/jwt.hex \
  --http-address 0.0.0.0 \
  --http-port 5052

Then start the validator client with the keys:

grandine vc \
  --network hoodi \
  --beacon-node-urls http://BN-HOST:5052 \
  --keystore-dir /path/to/validator_keys \
  --keystore-password-file /path/to/password.txt \
  --suggested-fee-recipient 0xYOUR-EXECUTION-ADDRESS

With Docker:

docker run -d --name grandine-vc \
  --network host \
  -v $HOME/.grandine-vc:/root/.grandine \
  -v /path/to/validator_keys:/validator_keys:ro \
  -v /path/to/password.txt:/password.txt:ro \
  sifrai/grandine:stable \
  vc \
  --network hoodi \
  --beacon-node-urls http://BN-HOST:5052 \
  --keystore-dir /validator_keys \
  --keystore-password-file /password.txt \
  --suggested-fee-recipient 0xYOUR-EXECUTION-ADDRESS

A separate validator client does not need an execution client, a JWT secret or P2P ports. Each beacon node it uses needs its own execution client.

Rules for options

• Options go after bn or vc. An option before the command is refused, for example grandine --network hoodi vc ... fails with --network must be specified after vc.
• vc refuses options that only the built-in beacon node acts on, for example --eth1-rpc-urls, --checkpoint-sync-url, --target-peers, --http-port, --builder-url or --track-liveness.
• bn refuses validator options, for example --keystore-dir, --enable-validator-api, --beacon-node-urls or --report-validator-performance.
• --args-file works with both: the options in the file are applied to the command they follow.
• --report-validator-performance reads blocks from the built-in beacon node, so vc refuses it.

How vc checks its beacon nodes

At startup, the validator client asks every beacon node for its genesis. All nodes that answer must report the same genesis validators root, and for a predefined network it must match that network. A node on a different network or chain stops the validator client with an error, because restarting would not fix it. If no node answers, it retries every 2 seconds:

no beacon node given with --beacon-node-urls reported genesis; retrying in 2 s
genesis agreed with the beacon nodes given with --beacon-node-urls (genesis validators root: 0x..., genesis time: ...)
checking remote beacon nodes
1 of 1 remote beacon nodes are ready to serve duties

After that, the validator client polls each node's health once per slot and follows each node's head through the event stream. How it picks a node for each duty is described in Redundancy and fallback beacon nodes.

Behavior that differs from the integrated validator client

• Fee recipient. A validator without a fee recipient, given with --suggested-fee-recipient or per validator in validators.yml, is not prepared as a proposer on the beacon nodes, is not registered with the builder, and gets no proposer preferences published. The beacon node then pays its own default fee recipient for local payloads. The integrated validator client falls back to the Grandine donation address instead.
• Builder. --builder-url belongs to the beacon node. On the validator client, --use-builder registers the validators with the builder behind the beacon nodes and accepts builder payloads. Without it only local payloads are requested and builder boost factors are ignored. It matters only until the Gloas fork, after which builders bid over gossip and need no registration. See Builder API and MEV.
• Doppelganger protection. --detect-doppelgangers does not need --track-liveness on vc, because liveness is asked from the beacon nodes. A Grandine beacon node only answers liveness requests when it runs with --track-liveness, so start such nodes with it. With the integrated validator client, --detect-doppelgangers still requires --track-liveness. See Doppelganger protection.
• Metrics. With --metrics, a separate validator client exposes only validator client metrics. If it runs on the same host as a beacon node that also has metrics on, give one of them another --metrics-port (default 5054). See Metrics.
• Graffiti. --disable-blockprint-graffiti is a beacon node option. On vc, set the graffiti with --graffiti or per validator.

Moving keys from the integrated validator client to grandine vc

The keys, validators.yml and the slashing protection database all live in the validator directory, <data-dir>/<network>/validator (by default ~/.grandine/<network>/validator). Keystores imported through the Keymanager API store their passwords in <data-dir>/<network>/secrets. Moving to a separate validator client means moving these, and making sure the old process no longer signs.

Warning: Stop the old process first, confirm that it has stopped, and only then start the new one. Never have both running with the keys, not even for a moment. If you run a service manager, disable the old service so that it cannot restart.

1. Stop Grandine and make sure it does not restart:

   sudo systemctl stop grandine
   sudo systemctl disable grandine
   

2. Back up the slashing protection data in the standard interchange format. Use the old data directory:

   grandine --network mainnet --data-dir /path/to/old-data-dir interchange export /path/to/slashing-protection.json
   

3. Move the validator directory to the validator client's data directory. Moving rather than copying makes sure only one copy is in use:

   mkdir -p /path/to/vc-data-dir/mainnet
   mv /path/to/old-data-dir/mainnet/validator /path/to/vc-data-dir/mainnet/
   mv /path/to/old-data-dir/mainnet/secrets /path/to/vc-data-dir/mainnet/   # only if it exists
   

   validators.yml stores absolute paths. Update the paths of keys imported through the API so that they point to the new location:

   sed -i 's#/path/to/old-data-dir/mainnet/#/path/to/vc-data-dir/mainnet/#g' \
     /path/to/vc-data-dir/mainnet/validator/validators.yml
   

   Keystores loaded from --keystore-dir keep their own paths. If the keystore directory moves too, update those paths, or remove their entries and let Grandine discover the keystores again.

4. Restart the node without keys, as grandine bn with the same beacon node options as before, minus the validator options. Add --track-liveness, because the doppelganger protection in step 5 asks the node for liveness. Without it, the validator client never starts performing duties. Make sure the validator client can reach its Beacon Node API.

5. Start the validator client with the new data directory and the same key options as before. Enabling doppelganger protection for the first start adds a safety check:

   grandine vc \
     --network mainnet \
     --data-dir /path/to/vc-data-dir \
     --beacon-node-urls http://BN-HOST:5052 \
     --keystore-dir /path/to/validator_keys \
     --keystore-password-file /path/to/password.txt \
     --suggested-fee-recipient 0xYOUR-EXECUTION-ADDRESS \
     --detect-doppelgangers
   

6. Check the logs for the loaded keys and for remote beacon nodes are ready to serve duties, and watch the validators attest in the next epochs.

If you cannot move the directory, for example because the validator client runs on a new machine, copy only what you need. Import the interchange file into the new data directory before the first start:

grandine --network mainnet --data-dir /path/to/vc-data-dir interchange import /path/to/slashing-protection.json

Then load the keys there with --keystore-dir or the Keymanager API, and delete them from the old machine.

To go back to the integrated validator client, follow the same steps in the other direction: stop grandine vc, move the validator directory back, and start grandine with the key options.

Troubleshooting

• --NAME must be specified after vc: the option is placed before the command. Move it after vc or bn.
• beacon node at URL is on a different network: --network (or the custom configuration) does not match the beacon node. Fix the network or the URL.
• no remote beacon node is fully synced: every node is syncing, optimistic, has its execution client offline, or has a head more than --max-empty-slots slots behind. Duties are still attempted, but may fail or be late. Check the beacon nodes.
• no remote beacon node can serve duties: no node could be reached. Check the URLs, firewalls and that the nodes' HTTP API is enabled on a reachable address.
• --track-liveness must be specified for --detect-doppelgangers with the built-in beacon node: add --track-liveness, or run the validator client separately.

Command line options

Options for keys, the Keymanager API and Web3Signer are listed on their own pages: Managing validator keys, Keymanager API and Web3Signer.

Options for grandine vc only:

• --beacon-node-urls (required): comma-separated Beacon Node API URLs to perform duties against.
• --publish-to-every-node (default all): duties to publish to every beacon node rather than only the first one that accepts them. See Redundancy and fallback beacon nodes.
• --use-builder (disabled): register validators with the builder behind the beacon nodes and accept its payloads (until Gloas).

Validator options for both grandine and grandine vc:

• --detect-doppelgangers (disabled): enable doppelganger protection.
• --report-validator-performance (disabled): log validator performance reports; integrated validator client only.
• --slashing-protection-history-limit (default 256): number of epochs of slashing protection data to keep.
• --suggested-fee-recipient (no default): default fee recipient for all validators.
• --graffiti (no default): default block graffiti.
• --default-gas-limit (default: the scheduled gas limit, or 60000000): default execution gas limit.
• --default-builder-boost-factor (default 100): the percentage by which a builder's bid is scaled before Grandine weighs it against the local payload.
• --max-empty-slots (default 32): how many slots a head may fall behind before it is considered stale.
• --data-dir (default ~/.grandine): parent directory for the data files, which go to <data-dir>/<network>.

See Command line reference for the full list.

Redundancy and fallback beacon nodes

A validator is only as available as the beacon node it uses. When that node restarts for an update, falls behind, loses its execution client or its peers, the validator misses duties. The safe way to add redundancy is to give one validator client several beacon nodes.

Grandine does this with the separate validator client, grandine vc, which accepts a list of beacon nodes in --beacon-node-urls. The integrated validator client always uses its own built-in beacon node. See Validator client for how to run the two parts separately.

Never run duplicate validator clients

Warning: Do not run two validator clients with the same keys "for redundancy", in any combination of clients, machines or data directories. Sooner or later the two sign different blocks or attestations for the same slot, which is a slashable offense. Each validator client only knows its own slashing protection database, so neither can stop the other. Slashing costs part of the stake and forces the validator out.

Redundancy belongs in the beacon node layer: several beacon nodes, one validator client. A missed attestation costs about as much as one attestation earns. A slashing costs far more. If the validator client itself has to fail over to another machine, do it by hand, following Moving validators between machines, with doppelganger protection enabled on the new machine.

Using several beacon nodes

List the beacon nodes, comma-separated, in order of preference:

grandine vc \
  --network mainnet \
  --beacon-node-urls http://BN-1:5052,http://BN-2:5052,http://BN-3:5052 \
  --keystore-dir /path/to/validator_keys \
  --keystore-password-file /path/to/password.txt \
  --suggested-fee-recipient 0xYOUR-EXECUTION-ADDRESS

The nodes can be Grandine (grandine bn). Other clients' beacon nodes should also work if they implement the standard Beacon Node API. Each needs its own execution client. Diversity helps: nodes on different hosts, with different execution clients, fail independently.

Every node must be on the same network. At startup the validator client asks each node for its genesis. Nodes that answer must agree on the genesis validators root, and for a predefined network it must be that network's. A node on a different network or chain stops the validator client with an error. Nodes that are down at startup are checked against the agreed genesis when they come back.

How a node is chosen

Health

Once per slot, the validator client polls every node and gives it one of these health levels, best first:

It also follows each node's head through the node's event stream (/eth/v1/events), and reconnects with a growing delay, up to one minute, when the stream drops.

Changes are logged:

beacon node at http://BN-2:5052/ is now Unusable
beacon node at http://BN-2:5052/ is now Ready

Order

For each duty, the validator client ranks the nodes by health first and by head second: among nodes of equal health, the one with the newest head goes first, and among nodes with equal heads, the one listed first in --beacon-node-urls. Incompatible nodes are left out. An unreachable node stays in the list as a last resort, because a failed poll does not mean the next request fails.

Requests for data then go to the nodes in that order until one succeeds: duties, attestation data, aggregates, sync committee data, validator indices and block production. A failure on one node falls back to the next one in the same slot, and is logged as a warning.

Some requests are special:

• Block production. A block is taken from the first node that builds on the head the other nodes agree on. A node with a different view of the head is passed over. If no node builds on that head, the first block produced is proposed anyway, because any block beats a missed slot.
• Attesting on a new head. When a node streams the slot's block as its new head before attestation time, the validator client attests to it without waiting for the attestation tick, and asks that node first, as it is the one known to have the block.
• Doppelganger liveness. Liveness is asked from every Ready node at once. A validator seen live by any of them counts as live.

Publishing

Signed messages are published according to --publish-to-every-node:

• With the default, all, every message is sent to every usable node at the same time. Publishing succeeds as soon as one node accepts it, and the slow nodes still get it. This spreads each message into the network from several places, so a single node with poor peers does not delay it.
• A list of duties publishes only those to every node. The others are sent to one node at a time, in ranking order, until one accepts them.
• The flag with no value sends everything to the nodes one at a time, in ranking order, until one accepts it.

The duty names are aggregates, attestations, blocks, payload-attestations, proposer-preferences, sync-committee-contributions and sync-committee-messages:

# Broadcast attestations and blocks, send the rest to one node
--publish-to-every-node attestations,blocks

# Send everything to the first node that accepts it
--publish-to-every-node

Regardless of the flag:

• when blocks are sent to one node at a time, the node that produced the block is tried first, as it is known to accept it;
• a blinded block from the builder goes only to the node that produced it, because only that node can unblind it;
• beacon committee and sync committee subscriptions, proposer preparation and builder registrations are always sent to every usable node, so that whichever node serves a later duty is ready for it.

Topologies

One validator client, one beacon node

The basic split setup: grandine bn on one host and grandine vc on the same or another host. It has no redundancy, but the two can be restarted separately. A beacon node update stops duties only while the node restarts.

One validator client, several beacon nodes

The recommended redundant setup. List two or three nodes. Keep the default --publish-to-every-node all. Update the beacon nodes one at a time, so that at least one is always Ready. Duties keep flowing while a node restarts, resyncs or loses its execution client.

  EL-1           EL-2           EL-3
    |              |              |
  BN-1           BN-2           BN-3
    |              |              |
    +--------------+--------------+
                   |
          Beacon Node API (HTTP)
                   |
              grandine vc        (validator keys, slashing protection)

Sentry nodes

The validator client runs on a host that is not reachable from the internet. It talks only to beacon nodes in a private network, and those nodes alone take part in the P2P network. This hides the validator's host from the network, and the redundant setup above protects against one sentry failing. Bind each node's Beacon Node API to its private address with --http-address, and firewall it so only the validator client can reach it.

Remote signing with redundant beacon nodes

Combine several beacon nodes with a Web3Signer, so that the keys live on a separate, locked-down signer. There is still exactly one validator client per key.

What the beacon nodes need

• HTTP API reachable by the validator client. For a Grandine node, --http-address (default 127.0.0.1) and --http-port (default 5052). Never expose the Beacon Node API to the internet.
• Liveness tracking for doppelganger protection. A Grandine beacon node answers liveness requests only with --track-liveness. Start every node with it if the validator client uses --detect-doppelgangers.
• A builder for MEV. With --use-builder on the validator client, every node needs a builder configured, for a Grandine node with --builder-url. See Builder API and MEV.
• Correct fee recipients. The validator client sends each node the fee recipient of every validator that has one. Set --suggested-fee-recipient or per-validator fee recipients on the validator client. A validator without one is not prepared on the nodes, and each node pays its own default.

Troubleshooting

• no remote beacon node is fully synced: no node is Ready. Duties are still attempted on the best node available, but its data may be outdated. Check each node's sync status, execution client and peers.
• no remote beacon node can serve duties: no node could be reached. Check the URLs, firewalls and --http-address on the nodes.
• beacon nodes at URL-1 and URL-2 are on different chains: the nodes disagree on genesis. One of them is on the wrong network.
• URL beacon node failed to ... warnings: a request failed on one node and was retried on the next one. Occasional warnings are normal. Constant ones point to a broken node.

Command line options

• --beacon-node-urls (required for vc): comma-separated Beacon Node API URLs, in order of preference.
• --publish-to-every-node (default all): comma-separated duties to publish to every node. Without a value, everything is sent to the first node that accepts it.
• --max-empty-slots (default 32): how many slots a node's head may lag behind the clock before the node counts as unusable.
• --use-builder (disabled): register validators with the builder behind the beacon nodes and accept builder payloads (until Gloas).
• --detect-doppelgangers (disabled): check, through the beacon nodes, that the keys are not live elsewhere before signing.

See Command line reference for the full list.

Managing validator keys

This page explains where Grandine gets validator keys from, how it finds keystores and their passwords, and how to add, remove and move keys safely. Everything here applies both to the integrated validator client (grandine) and to the separate one (grandine vc).

Key sources

Grandine loads keys from these sources, all of which can be combined:

Every key Grandine manages locally, and every remote key imported through the API, gets an entry in validators.yml. That file also holds per-validator settings. Keys that only a --web3signer-urls signer serves get no entry. See Web3Signer.

Where key files live

With the default data directory, the files for a network are under ~/.grandine/<network>/:

~/.grandine/hoodi/
├── validator/
│   ├── validators.yml                    validator definitions and per-validator settings
│   ├── slashing_protection.sqlite        slashing protection database
│   ├── api-token.txt                     Keymanager API token (with --enable-validator-api)
│   ├── 0x<pubkey>/keystore.json          keystores imported through the Keymanager API
│   ├── validator_key_cache_0x<hash>.json key cache (with --use-validator-key-cache)
│   └── keystores.json                    legacy key storage, from older versions only
└── secrets/
    └── 0x<pubkey>                        passwords of keystores imported through the API

Pass --data-dir to use another parent directory. The layout below it stays the same. Keystores loaded with --keystore-dir stay where they are and are never copied.

Loading keystores from a directory

--keystore-dir points to a directory of EIP-2335 keystores, such as the validator_keys directory that ethstaker-deposit-cli writes. It must be combined with exactly one of --keystore-password-file and --keystore-password-dir.

On every start, Grandine scans the directory:

• Only the top level is scanned. Subdirectories are not searched.
• Files ending in .json and files without an extension are read as keystores. Files with any other extension are ignored.
• Files whose names start with deposit_data are skipped with a warning, so the deposit data file can stay in the directory.
• Any other file that is not a valid keystore stops Grandine with Failed to load a keystore from file .... Keep only keystores in the directory, and no subdirectories without an extension.
• A keystore without a pubkey field is skipped with a warning.
• A keystore whose public key already has an entry in validators.yml is not added again. The existing entry, including its settings, wins.

New keystores are added to validators.yml as local_keystore entries with the keystore path and the password file path. Pass an absolute path to --keystore-dir, because the paths are recorded as given.

One password for all keystores

When all keystores share a password, as they do when they were generated in one run of ethstaker-deposit-cli, put the password in a file:

printf '%s' 'YOUR-KEYSTORE-PASSWORD' > /path/to/password.txt
chmod 600 /path/to/password.txt

grandine \
  --network hoodi \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /path/to/jwt.hex \
  --keystore-dir /path/to/validator_keys \
  --keystore-password-file /path/to/password.txt

One password per keystore

When the keystores have different passwords, put each password in its own file in a directory and pass --keystore-password-dir. Each password file is named after its keystore:

• for a keystore named NAME.json, the password file is NAME.txt;
• for a keystore without an extension, the password file has exactly the same name.

/path/to/validator_keys/keystore-m_12381_3600_0_0_0-1700000000.json
/path/to/validator_keys/keystore-m_12381_3600_1_0_0-1700000000.json
/path/to/passwords/keystore-m_12381_3600_0_0_0-1700000000.txt
/path/to/passwords/keystore-m_12381_3600_1_0_0-1700000000.txt

grandine vc \
  --network hoodi \
  --beacon-node-urls http://BN-HOST:5052 \
  --keystore-dir /path/to/validator_keys \
  --keystore-password-dir /path/to/passwords

Note: Grandine decides between the two modes by checking whether the password path is a directory. Passing a directory to --keystore-password-file therefore also works as a password directory. Passing both options at once is refused.

Password format

Grandine reads the whole password file and, as EIP-2335 requires, normalizes it to NFKD and removes control characters. A trailing newline is therefore ignored, as are tabs and other control characters anywhere in the password. Passwords in validators.yml (keystore_password_path or an inline keystore_password) are treated the same way.

Validator key cache

Keystores are encrypted with a deliberately slow key derivation function, so decrypting hundreds of keystores on every start takes time. --use-validator-key-cache makes Grandine save the decrypted keys in an encrypted cache file in the validator directory. On the next start, keys found in the cache are not decrypted again.

grandine \
  --network hoodi \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /path/to/jwt.hex \
  --keystore-dir /path/to/validator_keys \
  --keystore-password-file /path/to/password.txt \
  --use-validator-key-cache

How the cache works:

• It covers keystore files: the ones from --keystore-dir and the ones imported through the API. Web3Signer keys and the legacy storage are not cached.
• It is encrypted with the passwords of all cached keystores together, and its file name (validator_key_cache_0x<hash>.json) is derived from the keystores' UUIDs. A different set of keystores uses a different cache file. An old cache file left behind can be deleted.
• If the cache cannot be read, Grandine logs Unable to load validator key cache and decrypts the keystores as usual.

Note: The cache holds the same secret keys as the keystores, protected by the same passwords. Protect it like the keystores and include it in the same access controls.

Adding keys

There are three ways to add a validator:

1. Without a restart, through the Keymanager API. POST /eth/v1/keystores imports a keystore with its password and, ideally, its slashing protection history. The key starts signing right away and is kept across restarts. See Keymanager API.
2. Without a restart, through a Web3Signer. Import a remote key with POST /eth/v1/remotekeys, or add the key to a signer listed in --web3signer-urls and run with --web3signer-refresh-keys-every-epoch. See Web3Signer.
3. With a restart, through the keystore directory. Copy the keystore (and, with --keystore-password-dir, its password file) into --keystore-dir and restart Grandine. The directory is only scanned at startup.

Whichever way you choose, import the key's slashing protection history first if it ever signed anywhere else. See Slashing protection.

Removing and disabling keys

• Keys imported through the API are removed with DELETE /eth/v1/keystores. Grandine stops signing with them right away, deletes the keystore file, its password file and the validators.yml entry, and returns the keys' slashing protection data. Save that data if the key will run elsewhere.
• Remote keys are removed with DELETE /eth/v1/remotekeys.
• Keystores from --keystore-dir are read-only: the API reports them as readonly: true and refuses to delete them. To remove one, stop Grandine, delete the keystore file, and remove its entry from validators.yml or set enabled: false on it. Do both: an entry whose keystore file is gone stops Grandine at startup, and a keystore file left in the directory is discovered again if its entry is removed.

To keep a key on disk but stop using it, set enabled: false on its entry in validators.yml and restart. Grandine logs validator 0x... is disabled in validators.yml and will not be loaded. Discovery never re-enables a disabled entry. See Validator definitions.

Legacy key storage

Before validators.yml existed, Grandine kept keystores imported through the Keymanager API in a single encrypted file, keystores.json, in the validator directory, unlocked with the password in --keystore-storage-password-file.

New imports now go to separate keystore files, and existing installations migrate on their own:

• Keys in keystores.json get keystore_storage entries in validators.yml and keep loading. They still need --keystore-storage-password-file.
• Importing such a keystore through the API again moves the key out of keystores.json into its own keystore file.
• Deleting such a key through the API also needs --keystore-storage-password-file. Without it the request fails with password for decrypting keystores is missing, run Grandine with --keystore-storage-password-file to provide it.
• When the last key leaves keystores.json, Grandine deletes the file, and the password option is no longer needed.

Moving validators between machines

Moving a validator is the riskiest routine operation for a staker, because a short overlap between the old and the new signer is enough to get slashed. Follow this order:

1. Make sure the new machine is ready: the beacon node is synced, Grandine is configured, and it starts cleanly without the keys.

2. Stop the old validator client and make sure it cannot restart: disable its service, or remove the keys from it. If the old client is not Grandine, follow its documentation.

3. Export the slashing protection data from the old client. On Grandine:

   grandine --network mainnet interchange export /path/to/slashing-protection.json
   

4. Delete the keys from the old machine, or at least make sure nothing there loads them.

5. Import the slashing protection data on the new machine before its first start with the keys:

   grandine --network mainnet interchange import /path/to/slashing-protection.json
   

   Through the Keymanager API, you can pass the interchange data in the same request that imports the keystores instead.

6. Load the keys on the new machine and start it. Enable --detect-doppelgangers for extra safety. See Doppelganger protection.

7. Watch the logs and the validators' attestations for the next few epochs.

If you are not sure that the old machine is stopped for good, wait at least two epochs (about 13 minutes) after it last signed before you start the new one. A missed epoch costs a little. A slashing costs much more.

For moving keys from the integrated validator client to a separate one on the same machine, see Validator client.

Security

• Generate keys offline, keep the mnemonic offline, and never copy it to the staking machine.
• Make password files, the secrets directory and validators.yml readable only by the user that runs Grandine. Grandine creates validators.yml and API-imported password files with mode 600 itself.
• For large or valuable validator sets, keep keys off the staking machine with a Web3Signer.
• See Security best practices.

Command line options

• --keystore-dir (no default): directory with EIP-2335 keystore files to load at startup. Requires a password option.
• --keystore-password-file (no default): file with the password for all keystores in --keystore-dir. Conflicts with --keystore-password-dir.
• --keystore-password-dir (no default): directory with a password file per keystore. Conflicts with --keystore-password-file.
• --keystore-storage-password-file (no default): password for the legacy keystores.json storage of keys imported by older versions.
• --use-validator-key-cache (disabled): cache decrypted keys for faster startup.
• --data-dir (default ~/.grandine): parent directory of <network>/validator and <network>/secrets.

See Command line reference for the full list.

Validator definitions (validators.yml)

Grandine records the validators it manages in validators.yml, in the validator directory: <data-dir>/<network>/validator/validators.yml, which is ~/.grandine/<network>/validator/validators.yml by default. For each validator the file records where its key comes from (a keystore file, the legacy key storage, or a Web3Signer), whether it is enabled, and its per-validator settings such as the fee recipient and graffiti.

Grandine creates and maintains the file. You can also edit it by hand while Grandine is stopped. The integrated validator client (grandine) and the separate one (grandine vc) use it the same way. grandine bn runs no validators and does not read it.

What the file contains

The file records:

• keystores discovered in --keystore-dir;
• keystores imported through the Keymanager API;
• keys in the legacy keystores.json storage (see Migration from older versions);
• remote keys imported through POST /eth/v1/remotekeys, and web3signer entries you add by hand.

It is not always a complete list of running validators. Keys known only through a Web3Signer passed with --web3signer-urls are fetched from the signer on every start and are not written here. For them the signer is authoritative. Because they have no entry, they use the process-wide defaults (--suggested-fee-recipient, --default-gas-limit, --graffiti), and the Keymanager API rejects per-validator settings for them with 404. To configure such a validator on its own, give it an entry: import it through POST /eth/v1/remotekeys (which records an entry even for a key already loaded from a URL), or add a web3signer entry by hand.

How the file is populated

On every start, Grandine loads the file, then adds entries for keys it finds that have none yet:

• keystores in --keystore-dir;
• keys in the legacy keystores.json storage.

This is additive. An existing entry always wins: discovery never overwrites settings, changes the key source of an entry, or re-enables a disabled entry. While Grandine runs, the file changes only through the Keymanager API, and each change is written to disk at once.

With --in-memory, the file is neither read nor written, and the definitions exist only for the life of the process.

Format

The file is a YAML list with one entry per validator. Every entry has pubkey and type. The other fields are optional.

The three builder fields are resolved in this order: prefer_builder_proposals: true wins, then builder_boost_factor, then builder_proposals: false, and otherwise --default-builder-boost-factor applies. See Builder API and MEV.

local_keystore

An EIP-2335 keystore file on disk:

- enabled: true
  pubkey: "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"
  type: local_keystore
  keystore_path: /home/user/validator_keys/keystore-m_12381_3600_0_0_0-1700000000.json
  keystore_password_path: /home/user/passwords/keystore-m_12381_3600_0_0_0-1700000000.txt
  fee_recipient: "0x0000000000000000000000000000000000000001"
  graffiti: "my-staking-node"

The entry must give keystore_password or keystore_password_path. Without either, Grandine fails to start with validator 0x... declares neither keystore_password nor keystore_password_path.

A keystore stored under the validator directory, which is where keystores imported through the API go, can be deleted through the Keymanager API. A keystore stored anywhere else is read-only to the API.

web3signer

Signing is delegated to a Web3Signer:

- enabled: true
  pubkey: "0xb301803f8b5ac4a1133581fc676dfedc60d891dd5fa99028805e5ea5b08d3491af75d0707adab3b70c6a6a580217bf81"
  type: web3signer
  url: https://signer.example:9000
  root_certificate_path: /etc/grandine/signer-ca.pem
  client_identity_path: /etc/grandine/grandine-client.p12
  client_identity_password: "YOUR-P12-PASSWORD"
  request_timeout_ms: 5000
  fee_recipient: "0x0000000000000000000000000000000000000002"

The TLS fields and request_timeout_ms configure the HTTP client Grandine uses for all Web3Signer requests. Grandine builds a single Web3Signer client, so all enabled entries that set any of these fields must set the same values. Entries without them use that client too.

keystore_storage

The key lives in the legacy keystores.json storage. The entry needs nothing beyond pubkey:

- enabled: true
  pubkey: "0xa1d1ad0714035353258038e964ae9675dc0252ee22cea896825c01458e1807bfad2f9969338798548d9858a571f7425c"
  type: keystore_storage

File permissions

An entry may contain an inline password, so Grandine creates the file readable and writable only by its owner (mode 600). It writes the file atomically through a temporary file next to it. If you change the permissions, they are kept across later saves.

Editing the file by hand

Stop Grandine before you edit the file. Grandine keeps its own copy in memory and overwrites the file on the next Keymanager API change.

• To add a validator, add an entry. For a keystore, keystore_path and a password are enough. The key in the keystore must match pubkey.
• To retire a validator, set enabled: false. Deleting the entry is not enough when the key source is still there: a keystore left in --keystore-dir, or a key left in keystores.json, is added again on the next start.
• To remove a --keystore-dir validator for good, delete both the keystore file and the entry, or delete the file and set enabled: false. An enabled entry whose keystore file is missing stops Grandine at startup.

Grandine validates the file on startup and refuses to start when:

• two entries have the same pubkey;
• a graffiti is longer than 32 bytes;
• a keystore contains a key other than the declared pubkey;
• a keystore file or password file of an enabled entry cannot be read, or the password is wrong;
• enabled web3signer entries declare more than one distinct set of TLS or timeout options.

enabled: false keeps the key from its own entry out of the signer. It does not hide a key that a --web3signer-urls signer serves: that key is loaded from the URL anyway. Use --web3signer-public-keys to narrow the keys loaded from a URL.

Keymanager API changes

Changes made through the Keymanager API are recorded in the file right away:

• A keystore imported with POST /eth/v1/keystores is written to <validator directory>/0x<pubkey>/keystore.json, and its password to <data-dir>/<network>/secrets/0x<pubkey>. The entry is a local_keystore entry with these paths. Importing a key whose entry is disabled re-enables it.
• A remote key imported with POST /eth/v1/remotekeys gets a web3signer entry without TLS options.
• Deleting a keystore through the API removes its entry and deletes its keystore and password files. Only keys under the validator directory and in the legacy storage can be deleted. Keystores from --keystore-dir and keys loaded from --web3signer-urls are read-only.
• Deleting a remote key removes its entry. A key served by a signer in --web3signer-urls is loaded again on the next start.
• Fee recipients, gas limits and graffiti set through the API are stored on the validator's entry. A request for a validator without an entry is rejected with 404.

Migration from older versions

Older versions of Grandine kept keys imported through the API encrypted in keystores.json, unlocked with --keystore-storage-password-file, and kept proposer settings in a separate database in the validator directory. Both migrate automatically on the first start of a version that uses validators.yml:

• Keys in keystores.json keep loading through keystore_storage entries, and --keystore-storage-password-file is still required for them. Importing such a keystore through the API again moves the key to its own keystore file and removes it from keystores.json. Once the storage is empty, Grandine deletes the file, and the password option is no longer needed.
• Settings from the proposer settings database are copied to the matching validators.yml entries, and the database is removed once every setting has found its validator. Settings for validators that have no entry yet are kept in the database for a later start, and Grandine logs keeping the legacy proposer configs database.

Command line options

• --keystore-dir (no default): keystores discovered at startup and added as local_keystore entries.
• --keystore-storage-password-file (no default): password for the legacy keystores.json storage behind keystore_storage entries.
• --suggested-fee-recipient, --default-gas-limit, --graffiti, --default-builder-boost-factor: defaults for validators whose entry does not set the field.
• --data-dir (default ~/.grandine): validators.yml is in <data-dir>/<network>/validator.

See Command line reference for the full list.

Keymanager API

The Keymanager API lets you manage a running validator client over HTTP: list, import and delete keys, manage remote signer keys, set per-validator fee recipients, gas limits and graffiti, and sign voluntary exits. Grandine implements the standard Keymanager API, so tools written for it, such as staking dashboards, work with Grandine too.

Grandine calls this server the validator API. It is separate from the Beacon Node API: it has its own address, port and authentication token, and it runs wherever the validator client runs, in grandine with keys or in grandine vc. grandine bn has no validator client and refuses --enable-validator-api.

Enabling the API

The API is off by default. Turn it on with --enable-validator-api:

grandine \
  --network hoodi \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /path/to/jwt.hex \
  --keystore-dir /path/to/validator_keys \
  --keystore-password-file /path/to/password.txt \
  --enable-validator-api

Or on a separate validator client:

grandine vc \
  --network hoodi \
  --beacon-node-urls http://BN-HOST:5052 \
  --enable-validator-api

A validator client can start with no keys at all and get every key through the API.

By default the API listens on 127.0.0.1:5055. On startup Grandine logs where it listens and where the token is:

Validator API is listening on 127.0.0.1:5055, authorization token file: /home/user/.grandine/hoodi/validator/api-token.txt

With Docker, publish the port only on the loopback interface of the host, or keep it unpublished and use the API from inside the container's network:

docker run -d --name grandine-vc \
  -p 127.0.0.1:5055:5055 \
  -v $HOME/.grandine-vc:/root/.grandine \
  sifrai/grandine:stable \
  vc \
  --network hoodi \
  --beacon-node-urls http://BN-HOST:5052 \
  --enable-validator-api \
  --validator-api-address 0.0.0.0

Inside the container the token file is /root/.grandine/hoodi/validator/api-token.txt, which is $HOME/.grandine-vc/hoodi/validator/api-token.txt on the host.

Authentication

Every request must carry the API token as a bearer token:

Authorization: Bearer YOUR-API-TOKEN

A request without the header or with a wrong token fails with 401 or 400.

Where the token comes from

• By default, the token file is api-token.txt in the validator directory: <data-dir>/<network>/validator/api-token.txt, or ~/.grandine/<network>/validator/api-token.txt. If the file does not exist or cannot be read as a token, Grandine generates a random 256-bit token, writes it there as hex, and uses it. The token stays the same across restarts.
• With --validator-api-token-file, Grandine reads the token from that file and never writes it. If the file is missing or invalid, Grandine does not start.

A token file you provide must contain an even number of hex digits, at least 32 of them, and nothing else. A trailing newline makes the file invalid. Clients must send the token in lowercase. To create one:

openssl rand -hex 32 | tr -d '\n' > /path/to/api-token.txt
chmod 600 /path/to/api-token.txt

Warning: The token gives full control over the keys: whoever holds it can delete keystores, import keys and sign exits. Keep the token file readable only by the Grandine user and the tools that need it. Grandine writes the default file with the default file permissions of its user, so restrict them with chmod 600.

Exposing the API safely

Keep the API on 127.0.0.1 unless you have a reason not to. To use it from another machine, prefer an SSH tunnel to opening the port:

ssh -N -L 5055:127.0.0.1:5055 user@VC-HOST

If you do bind it to another interface with --validator-api-address, firewall the port so that only trusted hosts reach it. The API serves plain HTTP, so put it behind a TLS-terminating proxy if it must cross an untrusted network.

Browsers are allowed to call the API only from the origins in --validator-api-allowed-origins. By default the only allowed origin is the API's own listening URL, for example http://127.0.0.1:5055. Repeat the option to allow more origins, for example a local dashboard.

Endpoints

Successful responses use the standard formats: a JSON body with a data field, 202 Accepted for a successful POST of a setting, and 204 No Content for a successful DELETE of a setting. Errors return a JSON body with a message field. Bodies must be sent with Content-Type: application/json.

Every change through the API takes effect immediately, with no restart, and is recorded in validators.yml, so it survives restarts.

Examples

The examples set two shell variables first:

API=http://127.0.0.1:5055
TOKEN=$(cat ~/.grandine/hoodi/validator/api-token.txt)

List keys

curl -s -H "Authorization: Bearer $TOKEN" $API/eth/v1/keystores | jq

{
  "data": [
    { "validating_pubkey": "0x93247f22...", "readonly": true },
    { "validating_pubkey": "0xa1d1ad07...", "readonly": false }
  ]
}

readonly: true marks keys this endpoint cannot delete: keystores loaded from --keystore-dir and Web3Signer keys, which are listed here too. Keys imported through the API and keys in the legacy storage are readonly: false. Use GET /eth/v1/remotekeys to list and DELETE /eth/v1/remotekeys to remove Web3Signer keys.

Import a keystore

The request carries the keystores as JSON strings, the passwords in the same order, and optionally the slashing protection history as an EIP-3076 interchange JSON string. jq builds the body from files:

jq -n \
  --rawfile keystore /path/to/keystore-m_12381_3600_0_0_0-1700000000.json \
  --rawfile password /path/to/password.txt \
  --rawfile slashing /path/to/slashing-protection.json \
  '{keystores: [$keystore], passwords: [$password], slashing_protection: $slashing}' |
curl -s -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  --data @- \
  $API/eth/v1/keystores | jq

{ "data": [ { "status": "imported", "message": null } ] }

Each keystore gets a status:

• imported: the key is signing now;
• duplicate: the key is already loaded and enabled;
• error: the keystore could not be imported, with the reason in message, for example a wrong password or a key that is already a remote signer key.

Imported keystores are written to <data-dir>/<network>/validator/0x<pubkey>/keystore.json, and their passwords to <data-dir>/<network>/secrets/0x<pubkey> with mode 600. If the request carries slashing protection data, Grandine imports it before it starts signing with the keys. If the data is malformed or for another network, the whole request fails.

Warning: Always send the slashing protection history of a key that has signed anywhere before. Without it Grandine logs keystore import: slashing protection data is not provided! and has no record of what the key signed elsewhere.

Delete a keystore

curl -s -X DELETE \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"pubkeys": ["0xa1d1ad0714035353258038e964ae9675dc0252ee22cea896825c01458e1807bfad2f9969338798548d9858a571f7425c"]}' \
  $API/eth/v1/keystores > delete-response.json

jq -r .slashing_protection delete-response.json > slashing-protection.json

Grandine stops signing with the key, removes its validators.yml entry, and deletes its keystore and password files. The response holds a status per key and the keys' slashing protection data. Keep that data if the key will run anywhere else. The statuses are:

• deleted: the key was removed;
• not_active: the key is not loaded, but slashing protection data exists for it, which is returned;
• not_found: the key is not known;
• error: the key could not be deleted, for example key is read-only for a key from --keystore-dir.

Deleting keys from the legacy keystores.json storage requires --keystore-storage-password-file. See Managing validator keys.

Remote keys

List the Web3Signer keys:

curl -s -H "Authorization: Bearer $TOKEN" $API/eth/v1/remotekeys | jq

Import a key that a Web3Signer serves:

curl -s -X POST \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"remote_keys": [{"pubkey": "0xb301803f8b5ac4a1133581fc676dfedc60d891dd5fa99028805e5ea5b08d3491af75d0707adab3b70c6a6a580217bf81", "url": "http://SIGNER-HOST:9000"}]}' \
  $API/eth/v1/remotekeys | jq

A key Grandine holds locally is refused with key already exists in a local keystore, so that a local key is never handed to a signer. Importing a key that was loaded from --web3signer-urls but has no validators.yml entry records an entry for it, which lets you give it per-validator settings.

Delete a remote key:

curl -s -X DELETE \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"pubkeys": ["0xb301803f8b5ac4a1133581fc676dfedc60d891dd5fa99028805e5ea5b08d3491af75d0707adab3b70c6a6a580217bf81"]}' \
  $API/eth/v1/remotekeys | jq

Grandine stops using the key and removes its entry. If a signer listed in --web3signer-urls serves the key, it comes back on the next start, or at the next epoch with --web3signer-refresh-keys-every-epoch. See Web3Signer.

Fee recipient, gas limit and graffiti

PUBKEY=0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a

# Set
curl -s -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"ethaddress": "0xYOUR-EXECUTION-ADDRESS"}' $API/eth/v1/validator/$PUBKEY/feerecipient
curl -s -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"gas_limit": "60000000"}' $API/eth/v1/validator/$PUBKEY/gas_limit
curl -s -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"graffiti": "my-staking-node"}' $API/eth/v1/validator/$PUBKEY/graffiti

# Show
curl -s -H "Authorization: Bearer $TOKEN" $API/eth/v1/validator/$PUBKEY/feerecipient | jq
curl -s -H "Authorization: Bearer $TOKEN" $API/eth/v1/validator/$PUBKEY/gas_limit | jq
curl -s -H "Authorization: Bearer $TOKEN" $API/eth/v1/validator/$PUBKEY/graffiti | jq

# Remove the per-validator value and fall back to the default
curl -s -X DELETE -H "Authorization: Bearer $TOKEN" $API/eth/v1/validator/$PUBKEY/feerecipient

How the endpoints behave:

• The GET endpoints return the value in effect: the validator's own setting, or else the process-wide default (--suggested-fee-recipient, --default-gas-limit, --graffiti). GET .../gas_limit falls back to the network's scheduled gas limit. GET .../feerecipient returns 404 when neither the validator nor the default sets one.
• The POST and DELETE endpoints need a validators.yml entry for the validator and return 404 without one. Keys loaded only from --web3signer-urls have no entry. Import them through POST /eth/v1/remotekeys first.
• A graffiti longer than 32 bytes is rejected with 400.

See Fee recipient, graffiti and gas limit for how these settings are used.

Sign a voluntary exit

Warning: A voluntary exit cannot be undone. Once it is published, the validator leaves the active set and cannot rejoin. See Voluntary exits, withdrawals and consolidations before you exit.

The endpoint signs an exit for a key the validator client holds, locally or through a Web3Signer, and returns it. It does not publish it:

curl -s -X POST -H "Authorization: Bearer $TOKEN" \
  $API/eth/v1/validator/$PUBKEY/voluntary_exit > exit.json

Without the epoch query parameter the exit uses the current epoch. Add ?epoch=EPOCH to set it. A key the validator client does not hold, or one the beacon chain does not know yet, returns 404.

To exit, submit the signed message to a beacon node:

jq -c .data exit.json |
curl -s -X POST -H "Content-Type: application/json" --data @- \
  http://127.0.0.1:5052/eth/v1/beacon/pool/voluntary_exits

Change the log level

These endpoints change logging without a restart, which on a validator client would cost a slot of duties. target is a module path and level one of error, warn, info, debug or trace:

curl -s -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"target": "validator", "level": "debug"}' \
  $API/eth/v2/debug/tracing/log_level

trace_level takes the same body, with target optional, and applies to the telemetry output enabled with --telemetry-metrics-url. See Logging.

Command line options

• --enable-validator-api (disabled): run the validator API server.
• --validator-api-address (default 127.0.0.1): address to listen on.
• --validator-api-port (default 5055): port to listen on.
• --validator-api-allowed-origins (default: the API's own listening URL): Access-Control-Allow-Origin values for browsers. Repeat the option for several origins.
• --validator-api-timeout (default 10000): request timeout in milliseconds.
• --validator-api-token-file (default <data-dir>/<network>/validator/api-token.txt, generated if missing): file with the bearer token. A file you pass must already exist.
• --keystore-storage-password-file (no default): password for the legacy keystores.json storage of keys imported by older versions. Needed to load those keys and to delete them.

See Command line reference for the full list.

Web3Signer

Web3Signer is a remote signing service. It holds the validator keys, and the validator client sends it signing requests over HTTP instead of holding the keys itself. That keeps the keys off the machine that talks to the network, lets one key store serve several validator clients in turn, and allows hardware security modules and cloud key vaults behind the signer.

Grandine supports Web3Signer in both the integrated validator client (grandine) and the separate one (grandine vc). Remote signing is the recommended setup for large or valuable validator sets.

Note: Grandine applies its own slashing protection to remote keys too, before it sends a signing request. Enable Web3Signer's slashing protection database as well. It protects the key even if a second validator client ever reaches the same signer.

Warning: A signer serving a key to two validator clients at once is as dangerous as running the key twice. Point exactly one validator client at each key.

Two ways to configure the signer

Choose one, or combine them:

• --web3signer-urls loads keys in bulk: Grandine asks each signer for the keys it serves and runs all of them, or the subset given with --web3signer-public-keys. This is the simplest option when the signer manages the validator set. These keys are not written to validators.yml.
• web3signer entries in validators.yml declare keys one by one, created by hand or with POST /eth/v1/remotekeys on the Keymanager API. Use them for per-validator settings, such as fee recipients, gas limits or graffiti, and for TLS options to reach the signer.

Loading keys with --web3signer-urls

grandine \
  --network hoodi \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /path/to/jwt.hex \
  --web3signer-urls http://SIGNER-HOST:9000 \
  --suggested-fee-recipient 0xYOUR-EXECUTION-ADDRESS

With the separate validator client:

grandine vc \
  --network hoodi \
  --beacon-node-urls http://BN-HOST:5052 \
  --web3signer-urls http://SIGNER-HOST:9000 \
  --suggested-fee-recipient 0xYOUR-EXECUTION-ADDRESS

Give the signer's base URL. Grandine calls /api/v1/eth2/publicKeys to list keys and /api/v1/eth2/sign/{pubkey} to sign, relative to the host, so any path in the URL is ignored. Several signers are given as separate values: --web3signer-urls http://signer-1:9000 http://signer-2:9000.

On startup Grandine logs the keys it got from each signer:

loaded 12 validator key(s) from Web3Signer at http://SIGNER-HOST:9000/

To run only some of the keys a signer serves, list them:

--web3signer-public-keys 0x93247f22...,0xb301803f...

The list applies to every --web3signer-urls signer. A listed key that no signer serves is not loaded.

When keys are fetched

• On startup, Grandine fetches the key list of every signer.
• Every epoch, it fetches again from any signer that has not answered with keys yet: one that was unreachable or returned an empty list. Grandine logs Web3Signer at URL did not return any validator keys. It will retry to fetch keys again in the next epoch.
• With --web3signer-refresh-keys-every-epoch, it fetches from every signer every epoch, so that keys added to a signer start signing without a restart. A refresh only adds keys. A key removed from the signer stays loaded until the next restart, and its signing requests fail in the meantime. A refresh also brings back a remote key deleted through the Keymanager API if a --web3signer-urls signer still serves it, or if it had a validators.yml entry when Grandine started.

Keys loaded from --web3signer-urls have no validators.yml entry, so:

• they use the process-wide defaults: --suggested-fee-recipient, --default-gas-limit and --graffiti;
• the Keymanager API rejects per-validator settings for them with 404;
• enabled: false in validators.yml does not stop them. Narrow the keys with --web3signer-public-keys instead.

To give one of them its own settings, give it an entry. Import it through POST /eth/v1/remotekeys (this records an entry even for a key already loaded from a URL), or add the entry by hand. The URL keeps loading all its other keys as before.

Declaring keys in validators.yml

A web3signer entry names a key and the signer that serves it:

- enabled: true
  pubkey: "0xb301803f8b5ac4a1133581fc676dfedc60d891dd5fa99028805e5ea5b08d3491af75d0707adab3b70c6a6a580217bf81"
  type: web3signer
  url: http://SIGNER-HOST:9000
  fee_recipient: "0x0000000000000000000000000000000000000002"
  graffiti: "my-staking-node"

Grandine loads these keys on every start, with no --web3signer-urls needed. A URL named only by entries loads only the keys declared for it, plus any key listed in --web3signer-public-keys that it serves. A declared key the signer does not serve is skipped, with a warning:

Web3Signer at URL does not serve declared validator 0x...; it will not be loaded

Remote keys imported through the Keymanager API get such entries automatically. See Keymanager API.

A key that Grandine holds locally, in a keystore or the legacy storage, is never sent to a signer, even if a signer also serves it.

TLS and mutual TLS

There are no command line options for TLS material. To reach a signer behind a private certificate authority, or one that requires a client certificate, set the options on a web3signer entry:

- enabled: true
  pubkey: "0xb301803f8b5ac4a1133581fc676dfedc60d891dd5fa99028805e5ea5b08d3491af75d0707adab3b70c6a6a580217bf81"
  type: web3signer
  url: https://SIGNER-HOST:9000
  root_certificate_path: /etc/grandine/signer-ca.pem
  client_identity_path: /etc/grandine/grandine-client.p12
  client_identity_password: "YOUR-P12-PASSWORD"
  request_timeout_ms: 5000

• root_certificate_path: a PEM certificate added to the trusted roots.
• client_identity_path and client_identity_password: a PKCS#12 file with the client certificate and key, for mutual TLS. An empty password is used if client_identity_password is omitted.
• request_timeout_ms: the timeout of each signing request, in milliseconds. Without it, --request-timeout applies (default 30000).

Grandine builds one HTTP client for all Web3Signer requests, so these options apply to every signer, including the ones in --web3signer-urls. All enabled entries that set them must set the same values. Grandine refuses to start if two entries differ. Entries without TLS options use the same client.

Performance

Remote signing adds a network round trip to every signature. Keep the signer close to the validator client, on the same host or a low-latency link. Signatures needed by a deadline in the slot (attestations, blocks, aggregates and sync committee messages) are all requested at once. The larger batches signed once per epoch, selection proofs and builder registrations, are limited to 32 requests in flight so that they do not flood the signer.

Command line options

• --web3signer-urls (no default): Web3Signer base URLs whose keys are loaded. Space-separated. Web3Signer is not used through the command line without it.
• --web3signer-public-keys (no default, which loads every key): comma-separated keys to load from the --web3signer-urls signers.
• --web3signer-refresh-keys-every-epoch (disabled): fetch the key list of every signer every epoch, adding keys without a restart.
• --web3signer-api-urls: deprecated alias of --web3signer-urls.
• --request-timeout (default 30000): default request timeout in milliseconds, used for signing requests unless request_timeout_ms is set.

See Command line reference for the full list.

Slashing protection

A validator is slashed when it signs two conflicting messages: two different blocks for the same slot, two attestations with the same target epoch, or one attestation that surrounds another. Slashing destroys part of the stake and forces the validator out of the network. It almost always happens to honest operators who run the same key in two places, or who move a key without its signing history.

Grandine keeps a local record of everything each validator has signed and refuses to sign anything that conflicts with it. This record is the slashing protection database. It is always on, for local keystores and for keys held by a Web3Signer.

How it works

Before Grandine signs a block or an attestation, it checks the message against the database and stores it in the same transaction:

• Blocks. A second block for a slot that already has a different signed block is rejected. So is a block for a slot lower than the lowest slot on record for that validator.
• Attestations. An attestation is rejected if it has the same target epoch as a stored one with a different signing root, if it surrounds a stored attestation or is surrounded by one, or if its source or target epoch is lower than the lowest one on record.
• Clock. Grandine stores the current epoch. If the system clock goes back to an earlier epoch, it refuses to sign until the clock catches up.

Identical messages (same slot or target and same signing root) are not signed twice either. When a message is refused, Grandine logs a warning such as:

slashing protector rejected slashable attestation (error: invalid attestation ...)
slashing protector rejected slashable beacon block (error: duplicate signed beacon block proposal ...)
slashing protector ignored duplicate attestation: ...

One such line after a restart or a reorg is normal. Repeated rejections for the same validator mean that something else may be signing with the key. Stop and investigate.

Grandine keeps the records of the last 256 epochs (--slashing-protection-history-limit) and prunes older ones once per epoch. The lowest epoch and slot still on record act as a low watermark, so pruning never allows an older, conflicting message.

Note: Keep Web3Signer's own slashing protection enabled as well. Grandine checks every message before it asks any signer, but a second, independent check costs little.

Where the database lives

The database is an SQLite file named slashing_protection.sqlite in the validator directory:

~/.grandine/<NETWORK>/validator/slashing_protection.sqlite

With --data-dir /path/to/data, it is /path/to/data/<NETWORK>/validator/slashing_protection.sqlite. The network name is appended to the data directory, so mainnet and Hoodi never share a database.

Grandine opens the database in exclusive locking mode, as EIP-3076 recommends. While Grandine runs, no other process can open it. This includes the grandine interchange subcommand, so stop the client before you import or export.

Older versions of Grandine kept the file in the beacon directory. On startup, Grandine moves it to the validator directory. When a schema migration needs it, Grandine also writes a backup file named interchange-<TIMESTAMP>.json into the validator directory before it changes the database.

Warning: With --in-memory, the slashing protection database exists only in memory and is lost when the process exits. Never run real validators with --in-memory.

Each validator client keeps its own database. A separate validator client (grandine vc) stores it in its own data directory. Never let two Grandine processes share a data directory, and never run the same keys in two processes, even if each has protection: the two databases do not know about each other.

Exporting and importing (EIP-3076)

The EIP-3076 interchange format is a JSON file that every major consensus client can read and write. Use it to move validators between machines or clients. Grandine reads and writes version 5 of the format.

The interchange subcommand

grandine [OPTIONS] interchange export FILE
grandine [OPTIONS] interchange import FILE

Options such as --network and --data-dir must come before interchange. The subcommand itself takes only the file path. This is the opposite of bn and vc, whose options come after the command name.

Export the slashing protection data of a mainnet node:

grandine --network mainnet interchange export slashing_protection.json

Import a file into a Hoodi node with a custom data directory:

grandine --network hoodi --data-dir /var/lib/grandine interchange import slashing_protection.json

Use the same --network and --data-dir that the node or validator client runs with. Otherwise Grandine reads or writes a different database. This applies to data from a separate grandine vc too: pass the validator client's own --data-dir.

On export, Grandine logs the number of records for each validator. If nothing was exported, it warns:

no records were exported. This may indicate an issue if active validators are present. Please verify your configuration settings.

That usually means the wrong --network or --data-dir. On import, it reports the totals:

interchange file imported (imported records: 1234, failed records: 0)

An import is rejected as a whole if the file's genesis_validators_root does not belong to the selected network, or if its format version is not 5. Within a valid file, Grandine imports every record it can and counts the rest as failed. Duplicate public keys in the file are accepted. Records that are already in the database cannot be stored twice, so importing the same file again reports them as failed. That is harmless.

For predefined networks, the subcommand works offline. For a custom network, Grandine needs the genesis state (for example from --configuration-directory or --genesis-state-file) to find the genesis validators root.

With Docker

The image's entry point is grandine, and it runs as root, so the data directory is /root/.grandine inside the container. Stop the running container first, then:

docker run --rm                         \
  -v $HOME/.grandine:/root/.grandine    \
  sifrai/grandine:stable                \
  --network mainnet                     \
  interchange export /root/.grandine/slashing_protection.json

The file appears as $HOME/.grandine/slashing_protection.json on the host.

Through the Keymanager API

The Keymanager API also carries interchange data. DELETE /eth/v1/keystores returns the slashing protection data of the deleted keys, and POST /eth/v1/keystores accepts it in the slashing_protection field. Use this when another tool moves keys between running clients.

Moving validators safely

The rule is simple: the old client must stop signing before the new one starts, and the new one must know what the old one signed.

1. Sync the new node first, so it is ready to take over.
2. Stop the old validator client, and make sure it cannot start again (disable its service, or remove its keys).
3. Export the slashing protection data from the old client.
4. Import the file into Grandine with grandine ... interchange import FILE.
5. Import the keys (see Managing validator keys).
6. Wait until at least one or two full epochs have passed since the old client's last attestation. You can check this on a block explorer.
7. Start Grandine. Consider doppelganger protection as a final check.

Waiting is cheap: a few missed attestations cost far less than a slashing. See also Migrating from another client.

If the database is lost

Without its database, Grandine has no memory of what the validators signed. It creates a new, empty database and would sign anything that does not conflict with messages it signs from then on. If the old messages are still recent, a new one may conflict with them.

What to do:

1. Restore from a backup. If you have a recent interchange export, or a copy of slashing_protection.sqlite taken while Grandine was stopped, restore it.
2. Look for automatic backups. Check the validator directory for interchange-*.json files written during a migration. They are older, but still raise the low watermark.
3. If you have nothing, wait. Keep the validators offline for at least two full epochs after their last attestation and proposal, as shown on a block explorer. New attestations then have higher source and target epochs than anything signed before, and new blocks are for later slots, so they cannot conflict. Enable doppelganger protection when you start again.

Tip: Back up the slashing protection data regularly, and always before an upgrade or a move. Stop Grandine, then either run interchange export or copy slashing_protection.sqlite. A copy taken while Grandine runs may be inconsistent.

Command line options

• --slashing-protection-history-limit (default: 256): number of epochs of signing history to keep in the database.
• --data-dir (default: $HOME/.grandine): parent directory of the per-network data, including the validator directory that holds the database.
• --network (default: mainnet): selects the per-network directory and the genesis validators root that interchange files must match.
• --in-memory (default: disabled): keeps all data, including slashing protection, in memory only.

See the command line reference for the full list.

Doppelganger protection

A doppelganger is a second copy of your validator: the same key, running somewhere else at the same time. It may be an old machine you forgot to stop, a backup that started on its own, or a failover script that fired too early. Two copies of a key will sooner or later sign conflicting messages, and the validator will be slashed. Slashing protection cannot prevent this, because each copy has its own database.

Doppelganger protection is a safety check at startup. Before Grandine lets a validator sign, it watches the network for a while to see whether that validator is already active. If it is, Grandine does not sign anything and shuts down.

When to use it

Enable it whenever there is any chance that a key runs elsewhere:

• after moving validators from another machine or another client;
• after restoring from a backup;
• in setups with manual or automatic failover.

The cost is a few missed epochs of attestations on every start (see The cost). Many operators keep it on all the time and accept that cost on each restart.

Warning: Doppelganger protection is a last line of defense, not a license to run keys in two places. It can only see a duplicate that is online and attesting while the check runs. A duplicate that starts later, after the check has passed, is not detected. Never run the same keys in two validator clients, even briefly.

Enabling it

With the built-in validator client

When Grandine runs the beacon node and the validator client in one process, enable liveness tracking as well. The built-in beacon node uses it to see which validators are active:

grandine                                  \
  --network mainnet                       \
  --jwt-secret /path/to/jwt.hex           \
  --eth1-rpc-urls http://EL-HOST:8551     \
  --keystore-dir /path/to/validator_keys  \
  --keystore-password-file /path/to/password.txt \
  --track-liveness                        \
  --detect-doppelgangers

Without --track-liveness, Grandine refuses to start:

--track-liveness must be specified for --detect-doppelgangers with the built-in beacon node

With a separate validator client

grandine vc asks its beacon nodes through the standard liveness endpoint, POST /eth/v1/validator/liveness/{epoch}. It takes no --track-liveness option:

grandine vc                                   \
  --network hoodi                             \
  --beacon-node-urls http://BN-HOST:5052      \
  --keystore-dir /path/to/validator_keys      \
  --keystore-password-file /path/to/password.txt \
  --detect-doppelgangers

Every beacon node in --beacon-node-urls that the validator client may use must serve that endpoint. A Grandine beacon node serves it only when started with --track-liveness. Without it, the endpoint returns an error, the check cannot complete, and the validators never start their duties. The validator client logs:

doppelganger liveness check failed: ...

How it works

1. Tracking starts. At startup, every loaded key is put under watch. Keys added later, through the Keymanager API or a Web3Signer refresh, are put under watch at the next epoch boundary.

2. Liveness checks. Late in every slot, Grandine asks whether any watched validator was seen attesting or proposing in the previous or the current epoch. A beacon node with liveness tracking records this from attestations and blocks it sees on the network.

3. Waiting period. A validator stays under watch for 2 epochs (64 slots, about 13 minutes on mainnet) from the slot its watch started. During that time it performs no duties. Grandine logs:

   Validator 0x93...4a skipping attesting duty in slot 123456 since not enough time has passed to ensure there are no doppelganger validators participating on network. Validator will start performing duties on slot 123520.
   

4. Release. Once the waiting period is over and the latest checks found nothing, the validator starts performing its duties normally.

Some details:

• A key whose validator index is not yet known (for example, a deposit that has not been processed) cannot be checked. Grandine warns that it "cannot be checked until its index is known; it will not perform duties". Its waiting period starts when the index becomes known.
• If a liveness query fails, the release is postponed until a check succeeds. Grandine never releases a validator on a failed check.
• At the genesis slot of a new network, there is nothing to watch, so validators start at once.

Besides skipping duties, the signer itself refuses to sign for a watched validator, as a second safety net:

Doppelganger protection prevented validator 0x93...4a from signing a message since not enough time has passed to ensure there are no duplicate validators participating on network

When a doppelganger is detected

If any check finds a watched validator active on the network, Grandine stops. It does not restart itself, unlike after most other runtime errors. The log shows the public keys involved:

application runtime failed: Doppelganger validators detected on the network (public keys: [0x93...4a])

The process then exits with exit code 1. With the built-in validator client, this stops the whole process, including the beacon node.

What to do:

1. Do not restart Grandine until you have found the other instance.
2. Find and stop the other instance that uses the listed keys: an old server, a container, a backup, a failover partner, or another client.
3. If you cannot find it, remove the listed keys from Grandine and keep them offline. A key that another machine is signing with must not be started here.
4. Once only one instance remains, start Grandine again. The check runs again from the start.

Warning: Exit code 1 is also used for other startup failures, so a service manager cannot tell the difference. With systemd's Restart=always or Docker's --restart always, Grandine starts again, waits, detects the doppelganger again and exits again, in a loop. No messages are signed during this loop, but alert on the log line above so that you notice. See Running as a service.

The cost

On every start with doppelganger protection, each validator misses its duties for about 2 to 3 epochs: the 2-epoch waiting period, plus the time until the next check after it ends. The missed attestations cost roughly as much as being offline for the same time. A proposal or sync committee duty that falls in that window is also missed.

For short restarts, such as an upgrade, you can disable doppelganger protection if you are sure no other instance exists. Keep it on when keys have just moved.

Command line options

• --detect-doppelgangers (default: disabled): enables doppelganger protection. With the built-in beacon node, it requires --track-liveness.
• --track-liveness (default: disabled): makes the beacon node record which validators attest and propose, and serve POST /eth/v1/validator/liveness/{epoch}. A beacon node option, not accepted by grandine vc.

See the command line reference for the full list.

Fee recipient, graffiti and gas limit

When one of your validators proposes a block, three settings shape it:

• the fee recipient: the execution-layer address that receives the priority fees (tips) of the block and any MEV payment;
• the graffiti: 32 bytes of free text stored in the block;
• the gas limit: the gas limit the validator asks for. Each block can move the network's gas limit a little toward the proposer's preference.

Each can be set once for all validators on the command line, and overridden per validator in validators.yml or through the Keymanager API.

Fee recipient

Warning: Always set a fee recipient that you control. If you set none, the integrated validator client sends your block rewards from fees to the Grandine team's donation address (see Default behavior).

Set one address for all validators with --suggested-fee-recipient:

grandine                                   \
  --network mainnet                        \
  --jwt-secret /path/to/jwt.hex            \
  --eth1-rpc-urls http://EL-HOST:8551      \
  --keystore-dir /path/to/validator_keys   \
  --keystore-password-file /path/to/password.txt \
  --suggested-fee-recipient 0xYOUR_EXECUTION_ADDRESS

With Docker, add the same option after the image name:

docker run                                       \
  -v $HOME/.grandine:/root/.grandine             \
  sifrai/grandine:stable                         \
  --network mainnet                              \
  --suggested-fee-recipient 0xYOUR_EXECUTION_ADDRESS \
  ...

The address must be a valid 20-byte execution address. Use an address whose private key you hold, or a smart contract you have checked, such as a pool's fee distributor. It does not need to match the withdrawal address.

At startup, Grandine logs the default it will use:

suggested fee recipient: 0x...

Check this line after every configuration change.

Per-validator fee recipients

Add fee_recipient to a validator's entry in validators.yml (~/.grandine/<NETWORK>/validator/validators.yml by default):

- enabled: true
  pubkey: "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"
  type: local_keystore
  keystore_path: /path/to/validator_keys/keystore-0.json
  keystore_password_path: /path/to/secrets/keystore-0.txt
  fee_recipient: "0x0000000000000000000000000000000000000001"

Edit the file only while Grandine is stopped, or use the Keymanager API, which changes it for you at runtime.

Fee recipient precedence

For a validator run by the integrated validator client, the first match wins:

1. fee_recipient in the validator's validators.yml entry (also what the Keymanager API sets);
2. --suggested-fee-recipient;
3. the Grandine donation address.

When a Grandine beacon node builds a block for a validator, it uses, in order:

1. the address a validator client sent through POST /eth/v1/validator/prepare_beacon_proposer (remote validator clients send this; the integrated one does not need to);
2. the validators.yml entry on the beacon node, if it has one for that validator;
3. the beacon node's --suggested-fee-recipient;
4. the Grandine donation address.

Default behavior when none is set

• Integrated validator client (plain grandine with keys): Grandine falls back to its donation address, 0xe7cf7C3BA875Dd3884Ed6a9082d342cb4FBb1f1b. This address is used for local blocks and sent to the builder in validator registrations. The startup line suggested fee recipient shows it.
• Beacon node only (grandine bn): when a remote validator client does not send a fee recipient for a validator, the beacon node uses its own --suggested-fee-recipient, or else the donation address. Set --suggested-fee-recipient on beacon nodes that serve other validator clients.
• Separate validator client (grandine vc): there is no fallback address. A validator without a fee recipient, either from --suggested-fee-recipient or from validators.yml, is not reported to the beacon nodes through prepare_beacon_proposer, is not registered with a builder, and gets no proposer preferences published (from Gloas on). The beacon node then pays its own default fee recipient for blocks it builds locally. See Validator client.

Graffiti

Set a default graffiti with --graffiti:

grandine ... --graffiti "my staking setup"

The text must fit in 32 bytes (UTF-8). A longer value is rejected at startup.

--graffiti can be given more than once. Grandine then rotates through the values, one per block proposal:

grandine ... --graffiti "first" --graffiti "second" --graffiti "third"

Per-validator graffiti goes in validators.yml:

- enabled: true
  pubkey: "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"
  type: local_keystore
  keystore_path: /path/to/validator_keys/keystore-0.json
  keystore_password_path: /path/to/secrets/keystore-0.txt
  graffiti: "validator zero"

Precedence: the validators.yml entry (or the Keymanager API), then --graffiti, then none. Grandine validates the per-validator graffiti on startup and refuses to start if it is too long.

Client fingerprint ("blockprint")

When a Grandine beacon node builds a block, it appends a short client fingerprint to the graffiti if there is room. The fingerprint names the execution client and both clients' commits, followed by Grandine/<version>. For example, the graffiti test with a Besu execution client may become:

test BU4e2eGR6a37 Grandine/1.2.3

Parts that do not fit are left out, and your own text is never truncated. With no graffiti set, the block carries only the fingerprint. This helps researchers measure client diversity. To turn it off, start the beacon node with --disable-blockprint-graffiti. It is a beacon node option: with grandine vc, the remote beacon node decides.

Gas limit

By default, Grandine asks for the gas limit scheduled by the network configuration for the current epoch, or 60,000,000 when no schedule applies. Change the default for all validators with --default-gas-limit:

grandine ... --default-gas-limit 60000000

Per validator, set gas_limit in validators.yml:

- enabled: true
  pubkey: "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"
  type: local_keystore
  keystore_path: /path/to/validator_keys/keystore-0.json
  keystore_password_path: /path/to/secrets/keystore-0.txt
  gas_limit: 60000000

Precedence: the validators.yml entry (or the Keymanager API), then --default-gas-limit, then the scheduled limit, then 60,000,000.

The gas limit goes into builder registrations and, from the Gloas fork on, into the proposer preferences that Grandine publishes. If a configured limit exceeds the scheduled one, Grandine warns when it registers validators:

3 validators configured gas limit exceeds the recommended gas limit of 60000000

Tip: Change the gas limit only if you follow the community's coordination on it. Most operators should leave the default.

Changing settings at runtime with the Keymanager API

The Keymanager API changes these settings without a restart. Grandine stores the change in the validator's validators.yml entry, so it survives restarts. Enable the API with --enable-validator-api. It listens on 127.0.0.1:5055 by default and requires the bearer token stored in api-token.txt in the validator directory (or in --validator-api-token-file).

TOKEN=$(cat ~/.grandine/mainnet/validator/api-token.txt)
PUBKEY=0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a

Fee recipient:

# Read
curl -H "Authorization: Bearer $TOKEN" \
  http://127.0.0.1:5055/eth/v1/validator/$PUBKEY/feerecipient

# Set
curl -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"ethaddress": "0xYOUR_EXECUTION_ADDRESS"}' \
  http://127.0.0.1:5055/eth/v1/validator/$PUBKEY/feerecipient

# Remove the override (fall back to the default)
curl -X DELETE -H "Authorization: Bearer $TOKEN" \
  http://127.0.0.1:5055/eth/v1/validator/$PUBKEY/feerecipient

Gas limit:

curl -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"gas_limit": "60000000"}' \
  http://127.0.0.1:5055/eth/v1/validator/$PUBKEY/gas_limit

Graffiti:

curl -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"graffiti": "my staking setup"}' \
  http://127.0.0.1:5055/eth/v1/validator/$PUBKEY/graffiti

GET and DELETE work the same way for gas_limit and graffiti. A successful POST returns 202 Accepted, and a DELETE returns 204 No Content.

Notes:

• GET .../feerecipient returns the value that applies, including the default. With grandine vc and no fee recipient configured, it returns 404.
• GET .../gas_limit returns the effective gas limit, including the scheduled default.
• A POST or DELETE for a validator without a validators.yml entry returns 404. Validators loaded only from --web3signer-urls have no entry. See Web3Signer for how to give one an entry.

Command line options

• --suggested-fee-recipient (default: none; the integrated validator client and grandine bn fall back to the Grandine donation address): default fee recipient for all validators.
• --graffiti (default: none): default block graffiti, up to 32 bytes. Repeat it to rotate values.
• --disable-blockprint-graffiti (default: disabled): beacon node option. Do not append the client fingerprint to the graffiti.
• --default-gas-limit (default: the scheduled gas limit, or 60000000): default gas limit for all validators.
• --enable-validator-api (default: disabled): enables the Keymanager API used to change these settings at runtime.

See the command line reference for the full list.

Builder API and MEV

Specialized block builders assemble execution payloads that pay more than a payload your own execution client builds. They compete for the right to fill your block and pay the proposer for it. This value is called MEV (maximal extractable value). The Builder API is the standard interface between a consensus client and these builders.

Grandine supports the Builder API and connects to exactly one builder URL. To reach several relays, run a relay multiplexer such as mev-boost or mev-rs and point Grandine at it.

Using a builder is optional. Without one, Grandine always builds blocks with your own execution client, and you still receive the priority fees through your fee recipient.

How it works (before Gloas)

1. Registration. Once per epoch, Grandine signs a validator registration for each validator: the fee recipient, the gas limit and a timestamp. It sends the registrations to the builder, so relays know where to pay you.
2. Bidding. When one of your validators proposes, Grandine asks the builder for a payload header (/eth/v1/builder/header/...), with a 1 second timeout. At the same time, it asks your execution client for a local payload.
3. Selection. Grandine compares the builder's bid with the value of the local payload, adjusted by the builder boost factor. The more profitable one wins.
4. Reveal. If the builder wins, Grandine signs a blinded block and sends it to the builder, which returns the full payload and publishes the block.

If the builder has no bid, the request times out, or the circuit breaker is tripped, Grandine uses the local payload. You do not lose the slot because a relay is down. A builder can still fail after you sign a blinded block, by not revealing the payload in time, and then the block is missed.

Setting up mev-boost

1. Install and start mev-boost on the same machine or network, with the relays you choose. For example:

   mev-boost -mainnet -relay-check \
     -relays https://0xRELAY_PUBKEY@RELAY-HOST,https://0xRELAY_PUBKEY_2@RELAY-HOST-2
   

   By default mev-boost listens on localhost:18550; -addr changes it. See the mev-boost documentation for current flags and relay lists.

2. Start Grandine with --builder-url pointing at mev-boost, and a fee recipient you control:

   grandine                                   \
     --network mainnet                        \
     --jwt-secret /path/to/jwt.hex            \
     --eth1-rpc-urls http://EL-HOST:8551      \
     --keystore-dir /path/to/validator_keys   \
     --keystore-password-file /path/to/password.txt \
     --suggested-fee-recipient 0xYOUR_EXECUTION_ADDRESS \
     --builder-url http://127.0.0.1:18550
   

   With Docker, make sure the container can reach mev-boost, for example by putting both on the same Docker network and using --builder-url http://mev-boost:18550.

3. Check the logs. When a validator proposes, Grandine logs whether the builder delivered:

   received execution payload header from builder for slot 123456
   validator 4242 proposing beacon block with root 0x... in slot 123456 using the builder API
   

   or why it built locally:

   builder has no execution payload header available for slot 123456
   using more profitable local payload: local MEV: ..., builder MEV: ..., boosted builder MEV: ..., builder_boost_factor: 100
   cannot use Builder API for execution payload header: 4 consecutive missing blocks since head
   

--builder-api-url is the deprecated name of --builder-url. It still works, with a warning. If both are given, --builder-url wins.

Data format

--builder-format sets the encoding Grandine uses with the builder: ssz (the default) or json. With ssz, Grandine asks for SSZ but also accepts JSON responses. If the builder rejects SSZ registrations, Grandine retries them in JSON and keeps using JSON for registrations from then on. Set --builder-format json only if your builder or relay multiplexer has trouble with SSZ.

Circuit breaker

When the chain is unhealthy, for example after several missed slots, builders are more likely to fail. Grandine then stops asking the builder and builds locally. The checks run before every proposal:

• If more than --builder-max-skipped-slots (default 3) slots since the head block are empty, Grandine builds locally.
• If more than --builder-max-skipped-slots-per-epoch (default 8) slots in the last 32 slots (one rolling epoch) are empty, Grandine builds locally.

Both thresholds are exclusive: with the default of 3, the fourth consecutive missing block trips the breaker. When it trips, Grandine logs:

cannot use Builder API for execution payload header: 4 consecutive missing blocks since head
cannot use Builder API for execution payload header: 9 missing blocks in the last rolling epoch

--builder-disable-checks turns the circuit breaker off, so the builder is always asked. This is not recommended on mainnet.

Builder boost factor

When Grandine has both a builder's bid and a locally built payload, it scales the bid's value by the builder boost factor, a percentage, and then takes the payload that is worth more:

• 100 (the default): pick whichever pays more;
• 0: always build locally;
• 90: use the builder only if its bid, reduced by 10%, still beats the local payload. This favors local blocks a little, which helps censorship resistance;
• a very large value: always prefer the builder whenever it delivers a bid.

Set the default for all validators with --default-builder-boost-factor. Override it per validator in validators.yml:

- enabled: true
  pubkey: "0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a"
  type: local_keystore
  keystore_path: /path/to/validator_keys/keystore-0.json
  keystore_password_path: /path/to/secrets/keystore-0.txt
  builder_boost_factor: 90

Three per-validator fields exist. The first that applies wins:

1. prefer_builder_proposals: true: always prefer the builder (the factor becomes the maximum);
2. builder_boost_factor: N: use N;
3. builder_proposals: false: never use the builder (the factor becomes 0);
4. otherwise, --default-builder-boost-factor.

Validator registration

Registrations are sent once per epoch, in batches of up to 500, for every validator that has a fee recipient. They use the fee recipient and gas limit that apply to each validator (see Fee recipient, graffiti and gas limit). With the integrated validator client, a validator without its own fee recipient is registered with --suggested-fee-recipient, or else with the Grandine donation address, so always set your own.

A Grandine beacon node also accepts registrations from remote validator clients through POST /eth/v1/validator/register_validator, and forwards them to its builder together with its own.

Registration failures are logged as warnings:

failed to register validator batch: ...

Separate validator client

The builder options above are beacon node options. grandine vc refuses them: the builder is configured on the beacon nodes it connects to. The validator client opts in with --use-builder:

grandine vc                                      \
  --network mainnet                              \
  --beacon-node-urls http://BN-HOST:5052         \
  --keystore-dir /path/to/validator_keys         \
  --keystore-password-file /path/to/password.txt \
  --suggested-fee-recipient 0xYOUR_EXECUTION_ADDRESS \
  --use-builder

With --use-builder, the validator client signs registrations and sends them to its beacon nodes through POST /eth/v1/validator/register_validator. It also passes each validator's builder boost factor with its block requests. Without --use-builder, it asks only for local payloads (a boost factor of 0) and sends no registrations, because a beacon node without a builder rejects them.

With grandine vc, a validator without a fee recipient is not registered at all. See Validator client.

At the Gloas fork (ePBS)

The Gloas fork brings enshrined proposer-builder separation (ePBS, EIP-7732). Builders become part of the protocol: they bid for blocks over the peer-to-peer network, and the execution payload is published separately from the beacon block. At the time of writing, this Grandine version schedules Gloas on Sepolia and Hoodi, but not on mainnet. What changes in Grandine:

• Bids come from gossip. The beacon node picks the highest bid it has seen for the slot. --builder-url and validator registrations are no longer used for this. Grandine stops sending registrations once Gloas is active.

• Proposer preferences. Instead of registrations, the validator client publishes signed proposer preferences (fee recipient and gas limit) for its upcoming proposals, starting one epoch before the fork. Validators without a fee recipient get none.

• Boost factor. The builder boost factor still applies. Grandine compares the bid with the value of a local payload in the same way. With a factor of 0, it always builds the payload itself.

• Circuit breaker. Missing blocks are no longer the signal. Grandine instead counts canonical blocks whose builder did not reveal the payload in time, judged only for blocks that at least 60% of the committee attested. The same options set the thresholds: --builder-max-skipped-slots for consecutive withheld payloads and --builder-max-skipped-slots-per-epoch for the last rolling epoch. When it trips, Grandine builds payloads locally for one epoch, or less if builders recover first:

  builders withheld 4 payloads in a row, building payloads locally
  

  --builder-disable-checks disables this check too.

• --use-builder on vc matters only until Gloas. From Gloas on, a separate validator client always lets its beacon nodes consider builder bids, with the validator's boost factor.

Command line options

Beacon node options (not accepted by grandine vc):

• --builder-url (default: none): URL of the external block builder or relay multiplexer.
• --builder-api-url (deprecated): old name of --builder-url.
• --builder-format (default: ssz): data format for the Builder API, ssz or json.
• --builder-disable-checks (default: disabled): always use the builder, without circuit breaker checks.
• --builder-max-skipped-slots (default: 3): consecutive missing blocks (withheld payloads from Gloas on) tolerated before building locally.
• --builder-max-skipped-slots-per-epoch (default: 8): missing blocks (withheld payloads from Gloas on) in the last rolling epoch tolerated before building locally.

Options for both the beacon node and the validator client:

• --default-builder-boost-factor (default: 100): the percentage by which a builder's bid is scaled before Grandine weighs it against the local payload.
• --suggested-fee-recipient (default: none; plain grandine and grandine bn fall back to the Grandine donation address, grandine vc has no fallback): fee recipient used in registrations and payloads. Set your own, or priority fees and MEV payments can go to the donation address.
• --default-gas-limit (default: the scheduled gas limit, or 60000000): gas limit used in registrations.

Validator client option (grandine vc only):

• --use-builder (default: disabled): register validators with the builder behind the beacon nodes and accept its payloads. Relevant only until Gloas.

See the command line reference for the full list.

Voluntary exits, withdrawals and consolidations

This page covers how a validator's stake leaves the beacon chain or moves between validators:

• a voluntary exit stops a validator for good, and its full balance is then withdrawn;
• a BLS-to-execution change sets the withdrawal address of a validator that still has old BLS (0x00) withdrawal credentials;
• withdrawals pay balances to the withdrawal address automatically;
• execution-layer requests (EIP-7002 and EIP-7251) let the withdrawal address trigger exits, partial withdrawals and consolidations.

Warning: An exit cannot be cancelled or undone. Once a validator has exited, its key can never validate again. Practice on the Hoodi testnet first.

Withdrawals in brief

Withdrawals need execution-layer withdrawal credentials: 0x01 (classic) or 0x02 (compounding, from the Electra fork on). Validators created with 0x00 credentials must first switch with a BLS-to-execution change.

The beacon chain pays withdrawals automatically. No transaction and no fee is needed. Each block processes the next validators in index order (the "sweep"), up to 16 withdrawals per block on mainnet, and starts over from the first validator once it reaches the end. Depending on the size of the validator set, it takes several days for the sweep to come around to the same validator.

• Partial withdrawals (skimming). For a 0x01 validator, any balance above 32 ETH is paid out on each pass of the sweep. For a 0x02 validator, only the balance above 2048 ETH is.
• Full withdrawals. After a validator has exited and become withdrawable, the sweep pays its whole balance.

Grandine needs no configuration for any of this. It follows the protocol rules like every other consensus client.

Voluntary exit

The protocol accepts an exit only from a validator that has spent 256 epochs or more in the active set, which is about 27 hours on Mainnet. After the exit is included in a block, the validator joins the exit queue. It keeps performing its duties until its exit epoch, so keep it running until then. After it exits, it becomes withdrawable about 256 epochs (about 27 hours) later, and the next pass of the sweep pays the balance.

Grandine has no exit subcommand. You create a signed exit through the Keymanager API and submit it to a beacon node. This works with the integrated validator client and with grandine vc, for any key Grandine signs with, including keys held by a Web3Signer.

Step 1: create the signed exit

Start Grandine with the validator API enabled (--enable-validator-api). It listens on 127.0.0.1:5055 by default and requires the bearer token from api-token.txt in the validator directory. See Keymanager API.

TOKEN=$(cat ~/.grandine/mainnet/validator/api-token.txt)
PUBKEY=0x93247f2209abcacf57b75a51dafae777f9dd38bc7053d1af526f220a7489a6d3a2753e5f3e8b1cfe39b56f43611df74a

curl -X POST -H "Authorization: Bearer $TOKEN" \
  http://127.0.0.1:5055/eth/v1/validator/$PUBKEY/voluntary_exit \
  > exit.json

The response holds the signed exit message:

{
  "data": {
    "message": { "epoch": "123456", "validator_index": "4242" },
    "signature": "0x..."
  }
}

By default the exit is for the current epoch. Add ?epoch=EPOCH to the URL to choose another epoch. An exit with a future epoch only becomes valid when that epoch arrives.

This endpoint only signs. It does not publish the exit, so nothing happens until step 2. Keep the file private: anyone who holds it can publish it.

The request fails with 404 if the key is not loaded in this Grandine instance, or if the beacon chain does not know the validator yet.

Step 2: submit it to a beacon node

Send the data part of the response to the beacon node's pool endpoint:

jq '.data' exit.json > signed_exit.json

curl -X POST -H "Content-Type: application/json" \
  -d @signed_exit.json \
  http://127.0.0.1:5052/eth/v1/beacon/pool/voluntary_exits

Any consensus client's beacon node accepts it. A Grandine beacon node validates the exit, adds it to its pool and publishes it to the network. An invalid exit, for example one for a validator that has not been active for 256 epochs, is rejected with an error message.

Step 3: track the exit

Watch the validator's status with the Beacon Node API:

curl http://127.0.0.1:5052/eth/v1/beacon/states/head/validators/$PUBKEY

The status changes from active_ongoing to active_exiting once the exit is included, then to exited_unslashed, withdrawal_possible and finally withdrawal_done. A block explorer such as beaconcha.in shows the same.

Keep the validator running until its status is exited_unslashed. Until then it still has duties, and missing them costs rewards.

Other tools

Tools outside Grandine can also sign exits, for example offline from the mnemonic or a keystore. The ethstaker-deposit-cli does it with exit-transaction-keystore and exit-transaction-mnemonic, and ethdo can do it too. Submit their output with step 2.

BLS-to-execution change

Withdrawals need an execution address to go to, and 0x00 credentials do not contain one, so such a validator is never paid out. A one-time BLS-to-execution change, signed with the withdrawal key derived from your mnemonic, sets a 0x01 address. It can be done only once and cannot be changed later.

Grandine does not hold the withdrawal key, so it cannot sign this message. Generate it offline, for example with the ethstaker-deposit-cli generate-bls-to-execution-change command, then submit the resulting JSON array to the beacon node:

curl -X POST -H "Content-Type: application/json" \
  -d @bls_to_execution_change.json \
  http://127.0.0.1:5052/eth/v1/beacon/pool/bls_to_execution_changes

The endpoint takes a JSON array, so one file can carry changes for many validators. Each change is validated, and if some fail, the response lists them by their index in the array.

Warning: Double-check the execution address before you submit. A wrong address cannot be corrected, and everything the validator ever withdraws goes there.

Execution-layer requests (EIP-7002 and EIP-7251)

Since the Electra fork, the owner of the withdrawal address can act from the execution layer. These requests are transactions to system contracts on the execution chain, sent from the withdrawal address and paying a small, dynamic fee. They do not go through Grandine's APIs at all. The execution client puts them into the payload, and the beacon chain processes them in the block.

• Exit (EIP-7002). The withdrawal address can exit a validator without the validator's signing key. This helps when the signing key is lost or held by an operator you no longer trust.
• Partial withdrawal (EIP-7002). A 0x02 validator can withdraw part of its balance above 32 ETH on request, instead of waiting for the sweep.
• Consolidation (EIP-7251). Moves the balance of one validator into another with the same withdrawal address, so fewer validators can hold more stake (up to 2048 ETH each). A request whose source and target are the same validator switches it from 0x01 to 0x02 credentials.

Use a tool or interface that supports these contracts, such as the staking launchpad or a wallet with that feature, and check that it targets the right network. Grandine needs no option for them. After the request is included, track the validator's status as in step 3.

Related pages

• Keymanager API for the API token and other endpoints.
• Managing validator keys for removing keys after an exit.
• Becoming a validator for withdrawal credentials when you create keys.
• Beacon Node API for the pool and validator status endpoints.

Command line options

• --enable-validator-api (default: disabled): enables the validator (Keymanager) API, needed to sign exits.
• --validator-api-address (default: 127.0.0.1): address of the validator API.
• --validator-api-port (default: 5055): port of the validator API.
• --validator-api-token-file (default: api-token.txt in the validator directory): file with the bearer token.
• --http-address (default: 127.0.0.1) and --http-port (default: 5052): address of the beacon node API that accepts the signed exit.

See the command line reference for the full list.

Monitoring validator performance

A healthy validator attests in every epoch, proposes when chosen, and takes part in sync committees when selected. This page shows how to confirm that, using Grandine's logs, its performance reports, its Prometheus metrics, the Beacon Node API and an external explorer. It ends with a checklist.

For setting up Prometheus and Grafana in general, see Metrics. For log levels and formats, see Logging.

Duty log lines

Grandine logs each duty at the info level. With validators attached, you should see lines like these every slot or epoch:

loaded 4 validator key(s)
validators [4242, 4243] attesting in slot 123456
validators [4242] aggregating in slot 123456
validators [4243] participating in sync subcommittee 2 in slot 123457
validators [4243] aggregating in sync committee in slot 123457
starting block proposal task for validator 4242 at slot 123460
validator 4242 proposing beacon block with root 0x... in slot 123460

• loaded N validator key(s) appears once at startup. If you expected keys and see failed to load validator keys instead, check the keystore and password options.
• attesting in slot lists the validator indices that attested in that slot. Each active validator should appear once per epoch.
• A proposal ends with proposing beacon block with root .... With a builder it says using the builder API (before Gloas) or using builder N (from Gloas on).

Warnings to look for:

Performance reports

With the built-in beacon node, Grandine can print a summary of how its validators did. Start it with --report-validator-performance:

grandine                                  \
  --network mainnet                       \
  --jwt-secret /path/to/jwt.hex           \
  --eth1-rpc-urls http://EL-HOST:8551     \
  --keystore-dir /path/to/validator_keys  \
  --keystore-password-file /path/to/password.txt \
  --report-validator-performance

Once per epoch, while the node is synced, it logs:

attestation performance for epoch: 1000, expected: 4, produced: 4, correct target: 4 (100.00%), correct head: 4 (100.00%), included: 4 (100.00%), inclusion delays: ...
sync committee message performance for epoch: 1001, expected: 32, produced: 32, correct: 32, included: 32 (100.00%)

• The attestation report covers the epoch before the previous one, because attestations can still be included one epoch later.
• expected is the number of validators with an attestation duty, produced the number of attestations seen, and included the number found in blocks.
• correct target and correct head show how many votes matched the canonical chain. A low head rate often means late blocks or a slow node.
• inclusion delays summarizes how many slots it took for attestations to be included. A delay of 1 is ideal.

The report also warns about individual votes, for example:

validators [4242] voted for outdated head 0x... (by 2 slots) at slot 123456 (expected to vote for block 0x...)
validators [4242] voted for non-canonical block 0x... at slot 123456 (expected to vote for block 0x...)

If no validator is in a sync committee, the sync report says no sync committee members for epoch N with M registered validators.

Note: --report-validator-performance reads blocks from the built-in beacon node. It is not accepted with grandine bn, and grandine vc refuses it with --report-validator-performance reads blocks from the built-in beacon node. With a separate validator client, use an explorer or the Beacon Node API instead.

Prometheus metrics

Enable metrics with --metrics. Grandine serves them at http://127.0.0.1:5054/metrics by default. Change the address and port with --metrics-address and --metrics-port.

grandine ... --metrics
curl -s http://127.0.0.1:5054/metrics | grep -i validator

A minimal Prometheus scrape configuration:

scrape_configs:
  - job_name: grandine
    static_configs:
      - targets: ["127.0.0.1:5054"]

Validator metrics (also exported by a separate grandine vc):

Beacon node metrics that matter for validators (not exported by grandine vc):

Histograms appear with _bucket, _sum and _count suffixes. For example, a Prometheus alert on a stalled head:

- alert: GrandineHeadStalled
  expr: beacon_clock_slot - beacon_head_slot > 4
  for: 5m

Grandine does not export per-validator balances or effectiveness metrics. Use an explorer or the Beacon Node API for those.

Beacon Node API

The Beacon Node API (127.0.0.1:5052 by default) answers most "is it working?" questions directly:

# Sync status: is_syncing should be false, is_optimistic false, el_offline false
curl -s http://127.0.0.1:5052/eth/v1/node/syncing

# Peer count
curl -s http://127.0.0.1:5052/eth/v1/node/peer_count

# Validator status and balance (by public key or index)
curl -s http://127.0.0.1:5052/eth/v1/beacon/states/head/validators/0xPUBKEY

# Upcoming proposals in the current epoch
curl -s http://127.0.0.1:5052/eth/v1/validator/duties/proposer/EPOCH

A rising balance from one epoch to the next means the validator earns rewards.

Liveness tracking

With --track-liveness, the beacon node records which validators it has seen attesting or proposing in the current and the previous epoch, and serves the standard liveness endpoint:

curl -s -X POST -H "Content-Type: application/json" \
  -d '["4242", "4243"]' \
  http://127.0.0.1:5052/eth/v1/validator/liveness/EPOCH

Only the current and the previous epoch can be queried. is_live: true means the validator was seen on the network. Liveness tracking is also what doppelganger protection uses. Without --track-liveness, a Grandine beacon node returns an error for this endpoint.

beaconcha.in and other explorers

Block explorers such as beaconcha.in show each validator's attestations, proposals, sync committee duties and rewards, and can send alerts by email or through their mobile app. Look up your validators by public key or index.

Grandine can also push its own status to beaconcha.in's machine metrics once a minute. Create an API key at beaconcha.in/api/key-management, then pass the endpoint with your key, and optionally a machine name, to --remote-metrics-url:

grandine ... --remote-metrics-url "https://beaconcha.in/api/v1/client/metrics?apikey=YOUR_API_KEY&machine=MACHINE_NAME"

Grandine sends beacon node, validator (number of active and total validators) and system metrics. The beaconcha.in mobile app then shows the machine and can alert you when it goes offline. See Metrics.

Is everything fine? A checklist

Run through this list after setting up, after every upgrade, and whenever something looks wrong.

1. Version. The startup log line client version: ... shows the release you expect.
2. Execution client. It is synced, and Grandine reaches it. GET /eth/v1/node/syncing reports el_offline: false. See Connecting an execution client.
3. Beacon node synced. is_syncing: false and is_optimistic: false, and beacon_head_slot stays within a slot or two of beacon_clock_slot.
4. Peers. The peer count is stable and near the target (--target-peers). See Networking and peers.
5. Finality. beacon_finalized_epoch advances every epoch, and the log shows new finalized checkpoint regularly.
6. Keys loaded. The log shows loaded N validator key(s) with the expected N, and no failed to load validator keys.
7. Fee recipient. The startup line suggested fee recipient: 0x... shows your address, not the Grandine donation address. See Fee recipient, graffiti and gas limit.
8. Validators active. Their status is active_ongoing in the Beacon Node API or on an explorer.
9. Attesting. Each validator index appears in a validators [...] attesting in slot line once per epoch. With --report-validator-performance, included and correct target are close to 100%.
10. No warnings. No repeated cannot perform duties, slashing protector rejected or failed to publish messages.
11. Balances. Balances rise from epoch to epoch on an explorer.
12. Slashing protection backed up. You have a recent export. See Slashing protection.
13. Monitoring and alerts. Prometheus scrapes the metrics endpoint, or beaconcha.in monitoring is set up, and alerts reach you.
14. Service. Grandine runs as a service that restarts after a reboot. See Running as a service.

Some missed attestations are normal: late blocks, reorgs and restarts all cause them. A validator that misses most of its attestations for several epochs needs attention. See Troubleshooting.

Command line options

• --metrics (default: disabled): serve Prometheus metrics.
• --metrics-address (default: 127.0.0.1): address of the metrics endpoint.
• --metrics-port (default: 5054): port of the metrics endpoint.
• --remote-metrics-url (default: none): URL to push metrics to once a minute, for example beaconcha.in.
• --report-validator-performance (default: disabled): log attestation and sync committee performance once per epoch. Requires the built-in beacon node.
• --track-liveness (default: disabled): beacon node option. Record validator liveness and serve POST /eth/v1/validator/liveness/{epoch}.

See the command line reference for the full list.

Beacon Node API

The Beacon Node API is the standard HTTP REST interface of an Ethereum consensus client. It is defined by the Ethereum Beacon APIs specification. Validator clients, staking tools, block explorers, monitoring scripts and tools such as Vouch use it to read chain data and to submit blocks, attestations and other messages.

Grandine serves this API from the beacon node. You need it when:

• you run a separate validator client (Grandine grandine vc or another client's VC) against Grandine,
• you want to check the node's sync status, peers or chain head from scripts or monitoring,
• you connect third-party tools that speak the Beacon API.

The integrated validator client (plain grandine with validator keys) talks to the beacon node inside the same process and does not need the HTTP API. See How Grandine works for the difference between the two setups.

Grandine-specific endpoints that are not part of the specification are described in Grandine-specific endpoints. The server-sent event stream at /eth/v1/events has its own page: Event stream.

Enabling and binding the API

The API is enabled by default whenever Grandine runs a beacon node, that is, with plain grandine and with grandine bn. It listens on 127.0.0.1:5052, so by default only programs on the same machine can reach it. A standalone validator client started with grandine vc does not run a beacon node and does not serve this API. It has its own Keymanager API instead.

When the server starts, Grandine logs the address it listens on. Check that the API is up:

curl -s http://127.0.0.1:5052/eth/v1/node/version

{"data":{"version":"Grandine/VERSION-COMMIT/ARCH-OS"}}

To change the listen address or port:

grandine \
  --network mainnet \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /path/to/jwt.hex \
  --http-address 127.0.0.1 \
  --http-port 5052

To listen on all IPv4 interfaces, for example so that a validator client on another machine in your private network can connect, use --http-address 0.0.0.0. Only do this behind a firewall. See Security exposure.

To run a beacon node without the HTTP API, pass --disable-http-api. The integrated validator client keeps working, because it does not use HTTP.

Docker

Inside a container the API must listen on 0.0.0.0, otherwise Docker cannot forward the port. Publish the port on the host's loopback interface so it stays private to the host:

docker run \
  -p 9000:9000/tcp \
  -p 9000:9000/udp \
  -p 9001:9001/udp \
  -p 127.0.0.1:5052:5052/tcp \
  -v $HOME/.grandine:/root/.grandine \
  -v /path/to/jwt.hex:/root/.grandine/jwt.hex \
  sifrai/grandine:stable \
  --network mainnet \
  --checkpoint-sync-url CHECKPOINT-SYNC-URL \
  --eth1-rpc-urls http://EL-HOST:8551 \
  --jwt-secret /root/.grandine/jwt.hex \
  --http-address 0.0.0.0

The image's entry point is the grandine binary, so the options follow the image name directly.

CORS

Browsers only let a web page call the API if the API answers with a matching Access-Control-Allow-Origin header. By default Grandine allows a single origin: its own listening URL, for example http://127.0.0.1:5052. Web dashboards served from other origins are blocked by the browser.

Use --http-allowed-origins to allow other origins. It takes a comma-separated list:

grandine \
  --network mainnet \
  --http-allowed-origins http://localhost:3000,https://dashboard.example.com

The value * allows any origin. If you pass * together with other values, Grandine uses only the wildcard and logs a warning.

Note: CORS is a browser protection only. It does not stop curl or any other program that can reach the port. Use firewall rules to control who can connect.

Security exposure

The Beacon Node API has no authentication. Anyone who can reach the port can:

• read everything the node knows, including which validators it serves (through duties and subscriptions),
• submit blocks, attestations, exits and other messages,
• request expensive operations such as full beacon states, which costs CPU, memory and bandwidth,
• change runtime behavior through the non-standard endpoints, such as PATCH /features and the log level endpoints (see Grandine-specific endpoints).

Follow these rules:

• Keep the default --http-address 127.0.0.1 unless another machine must connect.
• If a remote validator client needs access, allow only that machine with a firewall, or connect it over an SSH tunnel or a VPN such as WireGuard.
• Never expose port 5052 to the internet. If you must serve the API publicly, put a reverse proxy in front of it that adds TLS, authentication and rate limits, and that blocks the non-standard and debug endpoints.
• Grandine does not terminate TLS itself. Use a reverse proxy if you need HTTPS.

See Security best practices for the wider picture.

Timeouts

Each request must produce a response within --timeout milliseconds. The default is 1000000 (1000 seconds). When a request takes longer, Grandine aborts it and answers with 408 Request Timeout. The timeout applies to producing the response, so long-lived event streams are not cut off by it.

Grandine does not limit the size of request bodies.

Response formats: JSON and SSZ

All endpoints return JSON by default. Some endpoints can also return SSZ, the compact binary encoding used on the Ethereum peer-to-peer network. SSZ responses are smaller and faster to produce and parse, which matters for large objects such as beacon states and blocks.

To ask for SSZ, include application/octet-stream in the Accept header:

curl -s -H 'Accept: application/octet-stream' \
  -o state.ssz \
  http://127.0.0.1:5052/eth/v2/debug/beacon/states/finalized

If the Accept header lists application/octet-stream anywhere, Grandine returns SSZ. Otherwise it returns JSON. An SSZ response body is the bare data object. Metadata that JSON carries in the body, such as version, is still sent in response headers.

These endpoints can return SSZ:

SSZ request bodies

These endpoints accept a request body in SSZ when you send Content-Type: application/octet-stream:

• POST /eth/v1/beacon/blocks and POST /eth/v2/beacon/blocks
• POST /eth/v1/beacon/blinded_blocks and POST /eth/v2/beacon/blinded_blocks
• POST /eth/v2/beacon/pool/attestations
• POST /eth/v1/validator/aggregate_and_proofs and POST /eth/v2/validator/aggregate_and_proofs
• POST /eth/v1/beacon/pool/payload_attestations
• POST /eth/v1/beacon/execution_payload_bids
• POST /eth/v1/beacon/execution_payload_envelopes
• POST /eth/v1/validator/proposer_preferences
• POST /eth/v4/validator/blocks/{slot}

All other request bodies must be JSON with Content-Type: application/json.

Tip: Some validator clients send JSON requests without a Content-Type header. If yours does and its requests are rejected, enable the PatchHttpContentType runtime feature, which fills in application/json when the header is missing. See Runtime features.

The Eth-Consensus-Version header

Objects such as blocks and states have a different shape in each fork. The header Eth-Consensus-Version names the fork in lowercase: phase0, altair, bellatrix, capella, deneb, electra, fulu or gloas.

• Responses. Endpoints that return fork-specific objects set Eth-Consensus-Version and also include a version field in JSON bodies. Block production endpoints also set Eth-Consensus-Block-Value, Eth-Execution-Payload-Blinded and Eth-Execution-Payload-Value.
• Requests. When you publish an SSZ body, you must send Eth-Consensus-Version so that Grandine knows how to decode it. The v2 endpoints (POST /eth/v2/beacon/blocks, POST /eth/v2/beacon/blinded_blocks, POST /eth/v2/beacon/pool/attestations, POST /eth/v2/validator/aggregate_and_proofs) require the header for JSON bodies too. The v1 block publishing endpoints accept JSON without it.

Responses during sync

GET /eth/v1/node/health returns 200 OK when the node is synced and 206 Partial Content while it is syncing.

Most validator duty endpoints under /eth/v1/validator/... and /eth/v2/validator/... answer 503 Service Unavailable while the node is syncing, as the specification requires. Proposer duties, prepare_beacon_proposer, and the v3 and v4 block production endpoints are served without this check.

Responses about chain data include execution_optimistic and, where it applies, finalized. An optimistic response means the execution client has not yet verified the payloads involved. Treat such data as provisional.

Supported endpoints

Grandine implements the endpoints below, grouped as in the specification. Path parameters:

• {state_id}: head, genesis, finalized, justified, a slot number, or a hex state root.
• {block_id}: head, genesis, finalized, a slot number, or a hex block root.
• {validator_id}: a validator index or a hex public key.

Endpoints marked Gloas belong to the upcoming Gloas fork (enshrined proposer-builder separation). They are only useful on networks where that fork is active.

Beacon

Rewards

The attestation rewards endpoint (POST /eth/v1/beacon/rewards/attestations/{epoch}) is not implemented. For per-validator reward and penalty breakdowns, see the Grandine-specific /validator/statistics endpoint in Grandine-specific endpoints.

Builder

Config

Debug

Full states are large. On mainnet a JSON state is hundreds of megabytes. Prefer SSZ, and do not poll this endpoint.

Grandine also serves two non-standard endpoints under /eth/v2/debug/tracing/ for changing the log and trace levels at runtime. They are described in Grandine-specific endpoints.

Events

See Event stream.

Node

Validator

"Sync check: yes" means the endpoint returns 503 while the node is syncing.

POST /eth/v1/validator/liveness/{epoch} needs liveness tracking, which is off by default. Start the node with --track-liveness to enable it. Validator clients use this endpoint for doppelganger protection.

Not supported

• Light client endpoints (/eth/v1/beacon/light_client/...) are not served. Grandine does not act as a light client server.
• Keymanager API endpoints (/eth/v1/keystores, /eth/v1/remotekeys and the per-validator fee recipient, gas limit and graffiti endpoints) are not part of this server. They are served by the validator API on a separate port with token authentication. See Keymanager API.
• Selection endpoints for distributed validators (POST /eth/v1/validator/beacon_committee_selections and POST /eth/v1/validator/sync_committee_selections) are routed but return 501 Not Implemented.
• Prometheus metrics are served by a separate server, not on port 5052. See Metrics.

Runtime features that affect the API

Some behavior is controlled by runtime features, which you turn on with --features. See Runtime features for the full list.

All standard endpoints are always served. The features above only gate the non-standard endpoints and add logging.

grandine --network mainnet --features LogHttpRequests,ServeCostlyEndpoints

Note: LogHttpBodies and PatchHttpContentType take effect only if they are enabled when the HTTP server starts, because Grandine adds their middleware while building the server. Turning them on later through PATCH /features has no effect on the API. The other features in this table are checked on every request and can be toggled at runtime.

Examples

The examples assume the API is on 127.0.0.1:5052. Pipe the output through jq for readable JSON.

Sync status

curl -s http://127.0.0.1:5052/eth/v1/node/syncing | jq

{
  "data": {
    "head_slot": "12345678",
    "sync_distance": "0",
    "is_syncing": false,
    "is_optimistic": false,
    "el_offline": false
  }
}

is_syncing: false with a small sync_distance means the node follows the chain head. is_optimistic: true means the execution client has not verified the head yet. el_offline: true means the beacon node cannot reach the execution client. See Connecting an execution client.

Health check

curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:5052/eth/v1/node/health

200 means synced, 206 means syncing. Load balancers and container health checks can use this endpoint directly.

Head block

curl -s http://127.0.0.1:5052/eth/v1/beacon/headers/head | jq '.data.header.message'

The slot field should increase by one every 12 seconds on mainnet.

Finality checkpoints

curl -s http://127.0.0.1:5052/eth/v1/beacon/states/head/finality_checkpoints | jq

On a healthy network the finalized epoch is two epochs behind the current epoch. If it falls further behind, the network is not finalizing.

You can also use this endpoint to cross-check a checkpoint sync: compare the finalized root with that of another trusted node or a block explorer.

Validator status

By index or by public key:

curl -s http://127.0.0.1:5052/eth/v1/beacon/states/head/validators/VALIDATOR-INDEX-OR-PUBKEY | jq

The status field shows where the validator is in its lifecycle, for example pending_queued, active_ongoing, active_exiting, exited_unslashed or withdrawal_done.

For several validators at once:

curl -s -X POST http://127.0.0.1:5052/eth/v1/beacon/states/head/validators \
  -H 'Content-Type: application/json' \
  -d '{"ids": ["VALIDATOR-INDEX-1", "VALIDATOR-INDEX-2"]}' | jq

Peers

curl -s http://127.0.0.1:5052/eth/v1/node/peer_count | jq
curl -s http://127.0.0.1:5052/eth/v1/node/peers | jq '.meta.count'

A low peer count usually points to a port forwarding problem. See Networking and peers.

Node identity

curl -s http://127.0.0.1:5052/eth/v1/node/identity | jq '.data | {peer_id, enr}'

Use the ENR to add this node as a static or trusted peer elsewhere.

Proposer duties for the current epoch

EPOCH=$(( $(curl -s http://127.0.0.1:5052/eth/v1/beacon/headers/head \
  | jq -r '.data.header.message.slot') / 32 ))
curl -s http://127.0.0.1:5052/eth/v1/validator/duties/proposer/$EPOCH | jq

Download the finalized state in SSZ

curl -s -H 'Accept: application/octet-stream' \
  -o finalized_state.ssz \
  http://127.0.0.1:5052/eth/v2/debug/beacon/states/finalized

Command line options

These options apply to the beacon node (grandine and grandine bn):

• --http-address (default: 127.0.0.1): address the API listens on.
• --http-port (default: 5052): port the API listens on.
• --http-allowed-origins (default: the API's own listening URL): comma-separated list of allowed CORS origins; * allows any origin.
• --timeout (default: 1000000): request timeout in milliseconds.
• --disable-http-api (default: off): run the beacon node without the HTTP API.
• --max-events (default: 100): number of events buffered per topic for /eth/v1/events.
• --track-liveness (default: off): enable liveness tracking for /eth/v1/validator/liveness/{epoch}.
• --features (default: none): comma-separated runtime features, such as ServeCostlyEndpoints.

See Command line reference for all options.

Event stream

The beacon node can push events to clients as they happen, so that tools do not have to poll. The endpoint is GET /eth/v1/events, part of the standard Beacon Node API. It uses server-sent events (SSE): the client opens one long-lived HTTP connection and the server writes a stream of text messages to it.

Validator clients subscribe to head and block events to time their duties and to notice reorganizations. Monitoring tools, block explorers and MEV software use the other topics.

Subscribing

Pass the topics you want in the topics query parameter as a comma-separated list:

curl -N 'http://127.0.0.1:5052/eth/v1/events?topics=head,finalized_checkpoint'

-N turns off curl's output buffering so that each event appears as soon as it arrives. Each event has an event line with the topic name and a data line with a JSON payload:

event:head
data:{"slot":"12345678","block":"0x...","state":"0x...","epoch_transition":false,"previous_duty_dependent_root":"0x...","current_duty_dependent_root":"0x...","execution_optimistic":false}

event:finalized_checkpoint
data:{"block":"0x...","state":"0x...","epoch":"385802","execution_optimistic":false}

Grandine sends periodic keep-alive comments so that proxies do not close an idle connection.

A request without topics fails with 400 Bad Request. An unknown topic name also makes the request fail.

The stream is not affected by --timeout, because the timeout applies only to producing the initial response.

Supported topics