Monitoring tools and considerations
When deploying and maintaining an Arbitrum chain, there are several key elements that need to be monitored. This page has two parts:
- Monitoring tools — scripts and services available to chain maintainers.
- What to monitor, by component — a reference of the specific metrics, onchain signals, and thresholds to watch for each part of the stack (sequencer, batch poster, validator, and chain fees), plus deployment and hardware guidance.
The metric names in this page are the internal Nitro metric paths (for example, arb/sequencer/backlog). On the Prometheus scrape endpoint, each / is replaced with _ (so arb/sequencer/backlog is scraped as arb_sequencer_backlog). See Enabling Nitro metrics for how to expose them.
Arbitrum chain verification script
The Arbitrum chain verification script retrieves information from an Arbitrum chain and its parent chain to verify that all parameters are configured correctly. After gathering the data, it generates a comprehensive report and issues warnings for any discrepancies detected. This tool is particularly useful after deploying and configuring an Arbitrum chain, to make sure that the onchain information has been correctly set.
The Arbitrum chain verification script is currently under active development and is considered a work-in-progress (WIP). Consequently, its findings should be approached with caution, as there is a potential for false positives.
The arbitrum-monitoring alerting suite
The arbitrum-monitoring repository provides three ready-to-run monitoring scripts that watch the critical liveness signals of an Arbitrum chain and can send alerts to Slack, with more monitors under active development. All three monitors read your chain's parameters (RPC endpoints and core contract addresses) from a shared config.json file, and can be run once (for example, on a schedule) or continuously.
Retryable monitor
Retryable tickets are messages sent from a parent chain and executed on the Arbitrum chain. Due to their asynchronous nature (they are executed several minutes after being created), if insufficient funds are provided at the time of creation, they might not automatically redeem (execute) upon arrival at the Arbitrum chain. When this occurs, a manual redemption of the ticket is required.
The retryable monitor tracks these tickets from creation through execution: it watches for ticket creation events on the parent chain, then checks each ticket's status on the Arbitrum chain — automatically redeemed, manually redeemed, still pending, or failed. It alerts on tickets that failed to redeem automatically and on tickets approaching their seven-day expiration window, so that no cross-chain message expires unexecuted.
Batch poster monitor
The batch poster monitor tracks batch posting activity and data availability. It alerts when no batches have been posted within the chain's expected time bounds while user transactions are pending, when the batch poster account balance falls below an estimate of about three days of posting costs, and when a backlog of unposted blocks accumulates. For AnyTrust chains, it also detects when the chain has fallen back to posting full calldata on the parent chain instead of DACerts, which indicates a problem with the Data Availability Committee.
Assertion monitor
The assertion monitor tracks the creation and confirmation of assertions on the chain's Rollup contract, supporting both BoLD and legacy (pre-BoLD) chains. It alerts when no assertions are being created despite chain activity, when assertions remain unconfirmed after the challenge period has elapsed, and on validator-related risks such as low validator participation, a base stake below the expected threshold on BoLD chains, or a disabled validator allowlist on legacy chains.
Data Availability Server (DAS) health checks
If you've deployed an AnyTrust chain with a Data Availability Committee, it is recommended to actively monitor the endpoints of the different configured DA servers. The How to deploy a DAS guide contains a section for testing both the RPC and REST endpoints of any given DAS, by using the anytrusttool available in Nitro.
Enabling Nitro metrics
Most of the signals below are exposed as Prometheus-compatible metrics by the Nitro node. Start the node with the --metrics flag to enable the metrics server, then scrape the endpoint at http://<metrics-server.addr>:<metrics-server.port>/debug/metrics/prometheus (default port 6070):
nitro --metrics \
--metrics-server.addr 0.0.0.0 \
--metrics-server.port 6070
For metrics-server flags, memory-related metrics, health-check patterns, and Kubernetes ServiceMonitor examples, see Node tuning and monitoring.
The signals that are not Nitro metrics — onchain events and account balances on the parent chain — are called out explicitly in each table below.
What to monitor, by component
The sections below group the elements to monitor by the component they belong to. Each table lists the signal, where it comes from, what it means, and a suggested condition to alert on. Suggested alert thresholds are starting points; tune them to your chain's activity and settlement layer.
Sequencer
The sequencer accepts transactions, orders them, and produces blocks. Two different "backlogs" are often confused: the sequencer's transaction backlog (transactions received but not yet sequenced into blocks) and the batcher backlog (sequenced messages not yet posted to the parent chain). They are separate metrics and live in separate components — the batcher backlog is covered under Batch poster.
| Signal | Source | What it means | Suggested alert |
|---|---|---|---|
arb/sequencer/backlog | Nitro metric (sequencer) | The sequencer's transaction backlog: transactions accepted but not yet included in a block. A sign of network health. | Sustained growth over several minutes |
arb/sequencer/active | Nitro metric (sequencer) | Whether this node is currently the active sequencer. Useful when running a sequencer coordinator with failover. | Value flaps, or no node reports active |
arb/feed/backlog/messages | Nitro metric (nodes with feed output enabled — the sequencer or a relay) | Messages retained by the broadcast server until it sees them covered by batches on the parent chain. Growth means batches aren't landing — not that a follower is lagging. | Sustained growth |
eth_syncing fields | RPC (any node) | While a node is syncing, eth_syncing returns fields such as batchSeen, batchProcessed, msgCount, and blockNum. See caveats below. | batchSeen − batchProcessed keeps growing |
eth_syncing returns false once a node is fully synced, so it is useful for tracking a node that is catching up, not for steady-state health. Its out-of-sync output is not a stable API and can change between Nitro versions, so do not build production alerting on the exact field shape. For a fully synced node, use the metrics above and an RPC liveness check (an eth_chainId call returning HTTP 200) instead. See eth_syncing for the full field reference.
Batch poster
The batch poster (the "batcher") posts sequenced transactions to the SequencerInbox contract on the parent chain as batches. It needs a funded account and must keep landing batches, or the chain's data stops being published to the parent chain.
| Signal | Source | What it means | Suggested alert |
|---|---|---|---|
arb/batchposter/estimated_batch_backlog | Nitro metric (batch poster) | The batcher backlog: sequenced batches waiting to be posted to the parent chain. | Sustained growth over several minutes |
arb/batchposter/wallet/eth | Nitro metric (batch poster) | The batch poster account balance, in ether. Monitor fund usage directly from this metric. | Below your funding threshold |
arb/inbox/latest/batch | Nitro metric (any node) | The latest batch number the node has seen in the SequencerInbox. Stalls if batches stop landing. | No increase while the chain has activity |
SequencerBatchDelivered event | Parent chain (SequencerInbox) | Emitted each time a batch is delivered onchain. The onchain confirmation that batches are landing. | No event within your expected posting interval |
Monitor the batch poster account balance directly through arb/batchposter/wallet/eth (and the validator balance through arb/staker/balance, below) rather than estimating monthly spend from configs and gas multipliers. Direct balance monitoring reflects real usage; config-based estimation drifts from reality as parent-chain fees move. There is no automatic mechanism to refund these accounts, so keep them overfunded and alert well before they run dry.
If batches stop being posted, see Batch poster troubleshooting for common causes and tuning flags.
Validator
Validators post and confirm assertions (historically called "RBlocks" or "nodes") of the chain's state to the Rollup contract on the parent chain. The metrics differ depending on whether your chain runs the legacy staker or the BoLD staker.
If you are relying on arb/staker/action/last_success and seeing it go stale for long periods even while assertions are being created, this is expected on two counts. First, that gauge updates each time the staker successfully acts — during periods when the staker has no action to take, it stays flat even though the validator is healthy and polling. Second, it is a legacy-staker metric; on BoLD chains, assertion activity is tracked by the arb/validator/poster/* and arb/validator/scanner/* metrics instead. Prefer the assertion-progress and failure signals below over last_success as your primary health check, and corroborate with the onchain assertion events.
| Signal | Source | What it means | Suggested alert |
|---|---|---|---|
arb/staker/staked_node | Nitro metric (legacy staker) | The latest assertion this validator has staked on. Should advance as the chain progresses. | No advance while the chain has activity |
arb/staker/confirmed_node | Nitro metric (legacy staker) | The latest confirmed assertion this validator sees. | Stalls unexpectedly |
arb/staker/action/failure | Nitro metric (legacy staker) | Counter of failed staker actions. A better failure signal than a stale last_success gauge. | Any sustained increase |
arb/staker/balance | Nitro metric (staker) | The validator account balance, in ether. Monitor fund usage directly. | Below your funding threshold |
arb/validator/poster/assertion_posted | Nitro metric (BoLD staker) | Counter of assertions posted by this validator. The primary "assertions are being made" signal on BoLD chains. | No increase while the chain has activity |
arb/validator/poster/error_posting_assertion | Nitro metric (BoLD staker) | Counter of errors while posting assertions. | Any sustained increase |
arb/validator/scanner/latest_confirmed_assertion_block_number | Nitro metric (BoLD staker) | Parent-chain block number of the latest confirmed assertion. | No advance over an extended period |
arb/validator/validations/failed | Nitro metric (any validator) | Counter of failed block validations. | Any sustained increase |
NodeCreated or AssertionCreated events | Parent chain (Rollup contract) | The onchain confirmation that assertions are being created. Watch alongside the metrics above. | No event over an extended period |
Long periods of inactivity can disable the validator allowlist. If the latest confirmed assertion (or its first child, once one exists) is older than validatorAfkBlocks parent-chain blocks, the Rollup contract's validator allowlist can be permissionlessly disabled. validatorAfkBlocks is configured per chain — read it from your Rollup contract; a value of 0 disables this mechanism entirely. Alert on assertion inactivity well before this window elapses. For example, on Arbitrum One validatorAfkBlocks is set to 201600 (~28 days of parent-chain blocks).
Chain fees and load
The chain's base fee is a useful signal of sustained demand. An Arbitrum chain has a gas target (also called the speed limit), measured in gas per second, that acts as a threshold for pricing. When cumulative usage exceeds it, a backlog of excess gas accumulates and the L2 base fee rises using an approach similar to Ethereum's EIP-1559 pricing algorithm. The default gas target is 7,000,000 gas per second as of ArbOS 6 (it was 1,000,000 in ArbOS 0). A sustained base-fee spike therefore indicates demand is exceeding the speed limit — as a rough guide, 7M gas per second is on the order of 330 simple transfers per second, though the exact transactions-per-second figure depends on the gas cost of the transactions being run.
| Signal | Source | What it means | Suggested alert |
|---|---|---|---|
arb/block/basefee | Nitro metric (any node) | The current L2 base fee. Spikes above the minimum indicate the chain is congested. | Sustained elevation above baseline |
ArbGasInfo.getGasBacklog() | Precompile (RPC) | The backlogged amount of gas burnt in excess of the speed limit. Drives the base fee up. | Sustained growth |
ArbGasInfo.getGasAccountingParams() | Precompile (RPC) | Returns the chain's speed limit, pool size, and block gas limit. Use to confirm configuration. | Values differ from expected configuration |
For guidance on choosing and changing the gas target, see Manage gas target.
Deployment guidance
Do not run the sequencer and a validator on the same node. They have different resource profiles and different failure modes, and co-locating them means a single machine failure takes down both liveness (sequencing) and safety (validation) at once. Running them separately also keeps their funding and monitoring independent:
- The batch poster account (
arb/batchposter/wallet/eth) pays to post batches to the parent chain. - Each validator account (
arb/staker/balance) pays to post and confirm assertions.
Fund and alert on each account separately, and keep each overfunded — there is no automatic top-up mechanism for either.
Appendix: reference hardware specifications
The following are the canonical hardware specifications used by Offchain Labs for Arbitrum One nodes. Treat them as a reference point for sizing your own chain; requirements scale with your chain's activity and history.
| Role | vCPU / cores | RAM | Storage | Reference instance |
|---|---|---|---|---|
| Sequencer | 10 cores | 90 GB | 6.6 TB NVMe | i7ie.3xlarge |
| Archive | 2 cores | 40 GB | 16 TB | — |
| Validator | 3 cores | 32 GB | 4 TB | — |
| RPC | 4 cores | 32 GB | — | — |
Fast, locally attached NVMe SSD storage is strongly recommended, especially for the sequencer and archive nodes, because disk latency directly affects performance as read volume grows. See State growth for more on storage sizing over time.