Monitoring & Maintenance
Nodes on Relix are not “set and forget” services. Whether you run a full node, an RPC gateway, or a validator, the long-term health of the network depends on operators noticing issues early and reacting in a controlled way. Monitoring and maintenance are the tools that turn a one-time setup into a reliable piece of infrastructure.
This section focuses on practical routines that apply to Relix Testnet today and will carry over to mainnet with minimal changes.
1. What you should always be watching
At a minimum, every Relix node should be tracked for four basic dimensions:
Liveness – is the node running and reachable?
Sync status – is it close to the chain head?
Resource usage – is CPU, memory, or disk close to its limits?
Network health – does it have enough peers and stable connectivity?
Typical metrics worth exporting to a monitoring system (Prometheus, Grafana, or similar):
Chain metrics
Current block height (and lag vs the public explorer)
Peer count
Time since last imported block
System metrics
CPU utilization
Memory usage and swap activity
Disk space and I/O pressure
Network in/out throughput
You do not need an overly complex setup on day one, but you should be able to answer simple questions at any time, such as:
“Is this node in sync with Relix Testnet?” “Did it crash or restart in the last hour?” “Are we close to running out of disk?”
If you cannot answer those quickly, monitoring is not yet where it needs to be.
2. Logs as a first line of diagnosis
Logs are your first window into what the node is doing:
Peer connections and disconnections
Block imports and consensus events
RPC errors, timeouts, or unusual load patterns
Recommended practices:
Keep logs structured and rotated
Use log rotation (logrotate or built-in mechanisms) to prevent disks from filling.
Store logs long enough to investigate issues, but avoid unbounded growth.
Separate concerns
System logs (kernel, SSH, firewall)
Node logs (Relix client)
Reverse proxy / RPC logs (Nginx, HAProxy, etc.)
Use log search tools
Even simple tools (grep, journalctl filters) go a long way.
For larger setups, consider centralizing logs in a stack such as Loki or Elasticsearch.
When you see symptoms like “node is behind” or “clients receive errors,” logs are usually the quickest way to identify whether the problem is resource exhaustion, a configuration error, or a network issue.
3. Alerts that actually matter
Alerting should be opinionated rather than noisy. A few well-chosen alerts will catch problems early without training you to ignore notifications.
Examples of useful alerts:
Node health
Process not running or repeatedly restarting.
Node block height lagging behind a reference value by more than N blocks.
Peer count near zero for an extended period.
Resource thresholds
Disk usage above 80–85%.
Memory usage consistently high with swap activity.
CPU pinned at 90–100% for prolonged periods.
Validator-specific signals
Missed blocks or participation falling below a defined threshold.
Gaps in signing activity for more than a few consensus rounds.
Alert destinations can be as simple as email, a Telegram bot, or messages in a private operations channel. The important part is that someone is responsible for reacting.
4. Planning and performing upgrades
Relix will evolve over time: protocol improvements, bug fixes, and performance updates are inevitable. Upgrades become routine if you establish a clear process:
Stay informed
Watch official channels for release announcements:
GitHub:
https://github.com/relixchainWebsite:
https://relixchain.comTelegram / X updates for testnet and mainnet changes.
Read the release notes
Check whether the update is:
Optional (bug fixes, performance only), or
Required for consensus (hard fork, protocol change).
Note any new flags, configuration changes, or data migrations.
Test on a non-critical node
If possible, upgrade a secondary full node first.
Verify that it starts, syncs, and behaves normally on Relix Testnet (chain ID
4127).
Upgrade production nodes one by one
For validators or public RPC nodes, avoid upgrading everything simultaneously.
Use repeatable steps (scripts, configuration management) to minimize human error.
Monitor closely after each upgrade
Keep an eye on logs, block height, and peers for at least a short window after the change.
A documented upgrade checklist is one of the simplest ways to prevent downtime caused by rushed or ad-hoc changes.
5. Backups and recovery strategy
While the blockchain data itself can be re-downloaded from the network, you still have assets that need protection:
Configuration files
Node config, ports, network settings.
RPC and reverse proxy configuration.
Keys
Validator signing keys (for validators).
Any operator keys used to interact with staking contracts or management tools.
Monitoring and deployment scripts
Templates and scripts used to provision and launch nodes.
Good backup habits:
Encrypt key backups and store them in multiple safe locations (not on the node itself).
Keep configuration and infrastructure code in version control.
Periodically rehearse recovery: build a new machine from scratch using only your documentation and backups.
If a disk fails or a server becomes unavailable, you should be able to:
Provision a new instance.
Install the Relix node software.
Restore configuration.
Sync the chain again.
For validators, import keys and rejoin consensus following best practices.
The calmer you feel about “what happens if this server disappears tomorrow,” the better your maintenance posture is.
6. Security as part of maintenance
Monitoring and maintenance are also about reducing risk over time:
Keep the OS patched
Apply security updates regularly.
Limit software installed on validator and RPC machines to what is strictly needed.
Harden access
Use SSH keys instead of passwords.
Restrict SSH to specific IPs or VPNs where possible.
Avoid sharing root access; use sudo with named user accounts.
Limit exposed ports
Only open the ports required for P2P and RPC.
Place public RPC endpoints behind a reverse proxy with rate limiting.
For validators, strongly consider a sentry architecture to shield the signing node from direct exposure.
Security is not a separate project; it is an ongoing series of small, disciplined decisions.
7. Capacity planning and scaling
As Relix usage grows, you may outgrow the initial “single node” setup. Monitoring data helps you anticipate when to scale:
If CPU or RAM is consistently high, consider:
Moving RPC workloads to dedicated nodes.
Adding more nodes behind a load balancer.
If disk usage grows quickly:
Plan ahead for larger SSDs or additional nodes.
Decide whether you need full historical data or can rely on pruning options when available.
For validator operators:
Consider separating roles early validator node for signing, full/RPC nodes for servicing traffic.
Use monitoring to understand peak times and adjust capacity accordingly.
Scaling is easier when it is proactive. Reacting after the disk is full or the node is already falling behind is always more painful.
8. Incident response mindset
Even with good preparation, incidents will happen: unexpected crashes, misconfigurations, or external issues. What separates a resilient operator from a fragile one is the response:
Stay methodical
Capture what changed recently (software, config, infrastructure).
Check monitoring and logs around the time of impact.
Reproduce on a test node if possible.
Prefer rollback over heroic patching
If a new change clearly caused the problem, rolling back to the last known-good state is often faster and safer than live debugging on a production validator.
Document what happened
Keep short, honest notes of the incident:
What went wrong
How it was detected
How it was resolved
What will be changed to avoid a repeat
Improve the system
Add alerts that would have caught the issue earlier.
Strengthen procedures that were unclear during the incident.
On a growing network like Relix, this kind of operational learning is just as important as new code.
9. Summary
Monitoring and maintenance are what turn a Relix node from “something that happens to be running” into reliable infrastructure:
Metrics and logs tell you what is happening.
Alerts make sure you look at the right things in time.
Upgrades, backups, and security routines keep the node healthy as the network evolves.
A calm, documented approach to incidents turns bad days into long-term improvements.
Whether you are operating a single testnet node or planning a multi-node mainnet setup, investing in these practices early will pay off in stability, fewer surprises, and more confidence when building on Relix.
Last updated