SyntropyStack — Manual and SyntropyCTL — Monitoring solution with Grafana, Prometheus, Node_exporter and Nginx — Part 4
In this tutorial, we will be creating a monitoring network with node_exporter, Prometheus, Grafana and Nginx (with Let’s Encrypt SSL certificates). To deploy our network, we will be using the Syntropy Stack, Docker and SyntropyCLI
Syntropy Stack allows us to quickly connect application services (Apache, nginx, Grafana, InfluxDB, MariaDB, etc.) with optimized and encrypted connections by default. It’s compatible with any device or service, running on a cloud, on-premise or edge locations.
Start Syntropy Agent with Docker
sudo docker run --network="host" --restart=on-failure:10 --cap-add=NET_ADMIN --cap-add=SYS_MODULE \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
--device /dev/net/tun:/dev/net/tun --name=syntropynet-agent \
-e SYNTROPY_API_KEY=CHANGE ME \
-e SYNTROPY_TAGS=CHANGE ME \
-e SYNTROPY_PROVIDER=CHANGE ME \
-e SYNTROPY_AGENT_NAME=CHANGE ME \
-e SYNTROPY_NETWORK_API='docker' \
-d syntropynet/agent:stableThe VM of different vendors register on the SyntropyStack interface thanks to the agent launched by Docker.
End-Points SyntropyStack
Certificat SSL LetsEncrypt
For my access to be secure, it was necessary to create a LetsEncrypt SSL certificate and pair it with a domain name. In this example, I use DuckDNS.
The creation of several sub-networks
Launch services on each dedicated VM. Be careful by launching them on a different subnet
sudo docker network create --subnet 172.20.0.0/24 syntropynetsudo docker network create --subnet 172.21.0.0/24 syntropynetsudo docker network create --subnet 172.22.0.0/24 syntropynetHere are some explanations for each service:
To distinguish VM and Services, it is necessary to mount them on different IP address plans. Ports were not to be exposed to the Internet, except for 443.
Build 3x VM
First VM
Three Docker images will be required. We mount a Proxy Revers by exposing port 443 to be coupled with a LetsEncrypt SSL certificate + a domain name. All that will depend on the Grafana image.
sudo docker run --detach --net=syntropynet \
--name nginx-proxy \
--publish 80:80 \
--publish 443:443 \
--volume /etc/nginx/certs \
--volume /etc/nginx/vhost.d \
--volume /usr/share/nginx/html \
--volume /var/run/docker.sock:/tmp/docker.sock:ro \
jwilder/nginx-proxysudo docker run --detach --net=syntropynet \
--name nginx-proxy-letsencrypt \
--volumes-from nginx-proxy \
--volume /var/run/docker.sock:/var/run/docker.sock:ro \
--volume /etc/acme.sh \
--env "DEFAULT_EMAIL=mail@domain" \
jrcs/letsencrypt-nginx-proxy-companionsudo docker run --detach --net=syntropynet \
--name grafana \
--env "VIRTUAL_HOST=DuckDNSDomain" \
--env "VIRTUAL_PORT=3000" \
--env "LETSENCRYPT_HOST=DuckDNSDomain" \
--env "LETSENCRYPT_EMAIL=mail@domain" \
--env "GF_SECURITY_ADMIN_USER=admin" \
--env "GF_SECURITY_ADMIN_PASSWORD=syntropy" \
--env "GF_USERS_ALLOW_SIGN_UP=false" \
grafana/grafana- Nginx: is a web server which can also be used as a reverse proxy, load balancer, mail proxy and HTTP cache. The software was created by Igor Sysoev and first publicly released in 2004.
- Grafana: free software licensed Apache 2.0 that allows data visualization. It allows dashboards and graphs to be made from multiple sources including time databases such as Graphite, InfluxDB and OpenTSDB
Second VM
A fourth Docker image to launch promiseheus without exposing port 9090 to the internet.
sudo docker run --net=syntropynet -d --name prometheus -v $PWD/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus:latest- Prometheus: free computer monitoring software and alert generator. It records real-time metrics in a time series database based on the input point content exposed using the HTTP protocol.
Third VM
And finally a fifth image for Node-Exporter.
sudo docker run --net=syntropynet -d --name node-exporter quay.io/prometheus/node-exporter- Node_exporter: will expose all metrics from enabled collectors by default. This is the recommended way to collect metrics to avoid errors when comparing metrics of different families. For advanced use the node_exporter can be passed an optional list of collectors to filter metrics.
SyntropyCLI
SyntropyCTL is a pretty powerful tool that will allow you to build your network in a few command lines.
Create your Network SyntropyStack — There are many ways:
- Solution 1 -using SyntropyCTL command
syntropyctl create-networksyntropyctl get-networkssyntropyctl get-endpointssyntropyctl configure-endpoints --enable-all-services <Agent_ID>syntropyctl create-connections -j <network_name> <VM1_id> <VM2_id> <VM2_id> <VM3_id>
Result:
[
{
"agent_connection_id": 26278,
"agent_1_id": 640,
"agent_interface_1_id": 2675,
"agent_2_id": 641,
"agent_interface_2_id": 2676,
"network_id": 318,
"agent_connection_created_at": "2021-01-16T13:19:51.870",
"agent_connection_updated_at": "2021-01-16T13:19:51.870",
"agent_connection_modified_at": "2021-01-16T13:19:51.870",
"agent_sdn_policy_id": null,
"agent_connection_link_tag": "PUBLIC",
"agent_connection_status": "PENDING",
"agent_connection_status_reason": null,
"agent_connection_last_handshake": null,
"agent_connection_tx_bytes_total": null,
"agent_connection_rx_bytes_total": null,
"agent_connection_latency_ms": null,
"agent_connection_packet_loss": null
},
{
"agent_connection_id": 26279,
"agent_1_id": 641,
"agent_interface_1_id": 2676,
"agent_2_id": 642,
"agent_interface_2_id": 2677,
"network_id": 318,
"agent_connection_created_at": "2021-01-16T13:19:51.870",
"agent_connection_updated_at": "2021-01-16T13:19:51.870",
"agent_connection_modified_at": "2021-01-16T13:19:51.870",
"agent_sdn_policy_id": null,
"agent_connection_link_tag": "PUBLIC",
"agent_connection_status": "PENDING",
"agent_connection_status_reason": null,
"agent_connection_last_handshake": null,
"agent_connection_tx_bytes_total": null,
"agent_connection_rx_bytes_total": null,
"agent_connection_latency_ms": null,
"agent_connection_packet_loss": null
}
]- Solution 2 — using SyntropyNAC command and YAML template
connections:
VM2:
connect_to:
Grafana:
services:
- grafana
type: tag
Node_Exporter:
services:
- node-exporter
type: tag
services:
- prometheus
type: endpoint
name: MON4
state: present
topology: P2Msyntropynac configure-networks --dry-run MON4.yaml
Result:
Configuring network MON4
Created network MON4 with id 319
Created 2 connections for network MON4
Configured 2 connections and 2 subnets for network MON4
DoneGrafana
Now that all services are communicating together, we have the option to get to the Grafana interface by typing the address of our domain name.
In order for Grafana to reassemble the metrics that Promotheus receives from Node_Exporter, it is necessary to identify the Docker DataSource IP of the Prometheus server. By installing a template, here is the result:
In conclusion, this exercise allowed us to highlight that with different VM connected via an encrypted tunnel, it is possible to build an architecture as if you were in a local network safely;-)
Without using the UI platform, it’s also easy to build the network through the SyntropyCLI.
