SyntropyStack — Manual and SyntropyCTL — Monitoring solution with Grafana, Prometheus, Node_exporter and Nginx — Part 4

In this tutorial, we will be creating a monitoring network with node_exporter, Prometheus, Grafana and Nginx (with Let’s Encrypt SSL certificates). To deploy our network, we will be using the Syntropy Stack, Docker and SyntropyCLI

Syntropy Stack allows us to quickly connect application services (Apache, nginx, Grafana, InfluxDB, MariaDB, etc.) with optimized and encrypted connections by default. It’s compatible with any device or service, running on a cloud, on-premise or edge locations.

Start Syntropy Agent with Docker

sudo docker run --network="host" --restart=on-failure:10 --cap-add=NET_ADMIN --cap-add=SYS_MODULE \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
--device /dev/net/tun:/dev/net/tun --name=syntropynet-agent \
-e SYNTROPY_API_KEY=CHANGE ME \
-e SYNTROPY_TAGS=CHANGE ME \
-e SYNTROPY_PROVIDER=CHANGE ME \
-e SYNTROPY_AGENT_NAME=CHANGE ME \
-e SYNTROPY_NETWORK_API='docker' \
-d syntropynet/agent:stable

The VM of different vendors register on the SyntropyStack interface thanks to the agent launched by Docker.

End-Points SyntropyStack

Certificat SSL LetsEncrypt

For my access to be secure, it was necessary to create a LetsEncrypt SSL certificate and pair it with a domain name. In this example, I use DuckDNS.

Duck DNS (www.duckdns.org)

The creation of several sub-networks

Launch services on each dedicated VM. Be careful by launching them on a different subnet

sudo docker network create --subnet 172.20.0.0/24 syntropynetsudo docker network create --subnet 172.21.0.0/24 syntropynetsudo docker network create --subnet 172.22.0.0/24 syntropynet

Here are some explanations for each service:

To distinguish VM and Services, it is necessary to mount them on different IP address plans. Ports were not to be exposed to the Internet, except for 443.

Build 3x VM

First VM

Three Docker images will be required. We mount a Proxy Revers by exposing port 443 to be coupled with a LetsEncrypt SSL certificate + a domain name. All that will depend on the Grafana image.

sudo docker run --detach --net=syntropynet \
--name nginx-proxy \
--publish 80:80 \
--publish 443:443 \
--volume /etc/nginx/certs \
--volume /etc/nginx/vhost.d \
--volume /usr/share/nginx/html \
--volume /var/run/docker.sock:/tmp/docker.sock:ro \
jwilder/nginx-proxysudo docker run --detach --net=syntropynet \
--name nginx-proxy-letsencrypt \
--volumes-from nginx-proxy \
--volume /var/run/docker.sock:/var/run/docker.sock:ro \
--volume /etc/acme.sh \
--env "DEFAULT_EMAIL=mail@domain" \
jrcs/letsencrypt-nginx-proxy-companionsudo docker run --detach --net=syntropynet \
--name grafana \
--env "VIRTUAL_HOST=DuckDNSDomain" \
--env "VIRTUAL_PORT=3000" \
--env "LETSENCRYPT_HOST=DuckDNSDomain" \
--env "LETSENCRYPT_EMAIL=mail@domain" \
--env "GF_SECURITY_ADMIN_USER=admin" \
--env "GF_SECURITY_ADMIN_PASSWORD=syntropy" \
--env "GF_USERS_ALLOW_SIGN_UP=false" \
grafana/grafana
  • Nginx: is a web server which can also be used as a reverse proxy, load balancer, mail proxy and HTTP cache. The software was created by Igor Sysoev and first publicly released in 2004.
  • Grafana: free software licensed Apache 2.0 that allows data visualization. It allows dashboards and graphs to be made from multiple sources including time databases such as Graphite, InfluxDB and OpenTSDB

Second VM

A fourth Docker image to launch promiseheus without exposing port 9090 to the internet.

sudo docker run --net=syntropynet -d --name prometheus -v $PWD/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus:latest
  • Prometheus: free computer monitoring software and alert generator. It records real-time metrics in a time series database based on the input point content exposed using the HTTP protocol.

Third VM

And finally a fifth image for Node-Exporter.

sudo docker run --net=syntropynet -d --name node-exporter quay.io/prometheus/node-exporter
  • Node_exporter: will expose all metrics from enabled collectors by default. This is the recommended way to collect metrics to avoid errors when comparing metrics of different families. For advanced use the node_exporter can be passed an optional list of collectors to filter metrics.

SyntropyCLI

SyntropyCTL is a pretty powerful tool that will allow you to build your network in a few command lines.

Create your Network SyntropyStack — There are many ways:

  • Solution 1 -using SyntropyCTL command
syntropyctl create-networksyntropyctl get-networkssyntropyctl get-endpointssyntropyctl configure-endpoints --enable-all-services <Agent_ID>syntropyctl create-connections -j <network_name> <VM1_id> <VM2_id> <VM2_id> <VM3_id>

Result:

[
{
"agent_connection_id": 26278,
"agent_1_id": 640,
"agent_interface_1_id": 2675,
"agent_2_id": 641,
"agent_interface_2_id": 2676,
"network_id": 318,
"agent_connection_created_at": "2021-01-16T13:19:51.870",
"agent_connection_updated_at": "2021-01-16T13:19:51.870",
"agent_connection_modified_at": "2021-01-16T13:19:51.870",
"agent_sdn_policy_id": null,
"agent_connection_link_tag": "PUBLIC",
"agent_connection_status": "PENDING",
"agent_connection_status_reason": null,
"agent_connection_last_handshake": null,
"agent_connection_tx_bytes_total": null,
"agent_connection_rx_bytes_total": null,
"agent_connection_latency_ms": null,
"agent_connection_packet_loss": null
},
{
"agent_connection_id": 26279,
"agent_1_id": 641,
"agent_interface_1_id": 2676,
"agent_2_id": 642,
"agent_interface_2_id": 2677,
"network_id": 318,
"agent_connection_created_at": "2021-01-16T13:19:51.870",
"agent_connection_updated_at": "2021-01-16T13:19:51.870",
"agent_connection_modified_at": "2021-01-16T13:19:51.870",
"agent_sdn_policy_id": null,
"agent_connection_link_tag": "PUBLIC",
"agent_connection_status": "PENDING",
"agent_connection_status_reason": null,
"agent_connection_last_handshake": null,
"agent_connection_tx_bytes_total": null,
"agent_connection_rx_bytes_total": null,
"agent_connection_latency_ms": null,
"agent_connection_packet_loss": null
}
]
  • Solution 2 — using SyntropyNAC command and YAML template
connections:
VM2:
connect_to:
Grafana:
services:
- grafana
type: tag
Node_Exporter:
services:
- node-exporter
type: tag
services:
- prometheus
type: endpoint
name: MON4
state: present
topology: P2M
syntropynac configure-networks --dry-run MON4.yaml

Result:

Configuring network MON4
Created network MON4 with id 319
Created 2 connections for network MON4
Configured 2 connections and 2 subnets for network MON4
Done

Grafana

Now that all services are communicating together, we have the option to get to the Grafana interface by typing the address of our domain name.

In order for Grafana to reassemble the metrics that Promotheus receives from Node_Exporter, it is necessary to identify the Docker DataSource IP of the Prometheus server. By installing a template, here is the result:

In conclusion, this exercise allowed us to highlight that with different VM connected via an encrypted tunnel, it is possible to build an architecture as if you were in a local network safely;-)

Without using the UI platform, it’s also easy to build the network through the SyntropyCLI.

Video tutorial

GitHub Tutorial

syntropy-devops-integrations/grafana-prometheus-node_exporter-mixed-2 at main · SyntropyNet/syntropy-devops-integrations (github.com)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Laurent ✤

Laurent ✤

Ambassadeur Français chez Syntropy (NOIA Network)