Tutorial: Utilizing VectorDotDev To Expose Log-Metrics To Prometheus

  • July 2, 2021

In this age of fast-growing advancement in cloud implementations, there is a great need to manage logs effectively. In some cases, you have to study the metrics and know what the system is about; it helps in understanding your system to take decisions, post-mortem analysis and several other interesting functions. In this piece, we would address how Vectordotdev can be an effective tool for performing log-to-metric actions and finally push to another tool called Prometheus via scraping. Prometheus collects metrics from targets by scraping metrics HTTP endpoints.  In Prometheus terms, an endpoint you can scrape is called an instance, usually corresponding to a single process.

First off, you should understand that Vector is a high-performance, end-to-end (agent & aggregator) observability data pipeline that puts you in control of your observability data. It directly orchestrates the operation of collecting, transforming and routing all your logs, metrics, and traces to any vendors you want today or tomorrow. Vector enables dramatic cost reduction, novel data enrichment, and data security where you need it, not where is most convenient for your vendors. Additionally, it is open source and up to 10x faster than every alternative in the space.

Use cases

  • Reduce total observability costs.
  • Transition vendors without disrupting workflows.
  • Enhance data quality and improve insights.
  • Consolidate agents and eliminate agent fatigue.
  • Improve overall observability performance and reliability.

 

Prerequisites 

  1. Read this quickstart guide to see some more details 
  2. Install Vector on the server/service or virtual machine that is releasing the logs. Choose the operating system that tallies your existing service. 
  3. Install Nginx to run as a web server on your machine either locally or in the cloud with an EC2 instance or probably ECS service – it depends on your use case.
  4. Confirm infrastructure is complete and setup preferably on AWS and you could do this with IAC or manually; use whatever works for you (Optional but paramount if you are hosting on the cloud and not locally)
  5. Fundamentally understand how to use/read JSON, TMOL and maybe YAML. 

 

Launch The Steps

1.Since the webserver is our log source, let us confirm that it is running using this command:

 

2.Confirm and check the location of its logs with the following command:

 

3.Setup config file for vector in the nginx directory file with

 

And the structure is to specify the logs source, log to metric transform then the logs destination. Before moving to the main config file, let us do a quick test of logs display with the following configuration file:

 

4.Next thing is to test the configuration implementation with this command: sudo $(which vector) –config /tmp/vector.toml. It read a line from the access log and another from the arrow log, see the response below:

 

5.The next thing is to parse the logs in a format before the transform is implemented. Look at the config file change below:

 

Look up more on this via the documentation here

6.Prometheus does not accept logs, just metrics so it has to be converted. Vector has a transform service for the logs so it can be sent to Prometheus. We will update the config file here:

 

Also, check out this documentation

7.Finally, let us count the log components but on a total scale then see the response in our command line, it is the response that we can imagine Prometheus will be collecting for any observability processes.

 

Then the CMD response when we run the command curl localhost:80 is shown below:

 

8.Next, let us count strings on the line of the logs, we need to change some parameters on the configuration file shown as thus:

 

Then run curl localhost:80, the CMD response for the logs are shown thus:

 

9.Now you can bring Prometheus into the equation after the URL has been exposed by vector for scraping. We can use the Prometheus exporter sink feature provided by Vector using the configuration thus:

 

Then run the following command to get the timestamp, metric name, and time stamp

 

10.Furthermore, you could go ahead and go for specific metrics in the logs like status code responses in the webserver. We do this by setting a tag to the status code for the HTTP request then follow other config protocols. Look at the configuration below:

 

After running the required command just like the previous step, we have this output:

 

11.Then finally utilize the Prometheus scrape in your config file and run your command again, it should be available on Prometheus. Look at the config below and update:

 

Look up other docs here

Congratulations, you can count logs via Vector in Prometheus. I should add that you must run all the latest versions of the services and tools used. Now, remember, this article is not only for experts in the software space, even newbies could hop in and learn a lot and that is why I try to make everything clear both in layman and professional terms, so if you have any questions, shoot or you can also reach out to us.

Lastly, you can read more about vector here.