Streaming DNS Analytics Using The CloudFlare API and Streamdata.io

We are looking into the different ways in which our API operations can be automated, made more real time, and allowing us to better understand the different events that occur across our platforms each day, using Streamdata.io. In our hunt for high value APIs that could be proxied using Streamdata.io, and streamed using Server-Sent Events (SSE), we found ourselves learning more about what is available via the CloudFlare DNS API.

CloudFlare provides a DNS analytics API, which allows you to pull DNS logs by date range, and different time interval, allowing you to get a minute by minute update of the activity at the DNS level for any API. Which then would be super easy to then proxy using Streamdata.io and stream to dashboards, mobile devices, and even to train machine learning models.

You can use the following cURL command to establish a DNS stream:

curl -v "https://streamdata.motwin.net/https://api.cloudflare.com/client/v4/zones/{zone_id}/dns_analytics/report/bytime?dimensions=responseCode,queryName&metrics=queryCount&sort=+responseCode,-queryName&since=2018-03-01T12:00:00Z&until=2018-03-11T23:00:00Z&limit=100&X-Sd-Token={streamddata_token}" -H "X-Auth-Email: {your_email}" -H "X-Auth-Key: {cloudflare_api_key}" -H "Content-Type: application/json"

CloudFlare provides other additional Zone analytics APIs, in addition to these basic set of aggregate analytics. We are playing around with more of their APIs, learning what is possible as we go along, looking for other opportunities for streaming DNS related data. Virtually every feature available via the CloudFlare interface has a supporting API, making it a pretty rich source of real time data regarding the DNS layer forAPI operations. When it comes to developing and training machine learning models, and using them to help us make decisions as part of API operations, the data CloudFlare makes available represents a tremendous opportunity.

We’ll add CloudFlare to our stack of APIs that can be used to develop real time streams of operational level data, when it comes to our API operations. The DNS layer of operations provides a pretty significant space for turning into real time streams, evaluating them for meaningful events, then further enriched using the ML models we use as part of our monitoring and security efforts. Making this type of research, and the profiling of ready to go operational level APIs like CloudFlare pretty relevant to how we are defining our DevOps, and DevSecOps programs in 2018.

**Original source: streamdata.io blog

LEAVE A REPLY

Please enter your comment!
Please enter your name here