<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Hi Tim,</p>
<p>That absolutely depends on the amount of TAGs you use, and how
you aggregate, etc.<br>
I am collecting DSTAS, SRCAS, en DST AS per IP. And influx is not
even sweating a single drop....</p>
<p>We have a 4 Tbps of traffic during peak, and as well as pmacct
and influxdb or running very very smooth.<br>
</p>
<p>(With the mentioned aggregations I can see what a single customer
costs with Transit, Peering and IX (per IP even if needed) )<br>
And dst AS per port/description/ethernet name<br>
</p>
<p>From your mail i derive that you just pushed everything to influx
from flows, you have to be a bit smarter with the layout,
aggregations and continuous queries.<br>
(collect what you need)<br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 02-01-19 13:08, Tim Raphael wrote:<br>
</div>
<blockquote type="cite"
cite="mid:A3DA6044-9B62-419D-9AC9-CD5FE4F9E358@gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
I would advise against InfluxDB in this case - flow data has a
very high (and open) tag cardinality which is not suited to Influx
(although their recently new index format has improved this).
<div class=""><br class="">
</div>
<div class="">I’m currently pushing sFlow through Pmacct —>
Kafka —> Clickhouse (columnar store) with a summing merge
tree database engine.</div>
<div class="">Clickhouse is very fast for queries across columns
as well as aggregating down them (e.g. summing number of bytes).</div>
<div class=""><br class="">
</div>
<div class="">For example this is the results of a query of nearly
a year’s worth of MAC-to-MAC flows (7-tuple) queried for the
last 7 days between two given sets of MACs:</div>
<div class=""><i class=""><br class="">
</i></div>
<div class=""><i class="">2016 rows in set. Elapsed: 0.208 sec.
Processed 17.56 million rows, 1.03 GB (84.51 million rows/s.,
4.97 GB/s.)</i></div>
<div class=""><i class=""><br class="">
</i></div>
<div class="">There is also a Grafana datasource plugin for
Clickhouse :)</div>
<div class=""><i class=""><br class="">
</i></div>
<div class=""><i class="">- </i>Tim</div>
<div class=""><br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 2 Jan 2019, at 7:50 pm, H I Baysal <<a
href="mailto:hibaysal@gmail.com" class=""
moz-do-not-send="true">hibaysal@gmail.com</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="">PMACCT (Works Awesome)<br class="">
push to influxdb ( Works awesome)<br class="">
<br class="">
With some custom scripts to add/match interface
descriptions. And you can query whatever you want in
grafana :D<br class="">
And grafana has a nice API for rendering a
dashboardgraph to a PNG and you can send this png to
whatever chat/bot or mail you want.<br class="">
<br class="">
And all for free with 99% of accuracy.<br class="">
<br class="">
(Mucho gracias to Paulo :D )<br class="">
<br class="">
<br class="">
On 01-01-19 05:56, Avi Freedman wrote:<br class="">
<blockquote type="cite" class="">We do have a minimum
for commercial service that's more like $1500/mo but
we are coming out with a free tier in Q1 with lower
retention (among other deltas, but including fully
slice and dice flow analytics +BGP that it sounded
like Erik might be looking for).<br class="">
<br class="">
Feel free to ping me if anyone would like to help us
test the free tier in January.<br class="">
<br class="">
Thanks,<br class="">
<br class="">
Avi Freedman<br class="">
CEO, Kentik<br class="">
<br class="">
<blockquote type="cite" class="">Doesn't Kentik cost
like $2000 a month minimum?<br class="">
<br class="">
<br class="">
On Mon, Dec 31, 2018 at 11:57 AM Matthew Crocker
<<a href="mailto:matthew@corp.crocker.com"
class="" moz-do-not-send="true">matthew@corp.crocker.com</a>><br
class="">
wrote:<br class="">
<br class="">
<blockquote type="cite" class=""> +1 Kentik as
well, DDoS, RTBH, Netflow. Cloud based so I
don't have to<br class="">
worry about it.<br class="">
<br class="">
On 12/31/18, 11:37 AM, "NANOG on behalf of Bryan
Holloway" <<br class="">
<a href="mailto:nanog-bounces@nanog.org" class=""
moz-do-not-send="true">nanog-bounces@nanog.org</a>
on behalf of <a href="mailto:bryan@shout.net"
class="" moz-do-not-send="true">bryan@shout.net</a>>
wrote:<br class="">
<br class="">
+1 Kentik ...<br class="">
<br class="">
We've been using their DDoS/RTBH mitigation
with good success.<br class="">
<br class="">
<br class="">
On 12/31/18 3:52 AM, Eric Lindsjö wrote:<br
class="">
> Hi,<br class="">
><br class="">
> We use kentik and we're very happy. Works
great, tons of new<br class="">
features<br class="">
> coming along all the time. Going to start
looking into ddos<br class="">
detection<br class="">
> and mitigation soon.<br class="">
><br class="">
> Would recommend.<br class="">
><br class="">
> Kind regards,<br class="">
> Eric Lindsjö<br class="">
><br class="">
><br class="">
> On 12/31/2018 04:29 AM, Erik Sundberg
wrote:<br class="">
>><br class="">
>> Hi Nanog….<br class="">
>><br class="">
>> We are looking at replacing our
Netflow collector. I am wonder what<br class="">
>> other service providers are using to
collect netflow data off their<br class="">
>> Core and Edge Routers. Pros/Cons…
What to watch out for any info<br class="">
would<br class="">
>> help.<br class="">
>><br class="">
>> We are mainly looking to analyze the
netflow data. Bonus if it does<br class="">
>> ddos detection and mitigation.<br
class="">
>><br class="">
>> We are looking at<br class="">
>><br class="">
>> ManageEngine Netflow Analyzer<br
class="">
>><br class="">
>> PRTG<br class="">
>><br class="">
>> Plixer – Scrutinizer<br class="">
>><br class="">
>> PeakFlow<br class="">
>><br class="">
>> Kentik<br class="">
>><br class="">
>> Solarwinds NTA<br class="">
>><br class="">
>> Thanks in advance…<br class="">
>><br class="">
>> Erik<br class="">
>><br class="">
>><br class="">
>><br class="">
------------------------------------------------------------------------<br
class="">
>><br class="">
>> CONFIDENTIALITY NOTICE: This e-mail
transmission, and any<br class="">
documents,<br class="">
>> files or previous e-mail messages
attached to it may contain<br class="">
>> confidential information that is
legally privileged. If you are not<br class="">
>> the intended recipient, or a person
responsible for delivering it<br class="">
to<br class="">
>> the intended recipient, you are
hereby notified that any<br class="">
disclosure,<br class="">
>> copying, distribution or use of any
of the information contained in<br class="">
or<br class="">
>> attached to this transmission is
STRICTLY PROHIBITED. If you have<br class="">
>> received this transmission in error
please notify the sender<br class="">
>> immediately by replying to this
e-mail. You must destroy the<br class="">
original<br class="">
>> transmission and its attachments
without reading or saving in any<br class="">
>> manner. Thank you.<br class="">
><br class="">
<br class="">
<br class="">
<br class="">
</blockquote>
</blockquote>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</blockquote>
</body>
</html>