automatic rtbh trigger using flow data
H I Baysal
hibaysal at gmail.com
Thu Sep 6 08:40:29 UTC 2018
I also looked at PMACCT, however I have to say that I am way more
pleased with the current setup.
PMACCT relies on bgp session etc, while you actually get that info from
your (s)flow. No need for lookup.
And I really didn't like the idea of SQL like DB, I know i could push it
to influx. But i thought this could be done way simpler and i think i
did exactly that.
I store the "raw" info for 7 days, as detailed as possible.
then downsample for a week again, a month and a year.
And the picture rendering api function of grafana makes it possible for
me to do for example /dst AS# or /peer #AS or /as-path AS# in a
chatbot, and grafana actually pushes the graph to telegram in our case,
but it could be any bot.
So at peering conferences, i can see with 1 message how much traffic we
do either to a peer, destination or if traffic passes your AS.
I don't see any (pre-made) system beating this :P
On 01-09-18 01:51, Paweł Małachowski wrote:
> On Fri, Aug 31, 2018 at 11:09:19AM +0200, H I Baysal wrote:
>> My personal view is, as long as you can store your flow info in a
>> timeseries database (like influxdb and NOT SQL LIKE!!!!!!!) you can do
>> whatever you want with the (raw) data. And create custom triggers for
>> different calculations.
> For one of our customers I've deployed good old pmacct + MySQL
> (using memory engine) backend for DDoS detection purposes.
> It has some drawbacks (e.g. one has to frequently delete old
> records to keep tables fit and fast) but it allows asking complex
> SQL queries against these short term data (e.g. different detection
> logic per subnets) or precompute with triggers.
>> Flows are on the fly and are coming in constantly, you could have a
>> calculation like group by srcip and whatever protocol you want or just
> Beware of high cardinality issues when facing random src IP floods.
> BTW, once again pmacct (with some glue) is nice for feeding flow
> data into time series database. It can pre aggregate and pre filter
> low volume flows to reduce storage requirements.
More information about the NANOG