DDOS, IDS, RTBH, and Rate limiting

Fri Nov 21 12:50:39 UTC 2014

On 21 Nov 2014, at 15:17, Denys Fedoryshchenko wrote:

> Word stateful has nothing common with stateful firewall.Stateful 
> protocol. "a protocol which requires keeping of the internal state on 
> the server is known as a stateful protocol."

Correct - and NetFlow is not stateful, by this definition.

> And sure unidirectional/bidirectional is totally unrelated.

On the contrary, it is quite relevant.

> Cisco 65xx (yes, they are obsolete, but they run stuff wirespeed)

They are not obsolete - they perform very well with Sup2T and 
EARL8-based linecards.

> Aug 24 12:30:53: %EARL_NETFLOW-SP-4-TCAM_THRLD: Netflow TCAM threshold 
> exceeded, TCAM Utilization [97%]

This is from a 6500 with either an EARL6 or EARL7 ASIC, which had many 
caveats with regards to NetFlow, including a lack of packet-sampled 
control of flow creation - i.e., sampled NetFlow.  As part of the 
extended team which defined requirements for the EARL8 ASIC, which is 
utilized in the Sup2T and DFC-4 enabled linecards, I can assure you that 
this is no longer an issue with 6500s running EARL8-based Sups and 
linecards.

> Also on many Cisco's if you use UBRL, then you cannot use NetFlow, 
> just because they use same part of TCAM resources.

This is where TCAM carving comes into play.  Also, it is not so much an 
issue with newer hardware, per the above.  Also, URBL is not commonly 
used in ISP networks.

> Others, for example Juniper, are using sampling (read - missing data),

The largest networks in the world use sampled NetFlow every hour of 
every day for many purposes, including DDoS 
detection/classification/traceback.  It works quite well for all those 
purposes.

> just to not overflow resources, and has various limitations, such as 
> RE-DPC communication pps limit, licensing limit.
> For example MS-DPC is pretty good one, few million flows in hardware, 
> 7-8Gbps of traffic, and... cost $120000.

You get what you pay for.

> But still they can run fine mirroring, and fastnetmon will do it's 
> job.

On the contrary - SPAN nee port mirroring cuts into the 
frames-per-second budget of linecards, as the traffic is in essence 
being duplicated.  It is not 'free', and it has a profound impact on the 
the switch's data-plane traffic forwarding capacity.

Unlike NetFlow.

> In certain limits. You can't set flow-active-timeout less than 60 
> seconds in Junos 14 for example.

Platforms vary, this is true.  However, I have never run into an issue 
with an active flow timer of 60s, nor have I ever run into anyone who 
has done so.

> On some platforms even if you can, you just run in the limits of 
> platforms again (forwarding - management communications).

This is incorrect.

> Fastnetmon and similar tools popularity says for itself.

Yes, it does - they are far less popular that NetFlow, because they do 
not scale on networks of any size, nor do they provide traceback (given 
your lack of comments on traceback elsewhere in this thread, it appears 
that you aren't familiar with this concept).

> "What can be asserted without evidence can be dismissed without 
> evidence."

You make my point very well, thank you.  There is overwhelming evidence 
that NetFlow and similar forms of flow telemetry scale well and provide 
real, measurable, actionable operational value on networks of all types 
and sizes.  The reason for the popularity of flow telemetry is that it 
is low-opex (no probes to deply); low-capex (no probes to deploy); 
scales to tb/sec speeds; is practicable for large networks (no probes to 
deploy); provides instantaneous traceback (probes can't do this); and 
provides statistics on dropped traffic (probes can't do this, either).

> Sweet marketing buzzwords,

It's pretty obvious which half of this 'conversation' is focused on 
marketing; and it isn't mine.

> that is used together with some unclear calculations,

No calculations have been discussed during the course of this 
'conversation'.

> to sell suffering hosting providers various expensive tools,

I'm uninterested in selling anyone anything.  What I'm interested in 
doing is correcting the misinformation you are promulgating regarding 
the utility of flow telemetry coupled with open-source flow analysis 
systems.  There has been no mention of any commercial systems or 
products in my half of this 'conversation'.

> that is not necessary for them.

Again, the benefits of flow telemetry are quite clear for networks of 
any size.

> OPEX of fastnetmon is a small fee for qualified sysadmin, and often 
> not required, because already hosting operator should have him.

You obviously do not know what the term opex actually means, nor what it 
encompasses.

> I can agree only that arguing about this subject is waste of time.

Yes, it isn't a profitable use of time to argue with someone who does 
not have the degree of operational expertise nor experience to back his 
demonstrably incorrect assertions.

> where netflow just by design cannot outperform it

Again, this is a completely unsupported statement with no basis in fact, 
and it totally ignores the inherent characteristics of flow telemetry 
(instantaneous traceback, statistics on dropped traffic, scalability, 
low opex) which make it eminently suitable for these various 
applications.

To be clear - the particular tool you are doing such a poor job of 
advocating is in no way unique, and is completely orthogonal to the 
utility, capabilities, and scalability of flow telemetry.  If such tools 
were so superior to flow telemetry, they would've eclipsed flow 
telemetry as the preferred mechanism for achieving visibility into 
network traffic many years ago.

I am going to stop replying to your trolling, because you obviously do 
not have the requisite operational experience and depth/breadth of 
knowledge to even try to plausibly support your demonstrably-incorrect 
assertions.  One can only hope that such a potentially useful tool as 
FastNetMon isn't tarnished in the view of those who have read this 
thread due to such uninformed, erroneous misadvocacy.

-----------------------------------
Roland Dobbins <rdobbins at arbor.net>