<div dir="ltr">ok got it thank you Nick</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, May 5, 2022 at 3:43 PM Nick Olsen <<a href="mailto:Nick@141networks.com">Nick@141networks.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
Nehul,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
He was running the 15 code train. I think 15.1R6.7. But don't take that as fact. I just know it was 15 for sure.</div>
<div id="gmail-m_5628548544172173184appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_5628548544172173184divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Nehul Patel <<a href="mailto:nehul.patel@gmail.com" target="_blank">nehul.patel@gmail.com</a>><br>
<b>Sent:</b> Thursday, May 5, 2022 6:40 PM<br>
<b>To:</b> Nick Olsen <<a href="mailto:Nick@141networks.com" target="_blank">Nick@141networks.com</a>><br>
<b>Cc:</b> <a href="mailto:nanog@nanog.org" target="_blank">nanog@nanog.org</a> <<a href="mailto:nanog@nanog.org" target="_blank">nanog@nanog.org</a>><br>
<b>Subject:</b> Re: Strange behavior on the Juniper MX240</font>
<div> </div>
</div>
<div>
<div dir="ltr">Hi Nick,
<div><br>
</div>
<div>Thank you for the feedback on it. Would you please let me know which Juno OS version he had installed on the MX Chassis that works with the extended memory command of it? </div>
</div>
<br>
<div>
<div dir="ltr">On Thu, May 5, 2022 at 12:50 PM Nick Olsen <<a href="mailto:Nick@141networks.com" target="_blank">Nick@141networks.com</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
Friend of mine had this issue recently on an MX chassis running DPC's and RE-2000's.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
The extend memory command others have mentioned worked for him.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
His instance drove us crazy for a bit. The device would learn a route, show that it was installed (show routes) but traffic to said prefix would bounce net unreachable. We even pushed a static just for S&G's and that still didn't resolve it. It was a single
prefix that a customer had reported.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
Some things to consider, as others have mentioned.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<ol>
<li><span>IPv6 routes share the same space. And use more per-route. You can extend the life of this box (probably considerably) by dropping full tables for IPv6. Perhaps taking just a default (Same goes for v4).</span></li><li><span>It seems from your previous output that you're taking ~1 full v4 table. And 2x v6 tables. Do you really need a full table if you're only taking 1 v4 table? Consider switching to a default only? In my Colleagues case, he was taking 2 full tables of
v4 and v6 until he hit the same issue.</span></li><li>While you're RE's could use a nice upgrade too. Your linecards are actually the problem here. If you move to anything > DPC you get the trio chipset with much more FIB space (2 Million routes I believe?). I'd consider new RE's and new line cards for this
box. Which might also mean new switch fabric controllers.... Basically, we'd be talking a full overhaul sans the power supplies and chassis.</li><li>Consider taking a default + full routes. Then filtering > /24 (if you even have anything < /24 learned now) (/48 on IPv6).</li></ol>
<div>Start with the memory command first and see where that gets you. But keep a watchful eye out for this to happen again (as the DFZ grows). Eventually your only option will be to filter routes and rely on a default or upgrade.</div>
</div>
<div id="gmail-m_5628548544172173184x_gmail-m_-515104920706288533appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_5628548544172173184x_gmail-m_-515104920706288533divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> NANOG <nanog-bounces+nick=<a href="mailto:141networks.com@nanog.org" target="_blank">141networks.com@nanog.org</a>>
on behalf of Nehul Patel <<a href="mailto:nehul.patel@gmail.com" target="_blank">nehul.patel@gmail.com</a>><br>
<b>Sent:</b> Wednesday, May 4, 2022 3:56 PM<br>
<b>To:</b> <a href="mailto:nanog@nanog.org" target="_blank">nanog@nanog.org</a> <<a href="mailto:nanog@nanog.org" target="_blank">nanog@nanog.org</a>><br>
<b>Subject:</b> Strange behavior on the Juniper MX240</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div><br>
</div>
<div>Hi NANOG,</div>
<div><br>
</div>
<div>We are seeing some strange behavior on our Juniper MX240 Chassis it is randomly dropping the routes to the certain destination IP address getting the following errors on the MX240 Chassis</div>
<div><br>
</div>
<div>If Someone has seen these errors before please suggest how to resolve it</div>
<div><br>
</div>
<div><br>
May 4 12:42:00 cr01 newsyslog[44735]: logfile turned over due to size>1024K<br>
May 4 12:42:01 /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, err 5 (Invalid)</div>
<div>May 4 12:42:01 /kernel: RT_PFE: RT msg op 3 (PREFIX CHANGE) failed, err 5 (Invalid)</div>
<div>May 4 12:42:01 last message repeated 4 times<br>
May 4 12:42:01 fpc0 RT: IPv6:0 - 2600:40fc:1011::/48 (add rt entry into jtree failed)<br>
May 4 12:42:01 fpc0 RT-HAL,rt_entry_add_msg_proc,2028: rt_halp_vectors->rt_create failed<br>
May 4 12:42:01 fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto ipv6,len 48 prefix 2600:40fc:1011::/48 nh 1048576<br>
May 4 12:42:01 fpc0 RT-HAL,rt_msg_handler,540: route process failed<br>
May 4 12:42:01 fpc0 RT: Failed prefix add IPv6 - 2001:67c:20fc::/48 (No memory) on FE 0<br>
May 4 12:42:01 fpc0 RT: IPv6:0 - 2001:67c:20fc::/48 (add rt entry into jtree failed)<br>
May 4 12:42:01 fpc0 RT-HAL,rt_entry_add_msg_proc,2028: rt_halp_vectors->rt_create failed<br>
May 4 12:42:01 fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto ipv6,len 48 prefix 2001:67c:20fc::/48 nh 1048576<br>
May 4 12:42:01 fpc0 RT-HAL,rt_msg_handler,540: route process failed<br>
May 4 12:42:01 fpc0 RT: Failed prefix add IPv6 - 2606:2800:e004::/48 (No memory) on FE 0<br>
May 4 12:42:01 fpc0 RT: Failed prefix add IPv6 - 2a05:3181:ffff::/48 (No memory) on FE 0<br>
May 4 12:42:01 /kernel: RT_PFE: RT msg op 3 (PREFIX CHANGE) failed, err 5 (Invalid)<br>
May 4 12:42:01 /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, err 5 (Invalid)<br>
May 4 12:42:01 /kernel: RT_PFE: RT msg op 2 (PREFIX DELETE) failed, err 5 (Invalid)<br>
May 4 12:42:02 fpc0 RT: Failed prefix add IPv4 - 79.120.22/24 (No memory) on FE 0<br>
May 4 12:42:02 fpc0 RT: IPv4:0 - 79.120.22/24 (add rt entry into jtree failed)<br>
May 4 12:42:02 fpc0 RT-HAL,rt_entry_add_msg_proc,2028: rt_halp_vectors->rt_create failed<br>
May 4 12:42:02 fpc0 RT-HAL,rt_entry_add_msg_proc,2092: proto ipv4,len 24 prefix 79.120.22/24 nh 1048583<br>
May 4 12:42:02 /kernel: RT_PFE: RT msg op 1 (PREFIX ADD) failed, err 5 (Invalid)<br>
May 4 12:42:02 fpc0 RT-HAL,rt_msg_handler,540: route process failed<br>
</div>
<div><br>
</div>
<div>May 4 09:33:17 fpc0 RSMON: Resource Category:jtree Instance:jtree2-seg0 Type:free-pages Available:20 is less than LWM limit:1638, rsmon_syslog_limit()<br>
May 4 09:33:17 fpc0 RSMON: Resource Category:jtree Instance:jtree2-seg0 Type:free-dwords Available:1280 is less than LWM limit:104857, rsmon_syslog_limit()<br>
May 4 09:33:18 fpc0 RSMON: Resource Category:jtree Instance:jtree3-seg0 Type:free-pages Available:19 is less than LWM limit:1638, rsmon_syslog_limit()<br>
May 4 09:33:18 fpc0 RSMON: Resource Category:jtree Instance:jtree3-seg0 Type:free-dwords Available:1216 is less than LWM limit:104857, rsmon_syslog_limit()<br>
May 4 09:33:18 fpc1 RSMON: Resource Category:jtree Instance:jtree0-seg0 Type:free-pages Available:16 is less than LWM limit:1638, rsmon_syslog_limit()<br>
May 4 09:33:18 fpc1 RSMON: Resource Category:jtree Instance:jtree0-seg0 Type:free-dwords Available:1024 is less than LWM limit:104857, rsmon_syslog_limit()<br>
May 4 09:33:18 fpc1 RSMON: Resource Category:jtree Instance:jtree1-seg0 Type:free-pages Available:15 is less than LWM limit:1638, rsmon_syslog_limit()<br>
May 4 09:33:18 fpc1 RSMON: Resource Category:jtree Instance:jtree1-seg0 Type:free-dwords Available:960 is less than LWM limit:104857, rsmon_syslog_limit()<br>
May 4 09:33:18 fpc1 RSMON: Resource Category:jtree Instance:jtree2-seg0 Type:free-pages Available:19 is less than LWM limit:1638, rsmon_syslog_limit()<br>
May 4 09:33:19 fpc1 RSMON: Resource Category:jtree Instance:jtree2-seg0 Type:free-dwords Available:1216 is less than LWM limit:104857, rsmon_syslog_limit()<br>
May 4 09:33:19 fpc1 RSMON: Resource Category:jtree Instance:jtree3-seg0 Type:free-pages Available:17 is less than LWM limit:1638, rsmon_syslog_limit()<br>
May 4 09:33:19 fpc1 RSMON: Resource Category:jtree Instance:jtree3-seg0 Type:free-dwords Available:1088 is less than LWM limit:104857, rsmon_syslog_limit()<br>
May 4 09:33:19 fpc2 RSMON: Resource Category:jtree Instance:jtree0-seg0 Type:free-pages Available:15 is less than LWM limit:1638, rsmon_syslog_limit()<br>
May 4 09:33:19 fpc2 RSMON: Resource Category:jtree Instance:jtree0-seg0 Type:free-dwords Available:960 is less than LWM limit:104857, rsmon_syslog_limit()<br>
May 4 09:33:19 fpc2 RSMON: Resource Category:jtree Instance:jtree1-seg0 Type:free-pages Available:15 is less than LWM limit:1638, rsmon_syslog_limit()<br>
May 4 09:33:19 fpc2 RSMON: Resource Category:jtree Instance:jtree1-seg0 Type:free-dwords Available:960 is less than LWM limit:104857, rsmon_syslog_limit()<br>
</div>
<div><br>
</div>
<div>Any suggestions will be helpful</div>
<div><br>
</div>
<div><br>
</div>
<div>Please do let me know if you have any questions.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>Regards and thanks,</div>
<div> Nehul</div>
<div><br>
</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote></div>