<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Sep 19, 2021, at 16:17 , Victor Kuarsingh <<a href="mailto:victor@jvknet.com" class="">victor@jvknet.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="auto" class="">Owen,</div><div dir="auto" class=""><br class=""></div><div class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Sep 18, 2021 at 23:51 Owen DeLong <<a href="mailto:owen@delong.com" class="">owen@delong.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space" class=""><div class=""><blockquote type="cite" class=""><div class="">On Sep 18, 2021, at 12:34 , Victor Kuarsingh <<a href="mailto:victor@jvknet.com" target="_blank" class="">victor@jvknet.com</a>> wrote:</div><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Sep 18, 2021 at 2:39 PM John Levine <<a href="mailto:johnl@iecc.com" target="_blank" class="">johnl@iecc.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">It appears that Owen DeLong via NANOG <<a href="mailto:owen@delong.com" target="_blank" class="">owen@delong.com</a>> said:<br class=""><br class=""></blockquote><div class=""><br class=""></div><div class="">Glad you noted this. Thinking this was/is purely a hardware cycle problem related to normal/forced upgrade strategies. On that point, most hardware I know of from 2004 in larger networks is long fully depreciated and sweating assets 15+ years can happen, but I don't personally think this is the biggest issue.</div></div></div></div></blockquote><div class=""><br class=""></div>It’s certainly not purely hardware, but it also doesn’t require solving the entire problem across the entire backend.</div><div class=""><br class=""></div><div class="">What is urgently needed is to fix the customer-facing front end so that it speaks both protocols (or at least speaks IPv6).</div><div class=""></div></div></blockquote><div dir="auto" class=""><br class=""></div><div dir="auto" class="">This is a great question. So when we approached this (going back a while now) this is what I thought too. But I was responsible to solve this end to end. So just updating front end (CPEs, network routers, DHCP, provisioning, IP address planning, security, peering/transit, staff training, test harnesses/plans, etc) was not sufficient to turn on IPv6. </div><div dir="auto" class=""><br class=""></div><div dir="auto" class="">The high cost and time challenge issues were not upgrading backend servers to IPv6, but backend provisioning systems, tools, customer support tools, their training, and related items. These systems were far more complex and costly to address (especially when considering opportunity cost). Without all of this in place, we could not actually deploy IPv6 for production use (it’s not just Turing it on, its our ability to operate it down to the person answer the phone, the technician that installs and/or fixes/replaces items in home). </div></div></div></div></blockquote><div><br class=""></div>This sounds like an eyeball environment… Note that my question was in the context of the laggard content providers.</div><div><br class=""></div><div>Eyeball providers are going to have to do something eventually whether they like it or not and I’m a whole lot less worried about a forcing function for them.</div><div><br class=""></div><div>The major eyeball providers have basically completed this. Hopefully that means that the vendors you’re using have been forced to upgrade their code and such to accommodate by now.</div><div><br class=""><blockquote type="cite" class=""><div class=""><div class=""><div class="gmail_quote"><div dir="auto" class="">Also at that time vendor CPE hardware was not as far along on IPv6 readiness (approx mid-decade 2000s). Getting reliable code at that time was hard with a lot of brokenness (which also could not go into production until it was reliable). Perhaps this a non-issue today, but it certainly was not a something that was “ready to go” even 10-15 years. This (IPv6 readiness in CPE code) was also competing with other priorities often blowing up timelines which meant it had to wait as not releasing code into production on a strict timeline was not an option for business reasons.</div></div></div></div></blockquote><div><br class=""></div>Sure, but a lot of that has gotten better in the recent years, largely driven by some of the major eyeball ISPs.</div><div><br class=""><blockquote type="cite" class=""><div class=""><div class=""><div class="gmail_quote"><div dir="auto" class="">All said and done, we got it completed, but it far most costly than we first thought, and IPv4 costs did not go away after we were completed. Sure it’s possible to reduce those costs with an aggressive program, but it is not as simple as many think.</div></div></div></div></blockquote><div><br class=""></div>Agreed… IPv4 costs for you, as an eyeball provider, can’t really go away until you can start doing one or more of the following:</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>1.<span class="Apple-tab-span" style="white-space:pre"> </span>Raising your overall rates and providing a discount to IPv6-only customers</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>2.<span class="Apple-tab-span" style="white-space:pre"> </span>Adding a surcharge to your billing for customers still requiring IPv4 services</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>3.<span class="Apple-tab-span" style="white-space:pre"> </span>Turning off or reducing availability of IPv4 native services and moving more towards IPv4AAS</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>4.<span class="Apple-tab-span" style="white-space:pre"> </span>Turning off IPv4 services outright</div><div><br class=""></div><div>All of those basically depend on moving a few key laggard content providers forward.</div><div><br class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space" class=""><div class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div class="">As you noted John, its the plethora of software, support systems, tooling, and most important in many environments - legacy customer management and provisioning systems that can be the limiting factor. I recall looking back when leading IPv6 turn-up, those were the biger problems to solve for. Often such systems are extremely expensive to touch and working on them required prioritization against direct revenue generating projects (opportunity cost) . Replacing routers was just a money problem.</div></div></div></div></blockquote><div class=""><br class=""></div>Why do those systems have to be updated in order to deliver the service to the customer in both protocols?</div></div></blockquote><div dir="auto" class=""><br class=""></div><div dir="auto" class="">Addressed in above comment. </div></div></div></blockquote><div><br class=""></div>Eyeball… Now think about it in the context of a content provider… Especially one that is behind a CDN where it’s literally simply flipping the switch on the CDN</div><div>that says front end our stuff with both protocols.</div><div><br class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote"><div dir="auto" class="">When it comes to turning on IPv6 and reducing your dependency on IPv4, your mileage may vary (in terms of real costs, complexities and deployment). </div></div></div></blockquote><div><br class=""></div>Agreed… Much more variable for eyeballs, but still some variability for content providers, too. Nonetheless, I think it is generally quite simple for content providers today while there are still some issues that remain unsolved on the eyeball side of things.</div><div><br class=""></div><div>The good news is that once a few laggard content providers add IPv6 capabilities, eyeball providers can start treating IPv4 as (mostly) optional in a lot of areas, so that those which have deployed IPv6 to their customers in a meaningful way can start to shrink their IPv4 costs.</div><div><br class=""></div><div>Owen</div><div><br class=""></div></body></html>