FW: Summary/minutes from sunday evening?

Mon Jan 31 13:01:16 UTC 2005

-----Original Message-----
From: Matthew Petach [mailto:mpetach at gmail.com]
Sent: Monday, January 31, 2005 11:15 AM
To: Huopio Kauto
Cc: nanog at nanog.org
Subject: Re: Summary/minutes from sunday evening?

On Mon, 31 Jan 2005 11:00:55 +0200, Huopio Kauto <Kauto.Huopio at ficora.fi>
wrote:
> 
> So..
> 
> Could someone please summarize the Sunday evening meeting? Minutes anyone?
>

I'm not allowed to post to NANOG from this account...but if you'd
like to forward this message to the list, I've included my minutes
from the Sunday night session at the end of this message.

Thanks!

Matt

> --Kauto
> 
> Kauto Huopio - kauto.huopio at ficora.fi
> Information Security Adviser / CERT-FI -coordinator
> Finnish Communications Regulatory Authority / CERT-FI
> tel. +358-9-6966772, fax. +358-9-6966515
> CERT-FI duty desk +358-9-6966510 / http://www.cert.fi
> 
> 

2005.01.30 Coordinating NANOG--input from community

Paul Vixie, moderator
Betty Burke, Merit
Steve Feldman, CNET
Martin Hannigan, Verisign
Paul Vixie, ISC
Dan Golding, the Burton Group

Betty Burke starts off.

NANOG needs to evolve.

Thanks to all who helped this effort
We all recognize the need for continous change
Our goals this evening:
 Get your input
 Ensure ongoing community coordination
My role at Merit--Special Projects

Why Merit?
NANOG grew out of of NSFNET backbone services
re-engineered and managed  by Merit 87 -95
Merit has been joining diverse technologies and organizations
  nationally/regionally since late 60s
Regional role: MichNet, Michigans largest ISP
 525 customers (affiliates)
 13 members (Michigans' 4-yr public universities)
 26 peers
National role: neutral organization for NANOG coordination

NANOG's early years to today
94 - 95 meetings planned by Elise Gerich
95 - 98 Bill Norton takes over, begins to get outside help
  with agenda in 97 convenes advisor group
98 program committee formalized under Craig Labovitz
2000 Susan Harris becomes coordinator

PC today:
 selects talks for agenda
 chaired by Susan Harris
 target is 18 members, currently at 17
Members selected by:
 self-nomination (volunteer) or PC nominations
  PC holds internal discussion, issues invitation to join

Thanks to the program committee
k claffy, CAIDA
Keven Epperson, LEvel3
Steve Feldmen, CNET
Elise Gerich, Juniper
Barry Greene, Cisco
Sue Hares, NextHop
Susan Harris, Merit
Crag Labovitz, Arbor
Bill Manning, EP.NET
Dave Meyer, U of O/Cisco
Bill Norton, Equinix
Dave O'Leary, Juniper
Ted Seely, Sprint
Stephen Stuart, Google
Rob Thomas, Cisco/Team Cyrmru
Curtis Villamizer
Bill woodcock

Mailing list growth
evolved from NSFNET Regional-Techs list
became NANOG in 1994
today: 7,919 subs, 10,000 receive mail

Mailing List Moderation
Sample annual volume: in 2004, 10,500 unique messsages
 sent to the list
Moderator sent ~460 private messages since 2000 (about 2/week)
135 dealt with spam management topics
 loosened up on this in mid-04
29 noop (moderation review would be helpful)
67 pseudonyms (many added .sig on request)
52 foul language
46 politics
44 personal attack

Neither Betty nor Susan are operators, so target may
have been off on the 29 non-op moderations.

29 addresses temporarily removed since 2000
 two warnings, three you're removed for 3-12 months,
  depending on violation
 People can re-subscribe upon request.  Nobody has
 ever been denied a request to resubscribe to the list.

NANOG attendance slide.  Less than 100 in 1994, to peak
of over 600 in 2001, back down to 400 now.

Only a few registration fee increases since inception.
Given hotel logistics, that's surprising, and difficult;
the goal is to try to keep the cost down so people don't
find it hard to manage to attend.
Get creative with sponsorships to help with costs.

That's it for formal information from Betty.

Steve Feldman will take over talking about the program
committee, and changes for the future.

NANOG talk selection process
Steve Feldman NANOG PC  CNET Networks
One of 17 Program Committee members

Ancient History
Originally the NANOG Chair did it all
Bill Norton started informal admisory committeee in 1997
Mailing list used to coordinate

Modern History
Advisory committee became Program Committee in 1998
Mailing iist discussion as proposals recieved
Formalized PC's talk selection process in 2004
Continuing refinements and improvement

Goals
Talks reflect attendees' needs
Audience
 ISPs (transit and end-user)
 Content Providers
 Large Enterprises
 Research/Education
 Vendors

Critera
Relevance: operational experience
Technical content (vs marketing)
Quality:
 Slides
 Language
 Speaker  (good with audience?)

Timeline
Call for Presentations, few months ahead
CFP Deadline
 6 weeks before meeting
Prgram Committee Review and Rating
PC Conference Call: Initial Selections (blind until this)
Initial notifications
 4 weeks before meeting
Late Submission Review (more later)
Agenda

This was a short period since last meeting, a bit rushed
due to holidays as well.

Call for Presentation
send to list

PC Review and Rating
Review starts after submission deadline
PC Review
PC Rating
each rated 1-5
comments
 blind to each other until call

Conflict of Interest Policy
Adopted May 2004
PC members must disclose all conflicts of interest during 
  discusssions
PC members recuse themselves when they deem it appropriate
 (some vendors have opted out)

Web Based Rating System
 allows topic to be rated 1-5, or skip, and add comments

Then a summary page is generated with comments and
ratings; that's what goes into the conf call

PC Conference Call
Review ratings and comments
 Decisions
Accept
Reject
Conditional
 request modifications
 possible PC member oversight
 Re-review after changes made
Late Submissions
Late submissions will be considered if: 
 exceptional quality or relevance
 available agenda slot
Less formal procedure
 PC mailing list discussion/conssneus
 Chair has some discretion

Agenda
schedule set by chair after talks are seleected
subject to change until sent to printer
minor changes still possible after that

Las Vegas Statistics
19 proposals considered:
Accepted 15
Rejected 4
Tentative: 1 (eventually accepted after revisions)
5 late subissions
 5 accepted
Talks plus BOFs for these numbers

Questions at end

Dan Golding with proposal
NANOG-reform working group

Why Reform NANOG?
Continuing problems with mailing list administration
 uneven moderation standards
 lack of transparency or reasoned decision making
 breakdown in mutual respect and collegiality
Also issues with NANOG meetings
 Program committee perceived to be out of touch
  with general operator community
 selection of topics is extremely uneven
 perception of powerlessness on the part of NANOG community

Time to stop complaining
A working group "NANOG-Reform" has been busy since the last
 NANOG meeting
Genesis after a spontaneous community meeting with Betty
  Burke of Merit concerning mailing list issues
 Representatives of various constituencies- former speakers,
  frequent attendees, previous hosts.  Participants work  for 
   vendors, operators, researchers, and others
Goal:
 come up with a prosal to fix what's wrong so we can 
  move forward
Work in progress: www.nanog-reform.org
The big proposal: Turn NANOG into membership org in 
  partnership with Merit

an elected board of directors/steering committee
 7 members, staggered 2 year terms
One member elected as Chair public face of NANOG org
Board responsibilities
 selection of list moderator and program committee
 sets policy for mailing list and conferences
 responsible for organization goals and direction
 working with committees, Merit staff, hosts, sponsors, 
  speakers and the larger community
budgetary oversight
 who votes for the commmittee?  anyone who has attended a
  conference in the last 12 months 
  (#insert membership-criteria)

Proposal specifics:
List administration committee
responsible for NANOG mailing list administrative issues
will coordinate use of peer pressure and other 
 techniques to keep the list "on-topic" (liberally defined)
must be technically knowledgable and have an
 understanding of all issues that can be discussed 
  on the list (collectively)
will implement BoD/SC policy for rare omposted
 moderation actions, which will be performed in a 
 transparent manner
Also FAQ keeper

Specifics:
Program committee
  selected BoD/SC for fixed terms
 responsible for conference program and agenda
 performs speaker recruitment and selection
 coordinate with Merit on venue selection and preparation
Merit Networks Role
 Hosting of NANOG web site and mailing list
 registration, fee collection, and financial management
 coordinate with local hosts for venue selection
 Ex-Officio representation on BoD

The Future
Where do we go from here?
 community buy-in: please endorse at www.nanog-reform.org
futher work: by-laws and initial elections
 cooperation of Merit and partner organizations like 
  ARIN and RIPE
Undesirable but possible outcomes
 Merit "taking input from the community" perpetually 
   --no action
 Continued erosion of NANOG mailing list value
 Split of community with rival ops conference in
  association with ARIN (like RIPE)

List of those responsible for putting together the
propsoal
Jim Bacher
Randy Bush
Steve Gibbard
Patrick Gilmore
Daniel Grollding
Mike Huges
Rodney Joffe
Simon Lockharte
Chirs Malayter
Ren Provo
and many others

Bill Norton, Equinix, first at mike.
there are already a number of competitive forums that
have grown out of the lack of peering coordinator
forums at NANOG, for example (Equinix peering forums)

Question about presentations missing the mark; but
only 4 were set aside, so not as though there are
a wealth of talks that are being passed over.

Rob Seastrom next up, not sure which org.
some presenters feel rebuffed from talking about
some topics.  does thank merit for sticking with
NANOG even when it wasn't solid financially.
Now, earning $350K/year, according to his back o
envelope calculations.
wants Merit to help keep nanog relevant to community
he talks about CBC, group he founded in Cambridge,
Cambridge Bandwidth Consortium; leadership and 
 direction should come from within the community, 
 not be imposed on it.

Betty notes Merit is a 501c3 non-profit; there is
enough in coffers to cover the logistics needed to
put on the conference; essentially, the reserve is
there to cover the next conference coming up in
case membership attendance is too low to pay the
hotel bills.
Maybe would help to share more details of the budget
with the membership, to show the financial liability
Merit assumes in covering NANOG.
RS is grateful for reserve, and feels it is a good 
thing, just to clarify.

Paul Vixie steps up to mike;
he has periodically spoken at NANOG when Sue asks;
he figures it's a bottom-of-the-barrel case when
he speaks.
The program committee is a volunteer effort,
they're not terribly empowered, they do their
work and then get back to their real jobs.  It
takes some amount of arm twisting to get the
presenters from the community up front to talk
about the projects they've been working on.
Paul thinks it would be better for there to be
50 talks they had to select among, instead of 19,
so that there could be a wider variety of talks.
Paul urges reform of program committee

Steve Wilcox from Telecomplete;
document on nanog-reform site is a work in progress,
he's got some suggestions he's been working on tweaking;
there's a main thrust, and then there's details; the
details can be worked out once the major idea is
decided on.  First get board in!
Second point, even if you don't have a criticism with
the current system, recognize the new system is a
community based system; that will help ensure that
going forward, there will be more direct community
input into the process.

Matthew Ringel of Tufts university.
talks that are relevant is his recommendation.
Might be good to have a best current practices
track.  Most people here have some hand/foot in
trying to improve processes, whether ops-eng, or arch.
Could pair like minded people for the BCPs, help

Chris Malyter, twtelecom, would like to see the
financials Betty talked about.  Would like to see
either quarterly or annual charts of where the money
goes.  Also notes nanog-reform is a working group; it's
a draft for consideration, a work in progress that people
should help contribute to.  Get on the mailing list,
and keep working with them to make it the best proposal
possible.
If there's 8,000 people on the list, and only 400
show up to conferences, how are we failing to atract
the other members, what is holding back their more
active participation.

Alex Rubenstein, NAC is up next.  Many people are
wondering where the money they pay goes, creates some
concern.  Would be nice to have a wrap-up summary after
each nanog.
program committee is doing public requests, but what
about making direct invitations to specific people for
programs.

Barry Greene repeats Andrew Partan's point that the
PC already tries to twist arms to get people to
present.  Barry then notes that 
what we are proposing is like APIA and APRICOT, 
should talk to the folks on those committees to make
sure we don't make the same mistakes.
Program committee, easy to say they should recruit
people, hard to actually get it done.
There's fresh talent coming into community, but PC
when they recruit only go back to the people they
already know.  Security side can pull from NSP-SEC,
for example.  Good to refresh program committee
constantly to keep them in touch with the talent
base that's out there.
Next question to Merit; is Merit in a position to
let go?  With APIA, APRICOT, and APNIC, if the parent
doesn't let go or refuses to cooperate, will make any
reform very difficult.

Betty responds to program committee challenge.  there
may be a certain lack of operator input, but they
really do try to go out and solicit people; the people
on the committee really do try.
Betty agrees that buget information sharing is definitely
something that can be done, and is probably overdue.
Would also be good to share survey information from
each meeting.  an independent compilation happens, but
they haven't reported it back to the community.
she commits to maintaining the dialog that is starting
at this point.
She doesn't see it as letting go, as much as evolving
in partnership; wants to come up with a structure that
works for the overall community.  Some room for change
on both sides of the table.  Merit is definitly on board
with the idea of change--this meeting came out of the
feedback from NANOG in Reston.

Steve Feldman agrees that going out and recruiting
talks is tough, but they could do more legwork on it.
Some of the best people for recruiting are perhaps
not those who actively operate, but those who touch
many different operators.
Daniel Golding agrees that other orgs also pull
from vendors to get talks from operators.
he notes that some PC members are on the reform list.
It's not a matter of kicking people out, it's just a
matter of getting fresh blood into the scene.
More former Merit people on the PC than current operators;
certainly some new fresh blood would help get a different
variety of talks.

Josh Snowhorn, NOTA, hosted twice, took about $185,000
to host it, so budget does take some looking at.  Josh
notes that he gets better discussions at the bar vs
listening to talk.
Notes that in RIPE, they have socials scheduled to get
people to talk; would be good to do that, rather than
just the beer and gear where you try to grab as much
gear as possible and then get out.
So, look at other orgs that work.  Multiple tracks,
more social interaction.
The social interaction helps establish a web of trust
between operators that helps us do our jobs more
effeciently and effectively.

Joe Provo, relaying from EFnet remotely.
Some suggestions about minutae.
One observation from several people; smaller hosting
and ISP networks not well served.  Tabulating the
surveys shows about attendees, not about why the
rest aren't attending.  That was from Dan Senie,
a frequent contributor.

Spencer ? from tektronix; speaking as a vendor, last
on the list, been coming since 2000, thanks for the work
done in the past.  Things at many different levels of
detail here; how many different reform proposals are
still waiting to come forward.

Dan golding replies that yes, it's not fully baked, it
still needs a bunch of work.  Spencer was worried this
might have gone through similar insanity like the IETF
at the time.

Mike Hughes, LINX.  They do good parties, by the way.
Agrees with Josh, largely.  The program committee member
list; note that the PC is from huge networks, generally.
they may have been small to start with, but they get
bought and grow, and the tendency is towards growth
into huge networks.  fresh blood is needed to get the
smaller folks flowing into the system.
Notes RIPE steering committee got together for a day
to talk face to face to hash out issues.  also notes
RIPE has several days of single track, then split into
separate tracks, to avoid having to repeat some
topics that need to go to whole group.
One of the key factors that got us here was the issue of
moderation; let's not necessarily turn this into a huge
reform effort that bogs down; we can look at addressing
some of the smaller points like the moderation issue
before the rest starts happening.

Chris Malayter from TDS again.  Bunch of forums starting
with other groups like equinix, terremark, S&D; what
about having forums happen in near conjunction, so
that people can get the best of both?  Nobody can afford
to go to all the separate forums, but if the meetings
could be grouped, it would essentially be like a start
at a multi-track system.
There's also MPLS-con that is held at the same time
as one of the NANOG meetings each year.  Rather than
competing against, perhaps they could partner together
or coordinate better?
Betty agrees that it would better to have good
cordination, but there are definite venue logistics
to be considered, to get enough bandwidth to support
NANOG.

Steve Feldman talks about hosting NANOG 2 years ago.
got dates, tried to find ballrooms of at least 10,000
sq ft to host it; one hotel sold space out from under
them at the last minute, had to start over with only
four months left.  deals sometimes fall apart, it's
a lot of hard work to try to schedule one.

Ren notes that RIPE uses amerstdam twice a year, and
then rotates one; that way, plumbing is re-used and
provides a constant that can be counted on for 
scheduling.

Chris Malayter says partners have been rebuffed from
hosting adjacent meetings with

Tony Kapella, was CW, now 
Testament to how open we can be, very very good.
The types of talks we've been attracting seem to
be self-selecting for similarities.  He notes that
other communities have activism internally that
becomes visible, and that doesn't seem to be happening
at NANOG.  Schools for example may not know that NANOG
exists, but that could talk about research efforts 
underway, or BCP's they implement.
Dan Golding asks aboug VOIP, Tony agrees it's relevant
but currently underrepresented.

Steve Gibbard, PCH, editor of reform proposal.  Want
to balance spencer's comment; balance minutae with
broad framework people could agree on; more effort
towards the broad view, not getting too tied up on
specific details at this stage.

Vijay Gill, AOL.  Easy to complain about lack of
presentations; more relevant, talk to people you know
who have something you'd like to hear to get talks
going.  Don't just leave it up to the PC.  You want
different talks, get off your ass and get your friends
to present.
Many relevant talks from the past isn't really relevant
anymore.  Much of our work is cookie cutter now; the
application layer is where more of the work is happening.
How is Skype affecting your network, for example

KC Claffy from CAIDA, been on PC for years.  Backs up
Vijay's comment to get your friends to bring interesting
talks.   Is it time to revisit the agenda part of the
system?  Can we really fix lack of operational content
by *adding* more tracks?
Look at research community; public archives of work out
there, SIGCOMM gets hundres of propsals, accepts very
few; we're actually not in bad shape for rejections
at all!  Just get a wider net cast for presentations!
KC claffy talked in 2003 about the top problems in
the internet; not one of the problems has gotten
better, have all gotten worse.  should she talk to
NANOG about these problems?  Is that a good topic
for her to talk about?  What is the best use of the
agenda time?

Dan responds, is price too low to get gripping content?
If it cost $50 more to get kick-ass content via incentives,
would we do it?

Mike Hughes notes that expense is shared equally; if
you start comping people based on travel, etc, you end
up with a two-tier system, which is bad.

RIPE has same challenge.  Difficult to get compelling
speakers.  People hold back due to concerns that their
work isn't up to par.  People should just go ahead an
contribute their work anyhow.

Steve Wilcox is back.  Other forums have similar issues,
similar people attend peering forums to talk about similar
issues, why not start finding areas of commonality and
combine to optimize time use?
Proposal isn't really about changing meeting, it's
about the overall structure.

Spencer from tektronix, VOIP was mentioned, dave meyers
did a talk on IMS, a VOIP technology on Tuesday morning.
The PC has been asking for VOIP presentations for the
past 4 NANOGs, and a PC member is giving this talk.
What can be done to bring things forward a bit?
What about poster sessions?  currently 3 or 4 'dot'
meetings, what about adding some more for VOIP BOF
meetings.

Miles? at mike, in past hasn't gotten impression that 
application layer wasn't really welcome at NANOG 
previously, it was network layer only.  are things
changing now?  What about video, audio, collaborative
environments, etc.  What about overstepping bounds by
expanding.

Rob Seastrom notes question and money often go hand in hand.
money isn't a free flow; many issues center around what does
it take to get more money flowing?  Hacks/kludges as well...
Also, perception of unapproachability previously about the
program committee.  This meeting is a huge step forward to
open up and start sharing information; too many assumptions
made otherwise.
Work towards a perception of approachability by the
program committee.

Joel Kraska from Cisco systems.  Two different sets of
people.  Philip Smith gives the same BGP talk every
NANOG, and there are always people watching; either
there's a bunch of newbies that are coming in, or
we are a bunch of slow learners.

Elise Gerich, Juniper networks.  Nobody other than Rob
has addressed the difference between the reform proposal
and the numbers Steve presented.  Have the numbers that
Betty and Steve presented answered some of the concerns
that the reform proposal wanted to fix?
Membership of program committee changes over time.
4 new members over the past year.  it's tough to recruit
new people, others have dropped off.   Perhaps that needs
to be shared.
Moderation of the list was another issue mentioned.
If you look at the overall number, 257 messages were
censored out of 10,000.  Is that just an overreaction?

Dan Golding responds; the list moderation issue is more
of concern to the community right now.  The manner in which
it has occurred is what started the ball rolling.
The PC isn't involved in the moderation decision; it
is even more of a black box.

Sylvie Pla? from Teleglobe; we are all operators here,
we all have operational issues.  Do we have clearance to
open the kimono from management?

David Barak, SBC, disagrees with having tracks.  he likes
being exposed to new things that he wouldn't have otherwise
seen; one of the few times tracks were tried, the more
interesting talks all happened at the same time.
Notes same clearance is needed to talk to a dozen people
as 10,000
Concludes by thanking everyone

Tony Hillard, broadwing--now that the proposal has been
put out, what are the next steps?

Closing remarks, from Betty.  Definitely next steps that
need to be taken.  Dialog we've had here tonight has been
outstanding, there are positive points in the reform
proposal.  Important comments online about current issues
facing the organizational structure of NANOG.  Critical
to take next evolutionary steps for change.
She will work with PC to outline next steps with Merit
to address some of the concerns raised tonight.  We're
in this together, let's avoid an us/them scenario; we
all want to see NANOG evolve to serve the community and
stay whole.  Want to be careful to not fragment the
community as we move towards the future.
Doesn't have all the answer, but will work with the PC
and the reform proposal authors to come up with a 
timeline of changes.
list moderation is a hot topic, there have been some
changes at merit on how the moderation is handled; you
have her committment to deal with those changes ASAP.
She again can't promise she has all the answers; the
dialog has been excellent, and should continue into
the future.

Martin Hannigan adjourns meeting at 2143 hours Pacific Time