Correct inclusion of rwhois info in WHOIS server output?

Wed Sep 7 23:27:57 UTC 2005

On Wed, 7 Sep 2005, Albert Meyer wrote:

> I've been talking to ARIN about the rwhois setup on our SWIPped blocks, and
> there appears to be a problem with the standard output from whois.arin.net.

Be carefull about using word "standard", there is no standard output for
whois plus both for ARIN and other RIRs output depends on the options used
when queries. In this case, I think you wanted to say that the problem you 
see is with their default output.

> The two rwhois clients I've tried are rwhois and jwhois.

jwhois is not an rwhois client. It does whois query on port 4321 but does
not actually interpret that data as real rwhois client would have.

> The rwhois client behavior is something like this:
> 1. Query whois.arin.net.
>  2a. If the response contains the name of an rwhois server, query that server 
> and
> return its output.
> 2b. If the response doesn't contain the name of an rwhois server, follow the
> links. Query every rwhois server you find and return all of the output.

I have not tried rwhois as a client for a while, but I believe you're not
describing it properly. Rwhois would do rwhois (port 4321) query on 
root.rwhois.net and would follow rwhois system referrals from there 
(which on the technical level is not the same as whois listed referral 
line). As of about year ago ARIN finally (after 4 or 5 years!) fixed 
those rwhois referrals (previously it would only work for very few blocks) 
so that rwhois server would follow to your own rwhois system.

I do want to note though that rwhois output at arin is not alwas insync
with their whois data (I think they generate it and once/day or less,
where as whois is updated twice/day) and does not list any contact
data - only organization name.

> The jwhois client behavior is something like this:
>
> 1. Query whois.arin.net.
> 2a. If the response contains the name of an rwhois server, query that server 
> and
> return its output.
> 2b. If the response doesn't contain the name of an rwhois server, return the 
> SWIP.

That seems reasonable for simple whois system which does not want to go
into all these complexities and rare cases of mixed SWIP and rwhois usage.

BTW - Completewhois behavior is to:
  1. Output swip data including every swip for multiple levels
  2. Look at if there is rwhois referral ReferralServer in the swip
     (AND also look at if rwhois server is listed by name in the
     comments and try to use that as well even if ReferralServer
     is not present) and if so do rwhois lookup (using rwhois
     library but without following referrals) to listed server.

> On blocks which are SWIPped to CoreNAP by an upstream provider, the response
> from whois.arin.net does not include an rwhois record. For example, if I 
> type:
>
> whois -h whois.arin.net 65.59.252.0

You might want to try whois -h whois.arin.net "+ 65.59.252.0"

> The whois server returns this:
>
> Level 3 Communications, Inc. LC-ORG-ARIN-BLK2 (NET-65-56-0-0-1)
>                                  65.56.0.0 - 65.59.255.255
> Core NAP, L.P. LVLT-CORENAP-NETBLOCK-03 (NET-65-59-252-0-1)
>                                  65.59.252.0 - 65.59.252.255
> VC Sterling, Inc. NET-65-59-252-0-1 (NET-65-59-252-0-2)
>                                  65.59.252.0 - 65.59.252.255
>
> Since there is no rwhois server listed here, rwhois clients don't necessarily
> manage to find the referral. rwhois apparently follows both links and returns
> results from every rwhois server it finds, but jwhois doesn't follow either
> link; it just returns the SWIP info. I believe that the correct response to 
> this query would be:

That is a problem with jwhois and not with default arin output which is
very simplistic and lists only list of swips without detail on each swip.
Jwhois should query the detail or use extended query ("+") to find all
the relevent information.

BTW, compare this to the output given by completewhois:
  whois -h whois.completewhois.com 65.59.252.1
(saved in http://www.completewhois.com/cgi-bin/whois.cgi?query=R33857865)

As you notice it tried to do query to both rwhois.level3.net and to
rwhois.corenap.com as well as displayed SWIP data. While admitedly
its too much information, that is how the system works - it provides
as much data as possible and lets user decide on what is relevent
to him/her.

But by automated means this really is a good example of how to confuse
the ip whois client, because you have 2 rwhois servers listed (for
top-most ip block and 2nd level deligation block) as well as SWIPs all
the way to the assignment to customer. If you want automated systems to 
use rwhois, then query to your own server would never have been made (the 
referral to follow would be rwhois.level3.net), actually let me correct 
myself - if rwhois.level3.net were to run rwhois server accessible to 
public, the  correct rwhois-only setup would be for L3 to enter rwhois 
referral to rwhois.corenap.com on their server (like ARIN does on 
root.rwhois.net). In this case client would find top-most rwhois server 
as rwhois.level3.net and follow from there and get to the lowest rwhois 
server available and give the results. But if you don't want automated 
systems to use rwhois data as primary, then the correct is to output the 
lowest swip level data in detail, i.e.
  whois -h whois.arin.net NET-65-59-252-0-1

> Level 3 Communications, Inc. LC-ORG-ARIN-BLK2 (NET-65-56-0-0-1)
>                                  65.56.0.0 - 65.59.255.255
> ReferralServer: rwhois://rwhois.level3.net:4321
> Core NAP, L.P. LVLT-CORENAP-NETBLOCK-03 (NET-65-59-252-0-1)
>                                  65.59.252.0 - 65.59.252.255
> ReferralServer: rwhois://rwhois.corenap.com:4321/
> VC Sterling, Inc. NET-65-59-252-0-1 (NET-65-59-252-0-2)
>                                  65.59.252.0 - 65.59.252.255

I'd be against any change like this because it will severely break a lot
of currently whois client software that know about this ARIN list data
and this would not be compatible with it.

> I've read through the apparently relevant RFCs (812, 954, 1714, 1834, 1835, 
> 1913, 1914, 2050, 2167, 3912) but did not find a clear specification of 
> correct WHOIS server output.

You're correct. Whois protocol has no standards for output format.
There were however several attempts to do that on top of whois with 
"whois+" and "rwhois" as well as RPSL.

> The people I talked to at ARIN say that the 
> configuration of whois.arin.net can be changed based on "significant 
> community consensus" but they suggested that the problem could be fixed by 
> rewriting the jwhois client (and any other client that doesn't follow links 
> to search for an rwhois server).

I agree with ARIN's assessment. But if I were doing coding for jwhois
I would not do it as you suggest and would insted write code that can
output the lowest level ARIN deligation (follow the lowest swip
in the list when its returned as above) and then look at that data and
see if that lowest swip has listed ReferralServer and follow that.

In your case it would not cause lookup to rwhois.corenap.com, but as
I said that is very very rare case that causes a lot of confusion for
pretty much every whois client except completewhois.

> I spent a fair amount of time looking 
> through the (apparently non-searchable) mailing list archive at 
> http://lists.arin.net/pipermail/dbwg/ and saw some discussion of rwhois 
> issues but I didn't manage to find information showing how the previous 
> change was initiated.

ARIN formed subgroup within DBWG to discuss rwhois issues about 3 years ago.

> Questions:
> 1. Does anyone agree that the present lack of rwhois server information in 
> the initial WHOIS response for SWIPped blocks is a problem?

My general feeling is that ARIN is wrong in providing default list response
rather then detailed data. But as far as rwhois not being listed in default
list response, I do not believe it to be appropriate either.

> 2. Can anyone think of a compelling reason why rwhois server information 
> should not be included in the initial response to a standard whois query for 
> all IP blocks, including SWIPped blocks, besides the fact that it is not 
> included now?

Yes, I can - see above.

> 3. Would this change (adding rwhois server information to the initial 
> response to a standard whois query for SWIPped blocks) break your scripts 
> that parse WHOIS output?

Yes, it would.

> 4. How disruptive was the change when rwhois server information was 
> initially added to WHOIS output?

Not disruptive because it was done to detailed data output and everyone
expected data there to be multiple "Field: data" lines and that new fields 
could be added in the future.

> 5. Was the issue fully thought through at that time, and the rwhois server 
> information intentionally left out of the initial response for SWIPped 
> blocks, or did this happen by accident?

I can't speak for ARIN, but I think they fully realized that they are
adding ReferralServer line only for detailed output and not list/summary.

> 6. Does anyone know where that change process was documented?

Ask ARIN, I don't think it is documented though.

-- 
William Leibzon
Elan Networks
william at elan.net