Starlink terminal data acquisition for network engineers

Sun Feb 7 22:45:22 UTC 2021

Thanks Eric, this is great information. I just installed my Starlink
"dishymcdishface" this weekend and it's humming along nicely, even in
the -25F (-51F with windchill) temps here in northern MN! I've got a
bunch of general monitoring of the link using a few Docker + grafana
widgets [1], but hadn't looked into direct metrics gathering from the
terminal yet. This will surely get me on that path :)

--Tim

[1] https://hub.docker.com/r/qlustor/speedtest_ookla-to-influxdb

Tim Nelson
Network Engineer / Cloud Services
Phone: (218)727-4332 x4501 / Fax: (866)716-0229
5019 Airport Road, Hermantown, MN 55811
Check us out at www.sangoma.com

On Sat, Feb 6, 2021 at 4:06 AM Eric Kuhnke <eric.kuhnke at gmail.com> wrote:
>
> I thought about posting this to only NANOG, but since a great concentration of beta testers of a technical/network engineering inclination are located in the Pacific NW, decided to also include the SIX chat list.
>
> You may have seen the Starlink android or ios consumer-friendly app, which displays network traffic, uptime/downtime, and other link stats. I believe this to be polled directly from the antenna unit itself over grpc.
>
> The beta antennas are always 192.168.100.1. If you are using your own router with the starlink beta system, in addition to its WAN interface being an ordinary DHCP client in cgnat IP space, you'll need to manually give it an address in that /24 and set up routing to reach the .1 IP as needed.
>
> reference: https://github.com/sparky8512/starlink-grpc-tools
>
> reference: https://github.com/fullstorydev/grpcui
>
> you'll need a fairly normal Linux or BSD box with:
>
> git
> go
> python3
> pip
>
> use pip to install grpcio and grpcio-tools
>
> install grpcurl: https://github.com/fullstorydev/grpcurl
>
> do a git clone of the starlink-grpc-tools url above, also take a look at its readme info
>
> get the dish's protoset file and write it to new file dish.protoset , this is an index of all data that can be polled
> cd /home/eric/starlink-grpc-tools
> /home/eric/go/bin/grpcurl -plaintext \
> -protoset-out dish.protoset \
> 192.168.100.1:9200 \
> describe SpaceX.API.Device.Device
>
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"get_history\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle | python parseJsonHistory.py
>
> output of the above looks like this:
> 2021-02-06T08:15:56,3600,19.92034332,14,2,0.3125,0,0,0.0,0
>
> full CSV header for the above:
> datetimestamp_utc,samples,total_ping_drop,count_full_ping_drop,count_obstructed,total_obstructed_ping_drop,count_full_obstructed_ping_drop,count_unscheduled,total_unscheduled_ping_drop,count_full_unscheduled_ping_drop
>
> since we are able to acquire the above in a comma-delimited csv format, it's fairly easy to write a script storing the integers from any one of those particular columns into a mariadb db, sqlite, influxdb, or whatever.
>
> the following will output about 3.8MB of text for the full history (I believe this to be the full copy of the ring buffer stored in RAM for the terminal's statistics) , pipe it into a text file if you want to manually look at it.
>
> /home/eric/go/bin/grpcurl -plaintext -d {\"get_history\":{}} 192.168.100.1:9200 SpaceX.API.Device.Device/Handle
>
>
> same as the above but human readable output
>
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"get_history\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle | python parseJsonHistory.py -v
>
> current counter:       299673
> All samples:           43200
> Valid samples:         43200
> Parsed samples:        3600
> Total ping drop:       20.03700998
> Count of drop == 1:    14
> Obstructed:            2
> Obstructed ping drop:  0.3125
> Obstructed drop == 1:  0
> Unscheduled:           0
> Unscheduled ping drop: 0.0
> Unscheduled drop == 1: 0
>
>
> see the get_history_notes.txt file for more info
>
>
> SOME EXAMPLE QUERIES
> these should match with what the json query is in the grpc GUI
> # get status
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"getStatus\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle
>
> # get device info
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"getDeviceInfo\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle
>
> # get history, this outputs a huge amount of data
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"getHistory\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle
>
>
> The following is copied/pasted from my notes file on things we can acquire, and then use a tiny shell script with awk, sed, regex, whatever as needed to trim out just the numbers, and put them somewhere else for polling by snmp extend
>
> /home/eric/go/bin/grpcurl \
> -plaintext \
> -d {\"getStatus\":{}} \
> 192.168.100.1:9200 \
> SpaceX.API.Device.Device/Handle
>
> notes on what's what:
> figures we care about to parse out and turn into just the integers
> snr can never be higher than 9
> fractionobstructed appears to be the percentage of the time that the view is obstructed, as long as the view
> remains unobstructed, this number appears to slowly decrement over time
> validS is valid seconds? i think
> the S is almost likely always Seconds of time
>
> "uptimeS": "304439"
> "snr": 9,
> "fractionObstructed": 0.0013524292,
> "validS": 61815.74,
> "last24hObstructedS": 53
> "downlinkThroughputBps": 47915.73,
> "uplinkThroughputBps": 34980.496,
> "popPingLatencyMs": 29.266666
>
>
> example of running the command above twice, the second time a few minutes after the first,
> to see that fractionObstructed does decrement itself over time
> first run: 0.0013524292
> second run: 0.0013467998
>
>
>