An Easy way to build a server cluster without top of rack switches (MEMO)

NAOTO MATSUMOTO naoto.mm at gmail.com
Fri Feb 20 14:50:52 UTC 2015


BTW: This scenario's combination has another portion for us like as below.

High Availability Server Clustering without ILB(Internal Load Balancer)
(MEMO)
http://slidesha.re/1vld6uB

--
Naoto MATSUMOTO

On Mon, Feb 16, 2015 at 12:23 PM, NAOTO MATSUMOTO <naoto.mm at gmail.com>
wrote:

> Hi Dan and ken.
>
> I respect your great works.
>
> Certainly, our scenario was network classics and it just does not "one
> size fits all" network architecture.
> Many people tried to built centralized and decentralized networks many
> years ago, some guys output
> implementation like this.
>
>
> Interconnect of K computer (torus fusion)
>
> https://www.fujitsu.com/global/Images/fujitsu-hpc-roadmap-beyond-petascale-computing.pdf
>
>
> I agree with you point. Our approach have to do more simple way on
> physical and logical
> network engineering, and change the mindset, I think.
> (e.g. network cabling procedure and troubleshooting and handling)
>
> But, some guys need more cost effective server cluster environment and
> they does't care
> network latency like Low-End Web Hosting.
>
>
> e.g. Intel Diversity of Server workloads http://bit.ly/1BgFH65 [JPG].
>
>
> Now, Many people do not use Dijkstra and automaton theory on the server
> side,
> but it is great mechanism for network durability if they controlled.
>
> The Ethernet NIC's bandwidth is increasing day by day, the cost is
>  decreasing too.
>
> I say again, our scenario is not "one size fits all" network architecture,
> but we believe that something will happen for some guys works. ;-)
>
> best regards,
>
> --
> Naoto MATSUMOTO
>
>
>
> On Sat, Feb 14, 2015 at 7:08 AM, Dan Eckert <daniel.eckert at microsoft.com>
> wrote:
>
>> I'm having a hard time seeing how this reduces cable costs or increases
>> network durability.  Each individual server is well connected to 3-4 other
>> servers in the rack, but the rack still only has two uplinks.  For many
>> servers in the rack you're adding 3-4 routing hops between an end node and
>> the rack uplink.
>>
>> Additionally, with only 2 external links tied to 2 specific nodes, you
>> introduce more risks.  If one of the uplink nodes fails, you've got to
>> re-route all of the nodes that were using it as the shortest path to now
>> exit through the other uplink node -- the worst case in the example then
>> increases from the original 4-hops-to-exit to now 7-hops-to-exit.
>>
>> As far as cable costs go, you might have slightly shorter cables, but far
>> more complex wiring pattern -- so in essence you're trading off a small
>> amount of cable cost for a higher amount of installation and
>> troubleshooting cost.
>>
>> Also, using this layout, you dramatically reduce the effective bandwidth
>> available between devices, since per-device links now have to be used for
>> backhaul/transport in addition to device-specific traffic.
>>
>> Finally, you have to manage per-server routing service configurations to
>> make this work -- more points of failure and increased
>> setup/troubleshooting cost.  In a ToR switch scenario, you do one config on
>> one switch, plug in the cables, and you're done -- problems happen, you go
>> to the one switch, not chasing a needle through a haystack of
>> interconnected servers.
>>
>> If your RU count is worth more than the combination of increased
>> installation, server configuration, troubleshooting, latency, and capacity
>> costs, then this is a good solution.  Either way, it's a neat idea and a
>> fun thought experiment to work through.
>>
>> Thanks!
>> Dan
>>
>>
>> -----Original Message-----
>> From: NANOG [mailto:nanog-bounces at nanog.org] On Behalf Of NAOTO MATSUMOTO
>> Sent: Wednesday, February 11, 2015 11:32 PM
>> To: nanog at nanog.org
>> Subject: FYI: An Easy way to build a server cluster without top of rack
>> switches (MEMO)
>>
>> Hi all!
>>
>> We wrote up TIPS memo "an easy way to build a server cluster without top
>> of rack switches" concept.
>>
>> This model have a reduce switches and cables costs and high network
>> durability by lightweight and simple configuration.
>>
>> if you interest in, please try to do yourself this concept  ;-)
>>
>>
>> An Easy way to build a server cluster without top of rack switches (MEMO)
>> http://slidesha.re/1EduYXM
>>
>>
>> Best regards,
>> --
>> Naoto MATSUMOTO
>>
>
>



More information about the NANOG mailing list