The Making of a Router

Fri Dec 27 10:14:39 UTC 2013

In talking about RAMBOOT I also realized the instructions are out of date
on the website.

The "ramboot" boot target script was updated since I created the initial
page to generate the correct fstab, and enable the root account, set a
hostname, etc. so you can actually use the OS until you create a new image.

I extracted the script from the initird to make it easier to grab:

http://ramboot.org/download/RAMBOOT/RAMBOOT-pre0.2/SYSLINUX/initrd/scripts/ramboot

Essentially, by adding a new "ramboot" script to
"/usr/share/initramfs-tools/scripts" along side "nfs" and "local" it
creates a new "boot=" target (since the init script looks for
"scripts/${BOOT}".

As mentioned on the website, the ramboot process needs a more complete
version of busybox (for tar archive support) and the mke2fs tool added to
"/usr/lib/initramfs-tools/bin/" so they will be available to the initrd.

Once you configure networking (see the "INSTALL/setup_network" script) you
can do an apt-get update and apt-get the packages you need from Ubuntu
12.04 LTS.

Example starting point:

apt-get install sudo
apt-get install nano
apt-get install ssh
apt-get install vlan
apt-get install bridge-utils

On Thu, Dec 26, 2013 at 8:27 PM, Ray Soucy <rps at maine.edu> wrote:

> The basic idea of RAMBOOT is typical in Embedded Linux development.
>
> Linux makes use of multi-stage boot process.  One of the stages involves
> using an initial ramdisk (initrd) to provide a base root filesystem which
> can be used to locate and mount the system root, then continue the boot
> process from there.
>
> For an in-memory OS, instead of the boot process mounting a pre-loaded
> storage device with the expected root filesystem (e.g. your installed HDD),
> you modify it to:
>
> 1) Create and format a ramdisk.
> 2) Create the expected directory structure and system files for the root
> filesystem on that ramdisk.
>
> The root filesystem includes the /dev directory and appropriate device
> nodes, the basic Linux filesystem and your init program.
>
> The easy way to do that is just to have a TAR archive that you extract to
> the ramdisk on boot, better yet use compression on it (e.g. tar.gz) so that
> the archive can be read from storage (e.g. USB flash) more quickly.
>
> Today, the initramfs in Linux handles a lot more than simply mounting the
> storage device.  It performs hardware discovery and loads appropriate
> modules.  As such the Debian project has a dynamic build system for
> initramfs that is run to build the initrd when a new kernel package is
> installed, it's called "initramfs-tools".
>
> You can manually build your own initramfs using the examples on the
> RAMBOOT website, but the point of RAMBOOT is to make building an in-memory
> OS quick and simple.
>
> RAMBOOT instead adds configuration to initramfs-tools so that each time a
> new initrd is generated, it includes the code needed for RAMBOOT.
>
> The RAMBOOT setup adds handling of a new boot target called "ramboot" to
> the kernel arguments.  This allows the same kernel to be used for a normal
> installation and remain unaffected, but when you add the argument
> "boot=ramboot" as a kernel option to the bootloader, it triggers the
> RAMBOOT process described above.
>
> Having a common kernel between your development environment and embedded
> environment makes it much easier to test and verify functionality.
>
> The other part of RAMBOOT is that it makes use of "Ubuntu Core".  Ubuntu
> Core is a stripped down minimal (and they really do mean minimal) root
> filesystem for Embedded Linux development.  It includes apt-get, though, so
> you can install all the packages you need from Ubuntu on the running system.
>
> RAMBOOT then has a development script to make a new root filesystem
> archive with the packages you've installed as a baseline.  This allows for
> you to boot a RAMBOOT system, install your desired packages and change
> system configuration files as desired, then build a persistent image of
> that install that will be used for future boots.
>
> I also have the start of a script to remove unused kernel modules, and
> other files (internationalization for example) which add to the OS
> footprint.
>
> You could build the root filesystem on your own (and compile all the
> necessary packages) but using Ubuntu Core provides a solid base and allows
> for the rapid addition of packages from the giant Ubuntu repository.
>
> Lastly, I make use of SYSLINUX as a bootloader because my goal was to use
> a USB stick as the bootflash on an Atom box.  Unfortunately, the Atom BIOS
> will only boot a USB device if it has a DOS boot partition, so GRUB was a
> no-go.  The upside is that since the USB uses SYSLINUX and is DOS
> formatted, it's easily mounted in Windows or Mac OS X, allowing you to copy
> new images or configuration to it easily.
>
> For the boot device I make use of the on-board vertical USB socket on the
> system board (typical for most system boards these days) and a low-profile
> USB stick.  I find the Verbatim "Store 'n' Go" 8GB USB stick ideally suited
> for this as it's less than a quarter-inch high after the USB adapter.
>
> RAMBOOT as a project is in the very early stages, so you should be
> comfortable with Linux before you build a system on it.  And I really feel
> it's more of an example than anything at this point.
>
> There are several advantages though:
>
> The most common point of failure on a Linux system is the storage device
> (either HDD or SSD).
> The biggest bottleneck in system performance is storage IO.
> Using a ramdisk eliminates both these concerns (in fact, even an Atom
> system has surprisingly great performance when run using a ramdisk).
>
> The result is that you get a very reliable, high-performance system.
>
> The other benefit to RAMBOOT is that the root filesystem is NOT
> persistent.  This means that like a Cisco device, every boot of the system
> brings you to a known working state OS-wise.  There are hundreds of system
> files in a Linux system; any one of which being modified could cause
> problems.  For both security and availability concerns a lot of effort is
> invested in detecting changes to system files and avoiding them.  With
> RAMBOOT the problem is easily avoided.
>
> A minimal system can fit within a 512MB ramdisk.  But with RAM being so
> cheap these days, I think even reserving up to 2GB of RAM for a ramdisk
> would be fine (e.g. for a 4-8GB system).
>
> Here is the hardware configuration I originally started RAMBOOT for in
> 2011 (wanted to avoid the cost of an HDD):
>
> $326.36 (shipping included):
> 1U rack-mount case Supermicro CSE-502L-200B
> Intel Atom D510 system board with dual Gigabit (Intel 82574L) Supermicro
> MBD-X7SPA-H-O
> 2GB RAM
> 8GB low-profile USB flash drive (which will connect to the internal USB
> port and be low enough to fit in the case) Verbatim "Store n Stay".
>
> No HD; the system will boot off the 8GB flash into RAM and run the OS on a
> ramdisk.
>
> Using the RAMBOOT release that's currently up, I can build a custom Linux
> in-memory OS in half a day.  I can easily update packages for security
> updates from the Ubuntu project and re-generate a new, updated, image in
> less time than that.  So the initial goal of being able to build something
> useful quickly was satisfied, at least.  My attention has now moved on to
> building a configuration management system, similar to Vyatta or VyOS and
> building a real distribution.  I was going to call it "Carrier-grade Linux"
> (cglinux.org), but given the momentum VyOS has I might try to help the
> VyOS community instead of doing something new on my own.
>
> For what it's worth, I'm actually working with the VyOS project to try and
> incorporate some of the RAMBOOT ideas into VyOS as an install option for
> in-memory only.
>
> If you make use of RAMBOOT I would love to hear about it. :-)
>
>
>
>
>
>
> On Thu, Dec 26, 2013 at 4:22 PM, Nick Cameo <symack at gmail.com> wrote:
>
>> Inline response exist,
>>
>> On 12/26/13, Ray Soucy <rps at maine.edu> wrote:
>> > You can build using commodity hardware and get pretty good results.
>> >
>> > I've had really good luck with Supermicro whitebox hardware, and
>> > Intel-based network cards.  The "Hot Lava Systems" cards have a nice
>> > selection for a decent price if you're looking for SFP and SFP+ cards
>> that
>> > use Intel chipsets.
>>
>> I like the supermicro as well however we have a couple of IBM x3250
>> with 2 pcie v3
>> x8 that are begging for a intel network card.
>>
>> > There might be some benefits in going with something like FreeBSD, but I
>> > find that Linux has a lot more eyeballs on it making it much easier to
>> > develop for, troubleshoot, and support.  There are a few options if you
>> > want to go the Linux route.
>>
>> This is very important to consider. I would be speculating, or even
>> worse, expecting
>> the same type of community support from the BSD verse that I have been
>> getting from the linux community.
>>
>> >
>> > Option 1: Roll your own OS.  This takes quite a bit of effort, but if
>> you
>> > have the tallant to do it you can generally get exactly what you want.
>>
>> If Free/OpenBSD is ruled out, I could crack open the LFS project. You only
>> have to do it once right? Or maybe just reach out to the gentoo community
>> for a stripped version, and build outwards.
>>
>> > The biggest point of failure I've experienced with Linux-based routers
>> on
>> > whitebox hardware has been HDD failure.  Other than that, the 100+ units
>> > I've had deployed over the past 3+ years have been pretty much flawless.
>> >
>>
>> SSD
>>
>> > Thankfully, they currently run an in-memory OS, so a disk failure only
>> > affects logging.
>> > If you want to build your own OS, I'll shamelessly plug a side project
>> of
>> > mine: RAMBOOT
>> >
>> > http://ramboot.org/
>> >
>> > RAMBOOT makes use of the Ubuntu Core rootfs, and a modified boot process
>> > (added into initramfs tools, so kernel updates generate the right kernel
>> > automatically).  Essentially, I use a kernel ramdisk instead of an HDD
>> for
>> > the root filesystem and "/" is mounted on "/dev/ram1".
>> >
>> > The bootflash can be removed while the system is running as it's only
>> > mounted to save system configuration or update the OS.
>> >
>> > I haven't polished it up much, but there is enough there to get going
>> > pretty quickly.
>>
>> Ummm, if it's ok with the community, can you kindly elaborate :). I am
>> not too fond of Debian since my horrible experience with Squeeze Desktop.
>> I would maybe like to try this using the combination of SSD, in memory,
>> and
>> Gentoo?
>>
>> >
>> > You'll also want to pay attention to the settings you use for the
>> kernel.
>> >  Linux is tuned as a desktop or server, not a router, so there are some
>> > basics you should take care of (like disabling ICMP redirects,
>> increasing
>> > the ARP table size, etc).
>>
>> Totally strip it as much as possible. If anyone has a Gentoo stripped
>> kernel
>> config that they would like to share, please do :).
>>
>> >
>> > I have some examples in: http://soucy.org/xorp/xorp-1.7-pre/TUNING
>> > or http://soucy.org/tmp/netfilter.txt (more recent, but includes
>> firewall
>> > examples).
>>
>> Will definitely look into all your sites.
>>
>> >
>> > Also a note of caution.  I would stick with a longterm release of Linux.
>> >  I've had good experience with 2.6.32, and 3.10.  I'm eager to use some
>> of
>> > the post-3.10 features, though, so I'm anxious for the next longterm
>> branch
>> > to be locked in.
>> >
>>
>> We are comfy with 3.4 right now...
>>
>>
>> > One of the biggest advantages is the low cost of hardware allows you to
>> > maintain spare systems, reducing the time to service restoration in the
>> > event of failure.  Dependability-wise, I feel that whitebox Linux
>> systems
>> > are pretty much at Cisco levels these days, especially if running
>> > in-memory.
>>
>> Really interested with the "in-memory", however, I would love to implement
>> it using gentoo as mentioned above.
>>
>>
>> Kind Regards,
>>
>> N.
>>
>
>
>
> --
> Ray Patrick Soucy
> Network Engineer
> University of Maine System
>
> T: 207-561-3526
> F: 207-561-3531
>
> MaineREN, Maine's Research and Education Network
> www.maineren.net
>

-- 
Ray Patrick Soucy
Network Engineer
University of Maine System

T: 207-561-3526
F: 207-561-3531

MaineREN, Maine's Research and Education Network
www.maineren.net