Unofficial FAQ for the /9/grid project

Q: What is the /9/grid?

A: A project begun by users of 4chan's technolo/g/y board to setup a free, decentralized grid computing network, available to users of any OS. By integrating Plan 9 based networking technologies (via Plan 9, Inferno, 9vx, or others) with our usual OSes, we can create transparent networks of whatever content and services we want.

Q: What is Plan 9? What is Inferno? What is 9vx? What is Plan 9 from User Space?

A: Plan 9 is the OS written by many of the original Unix authors, sponsored by Bell Labs, to create an OS that would allow for better networking, more user control, and the ability to share resources transparently between all computers connected to the grid. Despite the strength of the concepts, the corporate supervisors at AT&T/Bell labs didn't make much of an attempt to push it out to market, and so the technology branched out - most notably into Inferno, a new implementation of Plan 9 tech that can be run as a lightweight application inside any OS. Other projects which integrate Plan 9 tools and tech are 9vx, a version of plan9 that runs as a userspace application, and Plan 9 from User Space (plan9port). Inferno-os is the current repository of Inferno source code.

Q: What's the point? I can share files with bittorrent, and do remote access with SSH or putty.

A: Of course -- and you can get your news from a newspaper, you don't need to read websites and blogs! The networking and design of Inferno/Plan9 make it easy to do things that are hard to do with other protocols. The 9p/styx protocol can handle dynamic data streams just as easily as static files, for instance. Another reason for people to use Inferno/Plan 9 is that it puts complete control in your hands, you have a lot more power to configure stuff exactly how you want than with most 3rd party apps you could use for the same purposes. Also, there's stuff that really is pretty unique, like the ability to run CPU servers in plan 9 and do true distributed computing in Inferno. The most important reason, though, is that its fun - this is just cool software.

Q: How do I get started?

A: The easiest way is to download the Inferno install CD from Vita Nuova, see http://www.vitanuova.com/inferno/net_download4T.html. (You can also download full Plan 9 from http://plan9.bell-labs.com/plan9/download.html and install it in a virtual machine - Qemu works, and several others do also, although some require some tweaking. VMware requires disabling the cd-rom drive from the virtual machine's config file after install but before boot, for instance.) If you want to run a grid node, we recommend making a new user for inferno, installing Inferno as that user, then removing that user's admin privileges. This lets Inferno run 'sandboxed'. If you just want to see what inferno is and check some stuff out, this isn't necessary. Also, you DO have your data backed up and confidential personal data encrypted, right? That's basic computer use, regardless of whether you want to be a node. We believe its perfectly safe to operate a node on your everyday desktop, provided Common Sense is installed and running correctly.

Q: If I want to join the grid, do I have to keep my computer on all the time?

A: Absolutely not! The whole design of the grid is based around updating the list of connected nodes and services in real time, so as nodes come online and go offline, their services are automatically added and removed from the registries. The goal of the grid is to add on new network functions and services to your existing computer setup, with no loss of any existing functionality. Of course, you may find that you want to keep your node up most of the time, which is welcome and appreciated, especially if you are willing to help by providing a service registry for others to use.

Q: What's the most important thing to understand about Plan 9 and its relatives before I get started?

A: NAMESPACES AND NAMESPACE INHERITANCE. This is one of the primary ways these operating systems are different. Instead of a static view of the filesystem and network, you have a dynamic view, embodied in your current namespace. This is a huge topic and one you just HAVE to learn a bit about to use these systems. The easiest, simplest way to put it to a beginner is: what you do inside one program/window wont necessarily carry over to your other programs and windows. Anything that modifies your namespace - which is a lot of the most important stuff, such as mount and bind commands - applies only to the current process (such as a given shell window) and processes it launches. This may seem strange, and it will definitely cause you some confusion as you learn, but it is actually very cool and very powerful. As a simple example, you can open a shell in your current display, and issue a series of mount commands to setup a particular directory structure. Then, you do rio (plan 9) or wm/wm (inferno) to start a sub window manager session INSIDE your main session. Inside this sub session, all the mount commands you issued affect everything you do. Programs you start outside of the subsession wont see any of the mounts created by the shell thats 'next to them'.

Q: Okay, I just installed Inferno. What now?

A: When you start inferno, you get a semicolon prompt. Type ndb/cs to start the network connections server. Then type wm/wm to start the graphical interface. First off, spend a little time poking around. If you haven't used a text-based shell much, you might want to check out a guide to 'basics of the unix shell' or something -- even though Plan 9/Inferno is NOT unix/linux, a lot of the basic shell commands and grammar work similarly, so things like listing and moving around directories and copying files work about the same as in bash. (lc to list in columns, by the way. You're welcome.) If you don't want to bother with any of that, feel free to continue to the next question.

Q: How can I find out what's on the grid? How do I access grid resources?

A: The best way is to connect to a Grid Services Registry and see what's listed. At the moment of this writing, you can do this with (Inferno shell code, if it works you just get your prompt back in a few seconds:)

mount -A tcp!69.129.202.66!7888 /mnt/registry

which will attach the current registry to your filesystem at /mnt/registry (inside the inferno root directory). To list registered nodes and services, do

cat /mnt/registry/index

and you should see a list of available services. The information in that list provides instructions for how to connect to them. For instance, a line like:

tcp!fred-node!5055 myWallpapers9p at192.168.1.255

Would tell you that there was a node at a certain IP address serving a folder of wallpapers via standard (9p) protocol. To access the wallpapers, you'd do something like:

mkdir /tmp/gridwalls
mount -A tcp!192.168.1.255!5055 /tmp/gridwalls


And you would have connected to that node and attached its wallpapers folder to your filesystem at /tmp/wallpapers. Do:

lc /tmp/gridwalls

To see a list of stuff in the folder. To copy one of them to your local /mywallpapers folder (always INSIDE your inferno root) you would do:

fcp /tmp/gridwalls/NAMEOFPAPER.jpg /mywallpapers

Q: How do I add services to the grid? I wrote a great novel, how do I let people on the grid have access?

A: So, you've got a folder with your novel in it. Make a copy of that folder and put it inside your inferno root folder, call it /mybook or whatever. Make sure the permissions on it dont let the whole world write to it, only the owner (you) should be able to make writes, but anyone should be allowed to read it, if you want the grid to have access. First off, its assumed that you are connected to a services registry, mounted at /mnt/registry. Then you can use a single command to 'publish' your book to the grid:

grid/reglisten -A -r MyNovel9p at192.168.255.255 tcp!*!7676 {export /mybook &}

That command announces a service to the grid at a particular IP address and port (7676 in the example) and starts up a listener providing an export of the /mybook folder. Now anyone connected to the services registry you announced too can see that you are offering a service called MyNovel9p and know that they can connect to your IP address and download it. When you turn off the node, your service entry disappears.

Q: I want to play around with stuff, but I don't want to link up to the grid yet. How do I start a service to test without connecting it to the grid?

A: Here's the general way of starting a service in Inferno:

listen -A tcp!*!port {command args}

so for instance, suppose you wanted to start a listener that said hello to anyone who contacted the port 5511 with telnet or netcat:

listen -A tcp!*!5511 {echo HELLO}

the command to share a filesystem is export, and in Inferno/Plan 9 filesystems can do almost anything. So:

listen -A tcp!*!5995 {export /path/to/service &}

is the standard way of doing filesharing and other Cool Stuff.

Q: How do I start a registry of my own? How do people add services to it?

A: Glad you asked! Having lots of registry providers is the key to a failsafe, distributed system! Here's how you startup a registry service:

mkdir /mnt/registry2
mount -A -c {ndb/registry} /mnt/registry2
listen -A tcp!*!9897 {export /mnt/registry2}


Anyone can then mount your registry by doing:

mkdir /mnt/registry2
mount -A tcp!ip.of.registry.host!9897 /mnt/registry2


And they have a dynamic copy of the registry at /mnt/registry2 in their filesystem. To announce an already-running service to a registry, this is the command:

>/mnt/registry2/new {echo tcp!10.0.0.2!9001 ServiceName at10.0.0.2; while {sleep 3600} {sleep 3600};}&

Admittedly, thats a pretty crazy looking command. What's going on here? First off, the command is being directed at the 'new' file within the given registry, which is how new services get registered. What happens is that you write a text string to that file, and then keep the file open without ever closing the write, thats what the while sleep part does. That's how the registry removes dead services automatically -- when a node goes down, that node will no longer be keeping their write to the 'new' file open, and their entry is purged. Remember, that command announces an already running-service. If you don't have the service already running, start it with

listen -A tcp!*!port {command args}

(see above for a more specific example for sharing files)

Q: Why is all this code Inferno code? How do I do all this with Plan 9?

A: Right now Inferno is the most popular grid client/server, but Plan 9 can also access the grid registry and run and register services. Here's some Plan 9 code to mount a registry and start a service exporting the /pub directory and announce to the registry:

srv tcp!69.129.202.66!7888 gridregistry
mount /srv/gridregistry /n/gridregistry
aux/listen1 tcp!*!9001 /bin/exportfs -r /pub &
>/n/gridregistry {echo tcp!192.168.255.255!9001 MyPublicFiles at129.168.255.255; while (sleep 3600);}&

Q: Where can I learn more?

A: Try #plan9chan on irc.freenode.net for general help and discussion. http://www.kenya.servequake.com:8080/ has helpful tips. Also look for semi-daily threads on /g/, check the http://www.anonib.com/plan9/ board, browse the thread archives here, and of course check out the official plan 9 and Inferno websites and wikis.

Q: Are there any notable glitches/bugs to watch out for?

A: Sure, plenty. For some linux users, the Vita Nuova release of Inferno is somewhat unstable due to X errors. (Xorg and Inferno can fight for the blame.) You can expand 9vx to a full install by putting the binary on top of the full source tree, but not everything seems to work perfectly, notably contrib/install and how 9vx creates folders in the host os seem to result in folders with null permissions, which stops things from installing correctly. Plan 9 inside a VM (faq author uses qemu) seems to be one of the most stable setups, but use of a VM can complicate networking.

NOTE: For simplicity and convenience examples were given for making non-authenticated connections and providing non-authenticated services. This ought to be perfectly safe assuming sane file permissions and an updated Common Sense installed in your cranium. The grid as it grows will also make use of the authentication and encryption services built into Inferno and Plan 9. Feel free to setup your own auth server and provide authenticated services!

NOTE: The grid is based on principles of redundancy and no central point of failure. IT IS THE RESPONSIBILITY OF EACH NODE OPERATOR TO SET THEIR OWN POLICIES. There is no central authority or control. There are no laws or rules for you follow other than the laws of your country and your conscience. A cooperative and collaborative attitude is obviously helpful. There are no contracts to sign and no fees to pay. Please contribute knowledge, ideas, and useful scripts (especially useful scripts!) to the community. When we say 'no rules', that doesn't mean disruptive use of grid resources will be tolerated by node operators, and you have the right to set whatever rules you want FOR YOUR OWN NODE. Furthermore, following a set of common practices is incredibly useful: even if there isn't a rule 'forcing' you to do things according to Grid Policy, you will find that everything works better when people agree on common implementations.

NOTE: Sorry for any typos, inaccuracies, or Total Bullshit that has crept into this document. All errors are the responsibility of the author, but the consequences of anything you type into your own shell are on you, the person controlling your fingers. No guarantees, but your wasted time will be refunded by the universe, in some fashion or another.


special thanks to sqweek for helping us learn a few of the basics. anything that works right on this page is based on his examples and the inferno wiki tutorials such as: http://canto.hopto.org/wiki/A_Simple_Grid/index.html