This conversion has been an interesting experience: I have learned a lot of details about web servers and Linux that I only knew in abstract; I knew that there was a lot of good opensource software out there, but this project has really brought home how much is out there and how good some of it is; I have been reminded how desktop applications can be so much better than web applications, and of how polished and easy to use modern web applications are.

I love having a local website that is reachable from the internet. My DSL upload rate is pretty slow, so the difference between dropping some files on a local network drive and uploading them is huge. I am looking forward to moving my main website to my local web server. (We will have to improve our upload speeds to support chadweisshar.com and wsims.com)

I am enjoying having ownership of all of my data, but I am also feeling the burden of being responsible for keeping the data safe and the server running.

In the rest of this post, I will describe how I am using the web server and OwnCloud installation to replace all my remaining cloud services.

RSS Reader

Feedly, my cloud RSS reader, allowed me to export all of my subscriptions, but it didn’t allow me to export my tagged items. OwnCloud has a built in “news reader” app which works a lot like Feedly. Importing my feeds was easy. The tagged items were a bigger hassle. I went though them one by one and converted each item into a bookmark in the OwnCloud bookmark app. Tagged items in an RSS reader are really just bookmarks to the original URL and as I went though my old tagged items, I found several that were dead or degraded links. The internet is an ephemeral place and a bookmark is no substitute for a copy.

I also used the Feedly Android app. There is an Android app that links to the OwnCloud news reader, but I haven’t tried it yet.

Dropbox

OwnCloud has an application that works just like Dropbox. You install the OwnCloud client on your Windows/Mac/Linux machine and connect it to the OwnCloud server. The client program keeps a directory on your local machine synchronized with the server. You can install the client on as many machines as you want, so you effectively have a directory that is synchronized between all your machines. You can also access all these files from the OwnCloud webpage, so you can get them anywhere you have an internet connection.

OwnCloud is a bit better than Dropbox because it allows you to share a set of files with specific people and to provide read-only access to files.

Contacts

I had a bunch of contacts at Google because it synchronizes with my Android phone. Google let me export all my contact information, and OwnCloud imported it into its contact app.

Getting it hooked back up to my phone was a bit harder. There is an interface called CardDAV that OwnCloud supports, and you can get a CardDAV application for Android that will keep contacts in sync between Android and a CardDAV server. It doesn’t work quite as quickly as the built in Google synchronizer, but it does work. I also had to install a new Contact editor for Android as the built in editor would not use the CardDAV fields.

One thing that OwnCloud does better than Google is that it allows you to share any sub-set of your contacts with another person.

Calendar

My calendar was also at Google. The process to get a google calendar into OwnCloud is a bit strange. On Google calendar, you click on the calendar and select settings. On the setting screen, you will find the private ICAL link. Download this ICAL file. Upload that file to the OwnCloud file webapp click to open it in a browser. That imports the calendar into OwnCloud. It is visible and editable on the OwnCloud website.

I wanted to be able to view and edit it on my Android phone. The built in calendar is only setup to view Google calendars. But there is a pay app (the first software I have had to pay for in this process) called CalDAV Sync that will put the OwnCloud calendar on the Android calendar app. The process was straightforward and edits from the web or phone are synced.

Passwords

I stored all my passwords online at LastPass. They are a good service, but I switched to using KeePass which is a Windows/Mac/Linux application and stores the passwords in an encrypted file on my hard-drive. I have it store the password file in the OwnCloud synced directory so that I have access to the file from anywhere.

I used the LastPass chrome extension to autofill and generate passwords. KeePass also has a chrome extension, though it takes a bit of effort to install and link to the KeePass application.

Google Docs

When I reviewed my Google Docs account, I didn’t find anything that needed to be shared with other people. I moved all the files to my computer.

Evernote

I had quite a few notes in Evernote, and I used it mostly on my phone. Replacing it was a bit difficult. First I got the OwnCloud app for Android ($0.99). This allows me to view and synchronize my OwnCloud files just like the OwnCloud Windows client. Then I got a text editor and a Word/Excel editor. I created a couple of text files in my OwnCloud shared directory and created shortcuts to them on the phone desktop. So, instead of taking notes with Evernote, I will append to my existing notes and todo text files. The OwnCloud app will sync them back to my server.

YouTube

I had hosted a couple videos of violin recitals on You Tube. I just moved these videos to my website and am hosting them from there.

Geni

The genealogy site Geni allowed my to export my data in GEDCOM format. I download a free and opensource program called Gramps which is not nearly as pretty or easy to use as Geni, but has more features. It allows exporting to a webpage, so I have replaced my link to Geni with a link to the pages generated by Gramps.

Hardware

A web server typically doesn’t need to be a powerful machine unless you are getting a lot of traffic. An ideal web server is probably a low cost, low power machine. A computer marketed as a home theater PC would work well. I had spare hardware from my last desktop computer upgrade so I used that. I did need to buy a power supply and found that an 80+ certified supply pays for itself in energy savings pretty quickly:

Assuming that the machine is going to idle at 180 w, I compared several power supplies. Our electricity costs 13 cents per kilowatt hour when all taxes and fees are included.

As you can see, the sweet spot is either bronze or gold and the power is a very significant cost to consider when starting up your own web server.

Static IP and Hostname

To get an address that the outside world can use to see my home web server, I need a Static IP from my DSL provider. CenturyLink will provide a single static IP for $6 per month. Getting a static IP can be done on a web page and took less than a half hour.

I was given the address 209.181.65.34. This is like having a phone number that other people can always use to call me. To add myself to the internet version of the phone book, you have to register a domain name that points to that address.

I did that through namecheap.com. This costs about $10 per year and I registered the name billandchad.com. This was also quick and easy. The default setting at namecheap was to point my name to one of their web servers that has a standard “squatter” page. I changed that to point to my static IP address. They also had ways to setup email addresses that would forward to another email account, but I set it up to send the mail directly to my machine.

Software

I decided to go with a Linux based machine. Both Windows and Linux can be used to run an Apache web server, but it is a little bit easier to find DNS and mail server software for Linux. Linux is also free.

I installed the latest version of Ubuntu (13.04). I installed the desktop version instead of the server version so that I could use the machine as a home theater PC.

Once the OS was installed and a user created, I installed an ssh server so that I could log in from my main desktop PC.

sudo apt-get install openssh-server

# setup a static ip address and hostname so that I can log in remotely from inside my local network
# Edit /etc/network/interfaces to look like this
# auto eth0
# iface eth0 inet static
#        address 10.0.0.2
#        netmask 255.255.255.0
#        gateway 10.0.0.1
#        broadcast 10.0.0.255
#        dns-nameservers 10.0.0.1 205.171.2.65
# Edit /etc/hostname to have one line with the name of the machine

sudo service networking restart

The “sudo” command runs the rest of the command as root (administrator). Ubuntu strongly recommends that you don’t create a root account and use the “sudo” command instead. The “apt-get” command is how you install new software in Ubuntu on the command line. With an ssh server, I can use putty (or something like it) to log into my server from my main Windows PC by going to its hostname or 10.0.0.2.

I setup my router to forward all incoming traffic on port 80, 443, 25, 465, 585, 993 and 995 to 10.0.0.2. Port 80 is used for http and port 443 is used for https, the rest are used for email. These settings mean that if someone online connects to billandchad.com it will connect to my home web server instead of being dropped by the router.

Next I installed Apache, php, and mysql. These three pieces of software make up a common web server configuration and has been named the “LAMP” stack. I also installed phpmyadmin which is a nice webapp for maintaining a mysql database.

sudo apt-get install lamp-server^
sudo apt-get install phpmyadmin

# This next command may not be necessary. Should have been done my phpmyadmin installer
sudo ln -s /etc/phpmyadmin/apache.conf /etc/apache2/conf.d/phpmyadmin.conf

# Edit /etc/apache2/conf.d/security and make following changes:
# ServerTokens Prod
# ServerSignature Off

Now that a web server is installed, you can point a web browser to billandchad.com and see a default web page served by Apache. Next I installed the OwnCloud web app. I had to add the ownCloud repository to apt:

# Add the following line to /etc/apt/sources.list.d/owncloud.list
# deb http://download.opensuse.org/repositories/isv:ownCloud:community/xUbuntu_13.04/ /

wget http://download.opensuse.org/repositories/isv:ownCloud:community/xUbuntu_13.04/Release.key
apt-key add - < Release.key
sudo apt-get update
sudo apt-get install owncloud

Use phpmyadmin to create the owncloud user and a database with the same name.

CREATE USER 'owncloud'@'localhost' IDENTIFIED BY 'password';
CREATE DATABASE IF NOT EXISTS owncloud;
GRANT ALL PRIVILEGES ON owncloud.* TO 'owncloud'@'localhost' IDENTIFIED BY 'password';

Now we can use the OwnCloud webapp to finish the installation and create OwnCloud users. To do this just point a browser to http://127.0.0.1/ownCloud.

USB Drive

We have a network shared drive to store music, photos, and other shared data. Ubuntu auto mounts the partitions on a USB drive when it is plugged in. However, these mount points are only created after a user logs into the machine. I don’t want to have to log in after a reboot, so I will create my own mount points.

# First get the UUIDs of the drive partitions
blkid
# next add lines to the /etc/fstab file. One line for each partition.
# The first column is the UUID from the first step
# The second column is where the drive will be mounted
# The third drive is the format type. NTFS is the windows standard
# 	UUID=1294CE3B94CE2159 /media/ChadsDrive ntfs-3g defaults
#	UUID=72601D93601D5EE3 /media/WilliamsDrive ntfs-3g defaults
#	UUID=50C0308BC0307974 /media/DataDrive ntfs-3g defaults
# make the mount points
sudo mkdir -p /media/DataDrive
sudo mkdir -p /media/ChadsDrive
sudo mkdir -p /media/WilliamsDrive
# re-mount drives
sudo mount -a

Now we can go to /media/DataDrive to and see the files on the USB drive. I’d like to be able to see this files from my windows machines too. To do this I’ll use Samba.

sudo apt-get install samba
# Add these lines to /etc/samba/smb.conf for each partition that should be shared
# [DataDrive]
#        path = /media/DataDrive
#        browseable = yes
#        writable = yes
#        guest ok = yes
sudo service smbd restart

Now we can access these shared drives by going to a Windows machine on the local network and pointing the file explorer to \\10.0.0.2\DataDrive.

DNS

DNS is what computers use to turn a human friendly name, like billandchad.com, into an actual IP address. When you get broadband internet service, the ISP provides you with a DNS server that your web browser uses to lookup addresses. This works fine, and when I lookup billandchad.com at CenturyLink’s DNS, it comes back with my static IP address. This is good, but causes a problem inside my home network. If I try to go to http://billandchad.com it goes to http://209.181.65.34. My DSL router sees that as my own external IP and drops the request (on the theory that you wouldn’t want to route your traffic through the external internet just to get back to a computer in your house). Of course that is exactly what I was trying to do, but there is a “better” way to do this.

I can setup my own DNS that will tell my local computers how to get to billandchad.com. So if I lookup billandchad.com I will get the address 10.0.0.2, but if anyone else looks up billandchad.com they will get 209.181.65.34.

The standard DNS server is called “bind” and it is a bit of a hassle to setup. I am going to first setup bind to just be a caching DNS for my local network. That means that it will do all the DNS lookups for my home computers. The first time a site is requested (say google.com), my web server will ask CenturyLink for the address. The second time a site is requested, it will have the answer cached. This will be quite a bit faster than going back to CenturyLink every time. Most modern DSL routers already have a DNS cache, and Windows also caches DNS entries, so the actual speed improvement for browsing will be small.

# install bind
sudo apt-get install bind9
# edit /etc/bind/named.conf.options to have the following.
# These are the DNS servers I will use when the site isn't cached
#        forwarders {
#                205.171.2.65;
#                8.8.8.8;
#                156.154.71.25;
#        };

Next, I will tell bind that if someone is asking about billandchad.com that it can provide the address itself. This makes my DNS server the “master” for billandchad.com. Of course, the only people using this DNS server are other computers in my house.

# edit /etc/bind/named.conf.local to:
#	zone "billandchad.com" {
#	        type master;
#	        file "/etc/bind/db.billandchad.com";
#	};
# create the file db.billandchad.com with:
#	$TTL    604800
#	@       IN      SOA     ns.billandchad.com. hostmaster.billandchad.com. (
#	                              2         ; Serial
#	                         604800         ; Refresh
#	                          86400         ; Retry
#	                        2419200         ; Expire
#	                         604800 )       ; Negative Cache TTL
#	;
#	@       IN      NS      ns.billandchad.com.
#	;
#	        IN      A       10.0.0.2
#	ns      IN      A       10.0.0.2
#	                MX      10 mail
#	                TXT     "Necropolis"
#	www             CNAME   ns
#
# Edit /etc/network/interfaces to point to 127.0.0.1 for DNS
sudo /etc/init.d/bind9 restart
sudo service networking restart

DHCP

Now I need to setup my own DHCP server so that I can tell all the computers in my house what DNS server to use.

My DSL router has a DHCP server built in, but it insists upon listing itself as the DNS server. So the first step is to turn off the DCHP server in the router.

Next I install and configure the default DHCP server for linux

sudo apt-get install isc-dhcp-server

#edit /etc/dhcp/dhcp.conf to have the following settings
# option domain-name "billandchad.com";
# option domain-name-servers 10.0.0.2, 205.171.2.65;
# option routers 10.0.0.1;
# option subnet-mask 255.255.255.0;
# option broadcast-address 10.0.0.255;
# authoritative;
# default-lease-time 7200;
# max-lease-time 86400;
# subnet 10.0.0.0 netmask 255.255.255.0 {
#   range 10.0.0.15 10.0.0.255;
# }

sudo isc-dhcp-server restart

Finally I can go to http://billandchad.com/owncloud from either my phone, my computer or another computer connected to the internet and get to my local web server.

My local web server is also providing several network services for the other computers in my house. When my desktop computer is turned on, it asks the network for an IP address. My web server responds that it is the DHCP server and hands out an IP address to my desktop computer. At the same time, it tells my desktop computer that it is the DNS server. When my desktop computer tries to go to billandchad.com, the web server tells it that the IP address is 10.0.0.2 and the connection is made internally.

If a computer on the internet goes to billandchad.com, a real DNS server will tell it that the IP address is 209.181.65.34. Connecting to that address will connect to my DSL router on port 80 or 443. My router will forward that request on to my web server. The apache server on my web server will respond to the request because it is listening on port 80.

Setting all of this up has been a good reminder of what is really going on behind the scenes to make the internet work.

1. Store my data locally
2. Backup my data (ideally an offsite backup)
3. Provide remote access to my data

The first part is easy. The second two are much harder because of how home internet service works.

Internet providers use dynamic IP addresses. Each customer gets a new address every few days/weeks. This is like having a phone but getting a new phone number every week. You could make phone calls to other people, but couldn’t really receive phone calls back because no one knows your number. With dynamic IP addresses, you can talk to other computers, but you have to start the conversation. The consequence of this arrangement is that you can’t really run a website from your home computer.

Broadband companies will sell you a static IP (like a permanent phone number) for a small monthly fee, but since very few people have a static IP addresses, software companies haven’t been motivated to make it easy to setup a home website. So, for most people, if you want to share photos or start a blog, you have to involve a third party like wordpress.com or facebook.

This is really too bad. The promise of the internet was that anyone could publish content that could be seen by anyone else. Now we have a few large companies that are in the business of publishing other people’s content and making money off it. Just like record companies and book publishers before them, many internet companies (Facebook, twitter, flickr, youtube, etc) make money by publishing the content created by other people.

But it didn’t need to be that way. There was really no reason that we couldn’t each have our own IP address and our own personal website. Windows could have made it easy to publish your own content to your own site. Finding and connecting to other people could have been as easy as looking up or sharing a phone number. But that isn’t how things turned out, and now it is quite a bit of hassle to setup your own website. I’ll have another post with a lot more detail about hardware and software setup, but here is the quick summary:

1. Buy a static IP. We have DSL from CenturyLink and they charge $5 per month to have a static IP.
2. Register your domain name and point it to that IP address. That costs about $10/year.
3. Setup a machine as the web-server. This machine should be left on all the time. Most any computer will do for a personal website.
4. Install apache or some other web server software on the machine.
5. Keep the OS and web server software up to date, do regular backups, make sure the machine stays on and working.

Once you have a website, it is possible to replicate most of the services that are provided by the cloud. I am going to use software called OwnCloud. OwnCloud is a webapp (a program that runs on a web server) that provides a way to store files, contacts, pictures, music and calendars on your website and share them with just the people you want to share them with. Since OwnCloud is running on my own web server, no third party has access to my data.

OwnCloud will replace the following cloud services:

• Music access: I was using Amazon to hold my music collection and provide access to it when I am away from home.
• File Sharing: I used Dropbox to distribute touch table games. OwnCloud even has read-only and per user sharing.
• Notes: I used Evernote to take notes on my phone. I’ll replace that with a text file shared from OwnCloud.
• RSS reader: I used Feedly as an RSS feed aggregator. There is a feed reader app for OwnCloud that provides most of the features I want.
• Calendar: I’ve been using Google to hold my calendar. There is a paid app to keep my OwnCloud calendar synced to my phone.
• Contacts: I have contacts stored in Google mail and Windows mail. There is a free app to keep my OwnCloud contacts synced to my phone.
• Google Docs: My own files can move to OwnCloud.
• Passwords: Lastpass stores all my passwords. I will switch to Keepass. It stores all the passwords locally instead of on a server. I’ll keep the encrypted password file in OwnCloud so that I have access when I am not at home. Keypass has a chrome plugin, so it will be similar to Lastpass.

Other services can be replaced with other software on my own web server or computer:

• Wishlist: I keep a wishlist at Amazon for my family to use. I have moved this wishlist to my own website.
• Feedly Tags: I tagged a bunch of blog posts with Google Reader and Feedly. I’ll move these tags to bookmarks in my browser.
• Geneology: I used Geni.com to create a family tree. I use it more to store information about the people who are alive now than to try to piece together the past. There are lots of geneology programs for the PC and none of my family ended up joining Geni to help me maintain the tree, so I can move this data out of the cloud once I find a good program.
• ChadWeisshaar.com: Hostmonster is my web hosting company. I put a lot of my information on chadweisshaar.com and they own the physical hardware that stores that information. This includes my blog, info and download pages for the software that I have written, databases used by some of my programs, public and private photo galleries and an off-site backup. Once my contract expires I will point chadweisshaar.com to my local web server.
• Embedded videos: I have hosted a couple of private videos at YouTube so that I could embed them into blog posts. I will need to host the videos locally and use Video.js and the associated wordpress plugin to embed the videos.
• Google mail: I already have an email account at chadweisshaar.com and will use that exclusively.

And there are some things that will have to stay on third party services.

• Facebook/Twitter/Google+/LinkedIn: I rarely post anything to these sites, and most of my posts are links to something on my website. I don’t see any alternative to these sites since they are my only connection to quite a few people.
• Google chrome account: I have an account in Google chrome that keeps my bookmarks and plugins synced between chrome installs.
• Google Docs: Some documents are shared with other people who will no be interested in switching services, so these documents will have to stay on the cloud.
• Google voice: I have a phone number on Google voice. I use it as my main contact number and receive calls, voice-mail and text messages on this number. There is no way to truly keep any of this information private. I can either trust Google, or I can trust T-Mobile. I am going to keep using Google voice for now. The next time I change plans at T-Mobile, I will try to get text messages included so that I can start dropping Google Voice. It is not that I trust T-Mobile much more than I trust Google. But there is a chance that Google Voice will go away like Reader did.
• SteamPowered: I have bought quite a few computer games from Steam. In reality these games are rented for the life of my account at Steam. I can make a backup of the game, but it still has to connect to the steam server to run. If Steam actually went out of business, someone would probably come up with a work around. So I will download and keep my own backup of all the games I have on Steam.

I was fortunate that both Google Reader and Catch allowed me to download my feeds and notes before they closed. I just had to spend the time to find replacements and transfer my data. Between these two closures and the revelations that world’s spy agencies are working really hard to monitor and record my data, I have decided that I would like to take back ownership of my data.

Not that long ago, before the “cloud”, all of our data (documents, photos, music, etc.) was stored exclusively on our own computers. This was sometimes a problem: we had to copy files to a floppy to move them to another machine; sometimes we lost data when a computer crashed. Today, internet companies make it easy to store our data on their computers. There is flikr for photos, Amazon will store your music, Dropbox will store your documents, Google and Facebook will keep track of your contacts and mail. And of course it is all “free”. This works pretty well most of the time. The corporations let us access the data from anywhere, and they are pretty good about making backups.

But there are lots of problems too: corporations come and go; they make mistakes and lose data, or hackers break in and steal data; sometimes they sell the data to advertisers, or give data to the government; sometimes they decide that you aren’t using their services correctly and take away your account. That is the trade off that we all make when we sign up for a “cloud” service. We are giving up control of our data for convenience.

I have adopted many of the cloud services myself, and it will be difficult to give up the convenience that they provide. The first step is to take stock of what services I am currently using.