My journey to perfect backups
There and backup again.
Over the years, the more digital photos I've gotten, the more I've begun to worry about about backups. Now with my music and home videos all located on my computer, it's grown from a worry into a paranoia. Last year when I bought a 1 terabyte drive I combined all by previous hard drives onto it but within the month I was jittery after hearing about other people whose new terabyte drives had died. I realized that I had balanced all my memories and documents on the tip of a single hard drive spindle. That is when I knew that I needed to resolve this once and for all.
Attempt 1- Subversion
Several years ago, fresh out of college I was familiar with the code version control system I'd been using. At that time Dreamhost, my web host, began providing complimentary Subversion (aka SVN) repositories. I setup a SVN repository and decided to check in all my personal documents and photos.
The theory was sound. My documents were being copied to an offsite location and supposedly recoverable. Wrong.
Its was painful and slow. It wasn't automated so I had to routinely check in changes. (I realize I could have scripted the check in). It also littered .svn directories with junk all over the place. Finally, if I moved files to different directories the file had to be uploaded again.
I'll admit that I'm pretty green when it comes the the powers of SVN but there is no doubt that I was subverting its purpose.
Attempt 2- Rsync and DeltaCopy
Soon afterwards I became acquainted with rsync. Dreamhost doesn't let me backup files I can't redistribute to to my web directory that I'm not allowed to redistribute (my MP3s). Even if I password protected it and blocked access from the world I wasn't allowed. So as a solution I setup my home server with rsync and used DeltaCopy on my windows machines to publish to the rsync server.
Sure, later Dreamhost came out with a personal backup option, but 50 gig was too confining and insufficient for my needs.
Vista for some reason wasn't allowing DeltaCopy access to all my own files (even when run as admin) so my backup never felt complete. Also the backup wasn't offsite and file recovery was completely manual. Also no versioning of previous files anymore.
Attempt 3- Rsync + SSH + TrueCrypt to a remote friend
One day at work I came up with the idea that I could just backup to my friend's computer. We both had home servers, unfortunately his was a Windows server, so I had to make my solution work on his windows server while mine was a Linux server.
So I chose several open source pieces. The plan was to have an SSH server running on each end. I'd run a script on my end the connected to the other server, mount a TrueCrypt volume to make sure the data was encrypted on the other end, run rsync to push the data across. Then unmounting it. Eventually I might have configured a virtual machine to simplying configuration in the future. However, 2 weeks into my tinkering the motherboard died on one of my test servers and I just couldn't get motivated to try to reconfigure it all over again.
If I could have gotten it to work it would have been quick with rsync's incremental updates, secure with the SSH and TrueCrypt combo, and off site. Unfortunately it was so incredibly difficult to setup that I knew my next step would have been impossible. There was no way I could get my family backups.
Attempt 4- SyncBack
So, finally I just bought an external hard drive and used SyncBack to duplicate my files to the drive. It was simple and it worked. It actually was simple enough that I actually maintained this system for a while.
The only 2 drawbacks were that the backup wasn't offsite and no way to get previous versions. I wasn't too concerned though, because every 6 months I planned on just swapping the external drive to my in-laws when I'd visit.
Final Solution- CrashPlan
I was satisfied with SyncBack and probably would have upgraded to the paid for version, but then I saw on LifeHacker a post for CrashPlan. It presented CrashPlan as a solution that would let you back up to a friend's computer over the internet and the end result was encrypted on the remote computer. Its even cross platform. To top the whole deal off: its free!
Its seems too good to be true. So next I'll present my testing experience and related observations.
Summary Chart
| Fast | Encrypted | Off Site | Easy | Versioning | Free | |
|---|---|---|---|---|---|---|
| SVN |  |
|
||||
| rsync | |
|
|
|||
| rsync + SSH + TrueCrypt | |
|
|
|
||
| SyncBack | |
|
|
|||
| CrashPlan | |
|
|
|
|
|
Link:Syncing Thunderbird Contacts and calendar with GCal
I'm using a useful tool called GCalDaemon. It's written in Java so it runs both on Windows and Linux (and I assume Macs).
GCalDaemon does 2-way sync between ICal files and Google Calenda. It also includes an LDAP server that allows you to query your GMail Contacts.
It has good documentation to help you get it setup.
Back in Ubuntu
With the release of Ubuntu 7.04 (Feisty Fawn) I'm back in Ubuntu full time. It also helps that my employer has enabled our Windows only VPN software to run in a virtual machine again (though that's a story for another day).
One minor annoyance that I missed was a way to switch my wallpaper automatically and a way to right-click on a folder to be able to search in that fold specifically. Yeah, I'm shallow. I also wanted this functionality with no compiling in a simple GUI. Very shallow.
Without further ado:
Drapes - a simple automatic wallpaper switcher (deb file available at GetDeb.net)
nautilus-search-tool - for the right-clicky goodness (deb file available at GetDeb.net)
Also, I really enjoy GetDeb.net. I highly recommend subscribing to the RSS feed there to find out about neat new offerings.
Querying Amarok Stats
I've been using Amarok to manage my music collection for a couple of months now. I love this open source stuff because they are never afraid of letting you look at their work.
For example Amarok manages your music files and keeps various statistics. The player doesn't list of an obvious way to display the cummulative file size of a group of songs. However, since the Amarok database uses SQLite its a piece of cake to write your own SQL queries.
So, after looking at the Amarok wiki I was able to create and paste the following script in the command line and get a quick printout of my genres and how many megabytes of files are in each genre (this is with Amarok 1.4):
dcop amarok collection query "select g.name, ((sum(filesize)/1024)/1024) as size from tags t, genre g where t.genre = g.id group by t.genre order by sum(filesize) desc;"
And the knowledge was set free…
I love that fact that the most classic literature is the most easy to acquire. Take Shakespeare's Works as an example. You can sit down ad read any one of his masterpieces immediately without having to to pay a cent. Why? Because Will Shakespeare has been long dead (and so is his lawyer, as the saying goes) so his works are part of the Public Domain.
I've used Project Gutenberg (Wikipedia Entry) for a while an quite enjoy having these classics at easy reach.
Today I stumbled across the project whose goal it to turn all those works in the Public Domain into AudioBooks its called LibriVox. I've always suspected that something like this existed somewhere, so I'm glad that I finally found it. You can even download using torrents since that gives you nice speedy downloads.
It also doesn't hurt to list some of my other favorite books online: The Standard Works (Bible, Book of Mormon, Doctrine & COvenants, and corresponding reference material like dictionaries and indices) and I also enjoy the LDS Collection of Speeches and Articles. These are great motivational and inspirational writings. I'm also happy to mention you can get them all as audio downloads as well! (No torrents yet though 
)
Audio books of interest there (for anyone)
Watching TV Over the ‘Net
I've recently moved. I got broadband internet setup as quickly as possible. However, I turned down various offers of cable programming.
Why? Because 1) I hate commercials, 2) theretoo much junk just on TV, and 3) you can get a lot of good stuff (legally) over the internet.
Here is my current selection of good online programming that I watch regularly:
I'll add more as I come across and use them. For example, I hear that ABC has a streaming service for episodes of Lost (though I haven't tried it yet)
Matz is a Saint
I'm a fan of Ruby. Its a great programming language and it just makes sense. It doesn't hurt that the Rails framework uses it too.
This morning I followed a link to some humor about Ruby, Perl, and Java and was surprised to discover that Yukihiro Matsumoto (aka Matz) is a member of the Church of Jesus Christ of Latter-Day Saints (sometimes refered to as 'The Mormons'). I doubted for a moment that the info was entirely accurate, but after seeing a third posting of the information I had to believe.
Why does any of this matter? I'm a Mormon too!
Little Ez
Yesterday morning, my wife mentioned that her back was hurting. Two hours later we were in the hospital and less than an hour after that my son was born.
I think we'll call him Ezra. He weighed 5 pounds and 5 ounces and measured 20 inches. He showed up a month earlier than expected, but seems healthy enough, just a little light. We'll fatten him up good.
Its a bit alarming that its -15 degrees fahrenheit outside, so I'm okay that he has to stay at the hospital for a few weeks. I'll be sure to post when he comes home. His mother is doing great (she's wonderful!) and beautiful as always.
Here's the requisite photo:
Feel free to compliment in the comments.
Update 12-23-2005: Ezra came home with us on Wednesday, December 21st. He had lost and regained weight and weighed 5 pounds and 9 ounces when he left the hospital. It's great having him safe at home. Also, to clear things up, Ezra is my second son, he has an older brother named Daniel who is a year and a half old.
Embedding TiddlyWiki in Thunderbird
I couldn't help but be impressed by post today about embedding TiddlyWiki in Outlook. Using my common technique of 'improving' other's ideas I had to check to see if I could pull off a similar trick with Mozilla Thunderbird (though I prefer the portable version).
Its a piece of cake to get working.
Set a TW file as your default start page
Here are the steps:
- Open Thunderbird (I'm using 1.5 though it works in 1.0 version)
- Select 'Options' under the 'Tools' menu
- Under the general tab, in the 'Thunderbird Start Page' put the path to you TW file. (The easiest way to get the path is to open the TW in Firefox and copy the address URL from Firefox into the location box in Thunderbird)
- Under the 'Privacy' tab and the 'General' sub-tab allow JavaScript in mail messages. (I unchecked the 'Block JavaScript in mail messages' option)
- Click 'Ok' to save the settings
View the TW in Thunderbird
This was the toughest part for me to find (though its rather obvious)
- Click in the 'Go' menu and select the 'Mail Start Page'
- Thats it
Here is the all important sceenshot:
Gotchas
- Whenever you open an email it navigates away from the TW page, so you'll want to make sure that you have autosave on.
Nifty benefits
- This keeps your TW nearby and it will mirror any changes you make to it outside of TW in Firefox
- You can even point it to a TW online like pytw
- TW acts just like you would expect with all your plugins ans TW goodness.
Tools to jump start programming the DOM and/or TW
First of all this post is intended for people that are new to playing with the DOM (document object model) The DOM is the structure in a web page.
Specifically I've posted this for my IS450 (Web Development Class) and TiddlyWiki friends though feel free to comment on any additions.
Literature
Before using any tools, I'd recommend getting a bit of knowledge under you belt.
I got started by reading DHTML Utopia: Modern Web Design by Stuart Langridge. The first four chapters are available online for free (94 of the 300 pages in the book!)
Quirksmode also has a very good page on JavaScript document events.
Tools
The major tip is that if you are coding a page do the following: make it work using Firefox and its corresponding tools, then make it work in Internet Explorer.
Firefox Extensions:
Web Developer
View the pages rendered source
Colorzilla
Measure It
I'll put up more stuff after class.