One thing that I cannot live without is a text editor, namely VIM.  VIM was first released as free and extended version of the VI program found on all Unix machines.  I first cut my teeth on VI when I started my sysadmin career, and switched to VIM a couple of years later.  VIM by default works mostly like VI, but i’ts highly configurable, and has a lot of power, especially when you start working with your .vimrc.  Since starting with the editor, I’ve carried my .vimrc from machine to machine, and company to company.  I’m continually updating it, and while there’s not a ton of stuff here, I thought I’d share it in case someone gets some value:

" Ed Salisbury's .vimrc
" Last Modified: 11/17/11

" Make VIM act like VIM, not VI
set nocompatible

" Set up TABs
set tabstop=4
set shiftwidth=4
set softtabstop=4
set expandtab
set smarttab
set autoindent
set smartindent

" Set up syntax highlighting
syntax on
filetype plugin on
filetype indent on
set formatoptions=qroc
set cindent

" Misc settings
set number       " Print line numbers
set background=dark
set pastetoggle=<F2>

I’d consider these settings a starting point for doing code/web development. Of course, there are tons of more complicated .vimrc files out there, but this should be good for starters. If you’re interested as to what each of these settings do, you can type :helpin VIM and it will display the help file.

Backups. No one likes doing them, but they’re too important not to do. Typically when I’m doing stuff for my websites, I will occasionally do a tar of a directory and save it off, or even set up a script that will do this nightly. Then I have issues with disks filling up, and not wanting to delete my backups for fear of losing something (like the 20 copies aren’t enough!)

Recently, I decided to put an end to my backup worries, and write a differential backup system. What is a differential backup? Go go gadget Wikipedia! Basically, a differential backup means that I won’t be filling up my drives any time soon, because I’m not doing a FULL backup each time — it’s only backing up the changes that have been made since the last full backup, which runs weekly and monthly. I’ve been running and tuning the script for a while, and wanted to make sure that it’s working properly. If you decide to use this script, you MUST test the restores with it, as I can’t be responsible if you lose data and the script somehow fails you. You have been warned! Without further ado, the script:

#!/usr/bin/perl
# Website Differential Backup
# Keeps several websites and their databases backed up
# by Ed Salisbury (ed@edsalisbury.net)
# http://www.edsalisbury.net
# (c)2009 Ed Salisbury, Some Rights Reserved
#
# Config File Format
# DIR /path/to/dir
# DB dbname
#
# Limitations:
# * Cannot do differential backups of databases
#
#
# License:
# Except where otherwise noted, this work is licensed under Creative Commons
#   Attribution ShareAlike 3.0.
#
# You are free:
#   * to Share - to copy, distribute and transmit the work
#   * to Remix - to adapt the work
#
# Under the following conditions:
#   * Attribution. You must attribute the work in the manner specified by the
#     author or licensor (but not in any way that suggests that they endorse
#     you or your use of the work).
#   * Share Alike. If you alter, transform, or build upon this work, you may
#     distribute the resulting work only under the same, similar or a
#     compatible license.
#   * For any reuse or distribution, you must make clear to others the license
#     terms of this work. The best way to do this is with a link to the
#     license's web page (http://creativecommons.org/licenses/by-sa/3.0/)
#   * Any of the above conditions can be waived if you get permission from the
#   * Nothing in this license impairs or restricts the author's moral rights.

use strict;
use warnings;

# Place to store backups
my $BACKUP_DIR = "/backup";
# Configuration file to read what to backup
my $BACKUP_CONF = "/usr/local/etc/backup.conf";
# Database user/pass
my $DB_USER = 'root';
my $DB_PASS = '**DBPASSWD**';

# Turn off output buffering
$|++;

# Tools
my $TAR = '/bin/tar';
my $GZIP = '/bin/gzip';
my $LS = '/bin/ls';
my $HEAD = '/usr/bin/head';
my $MKDIR = '/bin/mkdir';
my $RM = '/bin/rm';
my $MYSQLDUMP = '/usr/local/mysql/bin/mysqldump';

my @months = qw( Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec );
my @days = qw( Sun Mon Tue Wed Thu Fri Sat );
my @dirs;
my @dbs;

# Get current date
my ($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = localtime time;
my $datestamp = sprintf("%04d%02d%02d", $year+1900, $mon+1, $mday);
my $day = $days[$wday];

# Read config file
unless (open(CFG, $BACKUP_CONF))
{
    print "Error: Cannot open $BACKUP_CONFn";
    exit(1);
}
while (<CFG>)
{
    chomp;
    if (/^DIRs+(.*)$/i)
    {
        push (@dirs, $1);
    }
    elsif (/^DBs+(.*)$/i)
    {
        push (@dbs, $1);
    }
}
close (CFG);

# Do Directory Backups
foreach my $dir (@dirs)
{
    print "Backing up $dir... ";

    # Add backslashes for spaces
    $dir =~ s/ /\ /g;

    my $backup_src = $dir;

    # Convert slashes to underscores
    $dir =~ s///_/g;
    # Remove leading slash
    $dir = substr($dir,1);

    unless (-d "$BACKUP_DIR/$dir")
    {
        system("$MKDIR -p $BACKUP_DIR/$dir");
    }

    # Get datestamp of last full backup
    my $last_full = `$LS -t $BACKUP_DIR/$dir/${dir}_full* 2>&1 | $HEAD -1`;
    chomp($last_full);
    if ($last_full =~ /No such file or directory/)
    {
        $last_full = '';
    }
    my ($dev,$ino,$mode,$nlink,$uid,$gid,$rdev,$size,$atime,$mtime,$ctime,
        $blksize,$blocks) = stat($last_full);

    if ($mday == 1)
    {
        # Monthly Full Backup
        system("$TAR -cf $BACKUP_DIR/$dir/${dir}_full_monthly_$datestamp.tar $backup_src > /dev/null 2>&1");
        system("$GZIP -f $BACKUP_DIR/$dir/${dir}_full_monthly_$datestamp.tar");
        system("$RM $BACKUP_DIR/$dir/${dir}_full_weekly_*");
    }
    elsif ($wday == 0 || !$last_full)
    {
        # Weekly Full Backup
        system("$TAR -cf $BACKUP_DIR/$dir/${dir}_full_weekly_$datestamp.tar $backup_src  > /dev/null 2>&1");
        system("$GZIP -f $BACKUP_DIR/$dir/${dir}_full_weekly_$datestamp.tar");
    }

    # Daily Differential Backup
    if ($mtime)
    {
        my $newer = sprintf("%02d-%s", (localtime($mtime))[3], $months[(localtime($mtime))[4]]);
        system("$TAR --newer $newer -cf $BACKUP_DIR/$dir/${dir}_diff_daily_$day.tar $backup_src > /dev/null 2>&1");
    }
    else
    {
        system("$TAR -cf $BACKUP_DIR/$dir/${dir}_diff_daily_$day.tar $backup_src > /dev/null 2>&1");
    }
    system("$GZIP -f $BACKUP_DIR/$dir/${dir}_diff_daily_$day.tar");

    print "done.n";
}

# Do Database Backups
foreach my $db (@dbs)
{
    my $db_name = $db;
    $db = "db_$db";

    unless (-d "$BACKUP_DIR/$db")
    {
        system("$MKDIR -p $BACKUP_DIR/$db");
    }

    print "Backing up $db_name database... ";

    if ($mday == 1)
    {
        # Monthly Backup
        system("$MYSQLDUMP --user=$DB_USER --password='$DB_PASS' $db_name > $BACKUP_DIR/$db/${db}_monthly_$datestamp.sql");
        system("$GZIP -f $BACKUP_DIR/$db/${db}_monthly_$datestamp.sql");
        system("$RM $BACKUP_DIR/$db/${db}_weekly_*");
    }
    elsif ($wday == 0)
    {
        # Weekly Backup
        system("$MYSQLDUMP --user=$DB_USER --password='$DB_PASS' $db_name > $BACKUP_DIR/$db/${db}_weekly_$datestamp.sql");
        system("$GZIP -f $BACKUP_DIR/$db/${db}_weekly_$datestamp.sql");
    }

    # Daily Backup
    system("$MYSQLDUMP --user=$DB_USER --password='$DB_PASS' $db_name > $BACKUP_DIR/$db/${db}_daily_$day.sql");
    system("$GZIP -f $BACKUP_DIR/$db/${db}_daily_$day.sql");

    print "done.n";
}

Call the script something like /usr/local/bin/backup, and then create a config file /usr/local/etc/backup.conf, that has the following format:
DIR /www/domain1.com
DIR /www/domain2.com
DB domain1_wp_db
DB domain2_wp_db

(Substituting locations/db names appropriate for your environment of course). If you use a different config file than /usr/local/etc/backup.conf, be sure to change that line in the script. Also, you’ll need to change the mysql root password in the script to be whatever it is for your env (or make it non-root, whichever you want, it just needs to be able to do a mysqldump on the databases)

Next, set up your crontab to run nightly:

sudo crontab -e

Add the following line:

0 1 * * * /usr/local/bin/backup > /dev/null 2>&1

That should be it! Again, make sure to do some test restores to verify that it works ok! Here’s what a typical backup directory looks like after a couple of months running the script:

[ed@halo1:/backup/www_edsalisbury.net]$ ls -l
total 144240
-rw-r--r--  1 root root  5423759 Sep 11 01:00 www_edsalisbury.net_diff_daily_Fri.tar.gz
-rw-r--r--  1 root root  1839115 Sep  7 01:00 www_edsalisbury.net_diff_daily_Mon.tar.gz
-rw-r--r--  1 root root  5787136 Sep 12 01:00 www_edsalisbury.net_diff_daily_Sat.tar.gz
-rw-r--r--  1 root root  7193167 Sep 13 01:00 www_edsalisbury.net_diff_daily_Sun.tar.gz
-rw-r--r--  1 root root  3782727 Sep 10 01:00 www_edsalisbury.net_diff_daily_Thu.tar.gz
-rw-r--r--  1 root root  2455973 Sep  8 01:00 www_edsalisbury.net_diff_daily_Tue.tar.gz
-rw-r--r--  1 root root  3268897 Sep  9 01:00 www_edsalisbury.net_diff_daily_Wed.tar.gz
-rw-r--r--  1 root root 23705401 Aug  1 01:00 www_edsalisbury.net_full_monthly_20090801.tar.gz
-rw-r--r--  1 root root 33975458 Sep  1 01:00 www_edsalisbury.net_full_monthly_20090901.tar.gz
-rw-r--r--  1 root root 31453170 Sep  6 01:00 www_edsalisbury.net_full_weekly_20090906.tar.gz
-rw-r--r--  1 root root 28556424 Sep 13 01:00 www_edsalisbury.net_full_weekly_20090913.tar.gz

Not too bad, size-wise. For all of my sites, the backup directory is about 4 GB, and will stay around that size for quite a while. The differentials get deleted and so do the weekly full backups after the monthly full completes. This makes it so that the directory only really grows when there’s a new full monthly. If you find this script useful or have issues, be sure to post a comment!

Up until a couple of years ago, I used shared web hosting for serving up my various sites. I went through several of them because for whatever reason, they turned out to not be what I wanted. I finally came to the realization that I needed to get my own dedicated host. I found a good provider (The Planet), and started migrating over my websites (along with my friend Chris’ sites). Something I knew I would have to learn is how to set up Apache to be able to do virtual hosting. In this guide, I’ll show you how I did it, and the script I created to make things easier.

The first thing to do is to set up a logical directory structure — I hate putting web docs in some silly folder like /usr/local/apache2/htdocs — that’s always seemed like such a weird place to put things (I guess it goes against my feelings that anything in /usr shouldn’t be writable by users) Since this is a webserver, I wanted something closer to the root, like /www/<domain>. I also wanted to have a different directory structure and logging for each virtual host, I chose /www/logs/<domain>.

The next thing is to set up the apache config file to be able to include a separate file just for vhosts. Add this section somewhere in your httpd.conf file:

# Virtual Hosts
NameVirtualHost *
Include etc/httpd-vhosts.conf

Now, create httpd-vhosts.conf in the apache config dir, and create a manual entry to make sure it works:

    ServerName      www.mydomain.com
    ServerAdmin     webmaster@mydomain.com
    ServerAlias     mydomain.com
    DocumentRoot    /www/mydomain.com/docs
    DirectoryIndex  index.html index.php
    AccessFileName  .htaccess
    HostnameLookups Off
    ErrorLog        /www/logs/mydomain.com/error.log
    LogLevel        warn
    CustomLog       /www/logs/mydomain.com/access.log combined

(Be sure to change “mydomain.com” to your domain)

Create /www/<domain.com> and log files, as well as change ownerships properly:

# mkdir /www/logs/<domain.com>
# mkdir /www/<domain.com>
# chmod g+w /www/<domain.com>
# chown nobody:nobody /www/<domain.com>
# chown nobody:nobody /www/logs/<domain.com>

Next, create some sort of test file (like index.html) in the new doc directory.

When you’re ready, restart the webserver (usually just by running apachectl graceful) With any luck, you should be seeing the webpage you created when you go do www.<domain.com>. If not, look at the apache config for errors, and verify that the DNS entries are correct.

Now, as you were expecting, I have a script to be able to automate this. It’s pretty simple, but handy. I call it “add_vhost” and I put it into /usr/local/bin:

#!/usr/bin/perl -w
# AddVhost
# Add Virtual Host Configs for Apache
# by Ed Salisbury (ed@edsalisbury.net)
# http://www.edsalisbury.net
# (c)2009 Ed Salisbury, Some Rights Reserved
#
# License:
# Except where otherwise noted, this work is licensed under Creative Commons
#   Attribution ShareAlike 3.0.
#
# You are free:
#   * to Share -- to copy, distribute and transmit the work
#   * to Remix -- to adapt the work
#
# Under the following conditions:
#   * Attribution. You must attribute the work in the manner specified by the
#     author or licensor (but not in any way that suggests that they endorse
#     you or your use of the work).
#   * Share Alike. If you alter, transform, or build upon this work, you may
#     distribute the resulting work only under the same, similar or a
#     compatible license.
#   * For any reuse or distribution, you must make clear to others the license
#     terms of this work. The best way to do this is with a link to the
#     license's web page (http://creativecommons.org/licenses/by-sa/3.0/)
#   * Any of the above conditions can be waived if you get permission from the
#     copyright holder.
#   * Nothing in this license impairs or restricts the author's moral rights.

use strict;

my $DOCROOT = '/www';
my $LOGDIR = '/www/logs';
my $USER = 'nobody';
my $GROUP = 'nobody';
my $VHOST_CFG = "/usr/local/apache2/etc/httpd-vhosts.conf";

my $domain = $ARGV[0];

if (!$domain)
{
    print "Error: No domain specified!n";
    exit(1);
}
open (OUT, ">> $VHOST_CFG");
print OUT "n";
print OUT "<VirtualHost *>n";
print OUT "tServerName      www.$domainn";
print OUT "tServerAdmin     webmaster@$domainn";
print OUT "tServerAlias     $domainn";
print OUT "tDocumentRoot    $DOCROOT/$domainn";
print OUT "tDirectoryIndex  index.html index.phpn";
print OUT "tAccessFileName  .htaccessn";
print OUT "tHostnameLookups Offn";
print OUT "tErrorLog        $LOGDIR/$domain/error.logn";
print OUT "tLogLevel        warnn";
print OUT "tCustomLog       $LOGDIR/$domain/access.log combinedn";
print OUT "</VirtualHost>n";
close (OUT);

system("mkdir $LOGDIR/$domain");
system("mkdir $DOCROOT/$domain");
system("chmod g+w $DOCROOT/$domain");
system("chown $USER:$GROUP $DOCROOT/$domain");
system("chown $USER:$GROUP $LOGDIR/$domain");

Be sure to change the config variables to suit your environment. When you want to add a virtual host, simply run:

% sudo add_vhost domain.com

It will create the config and set everything up for you. All you need to do then is to restart the webserver, and you’re good to go!

I’ve been into emulation again lately, and I started with my first favorite system of all time, the Atari 8-bit computer. When I was growing up, I had 2 different 8-bit machines:

The Atari 400:

and later the Atari 130XE:

I did a lot of things on these machines, including learn BASIC, type in programs from magazines, and of course… play games! In order to play these games, I had a great joystick –

the Wico Command Control joystick:

My goal is to have as close as an experience as possible to the real thing with modern hardware, through emulation. To purists, this is considered sacrilege — they want the physical hardware, and emulation for them just doesn’t cut it. I guess I’m not a purist — I just like being able to play the old games! Since I’m now running Linux, the goal will be to get a working emulator going and everything working properly. Here’s what I mean by a proper emulator setup:

• Emulator should run in fullscreen mode, with no window elements showing
• 2 joysticks should work
• Everything should be able to be specified via the command line
• It should handle multiple disk programs as painless as possible
• It should have a standard method for exiting the emulator

The emulator I’ll be using for the Atari 8-Bit computers is atari800. In my opinion, it is the best emulator for the 8-bits, and will work for us just fine, with a few modifications. As a part of this project, I am also going to use the original joysticks that were used for the system.

Controllers:

For this project, I’m using the Wico Command Control joystick (see photo above). I used to use the Logitech gamepads that look like PS2 controllers, but after switching to the right controller, there’s no going back! The issue is, I need a way to convert them to USB to be able to use them on modern machines. After doing some searching, I found just the adapter, from a company called RetroZone. I bought the joystick and adapter off of eBay, and I couldn’t wait to plug it in and try it out! It seemed to work, but I noticed something odd – the fire button was being pressed every time I moved the joystick. I thought that it might be a linux thing, so I plugged it into my Windows laptop, and sure enough, it presses a different button every time I move the joystick. This wouldn’t be a problem normally, if the emulator I’m using didn’t allow you to use any button as the fire button. I have a fix for this issue though, which I’ll get to later.

Installation:

To install the standard version of the emulator (atari800), simply run the following (in Ubuntu):

% sudo apt-get install atari800

Then before you can actually run it, you’ll need a couple of things:

• ROM files
• Disk / Cartridge Images

The ROM files are dumps from the actual ROM chips found in the original hardware. There’s a few of them out there on the internet, and one place I’ve found that has them is TheOldComputer.com. All you really need are the following ROMs:

• Atari Rev B (400/800) NTSC
• Atari XL or XE NTSC

You’ll also need disk images or cartridge images. One cool site that I use frequently is called AtariMania.com. They have images and also artwork scans from the boxes/disks/manuals etc.

Now would be a good time to talk about how you’re planning on organizing your emulator directories. I personally use the following directory structure, but you’re welcome to use whatever you like:

/data/Software/Retro/Atari 8-Bit/roms
/data/Software/Retro/Atari 8-Bit/disks
/data/Software/Retro/Atari 8-Bit/carts
/data/Software/Retro/Atari 8-Bit/art_front
/data/Software/Retro/Atari 8-Bit/art_back
/data/Software/Retro/Atari 8-Bit/snap
/data/Software/Retro/Atari 8-Bit/titles

The last 4 directories are useful when running the emulator from a front-end, which I highly recommend. I’ll talk more about front-ends in a future blog posting.

To run the emulator, simply run the following:

% atari800

This will probably give you an error saying that it can’t find the ROM files. You will need to set up the configuration so that it can find your ROMs. To do this from within the emulator, hit F1, and go to “Emulator Configuration”. From there you can select the different ROMs and configure things how you like. To exit the emulator, press F9.

After it can find your roms, you’ll need to load a disk image. To do this from the command line, simply run:

% atari800 <name of the disk image>

(If there are spaces in the diskname, be sure to either put double quotes around the filename or use backslashes in front of the spaces or other special characters)

A couple of keys to get you started:

• F1 – Configuration Mode
• F2 – Option
• F3 – Select
• F4 – Start
• F5 – Reset
• F9 – Quit the Emulator

With any luck, you’re now able to play some games! A couple of issues you might be having: It’s windowed and small, and you want it to be fullscreen and big! It took a little bit of effort to get this going for me, but here’s how I achieved this. If you are running with more than one monitor, please see my guide on how to set up MetaModes to fix fullscreen issues.

First off, you will want to turn on NTSC mode. To do this, either do it in the emulator itself (hit F1, etc.) or by editing the config file. If you’re editing the config file, just set this line in ~/.atari800.cfg:

DEFAULT_TV_MODE=NTSC

If you’re like me, you want your gaming to be in fullscreen mode, not a window. One way of doing this is to create an alias that sets the resolution:

alias atari800='atari800 -width 1280 -height 1024'

Add this line to your .bashrc, and you’ll be in fullscreen gaming heaven! (Be sure to change the height and width as necessary for your monitor)

Customization:

At first, I just used the standard distribution version of atari800 – this worked well, but I had a couple of issues. First, the emulator treats every joystick button as the fire button. This was an issue with the joystick adapter I’m using, which sends a different button press for each direction (by design). Another issue I had was the fact that there needed to be an easy way to swap disks in a multi-disk set — I would like to be able to hit a function key for this.

I figured the easiest way to get these things fixed would be to compile from source. I started by downloading the source from CVS, and started working on compiling. (For my examples, I used version 2.1.0. To compile it, you’ll also need to get the SDL development libraries, which can easily by installed via Synaptic Package Manager (search for SDL). There’s lots of guides on compiling from source out there, but the simple method is:

cd <source dir>
./configure
make

Before I do any source code hacking, I always make sure that the source compiles first, that way I know it works before I start breaking stuff! Another thing to do is to always keep a backup copy of the source file you are editing. You want to be able to back out of any changes you make. On that note, let the hacking begin!

To fix the trigger issue, edit atari_sdl.c:
In the get_platform_TRIG() function, change the following lines:

int trig0, trig1, i;

to:

int trig0, trig1;

and:

for (i = 0; i < joystick0_nbuttons; i++) {
    if (SDL_JoystickGetButton(joystick0, i)) {
        trig0 = 0;
        break;
    }
}

to simply:

if (SDL_JoystickGetButton(joystick0, 0))
    trig0 = 0;

And do the same for joystick1 -- convert:

for (i = 0; i < joystick1_nbuttons; i++) {
    if (SDL_JoystickGetButton(joystick1, i)) {
        trig1 = 0;
        break;
    }
}

to:

if (SDL_JoystickGetButton(joystick1, 0))
    trig1 = 0;

To map the Rotate Disk function to F11, edit atari_sdl.c:

Add the following into the #include section:

#include "sio.h"

Then do a search for "F10", and add the following after the SDLK_F10 case:

case SDLK_F11:
    key_pressed = 0;
    SIO_RotateDisks();
    return AKEY_NONE;

After making either or both of these changes, run make again, and test. There's still one more change I'd like to make, but it will have to be at another time. I like to have the Escape key exit from all of my emulators. The idea is, any one should be able to run these, and having to remember which function key does what is a pain. I'll be sure to update this post when I make my change so that you can see how I did it.

Conclusion:

Hopefully this guide has been helpful to people interested in RetroGaming on Linux. Please feel free to share your experiences or issues -- I'm always interested to hear other people's thoughts and ideas. Have a system you'd like me to cover? Let me know and I'll do my best to accommodate you. I enjoy working with all gaming and computer platforms, albeit some of them are not quite as easy to set up.

One of the reasons people have a hard time making the jump to Linux is the fact that they have a hard time leaving the games they play in Windows. In this article, I’m going to describe some methods of getting around this hurdle, and make the transition to Linux easier. I’m not going to promise that your favorite game will work, but will give you the tools to be able to try. I’m also not going to be talking about native linux apps – those are pretty straightforward. This article will focus on Windows apps in Linux.

The first tool in the linux gamer’s arsenal is a utility called WINE. WINE stands for “Wine Is Not an Emulator”, in true unix style recursive acronyms. It’s job is to make your system look like Windows to your Win apps, giving users the ability to run programs, and for our purposes, games. The first step to running a windows program in linux is to install WINE (use apt-get / yum to install from the standard repositories), and then simply run ‘wine <name of the program.exe>’ – there’s a good chance that it will just run. I have noticed that for some reason game installers tend to be a little temperamental, so one way to get around this is to install on a Windows box and then copy the files to your unix machine over the network.

Another tool is called CrossOver Games, which is a commercially-supported version of WINE. If you don’t feel like messing with config files, this may be your best bet. I haven’t needed anything but the basic WINE installation, but I’ve heard good things about this product.

The games that I like are traditionally MMORPGs and emulated games. I’ve gotten both Star Wars Galaxies and World of Warcraft running, and Atari800, Stella, and MAME running so far. Also, I’ve gotten Spore to run. I really haven’t found the need to run much else in the gaming dept, so I’m set! The main issues that I’ve found with running games in linux are:

• Dual-Screen Issues
• Screen Resolution
• Keyboard Repeat Issues

Dual-Screen Issues

First off, I like to do gaming in full screen. I hate window elements on my game screen. The issue is when you have a dual monitor set up. The issue is, Xwindows handles resolutions differently than windows (one big screen vs. 2 smaller screens). This can be an issue since games will try to use the entire surface, and probably centering the content, making it completely unplayable (half on one screen, half on the other). In doing some searching, I found a solution to this problem — it’s called MetaModes. Note: I’m using an Nvidia chipset – not sure what to do if you’re using ATI. To enable MetaModes with TwinView, do the following:

# sudo cp /etc/X11/xorg.conf /etc/X11/xorg.conf.backup
# sudo vi /etc/X11/xorg.conf

Add the following line in the “Screen” Section:

Option "metamodes" "1280x1024,1280x1024; 1280x1024,NULL"

(This assumes you have 1280×1024 monitors — change as needed)

Restart your X server — I usually close all my apps, hit CTRL-ALT-F1, log in, and run:

# sudo /etc/init.d/gdm restart

This shouldn’t change anything with your normal desktop, but if you run a fullscreen app, it should only use the first monitor, and turn off the second. When the app is done, it will go back to the normal 2-screen layout. This setup works pretty well for me.

Screen Resolutions

One of the things you’ll find is that configuration apps written for windows won’t detect available resolutions correctly when running in Linux. Since most people run with LCDs lately (which look crap if you don’t run in the correct resolution), this can be a pain. Don’t expect the application to be able to find your ideal resolution. What I usually do is run the program and let it do it’s thing with the default res. Then I quit the program and look for a config file. Most games have a config file somewhere that states what resolution to run in. Edit the file manually and re-run the game. This should fix things so that it runs the way you want.

Keyboard Repeat Issues

Lately I’ve noticed an odd issue with keyboard repeating. It seems that there is an issue with the latest Xorg version and wine not playing nicely with keyboard repeats. If you do run into something like this, try disabling keyboard repeating from System->Preferences->Keyboard (in Ubuntu).

One really nice thing about gaming on Linux is the fact that you can keep your software on an NFS mounted filesystem. If you’re sharing an installation of WoW, for instance, only one person has to update it, and then it’s updated for everyone. It can take a little longer to load from the network, but I haven’t noticed many issues here. Another good thing here is the registry. While I usually try to stay away from games that use the registry, having it in linux is helpful because it’s not tied to one machine. You can put it on a remote drive, and it encapsulates the registry so that it could then run on other machines, making it a lot more portable.

Previously, I have showed you how to set up software to be able to rip DVDs to your hard drive, and I’ve also shown you how to copy your TiVo videos nightly. I’ve described the method I use to transcode the videos, but haven’t provided the script… Until now. After months of using and tweaking, here’s what I use. I call it VidProc.

#!/usr/bin/perl
# VidProc
# Transcode videos that have been either ripped by ripper or copied from a TiVo
#    to H.264 format
# by Ed Salisbury (ed@edsalisbury.net)
# http://www.edsalisbury.net
# (c)2009 Ed Salisbury, Some Rights Reserved
#
# External Utilities Required:
# * HandBrakeCLI
# * MPlayer/Mencoder
#
# Notes:
# * Settings for HandBrake and mencoder are what I have come up with after
#   doing a fair bit of research, and seem to work pretty well - if you have
#   any concrete suggestions on *better* general-purpose settings, let me know.
#
# License:
# Except where otherwise noted, this work is licensed under Creative Commons
#   Attribution ShareAlike 3.0.
#
# You are free:
#   * to Share -- to copy, distribute and transmit the work
#   * to Remix -- to adapt the work
#
# Under the following conditions:
#   * Attribution. You must attribute the work in the manner specified by the
#     author or licensor (but not in any way that suggests that they endorse
#     you or your use of the work).
#   * Share Alike. If you alter, transform, or build upon this work, you may
#     distribute the resulting work only under the same, similar or a
#     compatible license.
#   * For any reuse or distribution, you must make clear to others the license
#     terms of this work. The best way to do this is with a link to the
#     license's web page (http://creativecommons.org/licenses/by-sa/3.0/)
#   * Any of the above conditions can be waived if you get permission from the
#     copyright holder.
#   * Nothing in this license impairs or restricts the author's moral rights.

use warnings;
use strict;
use File::Copy;
use File::Basename;

sub system_int($);

# Locations
my $DEST = "/path/to/destdir";
my $QUEUE = "/path/to/queue.txt";

# Encoding Options
my $DEINT = "pp=md";    # Deinterlacer
my $VBITRATE = 1000;    # Video Bitrate
my $SAMPLE_RATE = 44.1; # Audio Sample Rate
my $ABITRATE = 192;     # Audio Bitrate
my $OVERSCAN = 6;       # Overscan size (# of lines)

# File Extensions
my $SRC_EXT = "mpg";
my $DEST_EXT = "mp4";

# External Utilities
my $HANDBRAKE = "/usr/local/bin/HandBrakeCLI";
my $MENCODER = "/usr/bin/mencoder";
my $MPLAYER = "/usr/bin/mplayer";

# Add backslashes to spaces
$DEST =~ s/ /\ /g;

while (1)
{
    # Read in the queue
    open(QUEUE, $QUEUE);
    my $line = <QUEUE>;
    my @fields;
    if ($line)
    {
        chomp ($line);
        @fields = split(/|/, $line);
    }
    else
    {
        print "Waiting for new stuff to show up in the queue...n";
        close(QUEUE);
        sleep(60);
        next;
    }
    close (QUEUE);

    my $input = $fields[0];

    print "Processing $inputn";

    # Fix spaces
    $input =~ s/ /\ /g;
    $input =~ s/'/\'/g;
    $input =~ s/&/\&/g;
    $input =~ s/;/\;/g;
    my @extlist = (".$SRC_EXT");

    my $base = basename($input, @extlist);

    if ($fields[1])
    {
        # DVD, since it has a title number
        my $title = $fields[1];
        my $output = "$DEST/${base}_" . $fields[1] . ".$DEST_EXT";

        system_int("$HANDBRAKE --input $input --output $output --title $title --turbo --encoder x264 --vb $VBITRATE --audio 1 --aencoder faac --mixdown 6ch --arate $SAMPLE_RATE --ab $ABITRATE --detelecine --decomb --loosePixelratio --markers --two-pass --x264opts ref=3:mixed-refs:bframes=6:weightb:direct=auto:b-pyramid:me=umh:subme=9:analyse=all:8x8dct:trellis=1:nr=150:no-fast-pskip=1:psy-rd=1,1");
    }
    else
    {
        # TiVo file, since it has no title number
        my $output = "$DEST/${base}.$DEST_EXT";
        my $line;
        my $width;
        my $height;

        # Get video dimensions
        print "Getting video information... ";
        my @id = `$MPLAYER -nojoystick -nolirc -vo null -ao null -identify -frames 0 $input 2>&1`;
        foreach (@id)
        {
            if (/ID_VIDEO_WIDTH=(d+)/)
            {
                $width = $1;
            }
            if (/ID_VIDEO_HEIGHT=(d+)/)
            {
                $height = $1;
            }
        }
        if (!$width || !$height)
        {
            print "FAILED!";
        }
        else
        {
            print "(${width}x$height)n";
        }

        # Crop overscan
        $height-=$OVERSCAN;

        my %croptest;

        # Detect crop region
        print "Detecting crop region... ";
        my @cropdetect = `$MPLAYER -nojoystick -nolirc -vo null -ao null -vf crop=$width:$height:0:$OVERSCAN,cropdetect -ss 600 -endpos 180 $input 2>&1`;
        foreach $line (@cropdetect)
        {
            if ($line =~ /-vf crop=([d:]+)/)
            {
                $croptest{$1}++;
            }
        }

        # Get the crop value with the most hits
        my @croplist = sort { $croptest{$b} <=> $croptest{$a} } keys (%croptest);
        my $crop = $croplist[0];

        if (!$crop)
        {
            print "FAILEDn";
            exit();
        }
        print "($crop)n";
        print '-' x 80 . "n";

        print "Encoding first passn";
        print '-' x 80 . "n";
        system_int("$MENCODER -ovc x264 -x264encopts pass=1:turbo:bitrate=$VBITRATE:bframes=1:me=umh:partitions=all:trellis=1:qp_step=4:qcomp=0.7:direct_pred=auto:keyint=300 -vf $DEINT,crop=$crop,scale=-1:-10,harddup -oac copy -ofps 30000/1001 $input -o /dev/null");

        print "nn";
        print '-' x 80 . "n";
        print "Encoding second passn";
        print '-' x 80 . "n";
        system_int("$MENCODER -ovc x264 -x264encopts pass=2:turbo:bitrate=$VBITRATE:bframes=1:me=umh:partitions=all:trellis=1:qp_step=4:qcomp=0.7:direct_pred=auto:keyint=300 -vf $DEINT,crop=$crop,scale=-1:-10:,harddup -oac copy -ofps 30000/1001 $input -o $output");
    }
    print "Done. Removing from queuen";
    print "nn";
    open(IN, $QUEUE);
    open(OUT, ">$QUEUE.tmp");
    while (my $qline = <IN>)
    {
        chomp ($qline);
        unless ($line eq $qline)
        {
            print OUT "$qlinen";
        }
    }
    close (IN);
    close (OUT);
    unlink($QUEUE);
    move("$QUEUE.tmp", $QUEUE);
}

# Interruptible System Command
sub system_int($)
{
    my ($cmd) = @_;
    my $pid = fork();
    my $rc;
    if ($pid == 0)
    {
        exec($cmd);
    }
    else
    {
        waitpid($pid,0);
        $rc = $?;
    }
    return $rc;
}

It uses the same queue file that the other scripts will write to, so think of this as a continuation of the previous articles. To use it, stick it somewhere like /usr/local/bin, and then change the locations to be wherever your queue is, and where you want your videos to end up. The transcoding settings can get a little hairy, so I would say don’t mess with the actual command lines unless you know what you’re doing (but, if you did, you probably wouldn’t need this script!)

External Utilities Needed:

HandBrakeCLI – this can be downloaded from here
mplayer/mencoder – This can be installed via apt-get

A couple of Notes/FAQs:

Overscan removal:
If you deal with letterboxed videos copied from TV, you’ll know what a pain overscan can be – Overscan is the generic term for the fuzzy line above the video that gets really annoying when watching on your monitor, but not on your TV. This can wreak havoc with detecting where to crop the video, so I remove it.

HandBrake *AND* mencoder?? Why?
I found that for some reason, Handbrake didn’t like working with the files that were copied from the TiVo – I dealt with lots of weird sync issues. I posted it on the Handbrake forum, but never got a solution. I ended up just using mencoder for the TiVo files, and Handbrake for the DVD rips, and left it at that.

Your setting XXX sucks – you should use YYY!
If you can state that for any type of program (animation, TV, letterboxed, HD, etc.) that this is correct (and that it doesn’t make the process take much longer), I’ll happily thank you and update the script (as well as give you a mention here.) I spent a bit of time trying to find settings that would work well for what I wanted, and ended up choosing these. They’re not going to have the best quality, but the size and the time to transcode were good, so that’s what I stuck with.

This guide will show you how to set up a script to copy programs from your Series 3 TiVo on a regular basis, without having to do a thing after it’s set up!

Information you will need:

• Your TiVo’s IP Address (needs to be static)
• Your TiVo’s Media Access Key (MAK) – this can be found on tivo.com after you’ve logged in
• The location where you want your files to be dumped (they are *big* – figure 1 GB per hour for SD, and like 5 GB per hour for HD)
• The location where the files will eventually be stored

The external utilities you will need:

• Curl – Install via sudo apt-get install curl
• Tivodecode – download from Sourceforge

You will also need to install the following CPAN perl modules:

• Net::TiVo
• XML::Simple
• Text::Unidecode

After you have all of that stuff installed and info gathered, it’s time to install my tivo_dump script:

#!/usr/bin/perl
# TiVo Dump
# Copy TiVo programs from a Series3 TiVo
# by Ed Salisbury (ed@edsalisbury.net)
# http://www.edsalisbury.net
# (c)2009 Ed Salisbury, Some Rights Reserved
#
# External Utilities Required:
# * curl
# * tivodecode
#
# External Perl Modules Required:
# * Net::TiVo
# * XML::Simple
# * Text::Unidecode;
#
# Usage:
# tivo_dump
#
# License:
# Except where otherwise noted, this work is licensed under Creative Commons
#   Attribution ShareAlike 3.0.
#
# You are free:
#   * to Share - to copy, distribute and transmit the work
#   * to Remix - to adapt the work
#
# Under the following conditions:
#   * Attribution. You must attribute the work in the manner specified by the
#     author or licensor (but not in any way that suggests that they endorse
#     you or your use of the work).
#   * Share Alike. If you alter, transform, or build upon this work, you may
#     distribute the resulting work only under the same, similar or a
#     compatible license.
#   * For any reuse or distribution, you must make clear to others the license
#     terms of this work. The best way to do this is with a link to the
#     license's web page (http://creativecommons.org/licenses/by-sa/3.0/)
#   * Any of the above conditions can be waived if you get permission from the
#     copyright holder.
#   * Nothing in this license impairs or restricts the author's moral rights.

use warnings;
use strict;
use Net::TiVo;
use XML::Simple;
use utf8;
use Text::Unidecode;

sub fix_chars($);

# User-Configurable Variables
my $HOST = "";      # Hostname/IP of the TiVo
my $MAK = "";       # The Media Access Key of the TiVo
my $VIDEO_DIR = ""; # Where the programs get saved
my $COPY_SUGGESTIONS = 0; # If you want to copy "Suggested" programs, set this to 1

my $USER = "tivo";
my $TMPFILE = "/tmp/$$.xml";
my $PROGRAMS_FILE = "/home/username/.tivo_programs";
$|++;

# External Utilities
my $CURL = "/usr/bin/curl";
my $TIVODECODE = "/usr/local/bin/tivodecode";

my @PREV;

# Connect to the TiVo
print "Connecting to TiVo at $HOST... ";
my $tivo = Net::TiVo->new(host => $HOST, mac => $MAK);
my @folders = $tivo->folders();
if (@folders)
{
    print "OKn";
}
else
{
    print "FAILn";
    exit();
}

# Load the file that has the previously saved programs
open(IN, $PROGRAMS_FILE);
while (<IN>)
{
    chop();
    push(@PREV, $_);
}
close (IN);

# Go through each folder on the TiVo
foreach my $folder (@folders)
{
    foreach my $item ($folder->{'xmlref'}{'Item'})
    {
        foreach my $video (@$item)
        {
            # Only process videos, not folders
            if ($video->{'Links'}{'Content'}{'ContentType'} eq "video/x-tivo-raw-tts")
            {
                # Choose video type based on the icon
                if ($video->{'Links'}{'CustomIcon'} && $video->{'Links'}{'CustomIcon'}{'Url'} eq "urn:tivo:image:suggestion-recording" && !$COPY_SUGGESTIONS)
                {
                    next;
                }
                if ($video->{'Links'}{'CustomIcon'} &&
                   ($video->{'Links'}{'CustomIcon'}{'Url'} eq "urn:tivo:image:in-progress-transfer" ||
                    $video->{'Links'}{'CustomIcon'}{'Url'} eq "urn:tivo:image:in-progress-recording"))
                {
                   next;
                }

                # Get program and episode titles
                my $program_title = $video->{'Details'}{'Title'};

                if ($video->{'Details'}{'EpisodeTitle'})
                {
                    $program_title .= " - " . $video->{'Details'}{'EpisodeTitle'};
                }

                # Get Program ID and Video URL
                my $video_url = $video->{'Links'}{'Content'}{'Url'};
                my $program_id = $video->{'Details'}{'ProgramId'};

                if (!$program_id)
                {
                    next;
                }

                # If previously copied, skip
                if (grep /^$program_id$/, @PREV)
                {
                    print "Skipping $program_title.n";
                    next;
                }

                # get details XML file
                print "Getting details for $program_title... ";
                my $details_xml = $video->{'Links'}{'TiVoVideoDetails'}{'Url'};

                system("$CURL --digest -s -k -u $USER:$MAK -c /tmp/cookies.txt -o $TMPFILE "$details_xml"");
                if (-f $TMPFILE)
                {
                    print "OKnProcessing details file... ";
                    my $xml = XML::Simple->new();
                    my $doc = $xml->XMLin($TMPFILE);
                    my %meta;
                    my $filepath;
                    my $filename;

                    # Get rating and convert to proper form
                    $meta{'tvRating'} = $doc->{'showing'}{'tvRating'}{'content'};
                    if ($meta{'tvRating'})
                    {
                        if ($meta{'tvRating'} eq "Y_7") { $meta{'tvRating'} = 'x1'; }
                        elsif ($meta{'tvRating'} eq "PG") { $meta{'tvRating'} = 'x4'; }
                        elsif ($meta{'tvRating'} eq "_14") { $meta{'tvRating'} = 'x5'; }
                        else { $meta{'tvRating'} = "x7"; }
                    }

                    # Get other data
                    $meta{'vActor'} = $doc->{'showing'}{'program'}{'vActor'}{'element'};
                    $meta{'vDirector'} = $doc->{'showing'}{'program'}{'vDirector'}{'element'};
                    $meta{'vProgramGenre'} = $doc->{'showing'}{'program'}{'vProgramGenre'}{'element'};
                    $meta{'vSeriesGenre'} = $doc->{'showing'}{'program'}{'series'}{'vSeriesGenre'}{'element'};
                    $meta{'seriesTitle'} = $doc->{'showing'}{'program'}{'series'}{'seriesTitle'};
                    $meta{'title'} = $doc->{'showing'}{'program'}{'title'};
                    $meta{'isEpisode'} = $doc->{'showing'}{'program'}{'isEpisode'};
                    $meta{'originalAirDate'} = $doc->{'showing'}{'program'}{'originalAirDate'};
                    $meta{'episodeTitle'} = $doc->{'showing'}{'program'}{'episodeTitle'};
                    $meta{'description'} = $doc->{'showing'}{'program'}{'description'};

                    if (defined $meta{'description'})
                    {
                        $meta{'description'} =~ s/s+Copyright.*$//;
                    }

                    # Process Titles

                    $meta{'episodeNumber'} = $doc->{'showing'}{'program'}{'episodeNumber'};

                    my $series_title = '';
                    my $episode_number = '';
                    my $episode_title = '';
                    my $title = '';

                    if ($meta{'seriesTitle'})
                    {
                        $series_title = fix_chars($meta{'seriesTitle'});
                    }
                    if ($meta{'episodeNumber'})
                    {
                        $episode_number = fix_chars($meta{'episodeNumber'});
                    }
                    if ($meta{'episodeTitle'})
                    {
                        $episode_title = fix_chars($meta{'episodeTitle'});
                    }
                    if ($meta{'title'})
                    {
                        $title = fix_chars($meta{'title'});
                    }

                    if ($series_title && $episode_number && $episode_title)
                    {
                        $filepath = "$VIDEO_DIR/$series_title";
                        $filename = "$filepath/$series_title - $episode_number - $episode_title";
                    }
                    elsif ($series_title && $episode_title)
                    {
                        $filepath = "$VIDEO_DIR/$series_title";
                        $filename = "$filepath/$series_title - $episode_title";
                    }
                    elsif ($title)
                    {
                        $filepath = "$VIDEO_DIR/$title";
                        $filename = "$filepath/$title";
                    }
                    else
                    {
                        $filepath = "$VIDEO_DIR";
                        $filename = "$filepath/Unknown";
                    }
                    print "OKn";

                    unless (-d $filepath)
                    {
                        print "Path $filepath doesn't exist, creating... ";
                        mkdir($filepath);
                        if (-d $filepath)
                        {
                            print "OKn";
                        }
                        else
                        {
                            print "FAILn";
                            exit;
                        }
                    }

                    # Get the video with curl
                    print "Getting video... ";
                    system("$CURL --digest -s -k -u $USER:$MAK -c /tmp/cookies.txt -o "$filename.tivo" "$video_url"");
                    if (-f "$filename.tivo")
                    {
                        my $filesize = (stat("$filename.tivo"))[7];

                        if ($filesize > 0)
                        {
                            print "OKn";

                            # Convert to MPG
                            print "Converting video to MPG format... ";
                            system("$TIVODECODE -m $MAK -o "$filename.mpg" "$filename.tivo" > /dev/null 2>&1");
                            if (-f "$filename.mpg")
                            {
                                print "OKn";
                                unlink "$filename.tivo";
                                open(OUT, ">>$PROGRAMS_FILE");
                                print OUT "$program_idn";
                                close(OUT);
                            }
                            else
                            {
                                print "FAILn";
                                exit();
                            }

                            # Output metadata file
                            print "Outputting metadata... ";
                            open (OUT, ">$filename.mpg.txt");

                            foreach my $key (keys %meta)
                            {
                                if ($meta{$key})
                                {
                                    if ($meta{$key} =~ /^ARRAY/)
                                    {
                                        foreach my $item (@{$meta{$key}})
                                        {
                                            unless ($item =~ /^HASH/)
                                            {
                                                print OUT "$key : " . fix_chars($item) . "n";
                                            }
                                        }
                                    }
                                    else
                                    {
                                        unless ($item =~ /^HASH/)
                                        {
                                            if ($key eq "originalAirDate")
                                            {
                                                print OUT "$key : " . $meta{$key} . "n";
                                            }
                                            else
                                            {
                                                print OUT "$key : " . fix_chars($meta{$key}) . "n";
                                            }
                                        }
                                    }
                                }
                            }
                            unlink($TMPFILE);
                            close(OUT);
                            print "OKn";
                        }
                        else
                        {
                            print "FAILn";
                        }
                    }
                    else
                    {
                        print "FAILn";
                        exit();
                    }
                }
                else
                {
                    print "FAILn";
                    exit();
                }
            }
        }
    }
}

# Convert any offending characters
sub fix_chars($)
{
    my ($data) = @_;

    $data = unidecode($data);

    $data =~ s/:/ -/g;
    $data =~ s///-/g;
    $data =~ s/\/-/g;
    $data =~ s/?/-/g;
    $data =~ s/*/-/g;

    return $data;
}

Put the script wherever you want it – (I usually put my scripts into /usr/local/bin) – Edit the script, and fill in the $HOST, $MAK, and $VIDEO_DIR variables with the data gathered in step 1. Then do a test run. It will take a while to copy the data. If it’s working fine, add to cron to have it copy the files regularly. I copy mine at midnight:

0 0 * * * /usr/local/bin/tivo_dump >/dev/null 2>&1

This says, run every day at midnight and send the output to /dev/null (otherwise you’ll get emails every day, which you may or may not want).

Note: Most of this script deals with metadata – it writes a text file (filename).txt with the metadata about the program, which is in pyTivo format. For more info, see my guide on how to set up pyTivo as a media server for TiVos.

I’ve been a perl programmer/scripter/hacker for many years now, and I will frequently use CPAN modules to be able to do the out-of-the ordinary. This small guide is intended to help people easily install these modules.

First, let’s get the CPAN module up to date:

$ sudo perl -MCPAN -e shell
(take defaults to set everything up)
cpan> exit
$ sudo perl -MCPAN -e shell
cpan> install Bundle::CPAN
(answer yes to the question about saving data)
cpan> reload cpan

Then, to install a module, it’s simply:
$ sudo perl -MCPAN -e shell
cpan> install <name of module>
cpan> exit

One of the issues I’ve found with having lots of data is the fact that I’m worried that a hard drive will fail, and I’ll lose something important. Since I did have that happen earlier this year, I am now determined to not let it happen again. I’ve been focusing a lot on resiliency – making it so that there are no single points of failure. Since I have been collecting a lot of hard drives, I decided to put them to good use, and set up some data replication onto multiple drives.

You might ask, why not just set up a RAID5? Or, alternatively, why not buy a Drobo and be done with it? I hear the first one a lot, especially since I deal with RAID day in and day out for my day job. The main issue I don’t do either of these things is that I’m cheap! I want to be able to use all of the hardware I have without tossing out old stuff. On my server, I have the following drives:

• disk1: 120 GB ATA
• disk2: 200 GB ATA
• disk3: 300 GB USB
• disk4: 400 GB Firewire
• disk5: 600 GB SATA
• disk6: 1000 GB USB
• disk7: 1500 GB USB

Try to set up a RAID with all of that! The problem with this is the disparate drive sizes (RAID requires disks to be the same size, and will use the lowest common denominator if not.) I’ve been ‘collecting’ hard drives for quite a while, and I like the fact that I can just go to the local electronics store and pick up another drive when I run out of space. This configuration does have a downside though, and that is, how do you decide where to put everything? I decided to sit down and figure out how much space I actually need for each of my data types, and came up with this list (taking approximate growth rates into consideration):

• Photos: 100 GB
• Music: 300 GB
• Movies: 1 TB
• TV: 400 GB
• Software: 150 GB

I then tried to map things out onto the drives I have:

• disk1: Photos
• disk2: Software
• disk3: Music
• disk4: TV
• disk6: Movies

This leaves two disks, disk5 (600 GB) and disk7 (1500 GB) left to be able to mirror to. Since I can’t use standard mirroring software, I’m not able to have a real-time mirror. This is OK for my purposes, as I don’t intend on having them be hot-swappable, etc. — I just want another copy of my data out there, without having to worry about making backups all the time.

I set up my server to only export the data disks (I don’t want to be able to write to the backups from my desktop, as the changes will be overwritten by the backups) I also set up my filesystem in a convenient manner — everything is a directory off of /data:

• /data/Photos
• /data/Video
• /data/Software
• /data/Audio

This makes it very easy to get to any of my data (try that in Windows!)

One inherent advantage of having an asynchronous mirror is the fact that it can serve as a temporary backup if you accidentally delete a file/directory, whereas with a normal mirror, you’d still have to back up to tape or some other drive to be able to get this data back.

To do the actual mirroring, I opted to create a simple rsync script that will copy the data nightly from cron. Nothing overly complex, but something that makes it easy to add new filesystems via a config file:

#!/usr/bin/perl
# Mirror
# Asynchronously mirror filesystems on a local machine
# by Ed Salisbury (ed@edsalisbury.net)
# http://www.edsalisbury.net
# (c)2009 Ed Salisbury, Some Rights Reserved
#
# External Utilities Required:
# * rsync
#
# License:
# Except where otherwise noted, this work is licensed under Creative Commons
#   Attribution ShareAlike 3.0.
#
# You are free:
#   * to Share — to copy, distribute and transmit the work
#   * to Remix — to adapt the work
#
# Under the following conditions:
#   * Attribution. You must attribute the work in the manner specified by the
#     author or licensor (but not in any way that suggests that they endorse
#     you or your use of the work).
#   * Share Alike. If you alter, transform, or build upon this work, you may
#     distribute the resulting work only under the same, similar or a
#     compatible license.
#   * For any reuse or distribution, you must make clear to others the license
#     terms of this work. The best way to do this is with a link to the
#     license's web page (http://creativecommons.org/licenses/by-sa/3.0/)
#   * Any of the above conditions can be waived if you get permission from the
#     copyright holder.
#   * Nothing in this license impairs or restricts the author's moral rights.

use warnings;
use strict;

# Configuration file for mirrors
# Example Config:
# /data/Photos/       /backup/Backup/Photos/
# /data/Software/     /backup/Backup/Software/
# /data/Audio/        /backup/Backup/Audio/
# /data/Video/        /backup/Backup/Video/
# /data/Misc/         /backup/Backup/Misc/
my $CONFIG = "/usr/local/etc/mirror.cfg";

open(CFG, $CONFIG);
my @lines = <CFG>;
close(CFG);

foreach my $line (@lines)
{
    my ($src, $dest) = split(/s+/, $line);
    system("/usr/bin/rsync -av --delete $src $dest");
}

I put this script into /usr/local/bin, and then created a config file /usr/local/etc/mirror.conf similar to this:

/data/Photos/       /backup/Backup/Photos/
/data/Software/     /backup/Backup/Software/
/data/Audio/        /backup/Backup/Audio/
/data/Video/        /backup/Backup/Video/
/data/Misc/         /backup/Backup/Misc/

I then added a cronjob to have it run nightly:

0 0 * * * /bin/backup > /dev/null 2>&1

A word of caution: Make *sure* you have the directories right! rsync is set up to *DELETE* files in the destination if the file isn’t in the source directory, so if you have it rsync to two different directories, it will delete files as needed in the destination to sync them up. You have been warned! I take no responsibility for you deleting your data!

Please let me know if you found this guide/script useful.

This guide will help you set up a media server to be able to play videos on your Series 3 TiVo, using pyTiVo. (Note: this post was moved from my old blog, but has been updated a bit)

First, install ffmpeg:

$ sudo apt-get install ffmpeg

Then, install git (needed to grab pyTivo):

$ sudo apt-get install git-core

Then, grab the latest version of the pyTivo script (Look at the Current Release page to verify that you have the right version):

$ cd /usr/share
$ sudo git clone git://repo.or.cz/pyTivo/wmcbrine.git
$ sudo mv wmcbrine pyTivo

Edit /usr/share/pyTivo/pyTivo.conf:

$ cd /usr/share/pyTivo
$ sudo cp pyTivo.conf.dist pyTivo.conf
$ sudo vi pyTivo.conf

The only thing that I changed is to comment out the default video share, and add my own:

[Movies]
type=video
path=/data/Video/Movies
[Television]
type=video
path=/data/Video/Television
[Music]
type=music
path=/data/Audio/Music

Run pyTivo:

$ sudo python /usr/share/pyTivo/pyTivo.py

Verify that things are working correctly — on the Tivo, go to “Now Playing List” and look for the shares.  If they appear and you can browse to them, you’re almost done!

One issue I had was with transferring the videos.  When I transferred them to the TiVo, it had a message of “unknown mpeg2 codec” or something to that effect.  Apparently the package for ffmpeg on Intrepid doesn’t have all of the codecs needed.  To install them, run the following:

$ sudo apt-get install libavcodec-unstripped-51

After I did this, videos were able to be transferred just fine.

The last thing to do is to make pyTiVo start automatically on bootup. To do this, simply copy the following script to /etc/init.d:

#!/bin/bash
# chkconfig: 2345 99 05
# description: pyTivo server

### INIT INFO
# Provides: pytivo
# Required-Start: $network
# Required-Stop: $network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-description: pyTivo server
# Description: Start and stop the pyTivo server.
### END INIT INFO

RETVAL=0

start() {
echo -n "Starting pyTivo: "
pgrep -f pyTivo.py
RETVAL=$?
[ $RETVAL -eq 0 ] && echo "pyTivo already running: Exiting" && exit 1

# this call actually starts pyTivo.
python /usr/share/pyTivo/pyTivo.py > /dev/null 2>&1 &
RETVAL=$?
[ $RETVAL -eq 0 ] && echo -n "done"
echo
return $RETVAL
}

stop() {
echo -n "Stopping pyTivo: "
pkill -f pyTivo.py
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && echo -n "done"
echo
return $RETVAL
}

# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
restart|reload)
stop
sleep 1
start
RETVAL=$?
;;
*)
echo "Usage: $0 {start|stop|restart}"
exit 1
esac
exit $RETVAL

Note: I didn’t write this – I just grabbed it from somewhere when setting it up – let me know if you wrote it so that I can give you credit!

After creating the script, make it executable:

$ sudo chmod u+x /etc/init.d/pytivo

Then create links to /etc/rc3.d and /etc/rc1.d:

$ sudo ln -s /etc/init.d/pytivo /etc/rc3.d/S99pytivo
$ sudo ln -s /etc/init.d/pytivo /etc/rc1.d/K99pytivo

This should make it so that pytivo starts automatically upon bootup.

Hopefully you’ve found this guide useful. Next time, I’ll be talking about how to make the videos show up perfectly on the TiVo, with appropriate titles, etc.