One thing that I cannot live without is a text editor, namely VIM.  VIM was first released as free and extended version of the VI program found on all Unix machines.  I first cut my teeth on VI when I started my sysadmin career, and switched to VIM a couple of years later.  VIM by default works mostly like VI, but i’ts highly configurable, and has a lot of power, especially when you start working with your .vimrc.  Since starting with the editor, I’ve carried my .vimrc from machine to machine, and company to company.  I’m continually updating it, and while there’s not a ton of stuff here, I thought I’d share it in case someone gets some value:

" Ed Salisbury's .vimrc
" Last Modified: 11/17/11

" Make VIM act like VIM, not VI
set nocompatible

" Set up TABs
set tabstop=4
set shiftwidth=4
set softtabstop=4
set expandtab
set smarttab
set autoindent
set smartindent

" Set up syntax highlighting
syntax on
filetype plugin on
filetype indent on
set formatoptions=qroc
set cindent

" Misc settings
set number       " Print line numbers
set background=dark
set pastetoggle=<F2>

I’d consider these settings a starting point for doing code/web development. Of course, there are tons of more complicated .vimrc files out there, but this should be good for starters. If you’re interested as to what each of these settings do, you can type :helpin VIM and it will display the help file.

Up until a couple of years ago, I used shared web hosting for serving up my various sites. I went through several of them because for whatever reason, they turned out to not be what I wanted. I finally came to the realization that I needed to get my own dedicated host. I found a good provider (The Planet), and started migrating over my websites (along with my friend Chris’ sites). Something I knew I would have to learn is how to set up Apache to be able to do virtual hosting. In this guide, I’ll show you how I did it, and the script I created to make things easier.

The first thing to do is to set up a logical directory structure — I hate putting web docs in some silly folder like /usr/local/apache2/htdocs — that’s always seemed like such a weird place to put things (I guess it goes against my feelings that anything in /usr shouldn’t be writable by users) Since this is a webserver, I wanted something closer to the root, like /www/<domain>. I also wanted to have a different directory structure and logging for each virtual host, I chose /www/logs/<domain>.

The next thing is to set up the apache config file to be able to include a separate file just for vhosts. Add this section somewhere in your httpd.conf file:

# Virtual Hosts
NameVirtualHost *
Include etc/httpd-vhosts.conf

Now, create httpd-vhosts.conf in the apache config dir, and create a manual entry to make sure it works:

    ServerName      www.mydomain.com
    ServerAdmin     webmaster@mydomain.com
    ServerAlias     mydomain.com
    DocumentRoot    /www/mydomain.com/docs
    DirectoryIndex  index.html index.php
    AccessFileName  .htaccess
    HostnameLookups Off
    ErrorLog        /www/logs/mydomain.com/error.log
    LogLevel        warn
    CustomLog       /www/logs/mydomain.com/access.log combined

(Be sure to change “mydomain.com” to your domain)

Create /www/<domain.com> and log files, as well as change ownerships properly:

# mkdir /www/logs/<domain.com>
# mkdir /www/<domain.com>
# chmod g+w /www/<domain.com>
# chown nobody:nobody /www/<domain.com>
# chown nobody:nobody /www/logs/<domain.com>

Next, create some sort of test file (like index.html) in the new doc directory.

When you’re ready, restart the webserver (usually just by running apachectl graceful) With any luck, you should be seeing the webpage you created when you go do www.<domain.com>. If not, look at the apache config for errors, and verify that the DNS entries are correct.

Now, as you were expecting, I have a script to be able to automate this. It’s pretty simple, but handy. I call it “add_vhost” and I put it into /usr/local/bin:

#!/usr/bin/perl -w
# AddVhost
# Add Virtual Host Configs for Apache
# by Ed Salisbury (ed@edsalisbury.net)
# http://www.edsalisbury.net
# (c)2009 Ed Salisbury, Some Rights Reserved
#
# License:
# Except where otherwise noted, this work is licensed under Creative Commons
#   Attribution ShareAlike 3.0.
#
# You are free:
#   * to Share -- to copy, distribute and transmit the work
#   * to Remix -- to adapt the work
#
# Under the following conditions:
#   * Attribution. You must attribute the work in the manner specified by the
#     author or licensor (but not in any way that suggests that they endorse
#     you or your use of the work).
#   * Share Alike. If you alter, transform, or build upon this work, you may
#     distribute the resulting work only under the same, similar or a
#     compatible license.
#   * For any reuse or distribution, you must make clear to others the license
#     terms of this work. The best way to do this is with a link to the
#     license's web page (http://creativecommons.org/licenses/by-sa/3.0/)
#   * Any of the above conditions can be waived if you get permission from the
#     copyright holder.
#   * Nothing in this license impairs or restricts the author's moral rights.

use strict;

my $DOCROOT = '/www';
my $LOGDIR = '/www/logs';
my $USER = 'nobody';
my $GROUP = 'nobody';
my $VHOST_CFG = "/usr/local/apache2/etc/httpd-vhosts.conf";

my $domain = $ARGV[0];

if (!$domain)
{
    print "Error: No domain specified!n";
    exit(1);
}
open (OUT, ">> $VHOST_CFG");
print OUT "n";
print OUT "<VirtualHost *>n";
print OUT "tServerName      www.$domainn";
print OUT "tServerAdmin     webmaster@$domainn";
print OUT "tServerAlias     $domainn";
print OUT "tDocumentRoot    $DOCROOT/$domainn";
print OUT "tDirectoryIndex  index.html index.phpn";
print OUT "tAccessFileName  .htaccessn";
print OUT "tHostnameLookups Offn";
print OUT "tErrorLog        $LOGDIR/$domain/error.logn";
print OUT "tLogLevel        warnn";
print OUT "tCustomLog       $LOGDIR/$domain/access.log combinedn";
print OUT "</VirtualHost>n";
close (OUT);

system("mkdir $LOGDIR/$domain");
system("mkdir $DOCROOT/$domain");
system("chmod g+w $DOCROOT/$domain");
system("chown $USER:$GROUP $DOCROOT/$domain");
system("chown $USER:$GROUP $LOGDIR/$domain");

Be sure to change the config variables to suit your environment. When you want to add a virtual host, simply run:

% sudo add_vhost domain.com

It will create the config and set everything up for you. All you need to do then is to restart the webserver, and you’re good to go!

I’ve been into emulation again lately, and I started with my first favorite system of all time, the Atari 8-bit computer. When I was growing up, I had 2 different 8-bit machines:

The Atari 400:

and later the Atari 130XE:

I did a lot of things on these machines, including learn BASIC, type in programs from magazines, and of course… play games! In order to play these games, I had a great joystick –

the Wico Command Control joystick:

My goal is to have as close as an experience as possible to the real thing with modern hardware, through emulation. To purists, this is considered sacrilege — they want the physical hardware, and emulation for them just doesn’t cut it. I guess I’m not a purist — I just like being able to play the old games! Since I’m now running Linux, the goal will be to get a working emulator going and everything working properly. Here’s what I mean by a proper emulator setup:

• Emulator should run in fullscreen mode, with no window elements showing
• 2 joysticks should work
• Everything should be able to be specified via the command line
• It should handle multiple disk programs as painless as possible
• It should have a standard method for exiting the emulator

The emulator I’ll be using for the Atari 8-Bit computers is atari800. In my opinion, it is the best emulator for the 8-bits, and will work for us just fine, with a few modifications. As a part of this project, I am also going to use the original joysticks that were used for the system.

Controllers:

For this project, I’m using the Wico Command Control joystick (see photo above). I used to use the Logitech gamepads that look like PS2 controllers, but after switching to the right controller, there’s no going back! The issue is, I need a way to convert them to USB to be able to use them on modern machines. After doing some searching, I found just the adapter, from a company called RetroZone. I bought the joystick and adapter off of eBay, and I couldn’t wait to plug it in and try it out! It seemed to work, but I noticed something odd – the fire button was being pressed every time I moved the joystick. I thought that it might be a linux thing, so I plugged it into my Windows laptop, and sure enough, it presses a different button every time I move the joystick. This wouldn’t be a problem normally, if the emulator I’m using didn’t allow you to use any button as the fire button. I have a fix for this issue though, which I’ll get to later.

Installation:

To install the standard version of the emulator (atari800), simply run the following (in Ubuntu):

% sudo apt-get install atari800

Then before you can actually run it, you’ll need a couple of things:

• ROM files
• Disk / Cartridge Images

The ROM files are dumps from the actual ROM chips found in the original hardware. There’s a few of them out there on the internet, and one place I’ve found that has them is TheOldComputer.com. All you really need are the following ROMs:

• Atari Rev B (400/800) NTSC
• Atari XL or XE NTSC

You’ll also need disk images or cartridge images. One cool site that I use frequently is called AtariMania.com. They have images and also artwork scans from the boxes/disks/manuals etc.

Now would be a good time to talk about how you’re planning on organizing your emulator directories. I personally use the following directory structure, but you’re welcome to use whatever you like:

/data/Software/Retro/Atari 8-Bit/roms
/data/Software/Retro/Atari 8-Bit/disks
/data/Software/Retro/Atari 8-Bit/carts
/data/Software/Retro/Atari 8-Bit/art_front
/data/Software/Retro/Atari 8-Bit/art_back
/data/Software/Retro/Atari 8-Bit/snap
/data/Software/Retro/Atari 8-Bit/titles

The last 4 directories are useful when running the emulator from a front-end, which I highly recommend. I’ll talk more about front-ends in a future blog posting.

To run the emulator, simply run the following:

% atari800

This will probably give you an error saying that it can’t find the ROM files. You will need to set up the configuration so that it can find your ROMs. To do this from within the emulator, hit F1, and go to “Emulator Configuration”. From there you can select the different ROMs and configure things how you like. To exit the emulator, press F9.

After it can find your roms, you’ll need to load a disk image. To do this from the command line, simply run:

% atari800 <name of the disk image>

(If there are spaces in the diskname, be sure to either put double quotes around the filename or use backslashes in front of the spaces or other special characters)

A couple of keys to get you started:

• F1 – Configuration Mode
• F2 – Option
• F3 – Select
• F4 – Start
• F5 – Reset
• F9 – Quit the Emulator

With any luck, you’re now able to play some games! A couple of issues you might be having: It’s windowed and small, and you want it to be fullscreen and big! It took a little bit of effort to get this going for me, but here’s how I achieved this. If you are running with more than one monitor, please see my guide on how to set up MetaModes to fix fullscreen issues.

First off, you will want to turn on NTSC mode. To do this, either do it in the emulator itself (hit F1, etc.) or by editing the config file. If you’re editing the config file, just set this line in ~/.atari800.cfg:

DEFAULT_TV_MODE=NTSC

If you’re like me, you want your gaming to be in fullscreen mode, not a window. One way of doing this is to create an alias that sets the resolution:

alias atari800='atari800 -width 1280 -height 1024'

Add this line to your .bashrc, and you’ll be in fullscreen gaming heaven! (Be sure to change the height and width as necessary for your monitor)

Customization:

At first, I just used the standard distribution version of atari800 – this worked well, but I had a couple of issues. First, the emulator treats every joystick button as the fire button. This was an issue with the joystick adapter I’m using, which sends a different button press for each direction (by design). Another issue I had was the fact that there needed to be an easy way to swap disks in a multi-disk set — I would like to be able to hit a function key for this.

I figured the easiest way to get these things fixed would be to compile from source. I started by downloading the source from CVS, and started working on compiling. (For my examples, I used version 2.1.0. To compile it, you’ll also need to get the SDL development libraries, which can easily by installed via Synaptic Package Manager (search for SDL). There’s lots of guides on compiling from source out there, but the simple method is:

cd <source dir>
./configure
make

Before I do any source code hacking, I always make sure that the source compiles first, that way I know it works before I start breaking stuff! Another thing to do is to always keep a backup copy of the source file you are editing. You want to be able to back out of any changes you make. On that note, let the hacking begin!

To fix the trigger issue, edit atari_sdl.c:
In the get_platform_TRIG() function, change the following lines:

int trig0, trig1, i;

to:

int trig0, trig1;

and:

for (i = 0; i < joystick0_nbuttons; i++) {
    if (SDL_JoystickGetButton(joystick0, i)) {
        trig0 = 0;
        break;
    }
}

to simply:

if (SDL_JoystickGetButton(joystick0, 0))
    trig0 = 0;

And do the same for joystick1 -- convert:

for (i = 0; i < joystick1_nbuttons; i++) {
    if (SDL_JoystickGetButton(joystick1, i)) {
        trig1 = 0;
        break;
    }
}

to:

if (SDL_JoystickGetButton(joystick1, 0))
    trig1 = 0;

To map the Rotate Disk function to F11, edit atari_sdl.c:

Add the following into the #include section:

#include "sio.h"

Then do a search for "F10", and add the following after the SDLK_F10 case:

case SDLK_F11:
    key_pressed = 0;
    SIO_RotateDisks();
    return AKEY_NONE;

After making either or both of these changes, run make again, and test. There's still one more change I'd like to make, but it will have to be at another time. I like to have the Escape key exit from all of my emulators. The idea is, any one should be able to run these, and having to remember which function key does what is a pain. I'll be sure to update this post when I make my change so that you can see how I did it.

Conclusion:

Hopefully this guide has been helpful to people interested in RetroGaming on Linux. Please feel free to share your experiences or issues -- I'm always interested to hear other people's thoughts and ideas. Have a system you'd like me to cover? Let me know and I'll do my best to accommodate you. I enjoy working with all gaming and computer platforms, albeit some of them are not quite as easy to set up.

One of the reasons people have a hard time making the jump to Linux is the fact that they have a hard time leaving the games they play in Windows. In this article, I’m going to describe some methods of getting around this hurdle, and make the transition to Linux easier. I’m not going to promise that your favorite game will work, but will give you the tools to be able to try. I’m also not going to be talking about native linux apps – those are pretty straightforward. This article will focus on Windows apps in Linux.

The first tool in the linux gamer’s arsenal is a utility called WINE. WINE stands for “Wine Is Not an Emulator”, in true unix style recursive acronyms. It’s job is to make your system look like Windows to your Win apps, giving users the ability to run programs, and for our purposes, games. The first step to running a windows program in linux is to install WINE (use apt-get / yum to install from the standard repositories), and then simply run ‘wine <name of the program.exe>’ – there’s a good chance that it will just run. I have noticed that for some reason game installers tend to be a little temperamental, so one way to get around this is to install on a Windows box and then copy the files to your unix machine over the network.

Another tool is called CrossOver Games, which is a commercially-supported version of WINE. If you don’t feel like messing with config files, this may be your best bet. I haven’t needed anything but the basic WINE installation, but I’ve heard good things about this product.

The games that I like are traditionally MMORPGs and emulated games. I’ve gotten both Star Wars Galaxies and World of Warcraft running, and Atari800, Stella, and MAME running so far. Also, I’ve gotten Spore to run. I really haven’t found the need to run much else in the gaming dept, so I’m set! The main issues that I’ve found with running games in linux are:

• Dual-Screen Issues
• Screen Resolution
• Keyboard Repeat Issues

Dual-Screen Issues

First off, I like to do gaming in full screen. I hate window elements on my game screen. The issue is when you have a dual monitor set up. The issue is, Xwindows handles resolutions differently than windows (one big screen vs. 2 smaller screens). This can be an issue since games will try to use the entire surface, and probably centering the content, making it completely unplayable (half on one screen, half on the other). In doing some searching, I found a solution to this problem — it’s called MetaModes. Note: I’m using an Nvidia chipset – not sure what to do if you’re using ATI. To enable MetaModes with TwinView, do the following:

# sudo cp /etc/X11/xorg.conf /etc/X11/xorg.conf.backup
# sudo vi /etc/X11/xorg.conf

Add the following line in the “Screen” Section:

Option "metamodes" "1280x1024,1280x1024; 1280x1024,NULL"

(This assumes you have 1280×1024 monitors — change as needed)

Restart your X server — I usually close all my apps, hit CTRL-ALT-F1, log in, and run:

# sudo /etc/init.d/gdm restart

This shouldn’t change anything with your normal desktop, but if you run a fullscreen app, it should only use the first monitor, and turn off the second. When the app is done, it will go back to the normal 2-screen layout. This setup works pretty well for me.

Screen Resolutions

One of the things you’ll find is that configuration apps written for windows won’t detect available resolutions correctly when running in Linux. Since most people run with LCDs lately (which look crap if you don’t run in the correct resolution), this can be a pain. Don’t expect the application to be able to find your ideal resolution. What I usually do is run the program and let it do it’s thing with the default res. Then I quit the program and look for a config file. Most games have a config file somewhere that states what resolution to run in. Edit the file manually and re-run the game. This should fix things so that it runs the way you want.

Keyboard Repeat Issues

Lately I’ve noticed an odd issue with keyboard repeating. It seems that there is an issue with the latest Xorg version and wine not playing nicely with keyboard repeats. If you do run into something like this, try disabling keyboard repeating from System->Preferences->Keyboard (in Ubuntu).

One really nice thing about gaming on Linux is the fact that you can keep your software on an NFS mounted filesystem. If you’re sharing an installation of WoW, for instance, only one person has to update it, and then it’s updated for everyone. It can take a little longer to load from the network, but I haven’t noticed many issues here. Another good thing here is the registry. While I usually try to stay away from games that use the registry, having it in linux is helpful because it’s not tied to one machine. You can put it on a remote drive, and it encapsulates the registry so that it could then run on other machines, making it a lot more portable.

Previously, I have showed you how to set up software to be able to rip DVDs to your hard drive, and I’ve also shown you how to copy your TiVo videos nightly. I’ve described the method I use to transcode the videos, but haven’t provided the script… Until now. After months of using and tweaking, here’s what I use. I call it VidProc.

#!/usr/bin/perl
# VidProc
# Transcode videos that have been either ripped by ripper or copied from a TiVo
#    to H.264 format
# by Ed Salisbury (ed@edsalisbury.net)
# http://www.edsalisbury.net
# (c)2009 Ed Salisbury, Some Rights Reserved
#
# External Utilities Required:
# * HandBrakeCLI
# * MPlayer/Mencoder
#
# Notes:
# * Settings for HandBrake and mencoder are what I have come up with after
#   doing a fair bit of research, and seem to work pretty well - if you have
#   any concrete suggestions on *better* general-purpose settings, let me know.
#
# License:
# Except where otherwise noted, this work is licensed under Creative Commons
#   Attribution ShareAlike 3.0.
#
# You are free:
#   * to Share -- to copy, distribute and transmit the work
#   * to Remix -- to adapt the work
#
# Under the following conditions:
#   * Attribution. You must attribute the work in the manner specified by the
#     author or licensor (but not in any way that suggests that they endorse
#     you or your use of the work).
#   * Share Alike. If you alter, transform, or build upon this work, you may
#     distribute the resulting work only under the same, similar or a
#     compatible license.
#   * For any reuse or distribution, you must make clear to others the license
#     terms of this work. The best way to do this is with a link to the
#     license's web page (http://creativecommons.org/licenses/by-sa/3.0/)
#   * Any of the above conditions can be waived if you get permission from the
#     copyright holder.
#   * Nothing in this license impairs or restricts the author's moral rights.

use warnings;
use strict;
use File::Copy;
use File::Basename;

sub system_int($);

# Locations
my $DEST = "/path/to/destdir";
my $QUEUE = "/path/to/queue.txt";

# Encoding Options
my $DEINT = "pp=md";    # Deinterlacer
my $VBITRATE = 1000;    # Video Bitrate
my $SAMPLE_RATE = 44.1; # Audio Sample Rate
my $ABITRATE = 192;     # Audio Bitrate
my $OVERSCAN = 6;       # Overscan size (# of lines)

# File Extensions
my $SRC_EXT = "mpg";
my $DEST_EXT = "mp4";

# External Utilities
my $HANDBRAKE = "/usr/local/bin/HandBrakeCLI";
my $MENCODER = "/usr/bin/mencoder";
my $MPLAYER = "/usr/bin/mplayer";

# Add backslashes to spaces
$DEST =~ s/ /\ /g;

while (1)
{
    # Read in the queue
    open(QUEUE, $QUEUE);
    my $line = <QUEUE>;
    my @fields;
    if ($line)
    {
        chomp ($line);
        @fields = split(/|/, $line);
    }
    else
    {
        print "Waiting for new stuff to show up in the queue...n";
        close(QUEUE);
        sleep(60);
        next;
    }
    close (QUEUE);

    my $input = $fields[0];

    print "Processing $inputn";

    # Fix spaces
    $input =~ s/ /\ /g;
    $input =~ s/'/\'/g;
    $input =~ s/&/\&/g;
    $input =~ s/;/\;/g;
    my @extlist = (".$SRC_EXT");

    my $base = basename($input, @extlist);

    if ($fields[1])
    {
        # DVD, since it has a title number
        my $title = $fields[1];
        my $output = "$DEST/${base}_" . $fields[1] . ".$DEST_EXT";

        system_int("$HANDBRAKE --input $input --output $output --title $title --turbo --encoder x264 --vb $VBITRATE --audio 1 --aencoder faac --mixdown 6ch --arate $SAMPLE_RATE --ab $ABITRATE --detelecine --decomb --loosePixelratio --markers --two-pass --x264opts ref=3:mixed-refs:bframes=6:weightb:direct=auto:b-pyramid:me=umh:subme=9:analyse=all:8x8dct:trellis=1:nr=150:no-fast-pskip=1:psy-rd=1,1");
    }
    else
    {
        # TiVo file, since it has no title number
        my $output = "$DEST/${base}.$DEST_EXT";
        my $line;
        my $width;
        my $height;

        # Get video dimensions
        print "Getting video information... ";
        my @id = `$MPLAYER -nojoystick -nolirc -vo null -ao null -identify -frames 0 $input 2>&1`;
        foreach (@id)
        {
            if (/ID_VIDEO_WIDTH=(d+)/)
            {
                $width = $1;
            }
            if (/ID_VIDEO_HEIGHT=(d+)/)
            {
                $height = $1;
            }
        }
        if (!$width || !$height)
        {
            print "FAILED!";
        }
        else
        {
            print "(${width}x$height)n";
        }

        # Crop overscan
        $height-=$OVERSCAN;

        my %croptest;

        # Detect crop region
        print "Detecting crop region... ";
        my @cropdetect = `$MPLAYER -nojoystick -nolirc -vo null -ao null -vf crop=$width:$height:0:$OVERSCAN,cropdetect -ss 600 -endpos 180 $input 2>&1`;
        foreach $line (@cropdetect)
        {
            if ($line =~ /-vf crop=([d:]+)/)
            {
                $croptest{$1}++;
            }
        }

        # Get the crop value with the most hits
        my @croplist = sort { $croptest{$b} <=> $croptest{$a} } keys (%croptest);
        my $crop = $croplist[0];

        if (!$crop)
        {
            print "FAILEDn";
            exit();
        }
        print "($crop)n";
        print '-' x 80 . "n";

        print "Encoding first passn";
        print '-' x 80 . "n";
        system_int("$MENCODER -ovc x264 -x264encopts pass=1:turbo:bitrate=$VBITRATE:bframes=1:me=umh:partitions=all:trellis=1:qp_step=4:qcomp=0.7:direct_pred=auto:keyint=300 -vf $DEINT,crop=$crop,scale=-1:-10,harddup -oac copy -ofps 30000/1001 $input -o /dev/null");

        print "nn";
        print '-' x 80 . "n";
        print "Encoding second passn";
        print '-' x 80 . "n";
        system_int("$MENCODER -ovc x264 -x264encopts pass=2:turbo:bitrate=$VBITRATE:bframes=1:me=umh:partitions=all:trellis=1:qp_step=4:qcomp=0.7:direct_pred=auto:keyint=300 -vf $DEINT,crop=$crop,scale=-1:-10:,harddup -oac copy -ofps 30000/1001 $input -o $output");
    }
    print "Done. Removing from queuen";
    print "nn";
    open(IN, $QUEUE);
    open(OUT, ">$QUEUE.tmp");
    while (my $qline = <IN>)
    {
        chomp ($qline);
        unless ($line eq $qline)
        {
            print OUT "$qlinen";
        }
    }
    close (IN);
    close (OUT);
    unlink($QUEUE);
    move("$QUEUE.tmp", $QUEUE);
}

# Interruptible System Command
sub system_int($)
{
    my ($cmd) = @_;
    my $pid = fork();
    my $rc;
    if ($pid == 0)
    {
        exec($cmd);
    }
    else
    {
        waitpid($pid,0);
        $rc = $?;
    }
    return $rc;
}

It uses the same queue file that the other scripts will write to, so think of this as a continuation of the previous articles. To use it, stick it somewhere like /usr/local/bin, and then change the locations to be wherever your queue is, and where you want your videos to end up. The transcoding settings can get a little hairy, so I would say don’t mess with the actual command lines unless you know what you’re doing (but, if you did, you probably wouldn’t need this script!)

External Utilities Needed:

HandBrakeCLI – this can be downloaded from here
mplayer/mencoder – This can be installed via apt-get

A couple of Notes/FAQs:

Overscan removal:
If you deal with letterboxed videos copied from TV, you’ll know what a pain overscan can be – Overscan is the generic term for the fuzzy line above the video that gets really annoying when watching on your monitor, but not on your TV. This can wreak havoc with detecting where to crop the video, so I remove it.

HandBrake *AND* mencoder?? Why?
I found that for some reason, Handbrake didn’t like working with the files that were copied from the TiVo – I dealt with lots of weird sync issues. I posted it on the Handbrake forum, but never got a solution. I ended up just using mencoder for the TiVo files, and Handbrake for the DVD rips, and left it at that.

Your setting XXX sucks – you should use YYY!
If you can state that for any type of program (animation, TV, letterboxed, HD, etc.) that this is correct (and that it doesn’t make the process take much longer), I’ll happily thank you and update the script (as well as give you a mention here.) I spent a bit of time trying to find settings that would work well for what I wanted, and ended up choosing these. They’re not going to have the best quality, but the size and the time to transcode were good, so that’s what I stuck with.

I’ve been a perl programmer/scripter/hacker for many years now, and I will frequently use CPAN modules to be able to do the out-of-the ordinary. This small guide is intended to help people easily install these modules.

First, let’s get the CPAN module up to date:

$ sudo perl -MCPAN -e shell
(take defaults to set everything up)
cpan> exit
$ sudo perl -MCPAN -e shell
cpan> install Bundle::CPAN
(answer yes to the question about saving data)
cpan> reload cpan

Then, to install a module, it’s simply:
$ sudo perl -MCPAN -e shell
cpan> install <name of module>
cpan> exit

This guide will help you set up a media server to be able to play videos on your Series 3 TiVo, using pyTiVo. (Note: this post was moved from my old blog, but has been updated a bit)

First, install ffmpeg:

$ sudo apt-get install ffmpeg

Then, install git (needed to grab pyTivo):

$ sudo apt-get install git-core

Then, grab the latest version of the pyTivo script (Look at the Current Release page to verify that you have the right version):

$ cd /usr/share
$ sudo git clone git://repo.or.cz/pyTivo/wmcbrine.git
$ sudo mv wmcbrine pyTivo

Edit /usr/share/pyTivo/pyTivo.conf:

$ cd /usr/share/pyTivo
$ sudo cp pyTivo.conf.dist pyTivo.conf
$ sudo vi pyTivo.conf

The only thing that I changed is to comment out the default video share, and add my own:

[Movies]
type=video
path=/data/Video/Movies
[Television]
type=video
path=/data/Video/Television
[Music]
type=music
path=/data/Audio/Music

Run pyTivo:

$ sudo python /usr/share/pyTivo/pyTivo.py

Verify that things are working correctly — on the Tivo, go to “Now Playing List” and look for the shares.  If they appear and you can browse to them, you’re almost done!

One issue I had was with transferring the videos.  When I transferred them to the TiVo, it had a message of “unknown mpeg2 codec” or something to that effect.  Apparently the package for ffmpeg on Intrepid doesn’t have all of the codecs needed.  To install them, run the following:

$ sudo apt-get install libavcodec-unstripped-51

After I did this, videos were able to be transferred just fine.

The last thing to do is to make pyTiVo start automatically on bootup. To do this, simply copy the following script to /etc/init.d:

#!/bin/bash
# chkconfig: 2345 99 05
# description: pyTivo server

### INIT INFO
# Provides: pytivo
# Required-Start: $network
# Required-Stop: $network
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-description: pyTivo server
# Description: Start and stop the pyTivo server.
### END INIT INFO

RETVAL=0

start() {
echo -n "Starting pyTivo: "
pgrep -f pyTivo.py
RETVAL=$?
[ $RETVAL -eq 0 ] && echo "pyTivo already running: Exiting" && exit 1

# this call actually starts pyTivo.
python /usr/share/pyTivo/pyTivo.py > /dev/null 2>&1 &
RETVAL=$?
[ $RETVAL -eq 0 ] && echo -n "done"
echo
return $RETVAL
}

stop() {
echo -n "Stopping pyTivo: "
pkill -f pyTivo.py
RETVAL=$?
echo
[ $RETVAL -eq 0 ] && echo -n "done"
echo
return $RETVAL
}

# See how we were called.
case "$1" in
start)
start
;;
stop)
stop
;;
restart|reload)
stop
sleep 1
start
RETVAL=$?
;;
*)
echo "Usage: $0 {start|stop|restart}"
exit 1
esac
exit $RETVAL

Note: I didn’t write this – I just grabbed it from somewhere when setting it up – let me know if you wrote it so that I can give you credit!

After creating the script, make it executable:

$ sudo chmod u+x /etc/init.d/pytivo

Then create links to /etc/rc3.d and /etc/rc1.d:

$ sudo ln -s /etc/init.d/pytivo /etc/rc3.d/S99pytivo
$ sudo ln -s /etc/init.d/pytivo /etc/rc1.d/K99pytivo

This should make it so that pytivo starts automatically upon bootup.

Hopefully you’ve found this guide useful. Next time, I’ll be talking about how to make the videos show up perfectly on the TiVo, with appropriate titles, etc.

The Issue:
You have external USB drives that you want to be mounted the same way every time, but the /dev/sdX device name keeps changing.

The Solution:
First, unmount the drive:
$ sudo umount /media/disk

Then, look in /dev/disk/by-id to find the correct device file:
/dev/disk/by-id/usb-WDC_WD30_00JB-00KFA0_DEF107679C83-0:0-part1

Add this to /etc/fstab:
/dev/disk/by-id/usb-WDC_WD30_00JB-00KFA0_DEF107679C83-0:0-part1 /mnt/data01 ext3 defaults 0 0

Create the mountpoint:
$ sudo mkdir /mnt/data01

Mount the volume:
$ sudo mount /mnt/data01

Now, this device should always be mounted on /mnt/data01.

One thing that I really enjoy is to be able to watch my DVDs from any computer in the house, or from (almost) any of the TVs, without having to deal with physical media.  The way I do this is to “rip” the movies to my media server, and then transcode them to a smaller format.  This guide will discuss part one of this — ripping a DVD.  I know there are several utilities which will do this for me, but since I have very specific needs, and like things to be as automated as possible, I wrote a script to do this (naturally).


In order to use this script, you will need to install a few things:

• lsdvd
• vobcopy
• libdvdcss2

The first two are easy – simply:

% sudo apt-get install lsdvd vobcopy

For the CSS library, you’ll need to first add medibuntu sources:

% sudo wget http://www.medibuntu.org/sources.list.d/jaunty.list --output-document=/etc/apt/sources.list.d/medibuntu.list
% sudo apt-get update
% sudo apt-get install medibuntu-keyring
% sudo apt-get update

Then install libdvdcss2:

% sudo apt-get install libdvdcss2

The next thing will be to download this script, and change $VOBDIR and $QUEUE according to where you want to put the VOB files and which text file to write the list of titles to transcode.

Here’s some examples on how to use it:

To rip a movie, choosing the longest title:

% ripper -n "My Movie"

To rip a movie, specifying the title number:

% ripper -n "My Movie" -t 5

To rip a TV Show disc – choosing only titles that are 20 minutes or longer:

% ripper -n "My TV Show" -m 20

Note: This script doesn’t convert from VOB files to anything else, it just copies the contents of the DVD, and writes to a queue file. Transcoding to other formats will come soon.

#!/usr/bin/perl
# DVD Ripper
# Uses vobcopy to rip DVDs
# by Ed Salisbury (ed@edsalisbury.net)
# http://www.edsalisbury.net
# (c)2009 Ed Salisbury, Some Rights Reserved
#
# External Utilities Required:
# * lsdvd
# * vobcopy
# * libdvdcss2 library
#
#
# Usage:
# ripper -n <name> -t <title num> -l -m <minlength>
#        -t: Title Number
#        -l: Longest title (default)
#        -m: Minimum title length (in minutes)
#
#
# Examples:
# Rip a movie, choosing the longest title
# % ripper -n "My Movie"
#
# Rip a movie, specifying the title number
# % ripper -n "My Movie" -t 5
#
# Rip a TV Show disc - choosing only titles that are 20 minutes or longer
# % ripper -n "My TV Show" -m 20
#
#
# Limitations:
# * Will not rip some DVDs (limitation of vobcopy)
# * Will not transcode the VOB files
#
#
# License:
# Except where otherwise noted, this work is licensed under Creative Commons
#   Attribution ShareAlike 3.0.
#
# You are free:
#   * to Share — to copy, distribute and transmit the work
#   * to Remix — to adapt the work
#
# Under the following conditions:
#   * Attribution. You must attribute the work in the manner specified by the
#     author or licensor (but not in any way that suggests that they endorse
#     you or your use of the work).
#   * Share Alike. If you alter, transform, or build upon this work, you may
#     distribute the resulting work only under the same, similar or a
#     compatible license.
#   * For any reuse or distribution, you must make clear to others the license
#     terms of this work. The best way to do this is with a link to the
#     license's web page (http://creativecommons.org/licenses/by-sa/3.0/)
#   * Any of the above conditions can be waived if you get permission from the
#     copyright holder.
#   * Nothing in this license impairs or restricts the author's moral rights.

use warnings;
use strict;
use Getopt::Std;
use File::Copy;
$|++;

sub usage();

# Where to store the VOB files
my $VOBDIR = "/data/Video/Rip";
# Queue for transcoding (not done by this script)
my $QUEUE = "/data/Video/queue.txt";

my $VERSION = "1.1";
my $LONGEST;

my $title_num = 0;
my $title_name;
my $min_time;
my $name;
my $mounted = '';
my $count = 0;
my $cmd;
my %opt = ();

# External Utilities
my $VOBCOPY = '/usr/bin/vobcopy';
my $LSDVD = '/usr/bin/lsdvd';
my $MOUNT = '/bin/mount';
my $GREP = '/bin/grep';
my $EJECT = '/usr/bin/eject';

getopts("v?n:t:lm:", %opt);

if ($opt{'?'})
{
    usage();
}

if ($opt{'v'})
{
    print "DVD Ripper ver. $VERSION by Ed Salisbury <ed@edsalisbury.net>n";
    exit();
}

# Check for installed external utilities
unless (-f $VOBCOPY)
{
    print "ERROR: vobcopy not installedn";
    exit(1);
}

unless (-f $LSDVD)
{
    print "ERROR: lsdvd not installedn";
    exit(1);
}

# Get Name
if ($opt{'n'})
{
    $name = $opt{'n'};
}
else
{
    print "Error: No name specifiedn";
    usage();
}

if ($opt{'l'})
{
    $LONGEST++;
}

if ($opt{'m'})
{
    $min_time = $opt{'m'};
}

if ($opt{'t'})
{
    $title_num = $opt{'t'};
}

if (!$min_time && !$title_num)
{
    $LONGEST++;
}

if ($LONGEST)
{
    print "Using longest title.n";
}

# Check for mounted DVD
print "Checking for DVD-ROM...";
while (!$mounted)
{
    $mounted = `$MOUNT | $GREP cdrom`;
    if (!$mounted)
    {
        if ($count < 20)
        {
            print ".";
            sleep(1);
            $count++;
        }
        else
        {
            print " not found.n";
            exit();
        }
    }
    else
    {
        print " found.n";
    }
}

my @titles_to_xcode;

if ($title_num)
{
    push (@titles_to_xcode, $title_num);
}
else
{
    print "Getting DVD title info... ";
    $cmd = "$LSDVD 2>/dev/null";
    my @lsdvd = `$cmd`;

    foreach my $line (@lsdvd)
    {
        if ($line =~ /^Disc Title: (.*?)$/)
        {
            $title_name = $1;
        }

        if ($min_time && $line =~ /^Title: (d+), Length: (d+):(d+):/)
        {
            my $length = ($1 * 60) + $2;
            if ($length > $min_time)
            {
                push (@titles_to_xcode, $1);
            }
        }

        if ($LONGEST && $line =~ /^Longest track: (d+)$/)
        {
            push (@titles_to_xcode, $1);
        }
    }
    print "done.n";
}

print "Ripping...n";
$cmd = "$VOBCOPY -m -o $VOBDIR -F64";
system($cmd);
print "done.n";

print "Moving to proper location... ";
move("$VOBDIR/$title_name/VIDEO_TS", "$VOBDIR/$name");
rmdir("$VOBDIR/$title_name");
print "done.n";

print "Updating queue... ";
open (QUEUE, ">>$QUEUE");
foreach my $title (@titles_to_xcode)
{
    print QUEUE "$VOBDIR/$name|$titlen";
}
close (QUEUE);
print "done.n";

print "Ejecting DVD-ROM... ";
system("$EJECT");
print "done.n";

sub usage()
{
    print "ripper -n <name> -t <title num> -l -m <minlength>n";
    print "t-t: Title Numbern";
    print "t-l: Longest title (default)n";
    print "t-m: Minimum title length (in minutes)n";
    exit();
}

Soon I’ll post my scripts for transcoding DVDs to H.264. Please let me know if you find this useful or have any comments/suggestions.

Update: I have posted a new guide and script for processing the DVD rips here.

After my hard drive crash a few months ago, I’ve been a bit paranoid about losing data.   By “a bit” I mean I’ve been sticking copies of my data on different machines to make sure I have various copies in case another hard drive dies, or a computer gets fried, melting everything inside, or my house burns down… Oh wait, I don’t have that scenario covered….  Time to fix that!  What I want is the ability to copy my data to an offsite location, and keep the backup up to date.  The main thing I care about is my family photos.   I was talking to my cousin Dave, and he had the same issue.  He wanted access to my photos, as well as have an offsite backup copy of his family’s photos.  I thought of a solution.  What if we could set up a two-way mirror between his machine and mine, and keep it updated nightly?  That way if he sticks something in the directory on his machine and vice versa, keeping a live backup.  I began doing some research, and I found the tool — Unison.

One slight wrinkle is the fact that he runs Mac OS X and I run Ubuntu Linux.  Unison will run on both, so it should be fine.

A couple of things to consider:

Mirroring over the internet is SLOW — if you want to keep a large quantity of files mirrored, it’s best to “seed” the mirror by doing a local copy to the drive.  Dave sent me his hard drive via snailmail, so this worked out.

This process will involve poking holes in your firewall, and setting up SSH keys with no password.  These are security risks (which can be minimized).  You have been warned.

To make things easy, I’ll specify the hostnames:

Mac Host: eve

Linux Host: twiki

One thing needs to be decided before we go on is which host will be initiating the transfer.   You can initiate from both sides, but to make it easy, I’ll just be initiating from my computer, twiki.

The first step is to verify connectivity between the two machines.  Since I’m initiating the transfer, I’ll be adding a hosts entry for eve.  I first have Dave go to www.whatismyip.com and have him tell me his IP address.  I put this into /etc/hosts on my machine:

<hismachinesipaddress>        eve

I try to first ping the address.  Most likely it will fail.  Dave will need to poke a hole in the firewall (set up port forwarding) – How to do this is outside the scope of this article, but www.portforward.com has a lot of info on how to do this.  Dave forwards port 22 (SSH) to his local IP address and saves settings.

Ping still doesn’t work, but SSH should.  I try sshing to the box, and voila, I get a password prompt.

The next step is to have Dave create an account for me.  To make it easy, I have him call it “ed” (same as on twiki)

After my account has been created, I’m able to login via ssh.  The next step is to set up an SSH key to make it so that I can ssh in without having to give a password.  On twiki, I run:

twiki% ssh-keygen -t dsa -f ~/.ssh/unison

(Press enter when it asks for a passphrase)

Add the contents of ~/.ssh/unison.pub to ~/.ssh/authorized_keys2 on eve.  Make sure it’s all on one line (no linebreaks)

To test it out, I ssh from twiki to eve, specifying the unison key:

twiki% ssh -i ~/.ssh/unison eve

It shouldn’t ask me for a password, so I’m good to go for this step.  If it does, make sure the permissions on the files in ~/.ssh are set to 600.

Next, we’ll install Unison on both machines:

On Eve:

• Get and install MacPorts
• Use MacPorts to install Unison:

eve% sudo /opt/local/bin/port install unison
--->  Fetching ocaml
--->  Attempting to fetch ocaml-3.11.0.tar.bz2 from http://distfiles.macports.org/ocaml
--->  Verifying checksum(s) for ocaml
--->  Extracting ocaml
--->  Configuring ocaml
--->  Building ocaml
--->  Staging ocaml into destroot
--->  Installing ocaml @3.11.0_0
--->  Activating ocaml @3.11.0_0
--->  Cleaning ocaml
--->  Fetching unison
--->  Attempting to fetch unison-2.27.57.tar.gz from http://distfiles.macports.org/unison
--->  Verifying checksum(s) for unison
--->  Extracting unison
--->  Applying patches to unison
--->  Configuring unison
--->  Building unison
--->  Staging unison into destroot
--->  Installing unison @2.27.57_0
--->  Activating unison @2.27.57_0
--->  Cleaning unison

On Twiki:

twiki% sudo apt-get install unison
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
unison
0 upgraded, 1 newly installed, 0 to remove and 11 not upgraded.
Need to get 0B/529kB of archives.
After this operation, 1262kB of additional disk space will be used.
(Reading database ... 150151 files and directories currently installed.)
Unpacking unison (from .../unison_2.27.57-1ubuntu1_i386.deb) ...
Processing triggers for man-db ...
Processing triggers for doc-base ...
Processing 1 added doc-base file(s)...
Registering documents with scrollkeeper...
Setting up unison (2.27.57-1ubuntu1) ...

Next, set up the profile on twiki:

twiki% mkdir ~/.unison
twiki% vi ~/.unison/default.prf

Put the following into default.prf (changing as appropriate):

root = /data/EdsPhotos
root = ssh://eve//DavesPhotos
rshargs = -C
sshargs = -i /home/ed/.ssh/unison
servercmd = /opt/local/bin/unison
ignore = Name _*
ignore = Name .*
batch = true
log = true

For security purposes, add the following to ~/.ssh/authorized_keys2 on eve.  This will only allow this ssh key to be used to run unison:

command="/opt/local/bin/unison",no-port-forwarding,no-X11-forwarding,no-agent-forwarding

On twiki, run unison:

twiki% unison

It will spit out a lot of data, but should start scanning and eventually copying data back and forth, depending on how much data there is.   In order to run nightly, add the following to your crontab on twiki:

0 0 * * * nohup /usr/local/bin/unison > /dev/null 2>&1

That should be it.  Any feedback would be appreciated.