Linux Mint 12 + Gnome 3

I like it.

I've just installed the Mint 12 RC and I'm surprised how much I'm liking Gnome 3. It's different from 2 but not startlingly so -- feels tidier, simpler.

http://blog.linuxmint.com/?p=1858

Movies Are Cool

I've just finished my first day at Weta Digital as a Production Engineer. The group I'm working with help to keep the world spinning in the correct direction so the creative folks can do their thing.

Lot's of smart and interesting people. Tons of things I need to learn still but I don't feel out of step there at all.

Someone asked me today if the movies seem less magic now that I've had a peek behind the curtain. It's had just the opposite effect -- the effort, the energy and the creativity is massive. Looks like the best kind of magic to me.

The Python Standard Library by Example -- get it

My copy of The Python Standard Library by Example arrived this morning and it's totally kick-ass.

I don't much go for paper style books, especially chunky looking tech books but the python standard library ages pretty well and this is a collection of usage examples which is always nice to have at hand.

It turns out in this case the chunky book format is the perfect thing for thumbing through and discovering things that have been in the standard library all this time but you never noticed before. I've already spotted about 3 things I expect to use this week.

And it's written by Doug Hellman, the fellow who introduced me to python (via Zope even) on a project we worked on together a long time ago and it really saved the day. I've been loving python ever since.

Get it.
- from Amazon
- or bookdepository.co.uk (which tends to be cheaper where I am)

Flickr Backups - PicBackFlick, but really digital autonomy

I've been working on a Flickr backup utility and it's far enough along now for other people to give it a go.

See http://reedwade.net/picbackflick/
code at https://github.com/reedwade/PicBackFlick

It's written in python, is a command line script, is suitable for running via cron, has a handy locally browsable HTML photo viewer, etc.

It works well and there's more features it needs which I'll be adding over time but if you feel like lending a hand then please do.

--

I like using tools like gmail and flickr but I don't like the idea of depending on their good nature to keep my stuff safe. Keeping my own copy of the originals and meta info in a place I control is only sensible.

So, this takes care of photos, now I need similar tools for:
- gmail (and contacts and calendars)
- twitter
- blogger
- linkedin

HP Mini Mobile Drive, er USB connector

Laptop vendors get silly ideas sometimes. Some (all?) HP Minis have a slot for "HP Mini Mobile Drive" which is a recessed USB connector but machined to fit a proprietarily shaped USB stick. I recently picked up some cheap little zero profile mico SD adapters and they shove up in there just fine.

So, not sure this really helps as I was already using this in the sd slot w/an sd adapter. But, now that's freed up if I want to copy things off my camera easier.

And since it is recessed it can stay there mostly all the time w/out getting in the way.





Jeeze, I need to dust my gear, sorry.




Update: Yeah, the handle broke off and so the card is wedged down there permanently. Or until I decide to dig it out -- still works tho.

I work with awesome folk

As described here--

http://computerworld.co.nz/news.nsf/news/schools-in-for-open-source-advocates

I think it's cool we even rate comment trolls who don't approve of educating students. Must be doing something right.

Part Two: Koha Development Environment

This page has moved to the Koha Wiki @ http://wiki.koha-community.org/wiki/Koha_Dev_Env_Setup and will not be updated here.

This section describes how to wire in a Koha checkout on your desktop to the Koha installation you've just created. It assumes you've already followed the steps in “Part One: Setting Up Koha”.

Step: NFS Export Your Home Directory

It would be nice if you didn't need to copy files from machine to machine and could do your editing and other work on your desktop machine that has all the right dev tools in place. We use NFS to export your home directory to the virtual machine.

You may need to install the nfs-server package on your desktop if it's not already. If your desktop machine doesn't have an /etc/exports file you'll need to do that. Then:

desktop$ sudo vi /etc/exports

adding this line:
/home 192.168.122.0/255.255.255.0(rw,sync,no_subtree_check,no_root_squash)

then export the filesystem

desktop$ sudo exportfs -a

You should take some time to understand what you've just done. Any machine on the 192.168.122.0 subnet (the one used by QEMU/KVM) can do anything it wants to any file under /home on your desktop machine. You could be a little more strict about this by exporting just a single directory like /home/reed/koha-dev-dir and you could be more specific about exporting to just the single host. It's a balance of convenience and security. Different working environments will have different requirements.

Now mount the filesystem on kohabox.

kohabox$ sudo vi /etc/fstab
adding one line
desktop:/home/USER /home/USER nfs

then
kohabox$ sudo mount /home/USER

replacing USER with your username, now cd to your newly mounted home directory

kohabox$ cd ~/

You should see that your home directory is now on kohabox also; you should be able to edit files there.

After making changes like this I have a habit of rebooting the virtual machine or at least fully log out and back in. You've mounted a file system atop one you were using and it's just an odd situation to be in.

Step: Check Out Koha via Git

If you don't already have it. Install the git package. Git can change a bit from version to version. I'm using v1.7 and you probably want that too. If you are new to git but have experience with csv or svn you will find the usage model can be a bit confusing but it's useful and powerful enough that you'll eventually come to appreciate it.

desktop$ mkdir ~/koha-src

desktop$ cd ~/koha-src

desktop$ git clone git://git.koha-community.org/koha.git

You'll now have a checkout of Koha in ~/koha-src/koha/


Step: Point This Koha Instance At Your Development Code Checkout

Now we need to modify the apache configs so that instead of using the packaged Koha code, the system is pointing to your code checkout in ~/koha-src/koha/

You will be editing 3 files in /etc/koha/ (As always, save copies of the original files in case you need to roll back.)

In /etc/koha/apache-shared.conf change: (set USER as appropriate)
SetEnv PERL5LIB "/usr/share/koha/lib"
to
SetEnv PERL5LIB "/home/USER/koha-src/koha"

In /etc/koha/apache-shared-opac.conf :

• replace /usr/share/koha/opac/htdocs with /home/USER/koha-src/koha/koha-tmpl
• replace /usr/share/koha/opac/cgi-bin with /home/USER/koha-src/koha/

In /etc/koha/apache-shared-intranet.conf :

• replace /usr/share/koha/intranet/htdocs with /home/USER/koha-src/koha/koha-tmpl
• replace /usr/share/koha/intranet/cgi-bin with /home/USER/koha-src/koha/

Restart apache and confirm it works.

Now make some changes in the code and see them in action.

Step: Next Steps...

[ I need to add pointers to the wiki where it talks about about getting going with git and submitting patches. -reed ]

Setting up Koha in a Virtual Environment + Using That as a Development Environment

This page has moved to the Koha Wiki @ http://wiki.koha-community.org/wiki/Setting_Up_Koha_in_Virtual_Environment_+_Using_That_as_a_Development_Environment and will not be updated here.

This describes one way of setting up Koha in a virtual machine and how to configure that for use as a development environment. Virtual machines are used here to create an operating environment which is repeatable and well defined. It assumes a working knowledge of Linux systems though my intent is to describe things with enough detail that not much experience should be required.

I've repeated this process a couple of times so it should at least work -- but I would be grateful for comments, corrections and improvements.

This is aimed at development and experimental use. It may or may not work for a production environment depending upon your needs and situation.

This is written in two posts, the first one here gets you a working Koha installation. Part Two sets you up to do some development against it.




I use Nginx as an HTTP routing engine. Nginx is a lightweight, high performance web server which is close to my heart. It's used here to forward HTTP traffic into your Koha machine(s). It is optional if you only need to connect from your desktop machine.

My desktop development machine is running a recent version of Ubuntu and many of the instructions I provide here are specific to that environment. I'm using QEMU/KVM for the virtual host but this scheme works nicely with VirtualBox as well. (Ever since they tattooed the very large Oracle logo over the top of VirtualBox I've been stepping away from using that as much as possible.)

To help avoid confusion, I'll use the desktop$ prompt in examples below for things you should type from your desktop development machine and kohabox$ for the virtual machine running Koha.

Step: Create a Virtual Machine

Install QEMU/KVM and the Virtual Machine Manager. You can do this using the Ubuntu Package Manager. Select: virt-manager and qemu-kvm (I think that's all you need—let me know if not so I can update these instructions), then 'apply' to install. I had a problem with running this right after installation so a reboot at this point is probably a good idea.

Download the installation cdrom image (iso file) from which your new virtual machine will be built. I'm using Debian 6 (squeeze) which at this moment is still officially in testing. Go to http://www.debian.org/devel/debian-installer/ and get the i386 netinst CD image. Save it someplace on your machine.

Now, start up Virtual Machine Manager. Once you've installed it this will probably be under the System Tools menu on your desktop.



Create a new machine and name it kohabox.



Select the Debian CD image you just downloaded as the install media.






The Debian installation screen will appear. Select 'graphical install' and answer most of the questions using the defaults (except where it makes sense to diverge like when selecting your country and time zone). For the hostname, use kohabox. Set the passwords to something you will remember. When asked to create a new user, use the same user name you use on your desktop machine.



At the 'Partition disks' step there is a 'Write changes to disk?' question. The default is No for safety but that's not useful, answer Yes there.

When you get to the 'Software selection' page, add SSH server, unset Graphical desktop environment.



If you run into problems or enter the wrong things or forgot the passwords you gave it just shut off the virtual machine, delete it and start over. That's part of the fun of virtual machines.

Step: Static IP address for your new machine

(This section is specific to QEMU/KVM.)

Your machine will have been assigned an IP address via the QEMU/KVM DHCP service. It would be nice if that address is one that never changes. One way to do that is to configure the DHCP service to always give the same address to that host.



Shut down your virtual machine and then:

desktop$ sudo virsh net-edit default

change the section that looks like--

<dhcp>
<range start="192.168.122.2" end="192.168.122.254" />
</dhcp>

to--

<dhcp>
<range start="192.168.122.2" end="192.168.122.100" />
<host mac="52:54:00:e4:af:84" name="kohabox" ip="192.168.122.101" />
</host>

with 52:54:00:e4:af:84 being the MAC address which was generated for the network device on you new machine.

Now, active the new configuration:

desktop$ sudo virsh net-destroy default

desktop$ sudo virsh net-start default



Then, add an entry to /etc/hosts like this (and skipping ahead, we're adding a kohaadmin and koha entry to be used later):

desktop$ sudo vi /etc/hosts

192.168.122.101 kohabox koha kohaadmin


Start up kohabox and confirm it is now using the 192.168.122.101 address.

Confirm you can ssh to it from your desktop:

desktop$ ssh kohabox


(NB: I've tested this a few times and normally it just works. I did have one case where it's needed a restart or two before the new address took for some reason.)


Step: sudo and desktop hosts entry

sudo doesn't seem to be installed on the virtual machine, so do that:

kohabox$ su -

Password:

root@kohabox# apt-get install sudo

root@kohabox# vi /etc/group

and add your user to the sudo group

root@kohabox# vi /etc/hosts

and add an entry for the desktop host

192.168.122.1 desktop

You may need to log out completely from the virtual machine then back in for the new group membership to take effect (or at least open a new shell).

Step: Install Koha

We're going to simplify our lives in a big way now by installing Koha from packages. This will install the dependent packages for us and just makes sure our pipes are really clean before we start tampering with it for our development work.

You should review http://wiki.koha-community.org/wiki/Debian before starting.

kohabox$ sudo vi /etc/apt/sources.list

adding this line

deb http://debian.koha-community.org/koha squeeze-dev main


kohabox$ wget -O- http://debian.koha-community.org/koha/gpg.asc | sudo apt-key add -

kohabox$ sudo apt-get update

kohabox$ sudo apt-get install koha-common mysql-server


This step will take a while. Take the time to spare a loving thought for Lars Wirzenius who organised the package build and Robin Sheat who now maintains it (and even one for our employer, Catalyst, which funded that work).

When done, create (or replace) /etc/koha/koha-sites.conf.

kohabox$ sudo vi /etc/koha/koha-sites.conf

DOMAIN=""
INTRAPORT="80"
INTRAPREFIX=""
INTRASUFFIX="admin"
DEFAULTSQL="/usr/share/koha/defaults.sql.gz"

You may want to review /usr/share/doc/koha-common/README.Debian to understand where config files are landing.

The koha-common package sets up a system for creating multiple Koha instances on a single machine. It's just as useful for creating a single one which we will name “koha”.

kohabox$ sudo a2enmod rewrite

kohabox$ sudo a2dissite 000-default

kohabox$ sudo koha-create --create-db koha


Once those are done you should be able to see something at http://kohaadmin/ and http://koha/ from your desktop machine. The admin user will be koha_koha, find the password by looking in /etc/koha/sites/koha/koha-conf.xml


Step: Configuring Your Koha Instance



Go to http://kohaadmin/ and login in as user koha_koha. Step through the web installer selecting default values except:

MARC flavour, no default given, use Marc21 (dunno if that's preferred to unimarc, I picked it because it was listed first)

On the “MARC Frameworks: MARC21” setting page, set the 3 Optional items:

• marc21_default_matching_rules
• marc21_fastadd_framework
• marc21_simple_bib_frameworks

In the bottom Optional section enable:

• auth_val
• parameters
• sample_itemtypes
• sample_z3950_servers options

You may choose to select others but it can be educational and tidier not to. For the following example to work correctly you should at least set those I've listed. Leaving one or two out can give you a configuration which could be confusing/frustrating for someone new to Koha. (I'd be happy for someone with more knowledge of this to suggest a better starting configuration or a pointer to better information here--thanks.)

For the indexing engine, go ahead and select Zebra. It's a commonly used complicating factor so you'll want it if you plan to do any useful development work. Plus it's already going to be configured for from the package installation.

Ok, you're done. You should now have a working but not very interesting Koha installation.

Step: Add A Book (for non-librarians)

I am assuming you are a software developer but not a librarian. Also, that you don't really have a library other than a stack of books in your home. Adding a book sounds simple enough but it's important to know that you must first create a catalogue entry for the publication then, separately, create a record representing the physical instance of the book. Getting right with that was a key ah-ha moment for me anyway.

Quick detour, before adding books you'll want turn on bar code auto assigning. Go to System preferences and find autoBarcode under the Cataloguing section. Set it to something other than 'not generated automatically'.

Second detour, you need to create a library for your book record to have a home. Do this from the “basic parameters” configuration page.

Ok, so, imagine you have a book in your hand and you've got the ISBN which is printed on it somewhere.




In the Koha staff interface, select 'Add MARC Record' (under Cataloguing on the home page). Then hit the button marked “z39.50 Search”. A new window pops up with various search options. You'll also notice a list of search targets. These were added during the configuration step. Each is a publicly accessible searchable collection of catalogue records. At the moment you have none and need to copy one for each book you want to catalogue.

Enter 1400079764 for the ISBN and hit the Search button. You should get at least one result. Select “import”. The search window will close and the MARC Record page will be filled in with a catalogue entry for the book.

Select Save. You may get a pop up message saying some required field (CONTROL NUMBER IDENTIFIER) is not filled in. You will also see that it's got a candidate value now filled in for you. Hit Save again and you should be good.

You will now see an item record entry titled “Items for The Anchor book of modern Arabic fiction / (Record #1)”. Selecting the date and bar code fields will cause them to be filled in with useful values. Feel free to set and others or not then hit “Add Item”.

Sweet, you now have a book in your library.

Visit http://koha/ and search on part of the title “arabic”... well that's odd, no results (maybe)...

This is because the Zebra search indexer has not run. This is run from cron every 5 minutes. So, just wait a bit then try again.

Step: Configure Nginx

You can safely skip this step. Routing traffic through Nginx allows clients from outside your desktop machine to connect to your Koha machine. It also allows you a central place to manage HTTP traffic to your virtual machines.

desktop$ sudo apt-get install nginx

Now create a configuration for request forwarding.

Create a new file called /etc/nginx/sites-enabled/forwards and put this in it:

server {
listen 80;
server_name koha kohaadmin;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://kohabox/;
}
}

Edit your desktop /etc/hosts file changing:

192.168.122.101 kohabox koha kohaadmin

to

192.168.122.101 kohabox

127.0.0.1 koha kohaadmin



Now restart (or start) nginx:

desktop$ sudo /etc/init.d/nginx restart

With a web browser, visit http://kohaadmin/ and http://koha/ again. You should see no difference.

[ I need to add an example here of how you would get to Koha from a 3rd machine to make this actually useful. This will be a bit complex because to depends a lot on the local network environment so need to lay out a few scenarios. -reed ]