Transition to Linux and Ubuntu: The Experience of a Quantitative Social Scientist

This post discusses my experience migrating to Ubuntu from Windows from the perspective of a quantitatively oriented researcher in psychology. It discusses (a) my general transition to open source tools for doing research; (b) examination of Cygwin; (c) choosing a Linux distribution; (d) preparing for installation of Ubuntu; (e) initial reflections on installing Ubuntu; and (f) a long list of various solutions to specific problems that I encountered whilst getting started with Ubuntu.

Context of transition

Anyone following this blog for a while will know that I've been progressively migrating to using open source software for doing research. My first foray into open source started in 2007 when I decided to replace SPSS with R.

As is the case with many open source tools, they have a habit of mutually reinforcing each other. The path went something like this (with links pointing to associated blog posts):

• LaTeX replaced Word as my scientific document format.
• Markdown replaced Word as my document format for note taking and informal documents.
• Bibtex, Jabref, and other tools replaced Endnote as a bibliographic reference management system.
• Vim replaced a host of other IDEs, and text editors; it even replaced MS Word, in that it became the editor for the plain text file formats (e.g., LaTeX, Sweave, Markdown, etc.) that I use the most.
• git replaced ad hoc version control systems.
• make replaced various manual processes for building final products.

For a general summary of this workflow (albeit, prior to the adoption of Vim), see this video that I gave at Melbourne R Users in late 2010.

However, I still needed to cut ties from Windows, the operating system I'd been using for 15 years. I'd been putting off adopting Linux for several reasons. I was concerned about:

• hardware support for my main computer,
• the temporary drop in performance that results from adopting a new operating system,
• difficulties collaborating with other researchers operating in the Windows world,
• lack of support for various key programs (e.g., Outlook; Inquisit), and
• the possibility that some tasks would be less efficient under Linux.

Despite these concerns, I decided to finally dive in.

Initial foray into Cygwin

About a month prior to making the switch I'd installed Cygwin on Windows. This gave me access to many Linux command-line programs. However, ultimately, while Cygwin whet my appetite, it failed to satisfy.

• By default, Cygwin does not always play nicely with non-Cygwin programs. File paths often get confused. If it can't play nicely with Windows programs, I might as well go all the way.
• There are issues of where to store files (Cygwin home or Windows user home).
• Some Linux programs are not included.
• The speed of the terminal and shell was often poor.
• In general, interacting with other programs was a frustrating experience.

While some of these challenges probably could have been overcome, it felt silly investing time in learning the idiosyncrasies of a system that only goes half-way to creating a genuine unix experience.

Choosing a Linux distribution

The first decision involved choosing a Linux distribution. I decided on Ubuntu for several reasons.

• It's a popular Linux distribution.
• It seems like a good distribution for people new to Linux.
• Hardware support looked encouraging.
• I'd heard it was well suited as a personal desktop
• I'd also heard that ultimately Linux is configurable and thus, the distribution does not provide major constraints; furthermore basic unix command-line programs should work in all distributions.

Preparing to install Ubuntu

I approached the installation of Ubuntu with trepidation.

• I read a lot about partitioning and dual-booting on Ubuntu documentation and psychocats.
• I backed up my data (of course) before starting installation.
• I thought a lot about setting up my partitions. In the end I decided on a dual-boot partition. I have a solid state drive which is really fast, but capacity is not that high (only 120gb). Thus, I had to be careful with assigning space. I gave 40gb to Windows and 80gb to Ubuntu. This required the removal of many unwanted files from the Windows partition.
• I created and trialled the Ubuntu Live USB to check for hardware support, and everything seemed to be working surprisingly well on my Dell Latitude laptop.
• I then installed Ubuntu, letting the Ubuntu installer handle the hard drive partitioning.

To my pleasant surprise, the installation was smooth, involving minimal choices. The Ubuntu installation successfully partitioned my hard drive, and the core features of my laptop still worked (sleep, hibernate, wifi, volume buttons, graphics card, and so on). Furthermore, I was still able to boot into Windows for when I really needed to use Windows-only software.

Reflection on the installation

Switching operating systems is painful even when the new operating system is better than the old one. Many little tasks that were once automatic need to be automated in a new way. Furthermore, to fully grok the new system, it is often necessary to do more than map the old workflow onto the new system.

For the most part the switch to Ubuntu has been smoother than I was expecting. The first week involved dealing with a wide range of fundamental issues related to software installation and system configuration. However, in general, these issues have been resolved by a few Internet searches and asking questions on Ask Ubuntu (a Stack Exchange Q+A site).

The following were some of the main issues I encountered:

• Linking with my university email / calendar Exchange server has been painful. After two weeks, this still remains my biggest pain; The email program Evolution basically works, but is painfully slow at times, often freezes, and often produces errors.
• Configuring the Unity desktop (the default behaviour of the 11.04 launcher is awful) took a bit of searching.
• Configuring so that I could move between laptop monitor configurations required some fiddling.
• Configuring my vimrc took some work. I had a number of file format issues related to text files having inappropriate line feeds (i.e., Windows text files instead of Unix text files).

Despite these issues I'm really enjoying using Ubuntu. Adopting a Vim + command-line workflow is so much easier, and I'm gradually incorporating some of the great Linux tools into my workflow.

Some reading

As part of the transition, I've also been reading a few books:

• Harley Hahn's guide to Unix and Linux - Harley Hahn
  • This one was quite pedagogical in flavour and a bit wordy at times, but did provide a helpful introduction to Linux.
• UNIX power tools - Jerry Peek, Tim O'Reilly, Mike Loukide
  • This book is massive.
  • It provides heaps of short articles outlining useful tricks for getting more out of Unix / Linux.
• Learning the bash Shell - Cameron Newham, Bill Rosenblatt

Looking forward

As I get settled into the new operating system I'm looking forward to really incorporating the tools available into my research workflow.

Some of the things I'm looking forward to (which I may blog about at some point, see here for Blog RSS subscription options) include:

• Writing some bash scripts and Vim commands to enable me to link bibtex citations to stored pdfs of journal articles;
• Taking advantage of the command-line for file manipulation, running R, creating LaTeX documents, and more; and
• Dabbling in sed, awk, Python for better text manipulation.

Now with the first two weeks of initial adjustment out of the way, the subtler phase of tweaking and refinement begins. If you've made the switch to Linux particularly within the context of research and data analysis, I'd be keen to hear your experience.

Assorted Resolved Issues

Overview

The rest of this post documents a wide range of problems that I encountered when getting started with Ubuntu and the solutions that I adopted. The caveat applies that I'm pretty new to Ubuntu, so the solutions may not be best practice. Furthermore, they may not make for interesting reading unless perhaps you've stumbled onto this post through Google searching for the specific problem.

Overhaul file system

Question

• What are the implications of Linux for how my file system should be organised?

Discussion

I used the transition to Linux to overhaul my file system. I was keen to use the terminal to access files. Good file names for terminal access are different to that for a point-and-click environment.

Some general principles that guided the new file system

• Remove spaces from file names
  • It's a pain to have to escape spaces (\)
  • I replaced spaces with hyphen "-"
• Convert file names to lower case where possible
  • Linux is case sensitive
  • It's quicker to type lower case
  • Consistency reduces confusion
• Prefer file and folder names unique in the first few letters
  • This facilitates tab completion
• Prefer short, descriptive, and memorable file names

I also found it particularly useful to do a general reflection on my file structure. I cleaned up the hierarchy so that all my core activities had a location in the file system.

Accessing Windows Drive from Ubuntu

Question

• Are there any issues with accessing the Windows OS partition from Ubuntu?

Discussion

• It's easy. It shows up as a mounted drive called "OS". It's path was /media/OS

Monitoring processes and use of system resources

Question

• What is the Ubuntu equivalent of Ctrl+Alt+Delete for viewing active processes and memory and processor usage?

Discussion

• There is a program called "System Monitor"

Command prompt for terminal colour

Question

It seems to me that it might be useful to make the command prompt a different colour to help make it clear where one command starts and output begins a little clearer.

• How can this be done?
• What are good colour options?

Discussion

• Sites discussing the issue
  • cyberciti
  • archlinux wiki
• The .bashrc file that came with Ubuntu had a setting that could be enabled to have a colour prompt.

Opening common binary files from command-line

Question

• How can common binary files (e.g., PDF, Word, Excel, PowerPoint) be opened from the command line?

Discussion

• gnome-open
  • This command can be followed by filenames, web addresses, directories, and will typically do some appropriate default action.
  • gnome-open . opens the current directory in Nautilus
  • alias go=’gnome-open’ or equivalent may be useful

Keyboard shortcuts for size window left or right half of screen

Question

I tend to work on a large monitor with space for Windows on the left and right side of the screen. I like to have a keyboard shortcut which sizes a window to the left or right of the screen.

• How can this be done on Ubuntu?

Discussion

• Ubuntu does support half Desktop sizing by dragging a window with the mouse to the left or right most side of the desktop.

• keyboard shortcuts offers some options for moving windows, including maximise vertically or maximise horizontally, but no option for positioning window on left half of screen.

• Ctrl+Alt+NumPad Key

  • I learnt the following on askubuntu
  • By default Ubuntu supports shortcut keys for window manipulation (e.g., Ctrl+Alt+NumPad4 maximises a window vertically on the left half of the screen and Ctrl+Alt+NumPad6 does the same but to the right.
  • The main annoying aspect of this is that these keys are not suitable for touch typing.

• Solution

  • Install Advanced Desktop Effects settings (Compiz with Compiz Config)
  • Start CompizConfig
  • Go to Window Manager - Grid
  • Change the key bindings (I chose the following, which have a certain consistency with Vim movement keys)
    • Super + h (left)
    • Super + L (right)
    • Super + K (maximise)
    • Super + J (restore)

Shortcut key to initiate web search

Question

On windows I had a global shortcut key (Windows+Z) to initiate a Google search.

• How can this be done on Ubuntu?

Discussion

• googlizer
  • This can be used to create command line programs for searching.
• Chrome
  • Start chrome
  • Type search term in address bar
  • Making google.com the home page can speed this up
  • Evaluation:
    • The responsiveness of this is not adequate (a search box is instant; chrome takes around 1 second to start up)
    • It results in the loss of the default Chrome homepage
• AutoKey
  • A program modelled on AutoHotKey might provide the desired functionality.
• I asked the question on askubuntu
  • Google Desktop Search with Desktop search disabled was a good choice.
  • I changed the default shortcut key to Ctrl+F12

Using a laptop with multiple external monitors

Question

• How can a laptop monitor be disabled and an external monitor enabled?
• I use two external monitors, one at home, and one at work. How can Ubuntu be configured to automatically detect which one is connected?

Discussion

• When I configure to just use my external monitor and not my laptop monitor and then unplug the external monitor, the laptop monitor is not activated. Thus, I can't use the mouse to reactivate the laptop monitor. It would be good if Ubuntu could automatically detect the removal of the external monitor, or alternatively I could have a shortcut key to activate laptop only mode.

• ubuntugeek describes configuring Ubuntu so that Ctrl+Alt+Backspace can force a restart of X

• A user asks about automatic switching of monitor configurations on Ubuntu forums

• More suggestions

• I asked the question on askubuntu

  • I added my answer to the question
  • Essentially it involves using a program called disper
  • disper -s and disper -S activate primary and secondary monitors respectively. I've configured these commands to be triggered by shortcut keys using CompizConfig.

Launching programs with keyboard shortcuts

Question

• What is a good strategy for launching programs from keyboard shortcuts?

Discussion

• CompizConfig makes it easy to create commands and assign shortcut keys.
• The Unity Launcher enables shortcut keys combining the Super key and a number or typing the Super key and typing a key word.

Restart to Windows

Question

My bootloader defaults to Ubuntu.

• Is there a way to shutdown Ubuntu and boot to Windows without having to make a selection from the bootloader?

Discussion

• I asked the question on Ask Ubuntu
  • I received a good answer, and I added further details on AskUbuntu.

Default terminal size

Question

• What is a good default Terminal size?

Discussion

I can always press F11 to get a full-screen terminal. However, sometimes it is useful to be able to have a reasonable size terminal come up that does not take over the whole screen.

I've configured the default profile to be 95 columns by 40 rows. I would choose more rows, but I want the setup to work on my smallest monitor (i.e., my laptop).

Prevent Unity menu from showing

Question

At first I thought the Unity menu was okay. However, I don't like:

• the screen real estate that launcher consumes
• the way that it defaults to placing left aligned windows indented with space to the launcher
• The size of the launcher icons (although at least this can be reduced)
• The way that it stays in focus sometimes for a period of time

Discussion

I asked the question on askubuntu and received a good suggestion.

In Compix Config set Unity - Hide Launcher to Auto-hide.

Configuring Ubuntu for running basic scripts

Question

• What is a basic strategy for running scripts in Ubuntu?
• How do permanently add the custom script to the path?
• Where should I store throw-away scripts?
• What is a good way to run custom scripts?

Discussion

• There is a discussion of where to store scripts here.
• A common place seems to be ~/bin

I did the following:

• mkdir ~/bin
• added export PATH="$PATH:~/bin" to .bashrc

Find file by name

Question

• How can I find a file by a name or part of its name within a specified directory (or subdirectories)?

Discussion

• Cyberciti provides a page explaining several options.

• find command

  • The following command would find all files with the txt extension in ~/dir
  • find ~/dir -name "*.txt"

Accessing Ubuntu drive from Windows on dual-boot machine

Question

I store my data on my Ubuntu drive.

• How can I access my Ubuntu drive from Windows?

Discussion

This page has three options, the last of which, looks like what I want.

• Ext2 Installable File System for Windows
  • This did not work on Windows 7 (perhaps this will change in the future)
• DiskInternals Linux Reader
  • Evaluation
    • This worked adequately
    • This has both the benefit and the problem that it does not alter the Linux partition
  • How did it work?
    • Start Windows 7
    • Install and Start DiskInternals
    • Save selected Linux partition files onto Windows partition (I had a special section)
    • View, edit, and change files
    • Restart Ubuntu and copy files from Windows partition back into appropriate location

Script fur renaming files in folder

Question

I sometimes download files sent to me and want to convert the file names into more appropriate ones.

The main conversions that I'd like to see are:

• replace spaces with dashes ("-")
• replace uppercase with lower case

Others may arise as time goes by.

The script should take a list of file names.

• How can this be done?

Discussion

I wrote the following script and called it cleannames

#!/bin/bash
for file in * ; do mv "$file" `echo "$file" | sed 's/ /-/g'`; done
for file in * ; do mv "$file" `echo "$file" | sed 's/.*/\L&/g'`; done

Minimise current window with keyboard shortcut

Question

• What is the keyboard shortcut to minimise a Window?

Discussion

• Alt+F9 minimises

• Alt+F10 maximise

• Launcher - System Settings - Shortcuts allows for custom shortcut keys

Terminator Terminal

Question

• Should I adopt Terminator Terminal?

Discussion

• At present I'm using Gnome Terminal.
• Various reviews suggest that Gnome terminal is a reasonable option in terms of speed and features.

Preventing automatic shading when watching flash videos

Question

I had an ABC video on full screen and after a few minutes, Ubuntu shaded the screen.

• How can this be prevented?

Discussion

• Power Management has some options
• I disabled the screensaver. That seemed to fix the problem.

Batch renaming of files

Question

Unix is case sensitive. It also encourages substantially greater use of the command-line. As such, it is useful to adopt file naming conventions that make it easier to work with files on the command-line.

• How can I batch rename files
  • replace underscore with dash
  • replace upper case letter with corresponding lower case letter

Discussion

• Some options
  • KRename
  • for loop
    • Tutorial on using for loops
  • rename

Convert underscore to dash:

`for file in * ; do mv $file `echo $file | sed 's/_//'` ; done`

Convert upper case letter This tutorial shows how to use the convmv command.

Kill a program

Question

Windows has Ctrl+Alt+Delete which facilitates killing a process.

• What does Ubuntu involve?

Discussion

• The system monitor lists processes and enables killing processes
• Ctrl+Alt+Backspace kills X (once this feature is activated)

Better console / terminal colours

Question

The default background for Ubuntu terminal is a type of dark purple. While it's "pretty", I find the contrast to be relatively low.

• What are better settings?

Discussion

I applied the following settings:

• Edit - Preferences:
  • Colors - Built-in schemes= white on black
  • Background = solid colour
• export TERM="xterm-256color" enables better colour support for Vim

Problems displaying full screen video

Question

• I find that full screen video on YouTube seems to freeze. How can this be fixed?

Discussion

There is a tutorial on YouTube on how to fix this.

sudo mkdir /etc/adobe
sudo su
sudo echo "OverrideGPUValidation = 1" >> /etc/adobe/mms.cfg"

Reducing Grub loader times

Question

I think it would be useful to reduce the default grub display time to something like 1 or 2 seconds. I almost always boot to Ubuntu, and if I want to boot to Windows, I have a command from within Ubuntu that changes the default boot for one occasion.

• Is this a good idea?
• How can I do this?

Discussion

• I think 2 seconds would make sense as this is long enough to easily enter manual mode if I am watching carefully.

• sudo vim/etc/default/grub

• I modified the value of GRUB_TIMEOUT and made it GRUB_TIMEOUT=2
• sudo update-grub

Set up git on Ubuntu

Question

• How do I configure git on Ubuntu?

Discussion

• General information on configuring git and github on Linux is available here