Jeromy Anglim's Blog: Psychology and Statistics


Tuesday, January 11, 2011

Reasons for Transitioning to Vim: Bringing LaTeX, R, Sweave and More under One Roof

This post describes the reasons for my transition to Vim.

Brief Background

Over the years I've used a lot of different text editors on Windows. In general, I've used whatever text editor came with a program. When I started using R, I moved from Rgui through Tinn-R, JGR, and others, finally settling in to really enjoy the StatET plugin in Eclipse. When I started using LaTeX, I tried TexnicCenter, WinEdt, and Texlipse. Each program had the standard Windows shortcut keys and often a few extra ones for tasks such as deleting lines and toggling comments. However, these novel shortcut keys varied between programs.

I've acquired some fairly good skills in using Windows' admittedly limited set of standard text editing features. For example, I was comfortable pressing Ctrl+Shift+Right to select a word. I even developed and learnt a custom set of key bindings that allowed me to use the navigation and cursor keys from the home key position (see my earlier post).

Nonetheless, this wasn't enough. I wanted something more powerful, more consistent, more "awesome".

Reasons for Adopting Vim

Frustration with Previous Tools

  • I was hitting a wall in Eclipse. The StatET plugin for Eclipse has some excellent features which make working with R really nice (see my earlier blog post However, there were a few tasks that I could not easily automate. For example, I couldn't get decent code folding or an outline view for Sweave files. I couldn't develop a quick keyboard shortcut to send the R code in \Sexpr{} to the R console. There was no key to switch consoles. I could go on. I could see no easy way to introduce the desired customisation. No doubt the customisations were possible, but it meant diving into a whole world of complexity or hoping that one day someone else would write a plugin to meet my particular needs. I could describe a similar experience with WinEdt.

Readiness for Vim

  • I'd already adopted a plain-text--command-line workflow. LaTeX for documents; R for data analysis; Beamer for presentations; Make for building programs; Markdown for writing blog posts. This meant that I could do most of my daily tasks in a text editor. There was also the possibility that with Vim I could bring in other tasks into a more integrated environment (e.g., outlining, task lists, etc.). When I was first learning LaTeX, R, the command-line, and so on, it was useful to have software that held my hand a little bit. After getting comfortable with such things, Vim is just one additional step.

  • The main markup and programming languages that I use already had decent support in Vim (e.g., R, LaTeX, etc.).

Vim Matched My Orientation

  • I like the philosophy of Vim. I touch type. I learn shortcut keys. I use AutoHotKey. I despise unnecessary movement of the hands to the cursor keys or the mouse. Vim is built for efficiency. Vim is built for using the keyboard. I'm happy to invest a little effort to get this efficiency.

  • I watched Derek Wyatt's awesome, funny, and informative videos on Vim. http://www.derekwyatt.org/vim/vim-tutorial-videos/vim-novice-tutorial-videos/ I related to what Derek was saying. I related to the "Notepad-wall". I liked an editor that saw the mouse as an "evil" "rat-tailed beast". I liked an editor that saw moving the hands from the home key position as a health hazard.

Great Features of Vim

  • Vim was invented in 1991 and Vi goes back much further (see Wikipedia)). It's still very popular. Whenever I learn shortcut keys and introduce software customisations, I am paying a present cost for future time savings. Future time savings come both from the fact that the software is still supported and that I don't drop the software because I find something better. Vim has been around for a long time, is open source, and has an active developer community. Thus, it feels like it will be around for a long time to come. Also, given that many coding gurus use Vim, I'm hoping I wont be switching to another tool any time soon (but I guess we'll have to see).

  • The Pragmatic Programmer says that you should learn at least one powerful text editor really well. The main two options seem to be Vim and Emacs. At this point I've been lured by the elegance of the mode-based approach to text editing in Vim. However, as with a lot of things, I feel the more important point is that I move my workflow to either Emacs or Vim.

  • The system for saving settings in Vim gives me more confidence that I can roll over my customisations easily to other machines. Eclipse always felt a bit mysterious regarding how and where customisations would be stored.

  • Vim appears to lack "deal-breakers". By deal-breakers I refer to something about a piece of software that is so frustrating that you just can't use it. While most software can be frustrating at times, it should be simple enough to customise the software to overcome the problem. With Vim customisation is a natural extension of the editor. With Eclipse, some useful customisation is possible, but there also feels like there is a wall.

  • Building on the previous point customisation is a natural extension of the software. Vim is similar to R in this respect. While a lot of programs have a more advanced scripting language to add customisations, they don't all facilitate the gradual transition towards greater use of this scripting language.

Conclusion

Expect to see several upcoming posts on Vim as I begin to incorporate Vim into my workflow using R, LaTeX, and other tools.

23 comments:

  1. A fascinating post Jeromy.
    I must admit I am still too timid to get off the fence and get my hands dirty with VIM (although the comfort of shortcuts is something I too hold dear to my heart).

    I'll be eager to read your future posts on this topic and see where it develops.

    Many thanks for sharing your experiences on this!

    Best,
    Tal

    ReplyDelete
  2. Hi Tal,
    I'm glad to hear it's of interest.
    It should be a fun journey.
    cheers,
    Jeromy

    ReplyDelete
  3. Good post Jeromy. I had a journey similar to yours. But ended in Emacs (org mode is the cherry on top of the ice cream). Now I would not go back.

    ReplyDelete
  4. Hi Jeromy- If you like Vim, but still want to play around with the possibility of other editors, I suggest trying Emacs with viper-mode. The viper documentation describes itself this way:

    "Technically speaking, Viper is a Vi emulation package for Emacs. It implements all Vi and Ex commands, occasionally improving on them and adding many new features. It gives the user the best of both worlds: Vi keystrokes for editing combined with the power of the Emacs environment."

    That really does sum it up nicely. And the power of Emacs is something worth exploring. One of the things you mentioned is customization... in that respect I am sure Emacs' reputation precedes itself as the King of Customization! Of course, some people may be afraid at first of the s-expression syntax of lisp, but it turns out that with the customization facility in emacs (just go the the menu bar and select Options > Customize Emacs) you don't really need to ever write any lisp at all! (Deep down you know you want to, though! :)

    And since you're doing work with R, another natural part of emacs you'll want to explore is ESS- Emacs Speaks Statistics which was designed specifically with R in mind (though it supports other things, too).

    Another secret about R is that it has some of its roots in lisp (scheme, specifically) and if you take a look at the R Language Definition it mentions lisp repeatedly in various places like this "R presents a friendlier interface to programming than Lisp does, at least to someone used to mathematical formulas and C-like control structures, but the engine is really very Lisp-like." So, that alone may be a good reason to look at Emacs with all its lispy-ness... it may not feel so alien afterall!

    The other feature you'll care about is support for LaTeX which you'll find works very well with Emacs. In fact, if you type C-\ you can set the input method to TeX so that when you type something like \bot you'll get the symbol you want, ⊥

    If you do try out Emacs I'd love to hear about your experiences. And feel free to write if you have questions setting anything up.

    ReplyDelete
  5. @Julian
    Thanks for your comment. I'm not sure where I'll end up. But Emacs and Vim (or perhaps both) are the obvious candidates.

    ReplyDelete
  6. @Aaron Wow. Thanks for the hugely helpful comment. It's good to know that viper mode would make any investment in Vim skills more transferable. And AUCTeX, org-mode, ESS, and lisp customisation are all big pluses for Emacs.

    ReplyDelete
  7. Interesting post.

    Just wondering if you have tried Notepad++ http://notepad-plus-plus.org/

    There is also an interesting plug-in NppToR for full R support (http://npptor.sourceforge.net/)

    I'm currently using it with a positive feedback.

    I'm also willing to put R, Sweave, LaTeX, etc under the same roof and Notepad++ may be weaker under this point of view (but I'm guessing since there is also a huge community using it).

    Paolo

    ReplyDelete
  8. Interesting post. Like pcavatore, I am a fan of Notepad++ on Windows. I found Eclipse too annoying. These days I use either Notepad++ or TeXWorks to edit R noweb files and generally use a batch file or shell script (depending on platform) to process the files. Processing is one click... I like the spell checker in TeXWorks.

    ReplyDelete
  9. Nice post Jeremy. I also use Notepad++ with NppToR (an AHK-based program to send R code chunks to the R console). I've gotten used to Notepad++'s keyboard shortcuts, but it has no support for Sweave documents, which is one reason I've never fully adopted Sweave. Looking forward to seeing more on this topic.

    ReplyDelete
  10. Hi Paolo, John, and Stephen,

    So that's three votes for Notepad++.
    I do have it installed and have used it to open the odd text file, but I've never really dived into it with gusto.

    It does look more intuitive than Vim with so many options just a menu away.
    However, I get the feeling that Vim will ultimately be more speedy.
    I was won over by these videos:
    http://www.derekwyatt.org/vim/vim-tutorial-videos/vim-novice-tutorial-videos/

    ReplyDelete
  11. Oh why, oh why, did we have to open this can of worms again. I was so happy using eclipse for all my R, SAS, SQL, Sweave, and LaTeX tasks. Okay, not really. Though I only jumped into eclipse in the past year, the software engineers raved about it (last job was in a java house). Of course here all the analysts use SAS, but needed more power for my position, so wasn't going to jump onto that bandwagon.

    Eclipse + R seemed right, having an affiliation to Visual Studios and Matlab. And it works. Nice little help menu, object browser, auto complete, folding, one click build, speedy. Okay those are the nice points. Now why is it that when I am working on a sweave document, all the IDE is stupid with regards to LaTeX. Where did the folds go, where did the quick commands for completion. Of course I when working on a LaTeX document, its a dream, but has some silly requirement that you need to have one per folder. You can't just switch between sweave and LaTeX. 80+mb (its at 167mb now) and this is all I get?

    So thank you for giving me a push off the diving board. And thank you Derek Wyatt for making VIM look sexy. I do not know what will happen, but here are my thoughts after a half a week of fiddling.

    I love the VIM philosophy and small foot print. Installing everything was rather easy, though needed python and other plugins for R interaction (I am on a windows machine, and will alway be). In testing, the communication between VIM and R is slow and laggy. Could be python, could be conque, could be user error, but I don't like it.

    Now there is Notepad++, which I use, and still use (at it is my default in windows and too lazy to associate things over). Actually I started with R with it and use to LaTeX it. I don't remember why I left. Local help? Context help? Application segregation? Sweave? Who knows.

    Maybe I'll stick with eclipse, but now I am looking at emacs. With ESS, it works with R out of the box with no apparent issues. AucTex looks like a dream with its inline preview (still need to get GS, to test it). The foot print looks comparable (both a little less than 20mb, when doing straight R). Oh and it has several flavors of viper mode (though not being acquainted with either VIM or emacs, I am treading lightly).

    Well thats my two bits, but interested in everyone else's workflow and tools. I just want an editor to be mine. My father is a marine, so I always think of the Rifleman's Creed:

    This is my rifle. There are many like it, but this one is mine. My rifle is my best friend. It is my life. I must master it as I must master my life. My rifle, without me, is useless. Without my rifle, I am useless.

    ReplyDelete
  12. @jphilbert
    awesome comment. I relate a lot to your experience.

    I'm just hearing the equivalent creed for a data analyst / programmer in my head:

    "This is my editor. There are many like it, but this one is mine. My editor is my best friend. It is my life. I must master it as I must master my life. My editor, without me, is useless. Without my editor, I am useless."

    Classic.

    ReplyDelete
  13. After 33 years of with Emacs, anything less powerful (e.g. Eclipse) just makes me want to throw the keyboard in frustration. If you are stuck with Eclipse, I recommend installing the Mulgasoft Emacs+ bindings at http://www.mulgasoft.com/emacsplus/update-site.

    ReplyDelete
  14. i use ubuntu, and i go have gvim, since like 3 years i have being using emacs for mixing R-latex-sweave. I would like to try vim, giving your opinion.

    what should i install, and how, in order to get R-latex-sweave working in vim?

    thanks

    ReplyDelete
  15. Hi cseljatib,
    I hear great things about emacs and ess.

    I use the Vim-R-Plugin:
    http://www.vim.org/scripts/script.php?script_id=2628

    I've added a few customisations.
    I have my vimrc and vim files on git
    https://github.com/jeromyanglim/JA_Vimfiles

    I'll post more details of my R - Vim setup probably March or April 2011 (once I'm reasonably happy with it).

    ReplyDelete
  16. Dear Jeromy

    Thanks for the post.

    what are the customizations you've written about? can we not use vim with R directly?

    Subbu

    ReplyDelete
  17. Hi Subhash,
    You can use Vim with R directly.
    However,
    * there were several default options in Vim-R that did not suit the way that I work;
    * I also added a few features that I wanted, like being able to send R code in a \Sexpr{} in Sweave
    * Vim is designed to make customisations like this easy.

    I'll post the details shortly.
    However, if you want to see the customisations as I develop them, check out:
    https://github.com/jeromyanglim/JA_Vimfiles
    See the section on r in my vimrc file and sections in r-plugin/common global vim with the word "Anglim"

    ReplyDelete
  18. Thanks Jeromy

    i use R with Tinn-R for all computation and Latex for documentation; but i use both separately. But getting sweave, R and latex together seems great. i am glad hearing of that. The thing is at the same time i heard both about vim and eclipse from your blog and i'm confused what to start with. i found your blog on using eclipse really good but since you've transitioned to vim, i'm wondering if eclipse is the right thing to start with. If its vim, How do i start - what to install /configure etc. i would be grateful if you could post it like your detailed post on Sweave-Eclipse-R-Latex post. Or would you suggest that until then i try Eclipse?

    ReplyDelete
  19. Hi Subhash,
    Sorry for the conflicting recommendations. My blog charts my own journey. I still think Eclipse and StatET provides a nice way to work with R.
    If you are coming from a Windows background or you are used to standard IDEs, Eclipse is likely to be more intuitive than Vim.
    StatET is also more developed and refined than Vim-R at present (although the vim-R plugin is under active development).

    I switched to Vim for at least two big reasons.
    I liked the editing environment that Vim provided in general and I could see how I could easily customise anything that I didn't like.

    In summary, I think the best tool for you probably depends on your background and your general goals.

    If you are looking to learn a serious programmer's editor, then you should have a good think about Emacs and Vim.
    Both are extremely flexible and customisable. Both have steep learning curves (at least if you want to reap their benefits).
    Both have quite distinct conventions to what you may be used to if you are a typical Windows or Mac user.
    This means that the value of using either tool comes from bringing as many of the text manipulation tasks as possible into one consistent environment.

    Assuming you want this, you are then confronted with the decision of whether to learn Emacs or Vim. Both have their die hard fans.
    My impression is that a lot more people using R, Sweave, and LaTeX use Emacs with ESS rather than Vim.
    As such you are going to get more developed tools for Emacs.

    However, I couldn't resist the asymptotic efficiency of Vim. And I'm happy to invest the time in learning Vim Script and related technologies to enable me to customise Vim to work the way that I want.

    Perhaps from this comment, you can start to get a sense of the issues involved.
    Vim and Emacs are relatively major commitments. If you are serious about adopting a serious unifying programmer's text editor and are looking forward to playing around and customising your tools, then check them both out. Play around with them. See which one you like best. Or perhaps you'll decide they're not for you (or not for you just yet).

    But yes, when I'm happy with my configuration with Vim and R, I will post some details of my configuration.

    ReplyDelete
  20. Hi jeromy

    i appreciate your prompt reply. thank you.

    i guess, i will start with eclipse-stat ET and move on. Like you did.

    One last thing- which IDE of eclipse to download? there are many IDEs on the eclipse download page: http://www.eclipse.org/downloads/

    i guess i ve followed installation instructions for stat-et and texclipse and the config settings mentioned by you. was only unsure about which eclipse IDE.

    Seek your advice.

    ReplyDelete
  21. Hi Subhash,

    The StatET site under installation notes says: "at least "Platform Runtime Binary" and JDT, both is included for example in the package "Eclipse IDE for Java Developers" or the classic Eclipse SDK"

    ReplyDelete
  22. Just Yesterday i installed eclipse, StatET and Sweave, because I decided to supplement my studies of statistics with writing posts ... and now I found out there is Vim :D... I guess I should check it out

    ReplyDelete
  23. @Adrian Yep. There are lots of choices. RStudio and Emacs with ESS are two other good options.

    ReplyDelete