Saturday, January 14, 2012

Yet Another Moses Installation Guide

Though Moses is a versatile MT system, its installation is still from stone age. Let me document here some of the key points to navigate through the installation of Moses. The intent is not to present a complete installation guide, but to highlight key issues that may crop up (as they cropped up for me). For a complete installation, this is probably the best guide. Another useful installation guide can be found here.

To install the Moses system, the following tools need to be installed. 
  • Language modelling toolkit (SRILM, IRSTLM, etc.)
  • GIZA++ package which contains GIZA++ and mkcls
  • Moses decoder (version 1.0 and above)

SRILM installation
  • The primary installation reference is the INSTALL document that ships with the tool.
  • Install all pre-requisites mentioned in the SRILM installation guide. On Ubuntu I had to install the following packages: csh, g++-multilib, tcl-dev
  • Set the environment variable SRILM to point to the base directory of the install package before building SRILM.
  • Following the instruction manual with the SRILM download should be enough once the pre-requisites are installed.    
  • The problems you may yet face are
    • Problem in identifying the architecture, especially if it a 64-bit machine. To make sure that the install script correctly identifies the architecture, set the variable MACHINE_TYPE in sbin/machine-type.
    • Problems with TCL compilation. You may not need the TCL user interfaces at all, so it may just be able ok to disable their compilation. Set the variable NO_TCL = X in the file common/your_architecture_specific_makefile.         
  • Make sure you have added the $SRILM/bin and $SRILM/bin/$MACHINE_TYPE to the PATH variable
  • Note: SRILM 1.7.1 and above are not compatible with Moses
IRSTLM installation
  • Ubuntu packages required: libtool make autoconf autotools-dev automake
  • The installation is pretty simple, just have to follow the installation guide
  • One caveat: Sometimes, it may be required to create a directory named 'm4' manually, if the first step mails

GIZA++ and mkcls installation
  • You get both if you download the giza-pp tool. 
  • Most straightforward installation. Download and 'make'.
  • Copy the binaries - GIZA++, mkcls, snt2cooc.out to a new directory. 
XMLRPC Server
  • XML RPC Server is required if you want to run a webservice providing translations. If you just want to get Moses running, you can skip this step.
  • Install the following packages: libxmlrpc-core-c3 libxmlrpc-core-c3-dev libxmlrpc-c3-dev libxmlrpc-c++4 libxmlrpc-c++4-dev 
Boost Library
The C++ Boost library  is required for installation of  Moses. Boost 1.48 has a serious bug which breaks Moses compilation. Unfornately, some Linux distributions (eg. Ubuntu 12.04) have broken versions of the Boost library.To fix this situation you can:
  • For Ubuntu 12.04: Remove boost 1.48 from your distribution and install Boost 1.46 which is available in the distribution. This works most of the time. If not, build Boost from source as described below.
  • To install Boost manually and making it work with Moses, follow the instructions in the section titled "Manually Installing Boost" on this page: http://www.statmt.org/moses/?n=Development.GetStarted
Moses installation
  • The primary installation reference is the INSTALL document that ships with the tool.
  • SRILM or IRSTLM need to be installed before Moses is installed
  • Make sure you have installed the packages  automake and libtool
  • Boost has to be installed
  • It is then a matter of just following the instructions. The command to be run is
  • /usr/bin/bjam --with-srilm=  --with-xmlrpc-c= --with-boost=
    • If the xml RPC is installed in /usr/bin, then the parameter would simply be '/usr'
    • --with-boost is required only when Boost is installed in a non-standard directory. The path should contain both lib/lib64 and include directories
Now Moses is ready to cross the Red Sea.


Alternative ways of installation Moses

If you fail to install from the source as mentioned above, then there are a couple of simpler alternatives you can try:

One, use the pre-compiled binaries provided by the Moses team: 
The pre-compiled version comes with IRSTLM and does not support XML-RPC to the best of my knowledge. However, it is handy to get started. 

If that too runs into trouble, then you can try using the virtual machine provided by the Moses team. 


If you are using Virtual Box, you can import the OVA images into VirtualBox. 
This guide many be useful for importing OVA images into VirtualBox:
http://www.maketecheasier.com/import-export-ova-files-in-virtualbox/

I have not tried the Virtual Images, so let me know if it works.