Installing Software on the HBSGrid

Many applications and programs are installed system-wide, and more are available through the modules system. If a popular and useful software title is not available we encourage you to contact us to see if we can get it installed for you. That way other users can also benefit from having the application available. Note that installation of some types of software is prohibited for security reasons; if in doubt please contact us to verify that the software you wish to install does not violate our security policies.

If we are unable to install the software you need, you may be able to install it yourself. The HBSGrid runs Red Hat Enterprise Linux (currently RHEL 7.5), and installing software on Linux without admin access is often less straight-forward than the process you might be used to on Windows or Mac. Below we describe several approaches to installing your own software on the HBSGrid, starting with the simplest and working our way up to more complex procedures. If you get stuck or need help installing softare on the HBSGrid please contact us and we will be happy to assist you.

Sections:

Compatible Stand-alone Programs

If you are lucky the application you need will have pre-built versions (often called "binaries") compatible with the HBSGrid system. (Currently Red Hat Enterprise Linux 7, also known as RHEL 7.) Examples include the PyCharm Python IDE, the  rclone file-sync utility, the fzf command-line fuzzy-finder, and the LibreOffice office suite. In this case you can download and extract the program, run any installer provided (if there is no installer just move the program wherever you like), and run the software directly from the terminal. If the software is available in multiple versions, pick one that mentions "RHEL", "RPM", "Linux x86-64 / AMD64", or just "Linux".

As an illustration, we can install PyCharm as follows:

  1. Connect to the HBSGrid via NoMachine, start Terminal from the menu, and download Pycharm:
    wget https://download.jetbrains.com/python/pycharm-community-2020.1.3.tar.gz
  2. You will notice that the file has a '.tar.gz' extension, indicating that it needs to be decompressed with 'tar':
    tar xvf pycharm-community-2020.1.3.tar.gz
  3. Submit an interactive job to run Pycharm on a compute node:
    bsub -Is pycharm-community-2020.1.3/bin/pycharm.sh

If all goes well you will see the Pycharm splash screen:

Pycharm splash screen

and you can start a new Python project using this popular IDE.

The Conda Package Manager

If the software you need is not available as a stand-alone binary compatible with the HBSGrid system another option is to install it using the Conda package manager. Conda started as a package manager for Python packages, but can be used to install other kinds of software as well. As an illustration we can install the glueviz data exploration system as follows:

  1. Open a terminal and load the anaconda software module:
    module load anaconda
  2. Create and an environment named 'glue' and install the 'glueviz' package:
    conda create --name glue glueviz
  3. Activate your environment and run the glueviz application:
    conda activate glue
    bsub -Is glue

If all goes well you will see the glue welcome screen:


Glue application window
and can start visualizing your data using this interactive tool

Conda has a decent selection of software in the default channels, and more are available in the conda-forge channel. It is somewhat Python-and-R-centric but has general-purpose software as well. Refer to the excellent Conda documentation for more about installing software and managing environments.

Building From Source

Some software will not be available as a stand-alone binary nor available in the conda repositories. You may still be able to build the software yourself if the source code is available. This often follows a standard ./configure --prefix=<mydir>, make, make install sequence, otherwise just follow the instructions provided with the software you are installing. As an illustration, we can build the development version of the R environment for statistical computing (note that released versions of R are already available on the HBSGrid) as follows:

  1. Open a terminal on the HBSGrid and run
    wget https://stat.ethz.ch/R/daily/R-devel.tar.gz
    tar xvf R-devel.tar.gz
    cd R-devel
    to download and extract the R source code.
  2. Create a directory to install into (or skip and choose an existing directory in step 3 below)
    mkdir -p ~/.local
  3. Start a bash shell on a compute node, configure and build R
    bsub -Is bash
    ./configure --prefix=</path/to/home/.local> --with-pcre1 --enable-R-shlib
    make
    replacing replacing </path/to/home/.local> with the directory you wish to install to. Note that the --with-pcre1 option was discovered by trial-and-error, and the --enable-R-shlib option is optional but needed if you want to use this build of R from Rstudio.
  4. Install R to the directory specified in the --prefix argument in step 3 by running
    make install
  5. If all went well, you can now run
    bsub -Is ~/.local/bin/R
    to test out the bleeding-edge features of this popular program!

Note that as with other applications you should not run resource-intensive tasks on the login nodes. Instead submit an interactive or batch job to run on a compute node.