HBSGrid Environment Changes For 2020-11-02 - The Happy Halloween Release

October 29, 2020

The following changes are being made to the HBSGrid cluster and RC environment. There are a large number of improvements here, which we hope you will like and will accelerate your research.

Default software version changes:

As a reminder from last month’s maintenance note, please note the following change to default software versions:

  • Python v3.8 will be the new default version (currently v3.6)
  • R v4.0 will be the new default version (currently v3.5)
  • RStudio v1.3 will be the new default version (currently v1.1)

Note that it is almost guaranteed that you will need up update or reinstall any personal or shared R packages or Python modules that you have install with the default version change.

We encourage you to test your scripts prior to this change by following the instructions on our website at https://grid.rcs.hbs.org/news/new-pythonr-default-versions. Should you uncover problems with the newer versions during your testing or even at a later time, as a part of Monday's maintenance we are making available a script which is an option for fallback purposes -- a script that enables users to set their own system-wide version defaults for many applications, including R, RStudio, and Python.

How will this maintenance affect you?

  • You will not be able to log in to the HBSGrid via terminal or NoMachine
  • All interactive jobs will be killed, as will any running Stata jobs (both interactive and batch)
  • Stata will be updated with the latest bug fixes
  • Software modules will be available for everyone by default
  • The GUI (NoMachine/Gnome) and terminal wrapper scripts will make use of and/or can redirected by software modules
  • A 'set user defaults' script will be provided for both GUI and terminal users to set one's own system-wide module defaults
  • RClone will be available for pull/pull/sync with cloud-based storage services (e.g. OneDrive, DropBox, Google Drive, etc)
  • Julia will be available as an additional general-purpose programming language
  • We will have fixed a bug breaking Gnome desktop user interface customizations
  • We will make available a software module that can be used specifically for creating and working with Conda environments
  • You will have a larger number of Gnome desktop background choices (all HBS-themed)
  • LSF log files will be written to the working directory during the job run, not after
  • LSF log files > 10 MB will be written to disk instead of being sent via email in the job reports
  • Files in the scratch space (/export/scratch) will be deleted if they haven’t been modified in over 60 days

What work is being performed?

  • All login nodes will be rebooted
  • The opt-in option for Lmod software modules will be set for no opt-in, on by default
  • Stata v16 will be patched to 16.1 Oct 2020 release
  • The wrapper scripts will be modified to use module and yield to any pre-loaded modules
  • A new Gnome menu option ( Applications > Other ) and shell script at /usr/local/app/scripts/ will be provided to set personal module defaults which persist across logins
  • RClone will be installed and available via software modules
  • The Julia programming language and module file will be installed
  • The XDG_RUNTIME_DIR problem will be temporarily fixed until it is resolved from RedHat/NoMachine
  • Loading the Anaconda/2020.07 module file will load the Conda framework for working with software (Conda) environments
  • We will add HBS Desktop background pictures to the choices Gnome Settings
  • An LSF parameter will be modified to change the written location behavior of LSF output and error log files (bsub -o and -e options)
  • An LSF parameter will be modified to change the cutoff threshold for emailed job reports
  • We will run the scratch filesystem retention script, which deletes files and empty directories that haven't been modified in the last 60 days

Updated documentation and instructions will be available on our website by Monday, if not earlier; and we will provide updated links in another communication.

We thank you for your anticipated cooperation as we make improvements to our research computing environment. Our next maintenance will occur on Monday, December 7th. If you have any questions/comments, please email us at research@hbs.edu.

Updated 10/29/2020