Advanced Modules Tutorial

Modules has a number of advanced features that can make working with modules even easier. Users can  build custom module files so that they can use a custom modules environment.

CHPC uses the Lmod version of modules produced by TACC. For more information, please visit the Lmod Homepage.

• Saving module environments
• Creating a custom module environment
• Writing module files

Saving module environments

The commands module save and module restore can be used to save a loaded module environment for later use. This is useful if one needs different environments at different times. To create the same default environment at every login, load the modules in ~/.cshrc[sh,csh] instead. However, if you are using different sets of modules repeatedly, saving a module environment can same time, especially if there are multiple modules needed in the environment.

Suppose you want to use the Amber14 package to perform some molecular dynamics experiments with a small molecule, a process that may require a compiler, MPI library, amber, the visualization program VMD, and gaussian09, and possibly access to the CSD database to obtain a structure, and the structure file converter openbabel.

$ module list
Currently Loaded Modules:
  1) csd/2015           3) mpich2/3.0.4   5) openbabel/2.3.2   7) gaussian09/EM64T.D01
  2) intel/2015.1.133   4) amber/14       6) vmd/1.9.1

In total, this comprises 7 modules; if the user needs this environment on a regular basis, instead of typing these in each time, the user can save this environment with the use of module save:

$ module save md-setup
Saved current collection of modules to: md-setup

If no name is specified, the name "default" will be used. To load this list again use module restore:

$ module purge
$ module list
No modules loaded
$ module restore md-setup
Restoring modules to user's md-setup
$ module list
Currently Loaded Modules:
  1) csd/2015           3) mpich2/3.0.4   5) openbabel/2.3.2   7) gaussian09/EM64T.D01
  2) intel/2015.1.133   4) amber/14       6) vmd/1.9.1

To see a list of your module savepacks, use the command module savelist:

$ module savelist
Named collection list:
  1) md-setup  2) default

The files for these savepacks are stored in $HOME/.lmod.d. If you delete this directory, you will lose all savepacks!

The same topic is also described in Lmod documentation.

Creating a custom module environment

A module environment is started by creating a directory, putting modulefiles into that directory in a specifically organized way, and then adding that directory to the $MODULEPATH variable using module use. Under Lmod, a modulefile is a simple lua script containing Lmod specific commands to set up environment variables. Other versions of modules use TCL/TK scripts; Lmod is compatible with these scripts, but the CHPC module installaton only supports modulefiles written in lua.  Writing modulefiles will be discussed in a later section.

The hierarchy of a modulefiles directory is straightforward. The top level directory should contain a folder called Core. This directory is the landing place for modules which have no particular dependencies. Within this directory should be additional directories, one for each package that the user anticipates using:

mymodules$ ls
Core Compiler MPI
mymodules$ cd Core
mymodules/Core$ ls
package1 package2 package3

Each folder inside Core should contain one or more modulefiles, named according to the version of the package it references:

mymodules/Core$ cd package1
mymodules/Core/package1$ ls
1.0.lua 1.2.lua 2.0.lua

There are two important details about the contents of this directory: 1) Lmod treats ALL files in this directory as if they are modulefiles, and 2) All files that do not have the extension .lua will be treated as TCL/TK scripts. This means that if you put a file in the directory modulefiles/Core that is NOT a modulefile, it will be treated like a modulefile. Attempting to load this will result in errors, and possibly unpredictable behaviour. Therefore, do not put anything besides modulefiles in a modulefile directory.

To use a modulefile directory, the command module use is utilized. This command modifies the environment variable $MODULEPATH by prepending the given path:

$ echo $MODULEPATH
/some/path/to/modulefiles
$ module use $HOME/mymodules/Core
$ echo $MODULEPATH
/home/u0123456/mymodules/Core:/some/path/to/modulefiles

The user then should be able to use any modules contained in users modulefiles directory.

There is a way to remove this directory from the path, but first, some important notes:

1) Be extremely careful when unloading paths from modules. Unloading the wrong path can result in a broken CHPC environment.
2) Adding the wrong path to modules will break your spider cache and cause major errors when using module commands.
3) Do NOT modify $MODULEPATH directly; if you do this you will break modules and cause yourself headaches.

The command to unload a modulepath is module unuse:

$ module unuse $HOME/mymodules/Core
$ echo $MODULEPATH
/some/path/to/modulefiles

For a more detailed explanation of module hierarchies, please visit the following page: Module hierarchies

Writing module files

Lmod's Introduction to Writing Module Files  page gives an alternative instruction to what is described below.

A detailed list of functions that can be used in a modulefile can be found at Lmod's Lua Modulefile Functions page. In addition to these functions, any valid lua code may be used in a modulefile, however, complex modulefiles that use extensive logic are discouraged for security reasons and the sake of simplicity.

A template for users to pattern their own modulefiles after can be found at the following path:

/uufs/chpc.utah.edu/sys/modulefiles/templates/template.lua

The most basic and most used function in a modulefile is prepend_path. The first argument is the variable that will be prepended; generally this is PATH or LD_LIBRARY_PATH. The second argument is the path to be prepended. Prepend_path will create the variable if it does not already exist (as may be the case for $MANPATH or $LD_LIBRARY_PATH).

The second basic function that users will find the most value in is setenv. This is essentially identical to prepend_path, except that instead of prepending to the path, the environment variable is replaced with the given value or created from scratch.

Users may find value in the help and whatis functions, which are useful for providing information to users on the contents and nature of a package, as well as any particular instructions which may apply to a particular module.

If users need help creating more complex modulefiles, please contact CHPC at helpdesk@chpc.utah.edu. You can also look at CHPC's module format guidelines. If your software comes with an environment setup script, CHPC has a scripting utility available that can automate the creation process, or consult Lmod's Converting shell scripts to module files page.

Here is a simple example of a modulefile written in lua for openbabel, which covers the basic commands a user might need. Here we also utilize the pathJoin function to combine paths and temporary variables version and base to reduce the amount of typing needed to create the module.

-- -*- lua -*-
help([[
Openbabel is a software for converting between different chemical structure formats. This module loads Openbabel.
Once loaded, the command to execute openbabel is "obabel"
]]) 

local version = "2.3.2"
local base = pathJoin("/uufs/chpc.utah.edu/sys/installdir/openbabel", version)

prepend_path("PATH",pathJoin(base,"bin")
prepend_path("LD_LIBRARY_PATH",pathJoin(base,"lib")
setenv("BABEL_LIBDIR",pathJoin(base,"lib")
setenv("BABEL_DATADIR",pathJoin(base,"data/data")

whatis("Name         : OpenBabel")
whatis("Version      : 2.3.2")
whatis("Category     : Application")
whatis("Description  : Chemical Structure Format Converter")
whatis("URL          : http://openbabel.org/")
whatis("Installed on : 05 Feb 2015")