Using installed software¶
Software modules¶
Software that is not part of the operating installation is made available for your use through the Lmod module command. There are two parts to installing software: copying the program files that constitute the software and configuring the software for use. The module system is used to configure the software for your use as you need it. This enables ARC to provide multiple versions of the same software and to provide packages that may have conflicting configurations.
ARC provided software modules¶
ARC provides modules for the software it installs. Some software is installed because it is essential to basic functionality and other software is installed at user request. The ARC installed software modules are ready to use when you log in.
User written software modules¶
Just as you can install some software yourself, so you can also write
modules to configure it. This is done by creating a directory for your
modules, creating a module file, and then using the use.own module that
ARC provides to make your own modules available for your use. More detail
can be found below under Creating your own
modules.
Basic use of modules¶
The most common use of the module command is to load a module, which
simply means configuring it for your use. To load a software module, you
need to know its name. If you want to use the CMake program, you would
use the module command load followed by the module name, which in this
case is also the software name.
$ module load cmake
To see which modules are currently loaded, you use the module command
list.
$ module list
Currently Loaded Modules:
1) cmake/3.21.3
Note that module list shows both a name and a version number, in this case
3.21.3. When you module load a package, you can also specify the version,
as in,
$ module load cmake/3.21.3
When you log out, all modules are unloaded, and when you next log in, you will need to reload any needed modules. See the Module collections section for details on how to load modules automatically upon login.
To reverse the load, you would
$ module unload cmake
Only one version of a module can be loaded at the same time, so you do not have to give the version number when unloading.
You may also wish to unload all modules, in which case you can use
$ module purge
which clears all modules.
Information contained in a module¶
Modules installed by ARC administrators contain, at minimum, the following information
- module name
- software version
- brief description of the software
- keywords to help in finding and categorizing the package
- license information
You can display that information with the module whatis command, as in,
$ module whatis cmake
cmake/3.21.3 : Name: CMake
cmake/3.21.3 : Description: CMake cross-platform make system.
cmake/3.21.3 : License information: https://cmake.org/licensing/
cmake/3.21.3 : Category: Utility, programming, compiler
cmake/3.21.3 : Package documentation: https://cmake.org/documentation/
cmake/3.21.3 : ARC examples: None provided
cmake/3.21.3 : Version: 3.21.3
You can see that there may be other information, but at a minimum the five listed attributes will appear for all packages.
Finding installed software¶
Loading software once you know the module name is easy; finding what software
is installed presents a bit of a challenge. However, the module command
has several ways to locate software packages that may be of use.
The most general is to search by keyword. These are words that appear
in the Description or in the Category. For example, some software has as
a category compiler, so an edited result from a keyword search showing
the cmake example above would appear like this.
$ module keyword
----------------------------------------------------------------------------
The following modules match your search criteria: "compiler"
----------------------------------------------------------------------------
cmake: cmake/3.13.2, cmake/3.17.3, cmake/3.21.3
CMake cross-platform make system.
. . . .
Note that the first line lists the module name, then after the colon it lists
the exact versions for which there are modules in a form that can be used
directly with module load. For some packages it may take more than one
line; on the line after the name and version, the Description is displayed.
Once you know the name of the module you want, you can obtain additional
information about it using module spider. If you compare the output of
module keyword and module spider for the cmake module, there is not
a lot of difference. However, if you compare the output of module keyword
hdf5 with that of module spider hdf5, you will see greater differences.
$ module keyword hdf5
----------------------------------------------------------------------------
The following modules match your search criteria: "hdf5"
----------------------------------------------------------------------------
hdf5: hdf5/1.8.21, hdf5/1.10.5
HDF5 data libraries and utilities.
meep: meep/1.14.0
Meep (or MEEP) is a free finite-difference time-domain (FDTD)
simulation software package.
ncl: ncl/6.6.2
NCAR command language for visualization.
. . . .
$ module spider hdf5
----------------------------------------------------------------------------
hdf5:
----------------------------------------------------------------------------
Description:
HDF5 data libraries and utilities.
Versions:
hdf5/1.8.21
hdf5/1.10.5
Other possible modules matches:
phdf5
----------------------------------------------------------------------------
To find other possible module matches execute:
$ module -r spider '.*hdf5.*'
----------------------------------------------------------------------------
For detailed information about a specific "hdf5" module (including how to
load the modules) use the module's full name.
For example:
$ module spider hdf5/1.8.21
----------------------------------------------------------------------------
The module spider will tell you what you need to know to load a module.
For example,
$ module spider hdf5/1.10.5
----------------------------------------------------------------------------
hdf5: hdf5/1.10.5
----------------------------------------------------------------------------
Description:
HDF5 data libraries and utilities.
You will need to load all module(s) on any one of the lines below before
the "hdf5/1.10.5" module is available to load.
gcc/4.8.5
gcc/8.2.0
intel/14.0.2
intel/18.0.5
intel/19.1
Help:
HDF5 is a data model, library, and file format for storing and managing
. . . .
The most important information here is that another module must be loaded
before the hdf5/1.10.5 module is available to load.
Module hierarchy¶
The HDF5 program is a library, meaning that it contains binary code used by other programs. It must be installed to match the compiler used to install the program that uses it. All of the modules listed in the spider output above are compilers. When you load a compiler module, it adds additional modules to the list of what is available to load. To see what is available before and after loading a compiler compare
$ module available
-------------------- Core applications including compilers ---------------------
ANTs/2.3.1 image-libraries/170303
R/3.6.1 image-libraries/190711 (D)
R/4.0.2 intel/14.0.2
. . . .
with
$ module load gcc
$ module available
--------------------- Applications compiled with GCC 8.2.0 ---------------------
ITK/5.1.0 image-libraries/190711.1 (D) openblas/0.3.9 (D)
boost/1.50.0 impi/4.1.3 openmpi/1.10.7
boost/1.70.0 impi/2018.4.274 (D) openmpi/3.1.4
. . . .
As you can see, a number of modules that were not there previously now appears before the 'Core applications' that were at the top of the list previously. The Core applications are still available, but they appear below the list of those compile with GCC 8.2.0.
This is called a hierarchy, in this case a compiler hieararchy. There
are also hierarchies for programs that use the message passing interface
(MPI), for Python libraries, and possibly for some other modules. Using
module spider will always identify modules that are included in a
hierarchy and tell you which module must be loaded to make it available.
Module collections¶
Sometimes you will need to have a number of modules loaded to perform a task. For example, if I am doing some statistics that uses data from satellite imaging, I might need to load all of these
$ module load gcc hdf5/1.8.21 netcdf-c netcdf-fortran gdal geos proj R
That is a lot of modules to load each time you want to do that kind of work,
especially if you are working on something else that requires hdf5/1.10.5,
which conflicts with hdf5/1.8.21. You can use a module collection to
make switching easy. A collection is a named set of modules that can be
restored with a single command; restoring a collection removes all the
currently loaded modules and replaces them with those in the collection.
The list of modules loaded by the command above is
$ module list
Currently Loaded Modules:
1) gcc/8.2.0 4) netcdf-c/4.6.2 7) geos/3.7.2
2) szip/2.1.1 5) netcdf-fortran/4.4.5 8) proj/6.1.1
3) hdf5/1.8.21 6) gdal/3.0.1 9) R/4.1.0
and you can save it to a collection called geo with
$ module save geo
You can verify this with
$ module purge
$ module list
No modules loaded
$ module restore
$ module list
Currently Loaded Modules:
1) gcc/8.2.0 4) netcdf-c/4.6.2 7) geos/3.7.2
2) szip/2.1.1 5) netcdf-fortran/4.4.5 8) proj/6.1.1
3) hdf5/1.8.21 6) gdal/3.0.1 9) R/4.1.0
You can list all the named collections using
$ module savelist
Named collection list :
1) fnirs 2) geo 3) mri
And to see what is included in any given collection, you can use
$ module describe fnirs
Collection "fnirs" contains:
1) gcc 2) fftw 3) matlab
Collections are stored as text files in the directory ${HOME}/.lmod.d in
a file whose name is the collection name. If you create a collection, then
decide the contents should be slightly different, simple save it again. If
you modify a collection, a backup file of the previous configuration will
be saved with the ~ character appended; that is,
$ ls ~/.lmod.d/mri*
mri mri~
If you decide you no longer want to have a particular collection, you should just remove the file(s), as in
$ rm ~/.lmod.d/mri*
Default modules¶
You can create a default collection that will be automatically restored each time you login by saving a collection with no name given.
$ module save
Saved current collection of modules to: "default"
You can modify the collection by saving again, and you can remove it with
$ rm ~/.lmod.d/default*
If you ever change your loaded modules and wish to return to the default
collection, you restore it the same way you restore a named collection.
$ module restore default
Creating your own modules¶
If you install software of your own, you may wish to create a module for it.
This is generally recommended over modifying your .bashrc file, as many
web sites and some software installation instructions recommend. It is also
fairly common for research groups to have software installed for all members
of the group to use.
We created a module called use.own that will add the directory ~/Lmod
to your module path. Any modules that you create within that directory
will be automatically available.
To make setting that up easier, and to provide you with an example module,
we provide the command my_modules_setup that you can run and which will
create the ~/Lmod directory and copy a sample module called hello to
it. You would then load the use.own module to make hello available
for loading.
$ my_modules_setup
$ module load use.own
$ module av hello
----------------------- Personal Lmod modules ------------------------
hello/2.10
Note that the directory ~/Lmod contains a directory that bears the name
of the module, which is usually the name of the software package but that
is not required. The files in that directory are what Lmod considers
versions of that software. So, the example has ~/Lmod/hello which in
turn contains 2.10.lua. The .lua indicates that the language used
for writing module files is called Lua. There are comments in 2.10.lua
that explain the sections.
If you would like more information about writing modules of your own, or
for what the various lines in 2.10.lua do, please do not hesitate to
write us at arc-support@umich.edu.