NetCDF format

From GHER

NetCDF stands for Network Common Data Form. It is self-describing, machine-independent data format, frequently used in ocean and atmospheric sciences.

General

Home page: http://www.unidata.ucar.edu/software/netcdf/
What is NetCDF: http://www.unidata.ucar.edu/software/netcdf/docs/faq.html#whatisit

Visualization of NetCDF files

There are several ways to have a quick view of the content of a NetCDF file:

ncBrowse (very easy to install on any platform)
Ncview (easy on Linux, tricky on Windows), installation detailed here
Ferret: an interactive computer visualization and analysis environment.
ODV: lets you explore and visualize CF, COARDS, GDT and CDC compliant netCDF datasets.
Panoply: the NASA data viewer for netCDF, HDF and GRIB data.

Installation of the NetCDF library

The following instructions describe the installation of the NetCDF 4 library, with the HDF5 storage format.

The installation of the packages and libraries was performed on nic3 in the directory /u/mc and on the computer gher01 in the directory /usr/local.

Download the sources

Download the NetCDF sources from Unidata webpage:

wget ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf.tar.gz

Extract the archive:

tar -xvf netcdf.tar.gz

Compilation with Intel Compilers

Intel compilers are available for free non-commercial download at http://software.intel.com/en-us/articles/non-commercial-software-download/

Install HDF5 and zlib packages

Download:

wget ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4/hdf5-1.8.2.tar.gz
wget ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4/zlib-1.2.3.tar.gz

Extract the archives:

tar -xvf hdf5-1.8.2.tar.gz
tar -xvf zlib-1.2.3.tar.gz

Install zlib first

cd /u/mc/zlib-1.2.3
./configure --prefix=/u/mc
make
make install

Install HDF5

cd /u/mc/hdf5-1.8.2
./configure --prefix=/u/mc
make
make check
make install

Notes:

if you want to be sure to have the last version of zlib and hdf5, check ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-4/
that the make operation can take quite a long time.
replace /u/mc by /usr/local (or something else) for an installation on you machine

Compile

Specify the compilation options you want:

export CC=icc
export FC=ifort

Launch the configuration

./configure --enable-netcdf-4 --with-hdf5=/u/mc/ --with-zlib=/u/mc/ --prefix=/u/mc/

where /u/mc can be replaced by the location you want.

Run the tests

make check

Note that some of the tests may fail, but usually that is not a problem.

Perform the installation

make install

Once the installation is performed, new directories will appear in /u/mc:

bin/
include/
lib/
share/

In particular, the netcdf library is located in lib:

nick3:/u/mc$ ls lib/
libhdf5.a      libhdf5_hl.so        libhdf5.la        libhdf5.so.6      libnetcdf_c++.a   libz.a
libhdf5_hl.a   libhdf5_hl.so.0      libhdf5.settings  libhdf5.so.6.0.1  libnetcdf_c++.la  pkgconfig
libhdf5_hl.la  libhdf5_hl.so.0.0.0  libhdf5.so        libnetcdf.a       libnetcdf.la

Compilation with gcc

sudo aptitude install libhdf4-dev libhdf5-serial-dev 
./configure --enable-hdf4  --enable-cxx-4  --with-udunits --with-pic
make 
sudo make install

Failed options

--with-libcf

*** No rule to make target `../../libsrc/libnetcdf.la', needed by `cfcheck'.  Stop.

--enable-netcdf4

*** Testing HDF4/NetCDF-4 interoperability...
/bin/bash: line 5: 28738 Segmentation fault      ${dir}$tst
FAIL: tst_interops2
================================================
1 of 60 tests failed
================================================

Operations on NetCDF files

ncdump

The command ncdump allows the user to examine the content of a NetCDF file.

Examples are available here: http://www.unidata.ucar.edu/software/netcdf/workshops/2011/utilities/NcdumpExamples.html

Display header information

ncdump -h time file.nc

Display a given variable

ncdump -v time file.nc

netCDF Operator (NCO)

NetCDF Operator (NCO) is a set of command-line programs that take netCDF files as input and perform operations on their content.

Modification of the attributes

ncatted command permits the modification of the attributes of a NetCDF file. Here are some examples:

ncatted -a missing_value,TSM,m,d,-999 northsea_TSM.nc

replaces the attribute missing_value of the variable TSM by -999 in the file northsea_TSM.nc

Add a missing_value attribute to a file

This example is particularly useful when one wants to set the _FillValue to -99 (or -999, ...) in a NetCDF file generated by Diva.

ncatted -a _FillValue,analyzed_field,c,f,-99 results.nc

The benefit is that ncview automatically detect the missing value and does not plot it.

Ncclamp

ncclamp is another command-line tool for the replacement of values in files.

Note that you may have to modify the makefile before compilation of the code.

ncclamp northsea_TSM.nc TSM nan -999 eq

replaces all the NaN's by -999

CDO

CDO stands for Climate Data Operator. A collection of command line Operators to manipulate and analyse Climate and NWP model Data.