Octave-netcdf

From GHER

(Difference between revisions)
Jump to: navigation, search
(Installation fails)
Line 1: Line 1:
This package aims to implement the netcdf interface of MATLAB in GNU Octave.
This package aims to implement the netcdf interface of MATLAB in GNU Octave.
-
It is an alternative to the [[NetCDF_toolbox_for_Octave|octcdf]] toolbox.
+
It is an alternative to the deprecated [[NetCDF_toolbox_for_Octave|octcdf]] toolbox.
For new code, it is recommended to use this package instead of octcdf. If you want use to a even more high-level and object-oriented interface, I would suggest you to use the package [[ncArray]] (which works with both the octcdf and netcdf package in octave as well as with matlab).
For new code, it is recommended to use this package instead of octcdf. If you want use to a even more high-level and object-oriented interface, I would suggest you to use the package [[ncArray]] (which works with both the octcdf and netcdf package in octave as well as with matlab).

Revision as of 20:15, 16 June 2015

This package aims to implement the netcdf interface of MATLAB in GNU Octave. It is an alternative to the deprecated octcdf toolbox.

For new code, it is recommended to use this package instead of octcdf. If you want use to a even more high-level and object-oriented interface, I would suggest you to use the package ncArray (which works with both the octcdf and netcdf package in octave as well as with matlab).


Contents

Requirements

Your should have the netcdf 4.1 library (or later) installed (including development headers) and the nc-config script in your path. This function should return the complete path of nc-config:

which nc-config

If this is not the case, then you have to install netcdf 4 or adapt the PATH environment variable:

export PATH="/path/to/your/netcdf/dir/bin:$PATH"

Installation

Inside octave, you can download and install the package by issuing:

 pkg install -verbose -forge -auto netcdf

Double-check if netcdf uses the correct netCDF library. If you did not use the -auto flag, you must load the package explicitly (include this in your .octaverc or must do this at every octave session):

 pkg load netcdf

Test the package:

 test_netcdf

This should return:

Using NetCDF library version "4.1.1 of Nov  7 2011 11:35:16 $"
run test_netcdf_constant..............   OK  
run test_netcdf_create................   OK  
run test_netcdf_low_level_interface...   OK  
run test_netcdf_datatypes.............   OK  
run test_netcdf_scalar_variable.......   OK  
run test_netcdf_attributes............   OK  
run test_netcdf_high_level_interface..   OK  
run test_netcdf_ncwriteschema.........   OK  
run test_netcdf_ncwriteschema_unlim...   OK  
run test_netcdf_ncwriteschema_chunking   OK  
run test_netcdf_ncwriteschema_group...   OK

Low-level interface

GNU octave does not yet support namespace. One need to use the function "import_netcdf" at the beginning of every function and script which uses the low-level interface in GNU octave to use the functions with a dot (e.g. netcdf.open) instead of an underscore (e.g. netcdf_open). For portability, it is recommended to use the high-level interface. An implementation of the following function is available:

netcdf_getConstant
netcdf_getConstantNames
netcdf_inqLibVers
netcdf_setDefaultFormat
netcdf_setChunkCache
netcdf_getChunkCache
netcdf_create
netcdf_open
netcdf_abort
netcdf_sync
netcdf_setFill
netcdf_inq
netcdf_inqUnlimDims
netcdf_inqFormat
netcdf_defDim
netcdf_renameDim
netcdf_defVar
netcdf_renameVar
netcdf_defVarFill
netcdf_inqVarFill
netcdf_defVarDeflate
netcdf_inqVarDeflate
netcdf_defVarChunking
netcdf_inqVarChunking
netcdf_defVarFletcher32
netcdf_inqVarFletcher32
netcdf_endDef
netcdf_reDef
netcdf_putVar
netcdf_getVar
netcdf_close
netcdf_inqAttName
netcdf_inqAttID
netcdf_inqAtt
netcdf_getAtt
netcdf_putAtt
netcdf_copyAtt
netcdf_renameAtt
netcdf_delAtt
netcdf_inqVarID
netcdf_inqVarIDs
netcdf_inqVar
netcdf_inqDim
netcdf_inqDimID
netcdf_inqDimIDs
netcdf_defGrp
netcdf_inqGrps
netcdf_inqGrpName
netcdf_inqGrpNameFull
netcdf_inqGrpParent
netcdf_inqGrpFullNcid
netcdf_inqNcid

High-level interface

An implementation of the following function exist:

nccreate     
ncdisp       
ncinfo       
ncreadatt    
ncread       
ncwriteatt   
ncwrite      
ncwriteschema

The octcdf package (version 1.1.6) currently includes a rudimentary implementation of ncread, ncwrite and ncinfo which behaves similarly to the matlab equivalent functions. These functions will be removed in octcdf as they have been re-implemented in the netcdf package with a much higher degree of compatibility and with the ability to use netcdf4 features. The functions defined in octcdf might shadow the functions in the netcdf package. If you use both packages (netcdf and octcdf version 1.1.6), it is therefore important to load the octcdf package before the netcdf package.


Right:

>> pkg load octcdf netcdf 
>> which ncinfo
'ncinfo' is a function from the file /home/abarth/octave/netcdf-1.0.0/ncinfo.m

Wrong:

>> pkg load netcdf octcdf
>> which ncinfo
'ncinfo' is a function from the file /home/abarth/octave/octcdf-1.1.6/ncinfo.m


Example

What is missing

  • Documentation could be expanded.

Oddities of the matlab netcdf interface

The function netcdf.inqVarFletcher32 returns the string 'NOCHECKSUM' or 'FLETCHER32' (upper-case) while similar functions like netcdf.inqVarChunking return strings (storage is 'contiguous' or 'chunked') in lower case (in matlab 2013a). According to the matlab documentation all these function should return strings in upper-case. The octave interface mimics currently this odd behavior. In any case, it is better to compare the output of these functions with strcmpi (which is case insensitive).

Troubleshooting

recompile with -fPIC

On Linux with AMD 64 bit you are seeing:

`a local symbol' can not be used when making a shared object; recompile with -fPIC
/usr/lib/gcc/x86_64-linux-gnu/3.4.6/../../../../lib/libnetcdf.a: could not read symbols: Bad value

well, this means that your have to recompile NetCDF with the option -fPIC:

./configure --with-pic


HDF5 library version mismatched error

Error while reading a netcdf 4 file:

Warning! ***HDF5 library version mismatched error***
The HDF5 header files used to compile this application do not match
the version used by the HDF5 library to which this application is linked.
Data corruption or segmentation faults may occur if the application continues.
This can happen when an application was compiled by one version of HDF5 but
linked with a different version of static or shared HDF5 library.
You should recompile the application or check your shared library related
settings such as 'LD_LIBRARY_PATH'.
You can, at your own risk, disable this warning by setting the environment
variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'. 
Setting it to 2 or higher will suppress the warning messages totally.
Headers are 1.8.8, library is 1.8.5
	    SUMMARY OF THE HDF5 CONFIGURATION
	    =================================

General Information:
-------------------
		   HDF5 Version: 1.8.5-patch1
[...]

make sure that octave and netcdf was compiled with the same HDF5 library.

Multiple NetCDF libaries or libaries at non-standard location

Check the output of the shell commands:

which nc-config 
nc-conflig --libs

Does this correspond to the library that you want to use?

If not, define your PATH and LD_LIBRARY_PATH (in your .bashrc and in your current shell)

export PATH="/path/to/the/right/bin:$PATH"
export LD_LIBRARY_PATH="/path/to/the/right/lib:$LD_LIBRARY_PATH"


Crash when setting _FillValue with ncwriteatt

The following code crashes Octave with the netcdf packages AND matlab (R2013a):

>> nccreate(fname,'u_obs','Dimensions',{'x',182,'y',149,'t',inf},'FillValue',-9999.) 
>> ncwriteatt(fname,'u_obs','_FillValue',-9999.)

------------------------------------------------------------------------
       Segmentation violation detected at Thu Mar 26 13:38:28 2015
------------------------------------------------------------------------
[...]

Stack Trace (from fault):
[  0] 0x00007f93e33b8226 /opt/matlab-R2013a/toolbox/matlab/imagesci/+netcdf/private/../../../../../bin/glnxa64/libnetcdf.so.7+00623142 nc4_rec_write_metadata+00001286
[  1] 0x00007f93e33a9909 /opt/matlab-R2013a/toolbox/matlab/imagesci/+netcdf/private/../../../../../bin/glnxa64/libnetcdf.so.7+00563465
[...]


Thus bug is probably due to a bug in the NetCDF library https://bugtracking.unidata.ucar.edu/browse/NCF-187, which is currently not fixed (as for 26 March 2015). This bug could also be reproduced with the current version of the NetCDF library 4.3.3.1.


As a work-around, one should use define the FillValue during the call of nccreate:

fname = 'test.nc'
delete(fname)
nccreate(fname,'u_obs','Dimensions',{'x',182},'FillValue',-9999.)


Installation fails

You get the error message:

netcdf_constants.h: In function ‘void init()’:
netcdf_constants.h:10:47: error: ‘NC_UBYTE’ was not declared in this scope

  netcdf_constants["NC_UBYTE"] = octave_value(NC_UBYTE);

You must use the NetCDF library in version 4.1 (or later).

Personal tools