Octave-netcdf

From GHER

(Difference between revisions)
Jump to: navigation, search
(recompile with -fPIC)
(Replaced content with " Go to https://github.com/Alexander-Barth/octave-netcdf/wiki")
 
Line 1: Line 1:
-
This package aims to implement the netcdf interface of MATLAB in GNU Octave.
 
-
It is an alternative to the deprecated [[NetCDF_toolbox_for_Octave|octcdf]] toolbox.
 
-
For new code, it is recommended to use this package instead of octcdf. If you want use to a even more high-level and object-oriented interface, I would suggest you to use the package [[ncArray]] (which works with both the octcdf and netcdf package in octave as well as with matlab).
+
Go to  
-
 
+
https://github.com/Alexander-Barth/octave-netcdf/wiki
-
 
+
-
= Requirements =
+
-
 
+
-
Your should have the netcdf 4.1 library (or later) installed (including development headers) and the nc-config script in your path. This function should return the complete path of nc-config:
+
-
 
+
-
which nc-config
+
-
 
+
-
If this is not the case, then you have to install netcdf 4 (preferably using your package manager). Under Debian/Ubuntu you can install NetCDF using the following command:
+
-
 
+
-
sudo apt-get install libnetcdf-dev
+
-
 
+
-
If NetCDF is installed in a non-standard location, [[Octave-netcdf#Libraries_in_non-standard_location|you must set the variables PATH and LD_LIBRARY_PATH]].
+
-
If you compile netCDF from source, be aware that:
+
-
 
+
-
* octave-netcdf requires netcdf4 features. Therefore do not use the option --disable-netcdf4 when configuring netCDF and make sure that you have the HDF5 libraries installed. For example under Debian/Ubuntu you can install HDF5 using the following command:
+
-
 
+
-
sudo apt-get install libhdf5-dev
+
-
 
+
-
* if you are only building static libraries, you must use the [[Octave-netcdf#recompile_with_-fPIC|configure option --with-pic]] on an AMD/Intel 64-bit processor.
+
-
 
+
-
= Installation =
+
-
 
+
-
Fedora users can simply install the octave-netcdf package using their package manager:
+
-
 
+
-
dnf install octave-netcdf
+
-
 
+
-
Check your software repos if octave-netcdf is not already included.
+
-
 
+
-
Otherwise, inside octave, you can download and install the package by issuing:
+
-
 
+
-
  pkg install -verbose -forge -auto netcdf
+
-
 
+
-
Double-check if netcdf uses the correct netCDF library.
+
-
If you did not use the <tt>-auto</tt> flag, you must load the package explicitly (include this in your <tt>.octaverc</tt> or must do this at every octave session):
+
-
 
+
-
  pkg load netcdf
+
-
 
+
-
Test the package:
+
-
 
+
-
  test_netcdf
+
-
 
+
-
This should return:
+
-
 
+
-
Using NetCDF library version "4.1.1 of Nov  7 2011 11:35:16 $"
+
-
run test_netcdf_constant..............  OK 
+
-
run test_netcdf_create................  OK 
+
-
run test_netcdf_low_level_interface...  OK 
+
-
run test_netcdf_datatypes.............  OK 
+
-
run test_netcdf_scalar_variable.......  OK 
+
-
run test_netcdf_attributes............  OK 
+
-
run test_netcdf_high_level_interface..  OK 
+
-
run test_netcdf_ncwriteschema.........  OK 
+
-
run test_netcdf_ncwriteschema_unlim...  OK 
+
-
run test_netcdf_ncwriteschema_chunking  OK 
+
-
run test_netcdf_ncwriteschema_group...  OK
+
-
 
+
-
= Low-level interface =
+
-
 
+
-
GNU octave does not yet support namespace. One need to use the function "import_netcdf" at the beginning of every function and script which uses the low-level interface in GNU octave to use the functions with a dot (e.g. netcdf.open) instead of an underscore (e.g. netcdf_open). For portability, it is recommended to use the high-level interface. An implementation of the following function is available:
+
-
 
+
-
netcdf_getConstant
+
-
netcdf_getConstantNames
+
-
netcdf_inqLibVers
+
-
netcdf_setDefaultFormat
+
-
netcdf_setChunkCache
+
-
netcdf_getChunkCache
+
-
netcdf_create
+
-
netcdf_open
+
-
netcdf_abort
+
-
netcdf_sync
+
-
netcdf_setFill
+
-
netcdf_inq
+
-
netcdf_inqUnlimDims
+
-
netcdf_inqFormat
+
-
netcdf_defDim
+
-
netcdf_renameDim
+
-
netcdf_defVar
+
-
netcdf_renameVar
+
-
netcdf_defVarFill
+
-
netcdf_inqVarFill
+
-
netcdf_defVarDeflate
+
-
netcdf_inqVarDeflate
+
-
netcdf_defVarChunking
+
-
netcdf_inqVarChunking
+
-
netcdf_defVarFletcher32
+
-
netcdf_inqVarFletcher32
+
-
netcdf_endDef
+
-
netcdf_reDef
+
-
netcdf_putVar
+
-
netcdf_getVar
+
-
netcdf_close
+
-
netcdf_inqAttName
+
-
netcdf_inqAttID
+
-
netcdf_inqAtt
+
-
netcdf_getAtt
+
-
netcdf_putAtt
+
-
netcdf_copyAtt
+
-
netcdf_renameAtt
+
-
netcdf_delAtt
+
-
netcdf_inqVarID
+
-
netcdf_inqVarIDs
+
-
netcdf_inqVar
+
-
netcdf_inqDim
+
-
netcdf_inqDimID
+
-
netcdf_inqDimIDs
+
-
netcdf_defGrp
+
-
netcdf_inqGrps
+
-
netcdf_inqGrpName
+
-
netcdf_inqGrpNameFull
+
-
netcdf_inqGrpParent
+
-
netcdf_inqGrpFullNcid
+
-
netcdf_inqNcid
+
-
 
+
-
= High-level interface =
+
-
 
+
-
An implementation of the following function exist:
+
-
 
+
-
nccreate   
+
-
ncdisp     
+
-
ncinfo     
+
-
ncreadatt   
+
-
ncread     
+
-
ncwriteatt 
+
-
ncwrite     
+
-
ncwriteschema
+
-
 
+
-
The octcdf package (version 1.1.6) currently includes a rudimentary implementation of ncread, ncwrite and ncinfo which behaves similarly to the matlab equivalent functions. These functions will be removed in octcdf as they have been re-implemented in the netcdf package with a much higher degree of compatibility and with the ability to use netcdf4 features. The functions defined in octcdf might shadow the functions in the netcdf package. If you use both packages (netcdf and octcdf version 1.1.6), it is therefore important to load the octcdf package '''before''' the netcdf package.
+
-
 
+
-
 
+
-
Right:
+
-
 
+
-
>> pkg load octcdf netcdf
+
-
>> which ncinfo
+
-
'ncinfo' is a function from the file /home/abarth/octave/netcdf-1.0.0/ncinfo.m
+
-
 
+
-
Wrong:
+
-
 
+
-
>> pkg load netcdf octcdf
+
-
>> which ncinfo
+
-
'ncinfo' is a function from the file /home/abarth/octave/octcdf-1.1.6/ncinfo.m
+
-
 
+
-
 
+
-
= Example =
+
-
 
+
-
* [[Write a slice at a time in a NetCDF file]]
+
-
 
+
-
= Oddities of the matlab netcdf interface =
+
-
 
+
-
The function netcdf.inqVarFletcher32 returns the string 'NOCHECKSUM' or 'FLETCHER32' (upper-case) while similar functions like netcdf.inqVarChunking return strings (storage is 'contiguous' or 'chunked') in lower case (in matlab 2013a). According to the matlab documentation all these function should return strings in upper-case. The octave interface mimics currently this odd behavior. In any case, it is better to compare the output of these functions with <tt>strcmpi</tt> (which is case insensitive).
+
-
 
+
-
= Troubleshooting =
+
-
 
+
-
== recompile with -fPIC ==
+
-
On Linux with AMD/Intel 64 bit you are seeing:
+
-
 
+
-
`a local symbol' can not be used when making a shared object; recompile with -fPIC
+
-
/usr/lib/gcc/x86_64-linux-gnu/3.4.6/../../../../lib/libnetcdf.a: could not read symbols: Bad value
+
-
 
+
-
well, this means that your have to recompile NetCDF with the compiler option -fPIC, which can be achieved by setting the following configure option:
+
-
 
+
-
./configure --with-pic
+
-
 
+
-
== HDF5 library version mismatched error ==
+
-
 
+
-
Error while reading a netcdf 4 file:
+
-
 
+
-
Warning! ***HDF5 library version mismatched error***
+
-
The HDF5 header files used to compile this application do not match
+
-
the version used by the HDF5 library to which this application is linked.
+
-
Data corruption or segmentation faults may occur if the application continues.
+
-
This can happen when an application was compiled by one version of HDF5 but
+
-
linked with a different version of static or shared HDF5 library.
+
-
You should recompile the application or check your shared library related
+
-
settings such as 'LD_LIBRARY_PATH'.
+
-
You can, at your own risk, disable this warning by setting the environment
+
-
variable 'HDF5_DISABLE_VERSION_CHECK' to a value of '1'.
+
-
Setting it to 2 or higher will suppress the warning messages totally.
+
-
Headers are 1.8.8, library is 1.8.5
+
-
    SUMMARY OF THE HDF5 CONFIGURATION
+
-
    =================================
+
-
+
-
General Information:
+
-
-------------------
+
-
  HDF5 Version: 1.8.5-patch1
+
-
[...]
+
-
 
+
-
make sure that octave and netcdf was compiled with the same HDF5 library.
+
-
 
+
-
== Multiple NetCDF libaries or libaries at non-standard location ==
+
-
 
+
-
Check the output of the shell commands:
+
-
 
+
-
which nc-config
+
-
nc-conflig --libs
+
-
 
+
-
Does this correspond to the library that you want to use?
+
-
 
+
-
If not, define your PATH and LD_LIBRARY_PATH (in your .bashrc and in your current shell)
+
-
 
+
-
export PATH="/path/to/the/right/bin:$PATH"
+
-
export LD_LIBRARY_PATH="/path/to/the/right/lib:$LD_LIBRARY_PATH"
+
-
 
+
-
 
+
-
== Crash when setting _FillValue with ncwriteatt ==
+
-
 
+
-
The following code crashes Octave with the netcdf packages AND matlab (R2013a):
+
-
 
+
-
>> nccreate(fname,'u_obs','Dimensions',{'x',182,'y',149,'t',inf},'FillValue',-9999.)
+
-
>> ncwriteatt(fname,'u_obs','_FillValue',-9999.)
+
-
+
-
------------------------------------------------------------------------
+
-
        Segmentation violation detected at Thu Mar 26 13:38:28 2015
+
-
------------------------------------------------------------------------
+
-
[...]
+
-
+
-
Stack Trace (from fault):
+
-
[  0] 0x00007f93e33b8226 /opt/matlab-R2013a/toolbox/matlab/imagesci/+netcdf/private/../../../../../bin/glnxa64/libnetcdf.so.7+00623142 nc4_rec_write_metadata+00001286
+
-
[  1] 0x00007f93e33a9909 /opt/matlab-R2013a/toolbox/matlab/imagesci/+netcdf/private/../../../../../bin/glnxa64/libnetcdf.so.7+00563465
+
-
[...]
+
-
 
+
-
 
+
-
Thus bug is probably due to a bug in the NetCDF library https://bugtracking.unidata.ucar.edu/browse/NCF-187, which is currently not fixed (as for 26 March 2015).
+
-
This bug could also be reproduced with the current version of the NetCDF library 4.3.3.1.
+
-
 
+
-
 
+
-
As a work-around, one should use define the FillValue during the call of nccreate:
+
-
 
+
-
fname = 'test.nc'
+
-
delete(fname)
+
-
nccreate(fname,'u_obs','Dimensions',{'x',182},'FillValue',-9999.)
+
-
 
+
-
 
+
-
== Installation fails ==
+
-
 
+
-
You get the error message:
+
-
 
+
-
netcdf_constants.h: In function ‘void init()’:
+
-
netcdf_constants.h:10:47: error: ‘NC_UBYTE’ was not declared in this scope
+
-
+
-
  netcdf_constants["NC_UBYTE"] = octave_value(NC_UBYTE);
+
-
 
+
-
You must use the NetCDF library in version 4.1 (or later).
+
-
 
+
-
 
+
-
== Libraries not found ==
+
-
 
+
-
The location of the necessary libraries (netCDF and HDF5) are normally detected using nc-config. If this does not work (e.g. using static libraries as this is the case for Mac OS X homebrew), you can also set CPPFLAGS and OCT_LINK_DEPS environement variable in a bash shell before starting octave from the same shell.
+
-
 
+
-
export CPPFLAGS="-I/path/to/netcdf/include -I/path/to/hdf5/include $(mkoctfile -p CPPFLAGS)";
+
-
export OCT_LINK_DEPS="-L/path/to/netcdf/lib -L/path/to/netcdf/lib -lnetcdf -lhdf5_hl -lhdf5 -lz -lm"
+
-
 
+
-
More information about the libraries necessary for netcdf can be found [http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-install/Linker-Flags.html here].
+
-
 
+
-
== Libraries in non-standard location ==
+
-
 
+
-
If libraries are installed manually (i.e. without the package manager) in a non-standard location, you must set several environment variables to install and run octave:
+
-
 
+
-
export PATH=/path/to/netcdf/bin/:$PATH
+
-
export LD_LIBRARY_PATH=/path/to/netcdf/lib/:$LD_LIBRARY_PATH
+
-
 
+
-
(by substituting /path/to/netcdf/...).
+
-
If you are unfamiliar with environment variables you might want to read the [https://help.ubuntu.com/community/EnvironmentVariables|Ubuntu documentation] or the [https://wiki.archlinux.org/index.php/Environment_variables|Arch Linux documentation].
+
-
The directory /path/to/netcdf/bin/ should contain the script nc-config and the directory /path/to/netcdf/lib/ should contain a file starting with libnetcdf.so (e.g. libnetcdf.so.7). The variables. Note that LD_LIBRARY_PATH has to be set every-time before you start octave.
+
-
 
+
-
By using the library NetCDF from your package manager (which installs in "standard" location), all these troubles would disappear.
+
-
 
+
-
= Issues and bugs =
+
-
 
+
-
If you found a bug or having trouble with this package, you can write to help@octave.org (which is the mailing list for both octave and octave-forge).
+
-
Please provide the following information:
+
-
 
+
-
* Version of octave and from where you got it (for example octave 4.0.0 from http://www.gnu.org/software/octave/download.html or octave 4.0.0 from the cygwin repository)
+
-
* Version of octave-netcdf and from where you got it (for example netcdf 1.0.0 from sourceforge or octave-netcdf 1.0.0 from Fedora's repository)
+
-
* Version of your NetCDF library
+
-
* Please indicate if you have multiple versions of the NetCDF library installed and if yes, please specify which version and at which path they are installed. You can use for example the commands:
+
-
 
+
-
locate libnetcdf.a
+
-
locate libnetcdf.so
+
-
 
+
-
* If you compiled netCDF from source, please specify the all used options of the ./configure script and the file config.log.
+
-
* Name and version of your operating system. For Linux, please specify the distribution (for example Windows 10 or Linux Fedora 42). If you use cygwin, please sent also the list of installed packages by running the command:
+
-
cygcheck -c &gt; all_packages.txt
+
-
and sent the file "all_packages.txt".
+
-
* For installation problems, please provide the output of the following commands:
+
-
 
+
-
which nc-config
+
-
nc-config --all
+
-
 
+
-
* A minimal code and minimal example file which reproduces the error.
+
-
* If you have problems reading an existing file, run <tt>ncdump -h file.nc > file.cdl</tt> and provide the file.cdl (where file.nc is the problematic file)
+
-
* Does your file conforms to the CF standard? Please check the file using http://puma.nerc.ac.uk/cgi-bin/cf-checker.pl and report if the checker finds any issues.
+
-
* The output of the script <tt>test_netcdf</tt>.
+

Latest revision as of 14:31, 21 September 2016

Go to https://github.com/Alexander-Barth/octave-netcdf/wiki

Personal tools