New Diva Features
Beta testers are welcomed:
- Correlated observational errors
- Better file structures (input and driver better separated from command) in 4D loops
- Automatic selection of solver (parallel, serial, iterative) depending on the problem type and size
...work in progress...
- Rtimesx_weighting.f90 : truncation of CDI.
- test_inpolygon.F90, dvreadfinal4D.F, dvwritefinal4D.F : bug fix when cutting 4D netcdf (deepest variables with time indexes higher than 1 were empty in the smaller file).
Released in June 2016 ; this is the current official version (diva-4.7.1)
- Major feature : bottom analysis is now possible. The distance is counted from the bottom ocean, derived from the interpolation of the topography topo_fine.grd. This topography can be different (and finer) than topo.grd used for the creation of contours.
- Major feature : conversion of EMODnet bathymetry to Diva-readable format with the tool "emobath2ghertopo".
- Major feature : Variable correlation length depending on the gradient of the depth. Advection field adapted to this relative length field. Suited for bottom analyses. Updated programs : diva3Ddat,divarlvargraddepth,rlvargraddepth.f90,divadoall,divaUVtopo,UVtopogen.f,divadocommit.
- Acknowledgements field in 3D and 4D netCDFs.
- New compilation option DIVAHUGEMEMORY. Enabled by default in divacompile_options, it allow the use of a finer mesh, and/or a larger domain. Particularly useful with variables characterized by low correlation lengths.
- divacutNCDF : correction on climatology bounds + dealing with very big obsid vector
- divacompile_options : new tests on nc-config, due to recent change in its behaviour
- dv3Dreadnc.F : warning if dimensions are incoherent between GridInfo.dat and the netcdf file
- divadoall,divadoNCDF,divadoNCYR : corrected handling of 3DNCinfo and 3DNClist files so that recreating a new 4D NC file is much easier
- divadoxml : removed because deprecated (use divadoxml-gui instead)
- Update of the user guide
Released in October 2015.
- Creation of very large 4D netcdf files (> 2Go)
- divacutNCDF : a tool to cut the domain of your final NetCDF
- divaselectorODV4 : more general detection of non-sdn profiles through the use of position
- dvvinterpodv : warning in case of varying time inside a single profile
- anatransf.F : generation of a correct reference field when a transformation (log, logit,...) is asked for the reference
- dv3Dinvtrsh : fieldatdatapoint.anl files are now correctly "inverse transformed" when a transfo is asked
- dv3Drefoanl,dvrefetransf : cleaning of the code (path, log files)
- dvrefetransf : correct transfo of reference field when a transfo is asked for the analyse
- refetransf.F : handling of exclusion values in ref transfo
- diva3Dwrt.F : correction of the NetCDF time axis in some winter season (ex. 1101) crossing new years (no problem if 1202)
- dv3Drefoanl : always recompute databins files, in order to avoid old files impacting the final netcdf dimensions
- boudarypoints.f : minor correction avoiding diva crash in very rare cases (if ispec=-113)
- divaselectorODV4 : buf fix for winter season was forgotten in 4.6.10
- Update of the user guide
Released in June 2015.
- In case of close measurements (space and time), the representativity errors can be mitigated via a weighting option. Particularly suited for time series.
- Increased compatibility with Mac OS
- Extraction of topography from GEBCO more robust and accurate
- Test of presence of bc (divatest)
- divadetrend : avoid error calculation during iterations when cpme or exerr is asked (increased speed)
- Progress towards no longer need to specify the PATH
- fitlsn.f : floating point exception warnings in some cases during optimization of CL
- dv3Drefoanl : forcing ireg to 0 in case of analysis with reference field
- dvrefetransf : now allow whitespaces in var (for datapoint ref files)
- divaselectorODV4 : extraction more robust when dealing with similar columns containing whitespaces
- divaweighting : weighting more robust for neighbouring layers of empty ones
- divaweighting : safeguard in case of param.par absence
- divaselectorODV4 : more robust detection of the depth column
- divaselectorODV4 : winter season across new year more robust. REMARK : the previous year is used for month(s) before January
- Update of the user guide
- New document : summary of Diva installation on Mac OS divaonmac.pdf
Released in March 2015.
- Creation of contours if they are absent (+ warning) but needed for cleaning/optimization/analysis
- Use of whitespaces in variable names is allowed
- Safeguards added when optimizing parameters with too few data
- obsid,obslon,obslat,obsdepth and obstime now included in the global 4D netcdf file
- Optimization of parameters + vertical filtering in Example4D
- Increased speed of SN optimization in some cases, by avoiding the useless calculation of analysis errors
- Diva now also handles time series (profile and time series have to be in separated ODV files)
- gebco2diva : subsampling 10x10 by default
- better compatibility with Mac OS
- ncdf2gher.f90 : generalization to variables other than "bat"
- dv3Drefoanl : now handles variable names with blanks
- dvdatatransf,dv3Dintrsh,datatransf.F,anatransf.F : handles variables names with blanks (when using transformation flags 11,12,...)
- main_tomvec.f90 : bug fixed when filtering CL vertically in some cases
- diva3Dwrt.F : correction of the time axis when month period is across two years (ex : dec-feb)
- dv3Dncwrt.F,diva3Dwrt.F : bug solved with climatology bounds when using 3D version
Patch of 14-03-2015
This patch allow the use of the data weighting option. In case of close measurements (space and time), the representativity errors can be mitigated via this weighting option. Particularly suited for time series. The script to download the patch and apply it to your diva-4.6.9 is here : (applypatch-14-03-2015-diva-4.6.9) . It has to be placed in your existing local directory ~/diva-4.6.9/ and then launched by ./applypatch-14-03-2015-diva-4.6.9. If the script is not executable, change the permissions with this command : chmod +x applypatch-14-03-2015-diva-4.6.9.
Released in December 2014.
Corrections and improvements from the Diva workshop 2014.
- Warning if depths in the wrong order in "contour.depth"
- Warning if no contour (level below sea bottom)
- Presence of dos2unix is checked in "divatest"
- Different reference field for each month period (parameter "refsameasmonthlist" in "divadoall")
- Driver : option -1 for "minimal number of data" (= never use data from other months)
- Example4D : calculation of errors (available in the output netcdf)
- Better handling of new ODV files (metadata containing depth,...)
- Divadoall.log extended to the standard error (stderr)
- Godiva : more complete error check
- Gher2ncdf and ncdf2gher tools (format conversion)
- Topography from diva on web : new tool to erase zones from a netcdf file (masktopo.nc)
- Clearer title in 4D netcdf (units, years)
- Update of documentation
New bug fixes
- Data extraction : problems with "pres" or "bottom.depth" in ODV file solved
- Binning of data much faster
- Script "asctobin" replaced by "divaasctobin"
- No more warnings at compilation
- Other small bugs were fixed
Patch of 24-12-2014
This patch allow the use of whitespaces in the variable names. The script to download the patch and apply it to your diva-4.6.8 is here : (applypatch-24-12-2014-diva-4.6.8) . It has to be placed in your existing local directory ~/diva-4.6.8/ and then launched by ./applypatch-24-12-2014-diva-4.6.8. If the script is not executable, change the permissions with this command : chmod +x applypatch-24-12-2014-diva-4.6.8.
Released in October 2014.
- Transformation of user relative length or advection fields files (ascii format) into the gher binary format, via a run of Diva (new script "asctobin")
New bug fixes
- Correction of time axis and climatology_bounds in Netcdf output files (diva3Dwrt.F,diva4Dwrt.F,dv4DYRwrt.F,dv3DncYRw.F)
- Correction of some attributes in 4D netcdf (databins, snr, cl, varbak) (dv3DncYRw.F, diva3Dsub)
- Update of driver files (also in Example4D)
Released in September 2014.
- Check for severe errors in DIVA 3D/4D (script "godiva") + simple errors and warnings
- Possibility of binning the data before the parameters optimization (script "divabin" + program "binning_lines.f90")
- Variable correlation length, depending on depth (script "divarlvardepth" + program "rlvardepth.f90")
New bug fixes
- Correction of the example in 4D (datasource)
- Correction of the script divaguessformODV4
- Exact match needed between variable name in "varlist" and its real name in the data file.
Released in April 2014.
New bug fixes
- "end of line" problems under Windows (file "datasource")
- Portability of scripts using the "sort" command
- Vertical filtering of correlation length : case of 1 and 2 layer(s)
- Wrong min and max values in the netcdf output file (error and analyzed field) when using some values of ispec
- Error field not written in the netcdf output file under some values of ispec
- Other small fixes
Release date, February 2014.
- Introduction of Logit transformation
- Use of a mask file to introduce a relative correlation length field in Diva2D
New bug fixes
- Minor bug corrections following the Diva workshop 2013
Released in June 2013.
- Two additional solvers: a parallel and iterative version
- Optimisations for large data sets
- Optimisations of file exchanges for use with ODV
- Highly optimised new version of the grid generator (typical reduction of a factor 10 in CPU time)
- Integration of faster error calculations into the 3D/4D version (using new ispec values)
- Clearer log files for debugging
(note: With the availability of gfortran, we decided to abandon the support of pure f77 compilations and this release needs Fortran 90 or Fortran 95 and -cpp preprocessing tools).
Example of performance gain using divabigtest (test with 1.5 million data points and a very fine mesh):
CPU time in seconds in 4.5.1 4.6.1 for - mesh 27 2 - analysis 318 159
The results presented in the right-hand side shows how the computation time evolves:
- for the computation when the number of observations increases.
- for the mesh generation when the correlation length is modified.
Version 4.5.1 was released in March 2013 and includes the following new features, partly based on the feedback we got from the DIVA workshop 2012 in Roumaillac.
- Advection constraint with linear decay rate and local sources (such as found for radioactive tracers and river discharges)
- divadetrend now allows one to change easily the order in which detrending is done (for example first years then months or the inverse)
- two new error calculations are provided:
- one quick version divacpme with better quality than the original quick version of the poor man’s error;
- the other divaexerr is an almost exact error calculation much faster than the exact calculation.
These two options will be implemented into the 4D version for version 4.6.0 so that error fields will be available with more reasonable CPU times for final climatology productions
- Simplification of installation and compilation with additional tests of correct installation
- Housekeeping of the code (simplifications, error messages, cleaning up of code, further optimisations, elimination of depreciated tools)
- New documentation largely augmented with examples and new tool descriptions
- Possibilities to call DIVA from other software via system calls, exemplified by a matlab function divagrid.m
- divadoxml adapted to new specifications from IFREMER
Released January 2012
- simple compiled programs replaced by awk
Several releases during 2009 and 2010
- 3D/4D version with
- Adapting and automatisation of data selection for DIVA input (including vertical interpolation from ODV4 spreadsheets)
- Loops on depth, parameters, seasons etc
- Parameter optimisation (robustness and vertical coherency)
- Automated plotting interface
- Preparing NetCDF files with CF compliance
- Faster and more robust mesh generation, optimisation of computational load (for incorporation of stripped version into ODV)
- Adequate contour generation from a topography mask (including tools to retrieve topography fields)
- Automated cross validation for choice of analysis parameters (including fast real cross validation)
- Some additional diagnostic tools (fitting of correlation length)
- Data-quality control (spatial coherence)
- Non-uniform correlation length
- Exact error calculation and poor man's error calculation (for coupling with ODV)
- Advection constraint
- Hydrostatic constraint in 3D mode
- Option to eliminate data that are not on the mesh before estimation of L and S/N
- Option to create a data set from which outliers are eliminated (outliers determined from a preliminary DIVA execution)
- Possibility to create a pseudo-velocity field so as to increase correlations along topography
- Option ot use a relative correlation length spatially changing depending on data coverage
- Option to calculate a lower limit for correlation length (based on average data distance)
- Additional output fields (data coverage)
- Detrending (with help of J. Carstensen) to deal with heterogeneous data distribution
- Error calculation for domain integrals (heat content, total biomass etc)
Release: May 2008
Implementation of new tools:
- divaUVtopo: creates pseudo velocity field based on topography.
- adapation of divafit to take the mean distance between data as an inferior limit.
- divadatacoverage now creates a file containing the mean distance (./output/meandistance.dat)
Release: November 2007
- Outputs for coupling with ODV
- netCDF support
- automated contour generation
- Topography generation
- Windows Version
- First tools to help calibrating parameters (divafit, cross validation techniques)
Release: October 2006
- Basic Diva gridding tool
- Simplified input files (param.par data.dat coast.cont)