HDF5 file format now in use at the BRDC

A significant improvement to the management of BRDC datasets has been introduced in the form of a change in the file format to HDF5. This change is valid as of the July, 2001 CD. The HDF5 file format is developed and maintained by the HDF Group at the National Centre for Supercomputing Applications, University of Illinois at Urbana-Champaign. Using HDF5 addresses the weaknesses of the previously used so-called BALTRAD format. Fundamental advantages of HDF5 are:

• Platform independence
• General-purpose format for virtually any scientific data
• Efficient built-in compression using ZLIB (gzip, WinZip)
• Binary tools for simplified file management by non-programmers
• APIs for C, Fortran, Java. Interfaces to Python, GNU-Octave, IDL, Vis5d
• Support for parallel I/O using Message Passing Interface (MPI)
• Excellent documentation and human support
• Free

Check out the documentation on how HDF5 is being used at the BRDC. This documentation is available as PostScript or PDF files, or as on-line HTML pages.

Use of HDF5 with meteorological data is an issue that has been addressed within the framework of COST-717 (Use of radar observations in hydrological and NWP models). HDF5 provides a superior platform to current WMO standards and can surely show the way to improved management of operational data. The BRDC datasets will serve to exemplify this.