Hierarchical Data Format (version 4 and 5)
Hierarchical Data Format (HDF) is a data file format designed by the National Center for Supercomputing Applications (NCSA) to assist users in the storage and manipulation of scientific data across diverse operating systems and machines. NCSA developed a library of callable routines and a set of utility programs and tools for creating and using HDF files. This work is now performed by The HDF Group (THG) (https://www.hdfgroup.org/).
HDF supports a variety of data types: scientific data arrays, tables, and text annotations, as well as several types of raster images and their associated color palettes.
There are two distinct varieties of HDF, known as HDF (version 4 and earlier) and the newer HDF5. These are described in more detail at the THG HDF web site.
Some of the features of HDF are:
- HDF makes it possible for programs to obtain information about the data from the data file itself, rather than from another source.
- HDF standardizes the format and descriptions of many types of commonly used data sets, such as raster images and scientific data.
- HDF is a platform independent file format. It can be used on many different computers, regardless of the operating system that machine is running.
- New data models may be added to HDF by either the development team or HDF users.
HDF files are also self-describing. For each data object in an HDF file, there are predefined tags that identify such information as the type of data, the amount of data, its dimensions, and its location in the file. The self-describing capability of HDF files has important implications for processing scientific data. It makes it possible to fully understand the structure and contents of a file just from the information stored in the file itself. A program that has been written to interpret certain tag types can scan a file containing those tag types and process the corresponding data. Self-description also means that many types of data can be bundled in an HDF file. For example, it is possible to accommodate symbolic, numerical, and graphical data in one HDF file.
HDF5 File Format Specification Version 3.0