We are a home for Earth science data and computing professionals. Our sessions bring together the community for hands-on, interdisciplinary deep dives as we explore "Innovation to Impact" this year. Learn more about ESIP: esipfed.org
Session Description HDF5 has quietly enabled breakthroughs across geosciences for over two decades. From NASA satellite missions to climate and weather modeling (as netCDF-4 backend) it has helped scientists to manage, share, and make sense of complex data seamlessly across various computing platforms. This session will present enhancements slated for the upcoming release 2.0 of the HDF5 library: new datatypes for ML/AI applications and complex numbers, new default settings when accessing cloud optimized HDF5 files, and easier use of advanced compression methods. The topics covered will also include latest features and improvements for accessing: (1) NASA's EOSDIS HDF5/netCDF-4, HDF-EOS5, and HDF4 data via OPeNDAP DMR++ web service in the AWS cloud; (2) cloud native HDF5/JSON-based format (HSDS) using Python.
Value to Session Participants ESIP community typically does not provide feedback to HDF software developers outside of ESIP meetings, probably because for many it is buried into their software stacks. However, many of the performance bottlenecks or workflow obstacles come from the data access software. Sessions like this one help educate broader ESIP community and also provide them with an opportunity to report their data access issues without having to provide comprehensive technical details.
Recommended Ways to Prepare for this Session No recommendations provided
