HSDS - Highly Scaleable Data Service¶
The HSDS is a Restful web service for working with HDF5 data.
HDF5 lets you store huge amounts of numerical data, and easily manipulate that data from NumPy. For example, you can slice into multi-terabyte datasets stored on disk, as if they were real NumPy arrays. Thousands of datasets can be stored in a single file, categorized and tagged however you want.
Where to start¶
Other resources¶
Introductory info¶
Installation¶
- Installation
- Installation using Docker with AWS S3 storage
- Installation with Docker on Azure
- Installation using Docker with Tencent Cloud object storage
- Installation using Docker with POSIX storage
- Installation using DCOS with AWS S3 storage
- Installation with Kubernetes on AWS
- Installation with Azure Kubernetes
- Deploy HSDS to AKS
- HSDS for AWS Lambda
- Function Creation
- General Usage
- Access Control
- Building the Lambda Image
- API Gateway
- Using h5pyd with HSDS Lambda
- Post Install Configuration
- Docker setup instructions
REST API¶
Use cases¶
- HSDS Use Cases
- View end-point metadata
- Extract a one dimensional subset along a single dimension
- Extract many one dimensional subsets along a single dimension
- Extract subsets from data sets that share common dimensions and grid (run parallel)
- Extract a two dimensional subset from a data set
- Extract a three dimensional subset from a data set