Data storage on Ceph
As part of our services, we offer access to Ceph, a distributed storage system. Ceph is a scalable, fault-tolerant, and high-performance platform and supports object, block, and file storage.
This offering is intended to be a fast, low-cost bulk storage service for research data. Because it is hosted locally at NYU and connected to the High Speed Research Network, speeds will be much better than going to the cloud, and by building our own storage, we can achieve a lower cost than commercial public cloud services.
Note that this service is not intended to replace the high performance storage options available on NYU's supercomputer, which are noticeably faster but more costly.
Access CephFS via our Data Transfer Nodes
Access to the distributed filesystem (CephFS) is possible via our Data Transfer Nodes (DTN). These are separate from HPC's data transfer nodes.
The filesystem is mounted on the machine giving you access via a standard Linux interface. SSH, SFTP, and rsync are available on the machine, as well as rclone and other standard tools.
Access management is configured via Linux extended access control lists (the setfacl command) which lets you grant access to specific users and groups. Access to the machine and group membership is mediated via NYU's Grouper system.
Please contact us to get access to CephFS and the DTNs: hsrn-support@nyu.edu. Note that this service is intended for research data, not admin or student desktop backups.
More information about CephFS and the DTNs
S3-compatible storage
We offer object storage on Ceph using an API compatible with Amazon S3. Please contact us if you are interested: hsrn-support@nyu.edu.
Nextcloud
We provide Nextcloud, a web platform similar to Google Drive. Nextcloud allows you to manage files from a web browser, share them with collaborators, and sync them to your device using a dedicated app.
The files are stored in our Ceph cluster and can be made available from the Data Transfer Nodes.
Nextcloud is currently being deployed, stay tuned for more general access. Please contact us if you are interest in taking part of our beta test: hsrn-support@nyu.edu.