/tag/storage
Service Status
Project Storage Data Migration The storage hardware serving the Research Project Storage file system has been experiencing intermittent disruptions since October 2023. In response, we acquired and installed new, upgraded hardware offering faster, more reliable access and capacity. However, because an Active File Management (AFM) connection between the old and new systems was used to facilitate automatic file transfers, the new system’s performance is being negatively impacted by the old hardware and causing accessibility issues for some users.
To mitigate these issues, Research Computing engineers are switching to an alternate backend migration process on February 26, 2024, at 9:00 a.m.
Using UVA’s High-Performance Computing Systems
Afton is the University of Virginia’s newest High-Performance Computing system. The Afton supercomputer is comprised of 300 compute node each with 96 compute cores based on the AMD EPYC 9454 architecture for a total of 28,800 cores. The increase in core count is augmented by a significant increase in memory per node compared to Rivanna. Each Afton node boasts a minimum of 750 Gigabytes of memory, with some supporting up to 1.5 Terabytes of RAM memory. The large amount of memory per node allows researchers to efficiently work with the ever-expanding datasets we are seeing across diverse research disciplines. The Afton and Rivanna systems provide access to 55 nodes with NVIDIA general purpose GPU accelerators (RTX2080, RTX3090, A6000, V100, A40, and A100), including an NVIDIA BasePOD.
Globus Data Transfer
Globus is a simple, reliable, and fast way to access and move your research data between systems. Globus allows you to transfer data to and from systems such as:
Laptops & personal workstations Rivanna/Afton HPC clusters Ivy Central Storage Lab / departmental storage Tape archives Cloud storage Off-campus resources (ACCESS, National Labs) Globus can help you share research data with colleagues and co-investigators, or to move data back and forth between a lab workstation and Rivanna/Afton or your personal computer.
Are your data stored at a different institution? At a supercomputing facility? All you need is your institution’s login credentials.
Drive Mapping on Windows or Mac OSX
Research Standard and Research Project storage can be mapped to your Windows or Mac computer as a remote drive. If you are off Grounds you must be running a VPN, such as the UVA Anywhere or the More Secure VPN from ITS. We recommend the More Secure VPN if that is available to you.
Windows Open a File Explorer page. In the left column, right click on This PC. In the drop-down box that appears, look for the Map Network Drive option. If you do not see this option, click on Show more Options, and then click on Map network drive….
Mission
Research Computing empowers UVA researchers to achieve more with cutting-edge computational resources. Our support team strives to create innovative solutions for researchers who need help solving complex optimization, parallelization, workflow, and data analysis issues. We build and maintain the University’s best computing platforms while educating the next generation of researchers on the power of advanced computing.
Commercial Data Sharing and Archiving Solutions
Box® Non-sensitive cloud storage UVA Box is a cloud-based storage and collaboration service that gives eligible members of the University community the ability to access, store, and share up to 1 TB of public / internal use University files securely—anywhere, anytime, on any device. Read more DropBox®/Sookasa® Highly Sensitive Data (PHI/PII) storage If you plan on storing highly sensitive data such as PHI or PII, UVA Health System offers a secure encrypted storage for Health System affiliated researchers, students, and staff. “DropBox Sookasa” is a free cloud-based service hosted on Dropbox that can be accessed over the internet on any device.
Rivanna Storage
There are a variety of options for storing large-scale research data at UVa. Public and Internal Use data storage systems can be accessed from the Rivanna and Afton high performance computing systems.
.tg {border-collapse:collapse;border-spacing:0;border-color:#ccc;} .tg td{font-family:Arial, sans-serif;font-size:14px;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#ccc;color:#333;background-color:#fff;} .tg th{font-family:Arial, sans-serif;font-size:14px;font-weight:normal;padding:10px 5px;border-style:solid;border-width:0px;overflow:hidden;word-break:normal;border-color:#ccc;color:#333;background-color:#f0f0f0;} .tg .tg-hy9w{background-color:#eceeef;border-color:inherit;vertical-align:top} .tg .tg-dc35{background-color:#f9f9f9;border-color:inherit;vertical-align:top} .tg .tg-0qmj{font-weight:bold;background-color:#eceeef;border-color:inherit;vertical-align:top} Storage Directories Name Quota Price Data Protection Accessible from Best Practices /home 50GB Free 1-week snapshots Rivanna/Afton /home is best used as a working directory when using Rivanna/Afton interactively. Slurm jobs run against /home will be slower than those run against /scratch. The /home directory is a personal storage space that is not shareable with other users.
Research Standard Storage
Overview Research Computing offers several budget options for storing public and internal use research data. Information Security at UVA provides details about data sensitivity
classifications.
The Research Standard storage provides users with a solution for research data storage and collaboration. Members in the same group have access to a shared directory created by
the team lead or PI. Group membership can be defined and managed through Grouper (requires VPN connection). Research Standard storage is mounted on the HPC cluster and can also be accessed on a personal computer with an SMB
mount, allowing for point-and-click file manipulation.
If you are not a researcher, UVA ITS offers Value storage for long-term storage of large scale data.
Integrated Sensor Networks in Arctic Alaska
Understanding the Changing Natural-Built Landscape in an Arctic Community Navigating the New Arctic (NNA) is one of The National Science Foundation’s 10 Big Ideas. NNA projects address convergence scientific challenges in the rapidly changing Arctic. Arctic research is needed to inform the economy, security and resilience of the Nation, the larger region and the globe. NNA empowers new research partnerships from local to international scales, diversifies the next generation of Arctic researchers, enhances efforts in formal and informal education, and integrates the co-production of knowledge where appropriate. This award fulfills part of that aim by addressing interactions among social systems, natural environment, and built environment in the following NNA focus areas: Arctic Residents, Data and Observation, Education, and Resilient Infrastructure.
Cloud Storage Solutions
Amazon Web Services Tiered object storage Amazon S3 and Glacier offer cloud-based, affordable, unlimited capacity for storage from anywhere. Advanced features include scalability, lifecycle management, encryption, and sharing. S3 is ideal for static files that need to be retrieved from any location (PDFs, images, video, etc.). Glacier is archival storage, perfect for grant compliance that reqires data retention. How RC can help: Lower pricing - UVA has an Internet2 discount available for educational use. Contact us to create an account for you or your research project. Cost estimates - Cloud storage is not free. Consideration should be made to the size of your files and how often they will be retrieved.
ITS `home1` directories unavailable on Rivanna
ITS has completed phase 1 of its network reconfiguration and ITS NAS storage volumes have been remounted on Rivanna. RC managed Research Standard and Research Project storage are also fully available. ITS home1 directories, including /nv/t* mounts, remain unavailable on Rivanna. Users will still be able to mount ITS home1 storage to their local workstations and transfer their data via Globus and the UVA Main DTN. We regret the inconvenience.
Pricing
Below is a schedule of prices for Research Computing resources.
High Performance Computing Allocations Type SU Limits Cost SU Expiration Standard None Free 12 months Purchased None $0.01 Never Instructional 100,000 Free 2 weeks after last training session A service unit (SU) resembles usage of a trackable hardware resource for a specified amount of time. In its simplest form 1 SU = 1 core hour, but the SU charge rate can vary based on the specific hardware used. Resources like GPUs and memory may incur additional SU charges.
Transfer Files Using Amazon S3
Setup You will need to install and configure the awscli package in order to access objects in S3.
Install the AWS CLI The AWS CLI is available through the pip/pip3 installer:
If you have administrator privileges type
pip install awscli $('#copybtn237465918').click(function(){ var $temp = $(""); $(“body”).append($temp); $temp.val($('#237465918').text()).select(); document.execCommand(“copy”); $temp.remove(); var $this = $(this); $(this).fadeOut(200); $(this).delay(2000); $(this).fadeIn(200); }); Otherwise use
pip install –user awscli $('#copybtn365729148').click(function(){ var $temp = $(""); $(“body”).append($temp); $temp.val($('#365729148').text()).select(); document.execCommand(“copy”); $temp.remove(); var $this = $(this); $(this).fadeOut(200); $(this).delay(2000); $(this).fadeIn(200); }); The project is open source, so you can also download the source at https://github.
ACCORD: Jupyter Lab
Back to Overview
Jupyter Lab allows for interactive, notebook-based analysis of data. A good choice for pulling quick results or refining your code in numerous languages including Python, R, Julia, bash, and others.
Learn more about Jupyter Lab
ACCORD: RStudio
Back to Overview
RStudio is the standard IDE for research using the R programming language.
Learn more about RStudio
ACCORD: Theia IDE
Back to Overview
Theia Python is a rich IDE that allows researchers to manage their files and data, write code with an intelligent editor, and execute code within a terminal session.
Learn more about the Theia Python IDE
How To Tips for Storage
Map your Research Standard or Research Project Storage to your Desktop Use Globus from the Command Line Work with files in Amazon S3
Globus Command-Line Interface
Install the Globus CLI The Globus CLI is available through the pip installer:
If you have administrator privileges type
pip install globus-cli Otherwise use
pip install –user globus-cli The project is open source, so you can also download the source at https://github.com/globus/globus-cli
If you would like to use the CLI from UVA HPC, please follow these directions below.
Authenticate using the Globus CLI Log in against your institutional authentication provider. In the case of UVA, we use NetBadge for signing in:
globus login This will open a page in your web browser where you select your institution and proceed to login:
Storage FAQs
Accidental File Deletions Why Lease Storage Leased Storage Options File Transfer with Globus Permission Denied Error with Globus Help! I deleted my files accidentally! What can I do? Your home directory contains a hidden directory .snapshot which is not visible even with ls -a. Change into that directory to find timestamped versions of your home directory. Snapshots are created once per day. For GPFS Research Project (leased) storage, the directory is /gpfs/gpfs0/project/.snapshots. For non-GPFS Research Project storage, the directory is /project/.snapshot. Neither Research Standard (leased) nor Rivanna/Afton scratch storage is backed up in any way. Why should I lease storage?
Frequently Asked Questions
General Do you have a general computing question?
Read our FAQ› Rivanna and Afton High Performance Computing Platforms
Read our FAQ › Ivy Secure Data Computing Platform
Read our FAQ › Storage Research Data Storage & Transfer
Read our FAQ ›
How To
General General tips and tricks for computational research.
General HowTos › Rivanna and Afton High Performance Computing platforms
HPC HowTos › Ivy Secure Data Computing Platform
Ivy HowTos › Storage Research Data Storage & Transfer
Storage HowTos ›
Research Data Storage
$(document).ready(function() { $("#status-message").hide(); updateStatusMessages(); setInterval(checkStatusMessages, 6000); }); function updateStatusMessages() { $.getJSON(‘https://tja4lfp3da.execute-api.us-east-1.amazonaws.com/api/messages' , function(data) { var messageData = ‘'; $.each(data, function(key, value) { var converter = Markdown.getSanitizingConverter(); var timestamp = value.body.substr(0,19); var messageBody = value.body.substring(timestamp.length+1).replace("<br>", "
“); var messageLength = messageBody.length; if ( messageLength = millisPerDay) { var datestr = now.toDateString(); } else { var datestr = post_date.toDateString(); } msg = “[” + datestr + “] “+ msg; } } $('#status-message-'+keywords[j]).html(converter.makeHtml(msg)); } }); }); }; There are a variety of options for storing research data at UVA. Public and internal use data storage systems can be accessed from the Rivanna and Afton high performance computing systems.
Data Transfer
Data transfer Public & Moderately Sensitive Data Transfer Secure Copy (scp) scp uses secure shell (SSH) protocol to transfer files between your local machine and a remote host. scp can be used with the following syntax:
scp [source] [destination]
scp SourceFile [email protected]:/scratch/mst3k
scp SourceFile [email protected]:/project/Grouper_group_name
Detailed instructions and examples for using scp are listed here.
Secure File Transfer Protocol (sftp) sftp is a network protocol for secure file management. Instructions and examples for using sftp are located here.
Graphical File-Transfer Applications Filezilla and Cyberduck, and MobaXterm are examples of open source SFTP client software for file management through an interactive graphical user interface.
Public and Moderately Sensitive Data Storage
/home /home is a free 50GB space provided to users of the HPC system and is visible from the login and compute nodes. /home is the default working directory when logging on to UVA HPC. Users can also access their home directory at /home/$USER, where $USER is an individual’s UVa computing ID.
/scratch /scratch is a Lustre high performance parallel filesystem accessible via the login and compute nodes.
UVA HPC’s scratch file system has a limit of 10TB per user. This policy is in place to guarantee the stability and performance of the scratch file system. Scratch is intended as a temporary work directory.
Research Project Storage
Overview The Research Project Storage file system provides users with a collaborative space for data storage and sharing. Members in the same group have access to a shared directory created by the team lead or PI. Group membership can be defined and managed through the Grouper (requires VPN connection). /project storage is mounted on the HPC cluster and runs on a new scale-out NAS file system.
How to request /project storage space
/project storage can be purchased for $60
/TB/YR by using this form. When filling out the form, the PI can specify the size of the /project directory and the name of an existing or new Grouper group that can access this space.
Slurm Job Manager
var cursor = true; var speed = 280; setInterval(() = { if(cursor) { document.getElementById(‘cursor’).style.opacity = 0; cursor = false; }else { document.getElementById(‘cursor’).style.opacity = 1; cursor = true; } }, speed); // Add event listener on keydown document.addEventListener(‘keydown’, (event) = { var name = event.key; if ( name === ‘y’ ) { window.location.href = “/quiz/slurm/"; } else if ( name === ‘n’ ) { $('#slurm-modal').modal(‘hide’); } }, false); SLURM Would you like to take an interactive SLURM quiz? y/N |
Overview UVA HPC is a multi-user, managed environment. It is divided into login nodes (also called frontends), which are directly accessible by users, and compute nodes, which must be accessed through the resource manager.
Sensitive Data Storage - Ivy
Overview The Ivy secure computing environment meets both HIPAA- and CUI-compliance standards and is ideal for storing highly sensitive research data. Ivy offers several storage options to fit your research computing needs.
Ivy Central Storage Ivy Central Storage (ICS) is a highly sensitive data parking zone and central storage pool with a capacity greater than 1PB. This storage space is available for researchers with highly sensitive data and can be mounted on an Ivy virtual machine (VM). For added security, files stored on ICS are read & write only. Executable files can be moved from ICS to VM storage.
Researchers can request space on ICS by first requesting an Ivy account using the Ivy request form.
User Guides
High Performance Computing Standard and high security HPC to run your code, generally written in R, Python or shell scripts.
Get Started › Secure Computing Secure virtual machines and interactive notebooks for processing HIPAA and other highly sensitive data.
Get Started › Storage Need large, or extremely large storage offsite or on grounds? Can you count in GB, TB, or PB? Learn more about storage options and pricing.
Get Started › Cloud Have an idea you’d like to test? Need an environment provisioned in short-order? We can help you build in the AWS cloud.
