Microservice architecture is an approach to designing and running applications. Such applications are typically run within containers, made popular in the last few years by Docker. Containers are portable, efficient, and disposable, and contain code and any dependencies in a single package. Containerized microservices typically run a single process, rather than an entire stack within the same computing environment. This allows portions of your application to be easily replaced or scaled as needed. Microservices at UVA Research Computing runs microservices in an orchestration environment named DCOS (Distributed Cloud Operating System), based on Apache Mesos and Apache Marathon. DCOS makes the deployment and management of many containers easy and scalable.
Setup You will need to install and configure the awscli package in order to access objects in S3.
Install the AWS CLI The AWS CLI is available through the pip installer:
If you have administrator privileges type
pip install awscli Otherwise use
pip install –user awscli The project is open source, so you can also download the source at https://github.com/aws/aws-cli
Rivanna Users have two options:
Load the awscli module:
module load awscli If you need a different version, install it in your user directory:
pip install –user awscli== Authenticate the CLI to Amazon Web Services Once the aws package is installed, you will need to configure it:
// Run your Cloud computing is ideal for running flexible, scalable applications on demand, in periodic bursts, or for fixed periods of time. UVA Research Computing works alongside researchers to design and run research applications and datasets into Amazon Web Services, the leader among public cloud vendors. This means that server, storage, and database needs do not have to be estimated or purchased beforehand – they can be scaled larger and smaller with your needs, or programmed to scale dynamically with your application.
Service Oriented Architecture A key advantage of the cloud is that for many services you do not need to build or maintain the servers that support the service – you simply use it.
In addition to providing free, in-person workshop training, UVA Research Computing staff teach for-credit courses. Below is a selection of courses that members of our group have taught, co-taught or provided guest lectures:
BIMS 8382: Introduction to Biomedical Data Science Spring 2017, Spring 2018
This course introduces methods, tools, and software for reproducibly managing, manipulating, analyzing, and visualizing large-scale biomedical data. Specifically, the course introduces the R statistical computing environment and packages for manipulating and visualizing high-dimensional data, covers strategies for reproducible research, and culminates with analysis of data from a real RNA-seq experiment using R and Bioconductor packages.
CS 6501: Distributed & Cloud Computing Spring 2017, Spring 2018
General Do you have a general computing question?
Read our FAQ› Rivanna High Performance Computing Platform
Read our FAQ › Ivy Secure Data Computing Platform
Read our FAQ › Storage Research Data Storage & Transfer
Read our FAQ ›
General General tips and tricks for computational research.
General HowTos › Rivanna High Performance Computing platform
Rivanna HowTos › Ivy Secure Data Computing Platform
Ivy HowTos › Storage Research Data Storage & Transfer
Storage HowTos ›
Rivanna The primary vehicle for high-performance computing since 2014 has been the Rivanna cluster. Rivanna is a heterogenous system consisting of approximately 186 x 20-core nodes with 128GB of RAM each, 25 x 28-core nodes with 256 GB of RAM, and 152 x 40-core nodes with 384GB each. Five “big memory” nodes offer 1TB of RAM with 16 cores each. All nodes are supported by a high-performance FDR Infiniband network using Mellanox hardware and some have EDR Infiniband. A number of nodes supporting specialty hardware are included; there are 8 nodes with NVIDIA K80 GPGPUs, 4 nodes with NVIDIA V100 GPGPUs, 1 node with NVIDIA V100, and 2 nodes with NVIDIA RTX2080TI boards.
UVA Research Computing can help you find the right system for your computational workloads. From supercomputers to HIPAA secure systems to cloud-based deployments with advanced infrastructure, various systems are available to researchers.
Are you submitting a grant proposal and need standard information about UVA research computing environments? Get it here. High Performance Computing - Rivanna A traditional high performance cluster with job scheduler, large file system, modules, and MPI processing. Get Started on Rivanna Secure Computing for Highly Sensitive Data - Ivy A multi-platform, HIPAA-compliant system for secure data that includes dedicated virtual machines (Linux and Windows), JupyterLab Notebooks, and Apache Spark.
UVA Research Computing provides training opportunities covering a variety of data analysis, basic programming and computational topics. All of the classes listed below are taught by experts and are freely available to UVa faculty, staff and students.
Upcoming Workshops Workshops for 2021 have been moved to online learning formats due to campus closures from COVID-19.
DATE WORKSHOP INSTRUCTOR 01/19/2021, 01/20/2021, 01/21/2021 High Performance Python Katherine Holcomb 02/04/2021 Introduction to Shiny Christina Gancayco 02/05/2021, 02/12/2021 Parallel Matlab Ed Hall 02/11/2021 Customizing Shiny Apps Christina Gancayco 02/12/2021 Optimizing R Code Jacalyn Huband 02/18/2021 Building Containers for Rivanna Ruoshi Sun 02/19/2021 Optimizing Matlab Code Ed Hall 02/25/2021 Minimal Containers Ruoshi Sun 02/25/2021 Biopython Karsten Siller 03/16/2021 Using Bioinformatics tools on Rivanna Gladys Andino 03/12/2021 Parallelizing R Jacalyn Huband 04/09/2021 R with MPI Jacalyn Huband Research Computing is partnering with the Research Library and the Health Sciences Library to deliver workshops covering a variety of research computing topics.
UVA Research Computing (RC) is a new program that aims to support computational biomedical research by providing advanced cyberinfrastructure and expertise in data analysis at scale. Our mission is to foster a culture of computational thinking and promote interdisciplinary collaboration in various data-driven research domains. We offer services related to high performance computing, cloud architecture, scientific programming and big data solutions. We also aim to promote computationally intensive research at UVA through collaborative efforts such as UVA’s own CADRE (Computation And Data Resource Exchange) and XSEDE (Extreme Science and Engineering Discovery Environment).
One of our driving philosophies is that researchers already have medical and scientific expertise, and should not have to become computing experts on top of that.
Amazon Web Services Tiered object storage Amazon S3 and Glacier offer cloud-based, affordable, unlimited capacity for storage from anywhere. Advanced features include scalability, lifecycle management, encryption, and sharing. S3 is ideal for static files that need to be retrieved from any location (PDFs, images, video, etc.). Glacier is archival storage, perfect for grant compliance that reqires data retention. How RC can help: Lower pricing - UVA has an Internet2 discount available for educational use. Contact us to create an account for you or your research project. Cost estimates - Cloud storage is not free. Consideration should be made to the size of your files and how often they will be retrieved.
High Performance Computing Standard and high security HPC to run your code, generally written in R, Python or shell scripts.
Get Started › Secure Computing Secure virtual machines and interactive notebooks for processing HIPAA and other highly sensitive data.
Get Started › Storage Need large, or extremely large storage offsite or on grounds? Can you count in GB, TB, or PB? Learn more about storage options and pricing.
Get Started › Cloud Have an idea you’d like to test? Need an environment provisioned in short-order? We can help you build in the AWS cloud.