/tag/howto
Redis - A Key/Value Store
« Return to Databases
redis is an in-memory, key/value store. Think of it as a dictionary with any number of keys, each of which has a value that can be set or retrieved. However, Redis goes beyond a simple key/value store as it is actually a data structures server, supporting different kinds of values. Some fundamental concepts: Can be used as a databse, cache, or message broker Supports multiple data types and structures Built-in replication Keys can be up to 512MB in size /userinfo/howtos/general/databases/ Getting Started Rivanna has access to an open Redis service in the HPC network:
MySQL - A basic relational database
« Return to Databases
MySQL is an open-source relational database management system. It supports standard SQL syntax and models. Some important concepts are: Tables Rows Keys Schemas Data types Selects Joins Indexes Other CRUD operations Getting Started After submitting a request for a MySQL database, a username and password be created for you, this information along with your endpoint name will be sent to you via our ticketing system. Store this information somewhere secure, and do not share this information with others.
User: <your-db-username> Pass: <your-db-password> Host: <mysql-shared-endpoint-name> Port: 3306 The MySQL service is backed by a pair of servers in HA replication mode.
Introduction to Databases
There are two main families of databases: Relational and NoSQL. Relational databases store information in an orderly, column, row, and table schema. They “relate” the tables together to present different views of the data. NoSQL databases are much less structured. This means they can store different data alongside each other – which makes things both easier to store but harder to query across. There are additional types of databases, such as ledger, time-series and others. Those are beyond the scope of this introduction. Relational Databases (RDBMS) Most users have at least heard of relational databases such as:
How To Guides for Rivanna Users
Guides Building compiled code Using make Building and running MPI Code Bioinformatics on Rivanna Convert Jupyter Notebook to PDF Custom Jupyter kernels Loading Modules in Jupyter Docker images on Rivanna Adding packages to a container Migrate Python packages Launch RStudio Server from a Singularity container More Documentation Connecting Using SSH Using a browser Using FastX Jobs / Slurm / Queues Slurm Overview Queues Storage and File Transfer Storage overview Data transfer methods Allocations Allocations Overview
Transfer Files Using Amazon S3
Setup You will need to install and configure the awscli package in order to access objects in S3.
Install the AWS CLI The AWS CLI is available through the pip/pip3 installer:
If you have administrator privileges type
pip install awscli $('#copybtn235764891’).click(function(){ var $temp = $(""); $(“body”).append($temp); $temp.val($('#235764891’).text()).select(); document.execCommand(“copy”); $temp.remove(); var $this = $(this); $(this).fadeOut(200); $(this).delay(2000); $(this).fadeIn(200); }); Otherwise use
pip install –user awscli $('#copybtn416528937’).click(function(){ var $temp = $(""); $(“body”).append($temp); $temp.val($('#416528937’).text()).select(); document.execCommand(“copy”); $temp.remove(); var $this = $(this); $(this).fadeOut(200); $(this).delay(2000); $(this).fadeIn(200); }); The project is open source, so you can also download the source at https://github.
How To Tips for Storage
Map your Research Standard or Research Project Storage to your Desktop Use Globus from the Command Line Work with files in Amazon S3
SSH Keys
Users can authenticate their SSH sessions using either a password or an ssh key. The instructions below describe how to create a key and use it for password-less authentication to your Linux instances.
About SSH Keys SSH keys are a pair of encrypted files that are meant to go together. One half of the pair is called the “private” key, and the other half is the “public” key. When users use the private key to connect to a server that is configured with the public key, the match can be verified and the user is signed in. Or, put it more simply, when data is encrypted using one half of the key, it can be decrypted using the other half.
Globus Command-Line Interface
Install the Globus CLI The Globus CLI is available through the pip installer:
If you have administrator privileges type
pip install globus-cli Otherwise use
pip install –user globus-cli The project is open source, so you can also download the source at https://github.com/globus/globus-cli
If you would like to use the CLI from Rivanna, please follow these directions below.
Authenticate using the Globus CLI Log in against your institutional authentication provider. In the case of UVA, we use NetBadge for signing in:
globus login This will open a page in your web browser where you select your institution and proceed to login:
Drive Mapping on Windows or Mac OSX
Research Standard and Research Project storage can be mapped to your Windows or Mac computer as a remote drive. If you are off Grounds you must be running a VPN, such as the UVA Anywhere or the More Secure VPN from ITS. We recommend the More Secure VPN if that is available to you.
Windows Open a File Explorer page. In Windows 10 first select This PC. This is not necessary for earlier versions of Windows. When you see the Computer tab, select that, then select Map Network Drive. When the dialog box appears, the default drive letter will be Z.
How To Guides for Ivy Users
Transfer Files to or from Ivy Using Globus
How To
General General tips and tricks for computational research.
General HowTos › Rivanna High Performance Computing platform
Rivanna HowTos › Ivy Secure Data Computing Platform
Ivy HowTos › Storage Research Data Storage & Transfer
Storage HowTos ›
