Lau Lab Server Standard Operating Procedure
Overview
The Lau lab has two servers, Threadripper and Matisse, each named after the CPU it houses. Both are physically located in Ken’s office, so they depend on the lab’s wired ethernet connection and power supply.
Their main job is processing single-cell RNA-sequencing data, and their hardware was chosen and tuned for that. They handle data that ordinary computers can’t because of memory and time limits. Both are CPU-heavy machines with weak GPUs, so don’t expect to do much GPU-accelerated image analysis or deep learning on them.
| Threadripper | Matisse | |
|---|---|---|
| IP address | 10.141.26.164 | 10.141.26.101 |
| CPU | AMD Threadripper 2990WX, 32-core / 64-thread @ 3.00 GHz | AMD Ryzen 3900X, 12-core / 24-thread @ 3.8 GHz |
| GPU | AMD FirePro W2100, 2 GB VRAM | AMD RX570, 4 GB VRAM |
| Memory (RAM) | 128 GB DDR4 | 64 GB DDR4 |
| Storage | 2 TB SSD, 500 GB SSD, 2× 2 TB HDD | 1 TB SSD |
| OS | Ubuntu 20.04 LTS | Ubuntu 20.04 LTS |
💡 Which server should I use? Matisse has fewer cores and threads, but its base clock is about 26% higher than Threadripper’s. It can’t run as many tasks in parallel, but it runs single-threaded tasks faster. Use Threadripper for large, highly parallel or memory-hungry jobs.
Server Etiquette
Check Usage Before You Start
Every time you connect to a server, check how busy it is with:
top
If someone is running a large job (high CPU and high memory usage), use the #server Slack channel to coordinate resources and timing, or switch to the other server.
Reading the top Output
The top section of the display shows system-level resources. The key fields are:
| Field | What it means |
|---|---|
| %Cpu | Computational load across all available threads |
| GiB Mem used | RAM used by all connected users. ⚠️ Critical: going over the limit (e.g. 125.8 GB on Threadripper) may automatically kill processes |
| GiB Swap avail Mem | Remaining headroom. The smaller this number, the higher the risk of running out of memory |
Useful top Shortcuts
| Key | Action |
|---|---|
E | Cycle display units for per-process memory (e.g. bytes → GB) |
U | Filter processes by username |
Shift + E | Cycle display units for the system-level summary at the top |
Shift + M | Sort processes by memory usage |
Shift + P | Sort processes by CPU usage |
q | Quit top |
Memory Rule of Thumb
⚠️ Do not exceed 30 GB of memory in a single session for standard, count-based scRNA-seq analysis.
30 GB is roughly what you’d use analyzing dozens of combined atlas datasets at once. If your data isn’t that large and you’re near this limit, take a second look at the notebooks you’re running, the datasets loaded into those environments and the variables you’re holding in memory.
If you’ve determined you truly need more than 30 GB and other users are connected to your server, post an advisory message in #server on Slack so others know. Failing to do so will earn you sad emojis 😢 and may push us toward automated resource restrictions, which limit everyone.
Connection and Environment Setup
Using the servers requires setup on your local machine (home desktop, laptop, lab desktop, etc.) and on your server profile. Once that’s done, you can connect and set up your analysis environment.
1. Connect from Your Local Machine
- Connect to the Vanderbilt VPN using Pulse Secure (only needed off-campus)
- Open a terminal:
- Windows: install Git Bash and open it
- Mac / Linux: open the built-in Terminal
-
Run the SSH command for your server:
Threadripper
ssh -L 1234:127.0.0.1:XXXX your-username@10.141.26.164Matisse
ssh -L 1234:127.0.0.1:XXXX your-username@10.141.26.101-
XXXXis a 4-digit port number above 9000 of your choosing. It’s the port the server uses to send data back to your machine, so remember it for later. - This command connects to the server and forwards that port to port
1234on your local machine.
-
- Enter your password when prompted. By default it’s the server name followed by your first and last initials, all lowercase (e.g.
threadripperab). Change it after your first login withpasswd.
💡 Tip: Save the SSH command in a text file (e.g.
connect.sh) and run it withbash connect.shso you don’t have to retype it every time.
2. Use tmux (Highly Recommended)
A dropped connection usually terminates everything you were running. tmux creates sessions you can detach from and re-attach to, so your work survives disconnects.
| Action | Command |
|---|---|
| Start a new session | tmux new -s scanpy_session (a green bar at the bottom means it worked) |
| Detach (send to background) | Ctrl + b, release, then d |
| Re-attach | tmux attach-session -t scanpy_session |
| List running sessions | tmux ls |
| Close a session completely | exit (while attached) |
Use ls at any time to see the files in your current directory.
3. Install Anaconda
- Download the installer:
curl -O https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh - Run it:
bash Anaconda3-2020.02-Linux-x86_64.sh - Respond to the prompts:
- Accept the license terms
- Keep the default install location
- Say yes to initializing conda / modifying your bash profile
- Disconnect (
exit) and reconnect to the server. If everything worked, you’ll see(base)at the left of your command prompt.
4. Create a Conda Environment
Environments keep each project’s Python packages separate, since different packages may need conflicting dependency versions.
- Create an environment (answer yes to any prompts):
conda create --name my_env python=3.7 - Activate it:
conda activate my_env - Make sure the environment is active before installing anything. Packages installed with
pip installorconda installare available only while that environment is active. - Install JupyterLab:
pip install jupyterlab
5. Launch JupyterLab on the Server
jupyter lab --no-browser --port=XXXX
⚠️
XXXXmust be the same port number you used in your SSH command.
This hosts JupyterLab on the server at that port, which is forwarded to your local machine.
Managing the Jupyter Process
| Action | Command |
|---|---|
| Suspend the process and get your terminal back | Ctrl + Z |
| Resume it in the background (keeps Jupyter running while you use the terminal) | bg |
| List background jobs | jobs |
| Bring it back to the foreground | fg |
| Stop Jupyter (while in the foreground) | Ctrl + C twice |
⚠️ After
Ctrl + Z, Jupyter is paused until you runbgorfg. Your browser tab will hang until you do.
6. Open Jupyter in Your Local Browser
- In any browser on your local machine, go to:
localhost:1234 - The first time, it will ask for a token. In your server terminal, find the output line that looks like
localhost:XXXX/?token=..., then copy everything aftertoken=into the browser. Make sure you don’t copy any extra spaces. - You should now see your JupyterLab environment, hosted on the server.
💡 All file paths in Jupyter are relative to the server directory where you ran
jupyter lab.
Exiting Server Operations
On your local machine:
- Save everything (
Ctrl + Sworks as usual) - Go to the Running tab in JupyterLab and shut down active notebooks
On the server terminal:
- Make sure the Jupyter process is in the foreground. Use
fgif needed. (If you can’t type commands, it’s already in the foreground.) - Press
Ctrl + Conce to bring up the exit prompt, then again to confirm - Type
exitto disconnect. You can now close the window.
Resetting Everything
If things get stuck and you need to start over, or you’re completely done with the server:
pkill -u your-username
⚠️ Warning: This kills all processes running under your username. Anything not saved to disk will be lost.
Copying Files Between Servers
Log in to the destination server first, then pull files from the other one.
Logged in to Matisse, copying from Threadripper:
scp username@10.141.26.164:/home/username/* /home/username
Logged in to Threadripper, copying from Matisse:
scp username@10.141.26.101:/home/username/* /home/username
To copy whole directories, add the -r flag:
scp -r username@10.141.26.164:/home/username/my_folder /home/username
Enjoy Reading This Article?
Here are some more articles you might like to read next:
- Google Gemini updates: Flash 1.5, Gemma 2 and Project Astra
- Displaying External Posts on Your al-folio Blog
- Lau Lab NAS Standard Operating Procedure
- a post with marimo snippets
- a post with plotly.js
- a post with image galleries
- a post with tabs
- a post with typograms
- a post that can be cited
- a post with pseudo code