Activating QIIME2 in Alpine & Importing Metadata¶
The point at which the workshop transitions from concepts to commands. Activate the QIIME2 module on Alpine, set up your working directory, copy the workshop metadata, and verify QIIME2 can read it via qiime metadata tabulate.
Before We Start: Setting Up the Environment for Windows Users¶
Windows computers sometimes have trouble copying and pasting multi-line commands into the Alpine terminal. This is because Windows uses different line-ending behavior than Linux or macOS. Windows often adds carriage returns at the ends of lines automatically, which can cause only one line of a pasted command to run in Alpine.
There are a couple of options if you are using a Windows computer. You can copy the commands into VS Code or another code editor and make them one line before pasting them into the terminal. After the workshop, you can also install QIIME 2 on your local computer; the copy-and-paste issue for multi-line commands does not seem to be as much of a problem when QIIME 2 is run locally.
For this workshop, we are going to add a code block to your .bashrc file. This code block will disable carriage returns so you can copy multi-line commands into the terminal. However, it also disables the Return key when trying to submit a command, so you will need to use Ctrl + J to run commands.
In OnDemand, go to your home directory. Then click Show Dot Files. After that, click the three dots next to the .bashrc file and choose Edit. At the bottom of the .bashrc file, add this code block:
# Modify copy and paste for Windows users, carriage returns
case $- in
*i*)
stty igncr 2>/dev/null
;;
esac
Before We Start item #2: Silence package warning¶
In OnDemand, go to your home directory. Then click Show Dot Files. After that, click the three dots next to the .bashrc file and choose Edit. At the bottom of the .bashrc file, add this code block:
First we need to load the workshop node (only use this line once, or there wont be enough allocated resources for everyone)¶
sinteractive --reservation=microbiome --time=04:00:00 --partition=amilan --nodes=1 --ntasks=6 --qos=normal
Note: This command will only work during this workshop. we have reserved these resources for this week only, with all future use of Alpine, you will need to use ainteractive --time=04:00:00 --ntasks=6
Activate the QIIME2 Module¶
Verify the install:
Set Up Your Working Directory¶
First, we'll build a directory to conduct the analysis in. We will do this in our scratch directory, but remember that the files in this directory are deleted after 90 days of inactivity, so if we wanted to retain these results we'd need to transfer them to projects for storage. We just need to run commands in the scratch directory because there are more resources available to us here.
Use the following commands (adding in your username for the first line) to log into our scratch directory, create a "decomp_tutorial" directory, and move into that directory.
Notice that in the command line, your location changed once you used cd. On your terminal screen, you went from:
$USER to decomp_tutorial
Sometimes half the battle is making sure you know where you are and where your files are... this means pwd becomes your best friend!
Use pwd to check that you are in the correct directory. If you are you should see /scratch/alpine/username@colostate.edu/decomp_tutorial
Metadata¶
Metadata is very, very important. It is all of the information associated with your samples. When you analyze your data, the metadata that you have associated with your samples is how you can notice trends (e.g., is there a trend by time, treatment, body site, geography, etc). Otherwise, you would have a bunch of microbiome data and nothing to associate it with! Metadata is all information that you as a researcher should record thoroughly for your own project. You should include anything that you think might have an effect on your hypothesis, such as environmental variables, host sex and age (if applicable), date of collection, any potential biological or technical confounders, etc. It's better to have too much metadata than not enough.
Why else do I need metadata? Metadata are created and maintained because they improve use of research data and enable their re-use. Good metadata can make up for human shortcomings. People forget and misplace things, and leave research projects, taking their knowledge of the research methodology and the data with them.
Some notes about metadata formatting:
Make sure to never include spaces in your column names. If you need a space, use an underscore instead. In some cases you can get away with using periods in place of spaces, but it's best to avoid them.
When you can, make your sample names informative because this can be helpful with the initial visualizations and troubleshooting. In some cases samples need to be anonymized, and that's okay. Also, never have duplicate sample names for different samples (even your controls need unique names)
Copy the Workshop Metadata¶
# Make a directory for the metadata using the mkdir command while you are INSIDE of the decomp_tutorial directory
mkdir metadata
Now, we will use cp command (this is a new one! this stands for "copy") to download the metadata for this study:
- don't forget that period at the end!
Rename the metadata using the mv (move) command
Validate the Metadata¶
Open metadata.qzv on view.qiime2.org and confirm the columns and types render the way you expect from the metadata decomposition tutorial. Take some mental notes about the kinds of variables you see - why do you think they will be important for this study?
Note that you can always edit your metadata file in Excel if needed and continue to use it in QIIME2. Converting your metadata file into a .qzv isn't always necessary because you can always look at it in Excel, but it is good practice for those just learning to use QIIME2.
Next: Importing Sequences