Getting Data Into R¶
The last block of Day 2 is a short demo of the bridge between QIIME2 and R: how to export the artifacts you've just produced into flat TSV / CSV files that R can ingest directly. The full export reference (every metric and every command you'll likely want for publication-quality figures) lives on the Publication Plots in R (Self-Study) page, work through that on your own afterward.
Why Export?¶
For downstream analysis outside of qiime2, it is helpful to export qiime2 data so it can be used with other data analysis tools such as R or Python.
We export QIIME 2 data by unzipping the .qza file because a .qza artifact is essentially a compressed (zipped) directory. Inside this archive is a structured collection of files, including the actual analysis data (e.g., diversity metrics), as well as supporting files such as provenance information, metadata, version information, and checksums.
The provenance folder records the full analysis history of the artifact, allowing reproducibility and transparency of all upstream steps. The checksums file ensures data integrity by verifying that the contents have not been altered. The data/ directory contains the primary output of interest (e.g., alpha diversity values or ordination results).
By unzipping the .qza file, we can directly access the underlying data files for use in downstream analyses outside of QIIME 2.
- .qza files are just zipped directories. They can be unzipped and you can take out the text file with the data you need for downstream analysis.
Note that there are multiple ways to export data from QIIME2 this tutorial goes over one way, the Publication Plots in R (self study) has examples of additional ways to export data.
R Stats and Figures Tutorial
This page demonstrates how the metrics were exported for the Publication Plots in R (Self-Study) page. There is a complete directory provided for you in the tutorial that includes this data. That way everything is already in the correct directory when you work through the R tutorial.
Exporting via unzip
The following steps for alpha diversity and beta diversity are examples of how to export via unzip.
Alpha diversity¶
We will export these alpha diversity metrics: shannon_vector.qza, observed_features_vector.qza, faith_pd_vector.qza, and eveness_vector.qza.
Create a new directory for the data we want to export
In the decomp tutorial directory on alpine make an export directory, we will put all of the metrics we want to export in this one directory to make dowloading them easier.
Unzip the shannon vector qza After unzipping, you’ll notice a UUID-named folder insideexport/shannon/. QIIME 2 stores each artifact inside a uniquely identified directory for provenance tracking. Within that folder, the data/ directory contains the actual Shannon diversity values (alpha-diversity.tsv), which is the file we will use for downstream analysis.
If you look into the directory that was just generated from unziping the shannon_vectory.qza you will see a data directory that is where the data we want to export lives.
Unzip the other alpha diversity metrics
# Observed Features
unzip core_metrics/observed_features_vector.qza -d export/observed_features
# Faith's PD
unzip core_metrics/faith_pd_vector.qza -d export/faith_pd
# Pielou's evenness
unzip core_metrics/evenness_vector.qza -d export/evenness
Clean up the unziped alpha diversity metrics Make an alpha diversity directory
# move into the export directory
cd export
# make a dedicated alpha diversity directory
mkdir alpha_div
# define alpha metrics
metrics=("shannon" "evenness" "faith_pd" "observed_features")
# copy their tsv files into alpha_div/
for metric in "${metrics[@]}"; do cp "$metric"/*/data/alpha-diversity.tsv "alpha_div/${metric}.tsv"; done
-
The rename matters (
cp): every alpha export writes toalpha-diversity.tsvby default, so you have to rename right after export inorder to correctly name the metric, and if you export additional metrics they could be overwritten. -
Organizing the files this way puts all alpha diversity metrics in one clean directory with consistent filenames, making them easier to import into R for downstream statistical analysis and visualization.
Beta Diversity¶
Lets repeat what we did witht he alpha diversity metrics with our beta diversity metrics.
# Bray Curtis
unzip core_metrics/bray_curtis_pcoa_results.qza -d export/bray_curtis
# Jaccard
unzip core_metrics/jaccard_pcoa_results.qza -d export/jaccard
# Unweighted Unifrac
unzip core_metrics/unweighted_unifrac_pcoa_results.qza -d export/unweighted_unifrac
# Weighted Unifrac
unzip core_metrics/weighted_unifrac_pcoa_results.qza -d export/weighted_unifrac
Clean up the unziped beta diversity metrics Make a beta diversity directory
# move back into the export directory
cd export
# make a directory dedicated to the extrated beta diversity metrics
mkdir beta_div
Copy the tsv files into the beta_div directory
# define beta metrics
metrics=("bray_curtis" "jaccard" "unweighted_unifrac" "weighted_unifrac")
# copy their txt files into beta_div/
for metric in "${metrics[@]}"; do cp "$metric"/*/data/ordination.txt "beta_div/${metric}.txt"; done
Now, if you want to download all of these metrics at once to your local computer, go to OnDemand and select the export directory. From there, you can download all of the unzipped files at once. Alternatively, you can navigate into the export directory and download only the alpha_div or beta_div folders, which contain just the TSV files.
Exporting via qiime2
The following steps for taxonomy export are done via qiime2 commands so an interactive session will need to be started and qiime2 will need to be activated.
Taxonomy¶
Load an interactive session
Load qiime2
Move in to the dada2 directory
Transpose the feature table (swap the rows (typically features) and columns (typically samples) of a feature table)
qiime feature-table transpose \
--i-table table_nomitochloro_nocontrol.qza \
--o-transposed-feature-table table_nomitochloro_nocontrol_transposed.qza
Turn the transposed table into a vizulization that be viewed in qiime2 view.
qiime metadata tabulate \
--m-input-file table_nomitochloro_nocontrol_transposed.qza \
--m-input-file seqs.qza \
--m-input-file ../taxonomy/taxonomy_gg2.qza \
--o-visualization tabulated_results.qzv
tabulated_results.qzv and open it on view.qiime2.org click the download tsv. This download can then be copy and pasted into your R analysis directory and loaded into R.
- This file has already been exported and put into the taxonomy directory in the r_decomp_tutorial direcotry for the R tutorial.
These are the files that are used in the R stats and publication demo.
Join them to your metadata on the sample-ID column and you're set up for ggplot2, vegan, phyloseq, or whatever downstream package you prefer.
Going Further¶
The Publication Plots in R (Self-Study) page contains qiime2 export commands for:
- All four α-diversity vectors (Shannon, Faith's PD, observed features, evenness)
- All four β-diversity distance matrices (Bray-Curtis, Jaccard, weighted & unweighted UniFrac)
- The taxonomy + feature table joined into a single tabulated visualization
It also walks you through copying the workshop's decomp_stats_figures.Rmd and R.Rproj into your working directory, knitting the report, and producing figures.
This completes Day 2. Continue to Day 3, Cow Dataset Pipeline.