diff --git a/README.md b/README.md index db07ecc3e..066754875 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ The output of augur is a series of JSONs that can be used to visualize your resu * [Technical documentation for Augur](https://nextstrain-augur.readthedocs.io/en/stable/installation/installation.html) * [Contributor guide](https://github.com/nextstrain/.github/blob/master/CONTRIBUTING.md) * [Project board with available issues](https://github.com/orgs/nextstrain/projects/6) -* [Developer docs for Augur](./DEV_DOCS.md) +* [Developer docs for Augur](./docs/contribute/DEV_DOCS.md) ## Quickstart diff --git a/docs/conf.py b/docs/conf.py index 9649daff5..d1d2ffea3 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -53,7 +53,7 @@ def prose_list(items): # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = ['recommonmark', 'sphinx.ext.autodoc', 'sphinxarg.ext', 'sphinx.ext.napoleon', 'sphinx_markdown_tables'] +extensions = ['recommonmark', 'sphinx.ext.autodoc', 'sphinxarg.ext', 'sphinx.ext.napoleon', 'sphinx_markdown_tables', 'sphinx.ext.intersphinx'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] @@ -61,7 +61,19 @@ def prose_list(items): # List of patterns, relative to source directory, that match files and # directories to ignore when looking for source files. # This pattern also affects html_static_path and html_extra_path. -exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store', + 'contribute/DEV_DOCS.md', + 'faq/colors.md', + 'faq/fasta_input.md', + 'faq/import-beast.md', + 'faq/lat_longs.md', + 'faq/seq_traits.md', + 'faq/translate_ref.md', + 'faq/vcf_input.md', + 'tutorials/tb_tutorial.md', + 'tutorials/zika_tutorial.md', + 'usage/augur_snakemake.md', +] # A string of reStructuredText that will be included at the end of every source # file that is read. This is a possible place to add substitutions that should @@ -78,6 +90,12 @@ def prose_list(items): # html_theme = 'nextstrain-sphinx-theme' +html_theme_options = { + 'logo_only': False, # if True, don't display project name at top of the sidebar + 'collapse_navigation': False, # if True, no [+] icons in sidebar + 'titles_only': True, # if True, page subheadings not included in nav +} + # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". @@ -88,3 +106,9 @@ def prose_list(items): html_css_files = [ 'css/custom.css', ] + +# -- Cross-project references ------------------------------------------------ + +intersphinx_mapping = { + 'docs.nextstrain.org': ('https://docs.nextstrain.org/en/latest/', None), +} diff --git a/DEV_DOCS.md b/docs/contribute/DEV_DOCS.md similarity index 99% rename from DEV_DOCS.md rename to docs/contribute/DEV_DOCS.md index cf483790e..77f83edf5 100644 --- a/DEV_DOCS.md +++ b/docs/contribute/DEV_DOCS.md @@ -1,6 +1,6 @@ -# Development Docs for Contributors +# Augur Development Docs for Contributors -Thank you for helping us to improve Nextstrain! This document describes: +Thank you for helping us to improve Augur! This document describes: - Getting Started - Contributing code diff --git a/docs/faq/augur_snakemake.md b/docs/faq/augur_snakemake.md deleted file mode 120000 index c45f33e95..000000000 --- a/docs/faq/augur_snakemake.md +++ /dev/null @@ -1 +0,0 @@ -../usage/augur_snakemake.md \ No newline at end of file diff --git a/docs/faq/community_hosting.md b/docs/faq/community_hosting.md deleted file mode 100644 index 79de1689b..000000000 --- a/docs/faq/community_hosting.md +++ /dev/null @@ -1,45 +0,0 @@ -# Sharing your analysis - -[nextstrain.org](https://nextstrain.org) has a feature that allows you to share your own analysis through the nextstrain.org website. -This works using github as a repository for your analysis files. -To share an analysis, you need to create a repository. -The name of the repository will be what is used to access the results. -Within this repository, there should a folder called auspice which contains the output json files of the augur pipeline. -Importantly, the name of these files has to start with the name of the repository. -If so, you should be able to access your analysis via the nextstrain community feature. - -As an example, lets look at one of our nextstrain community analysis. -The following link shows you an analysis we made a few month ago of many influenza B sequences: - -[nextstrain.org/community/neherlab/allflu/B_ha](https://nextstrain.org/community/neherlab/allflu/B_ha) - -The analysis files are hosted on our github page in the reposity - -[github.com/neherlab/allflu](https://github.com/neherlab/allflu) - -In this repository, you'll find the folder `auspice` which contains the files -``` -allflu_B_ha_meta.json -allflu_B_ha_tree.json -``` -Note that all files start with "allflu" which matches the name of the repository. -In fact, there are multiple analysis in this folder. The corresponding files all start with "allflu" but they differ in the viral lineage they correspond to: -``` -allflu_B_ha_meta.json -allflu_B_ha_tree.json -allflu_h1n1_ha_meta.json -allflu_h1n1_ha_tree.json -allflu_h1n1pdm_ha_meta.json -allflu_h1n1pdm_ha_tree.json -allflu_h3n2_ha_meta.json -allflu_h3n2_ha_tree.json -``` -All these can be accessed as - -[nextstrain.org/community/neherlab/allflu/h1n1_ha](https://nextstrain.org/community/neherlab/allflu/h1n1_ha) - -[nextstrain.org/community/neherlab/allflu/h3n2_ha](https://nextstrain.org/community/neherlab/allflu/h3n2_ha) - -[nextstrain.org/community/neherlab/allflu/h1n1pdm_ha](https://nextstrain.org/community/neherlab/allflu/h1n1pdm_ha) - - diff --git a/docs/faq/faq.rst b/docs/faq/faq.rst index bcb6e83d0..9dfd5bb5b 100644 --- a/docs/faq/faq.rst +++ b/docs/faq/faq.rst @@ -10,15 +10,7 @@ common questions and problems users run into. :maxdepth: 1 :glob: + what-is-a-build metadata - translate_ref clades - community_hosting - import-beast - colors - lat_longs Specifying `refine` rates - Using Augur and Snakemake - vcf_input - fasta_input - seq_traits \ No newline at end of file diff --git a/docs/faq/what-is-a-build.md b/docs/faq/what-is-a-build.md new file mode 100644 index 000000000..405c67163 --- /dev/null +++ b/docs/faq/what-is-a-build.md @@ -0,0 +1,32 @@ +# The concept of a 'build' + +Nextstrain's focus on providing a _real-time_ snapshot of evolving pathogen populations necessitates a reproducible analysis that can be rerun when new sequences are available. +The individual steps necessary to repeat analysis together comprise a "build". + + +Because no two datasets or pathogens are the same, we build Augur to be flexible and suitable for different analyses. +The individual Augur commands are composable, and can be mixed and matched with other scripts as needed. +These steps, taken together, are what we refer to as a "build". + + +### Example build + +The [Zika virus tutorial](https://docs.nextstrain.org/en/latest/tutorials/zika.html#build-steps) describes a build which contains the following steps: + +1. Prepare pathogen sequences and metadata +2. Align sequences +3. Construct a phylogeny from aligned sequences +4. Annotate the phylogeny with inferred ancestral pathogen dates, sequences, and traits +5. Export the annotated phylogeny and corresponding metadata into auspice-readable format + +and each of these can be run via a separate `augur` command. + +If you look at the [other tutorials](https://docs.nextstrain.org/en/latest/tutorials/index.html), each one uses a slightly different combination of `augur` commands depending on the pathogen. + +### Snakemake + +While it is possible to run a build by running each of the individual steps, we typically group these together into a make-type file. +[Snakemake](https://snakemake.readthedocs.io/en/stable/index.html) is "a tool to create reproducible and scalable data analyses... via a human-readable, Python-based language." + +> Snakemake is installed as part of the [conda environment](https://docs.nextstrain.org/en/latest/guides/install/local-installation.html#install-augur-auspice-with-conda) or the [docker container](https://docs.nextstrain.org/en/latest/guides/install/cli-install.html). +If you ever see a build which has a "Snakefile" then you can run this by typing `snakemake --cores 1` or `nextstrain build --cpus 1 .`, respectively. diff --git a/docs/index.rst b/docs/index.rst index d1e5669f8..ead0a788c 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -5,9 +5,19 @@ Augur: A bioinformatics toolkit for phylogenetic analysis *One held to foretell events by omens.* (`Merriam-Webster `__) +.. note:: + The documentation you are viewing is Augur's reference guide, which means it is information-oriented and targeted at users who just need info about how Augur works. + + * If you have a question about how to achieve a specific goal with Augur, check out our :doc:`Augur-focused How-to Guides section ` in the main Nextstrain documentation. + * If you want to learn the basics of how to use Augur from scratch, check out our :doc:`Zika tutorial ` in the main Nextstrain documentation. + * If you want to understand how Augur fits together with Auspice to visualize results, check out our :doc:`Data Formats section ` in the main Nextstrain documentation. + + + Augur is a bioinformatics toolkit to track evolution from sequence and serological data. It provides a collection of commands which are designed to be composable into larger processing pipelines. Augur originated as part of `Nextstrain `__, an open-source project to harness the scientific and public health potential of pathogen genome data. +All source code is available on `GitHub `__. .. note:: We have just released version 6 of augur -- `check our upgrading guide `__ @@ -26,7 +36,6 @@ The ``refine`` step is necessary to ensure that cross-referencing between tree n The different augur modules can be strung together by workflow managers like snakemake and nextflow. The nextstrain team uses `snakemake `__ to run and manage the different analysis that you see on `nextstrain.org `__. - .. toctree:: :maxdepth: 2 :caption: Table of contents @@ -37,7 +46,6 @@ The nextstrain team uses `snakemake _Make sure that these have corresponding entry in the lat-longs TSV file supplied to `export`. See how to do this [here](/faq/lat_longs)._ +> _Make sure that these have corresponding entry in the lat-longs TSV file supplied to `export`. See how to do this [here](https://docs.nextstrain.org/en/latest/guides/bioinformatics/lat_longs.html)._ + --- @@ -528,5 +529,6 @@ In Auspice v2, all values are now displayed exactly as they arrive, allowing use Don't forget to also change them in any custom lat-long and/or coloring files you are using. We've also become stricter about the format of the files that pass in color and lat-long information. Previously, it didn't matter if columns were separated by spaces or tabs - now, they must be separated by tabs. -You can find out more about how to add [custom coloring](/faq/colors) and [lat-long](/faq/lat_longs) values. +You can find out more about how to add [custom coloring](https://docs.nextstrain.org/en/latest/guides/bioinformatics/colors.html) and [lat-long](https://docs.nextstrain.org/en/latest/guides/bioinformatics/lat_longs.html) values. + If you use the command `parse` to generate a metadata table from fields in a fasta header, you can use the flag `--prettify-fields` to apply some prettifying operations to specific metadata entries, see the documentation [`parse`](/usage/cli/parse). diff --git a/docs/releases/v6.md b/docs/releases/v6.md index 0d75a53a6..9bdbcec9d 100644 --- a/docs/releases/v6.md +++ b/docs/releases/v6.md @@ -46,7 +46,7 @@ Users can ask for this output and specify a file name using `--output-sequences` Deprecation warning: The argument `--output` is now deprecated. Please use `--output-node-data` instead. ## Import BEAST MCC trees -We now have instructions and functionality to import BEAST trees, see [here](/faq/import-beast). +We now have instructions and functionality to import BEAST trees, see [here](https://docs.nextstrain.org/en/latest/guides/bioinformatics/import-beast.html). ## Prettifying of strings Previous auspice version "prettified" metadata strings (like changing 'north_america' to 'North America'). @@ -110,4 +110,4 @@ We've tried to use redirects to ensure that all the old links continue to work. * Errors in formatting of input files (e.g. metadata files, Auspice config files) weren't handled nicely, often resulting in hard-to-interpret stack traces. We now try to catch these and print an error indicating the offending file. -* Tests using Python version 2 have now been removed. \ No newline at end of file +* Tests using Python version 2 have now been removed. diff --git a/docs/tutorials/tb_tutorial.md b/docs/tutorials/tb_tutorial.md index 8b9b095fe..c40b2965e 100644 --- a/docs/tutorials/tb_tutorial.md +++ b/docs/tutorials/tb_tutorial.md @@ -7,7 +7,7 @@ As in the Zika fasta-input [tutorial](zika_tutorial), we'll build up a Snakefile ## Setup -To run this tutorial you'll need to [install augur](../installation/installation) and [install Snakemake](https://snakemake.readthedocs.io/en/stable/getting_started/installation.html). +To run this tutorial you'll need to [install augur](../guides/install/augur_install.md) and [install Snakemake](https://snakemake.readthedocs.io/en/stable/getting_started/installation.html). ## Build steps Nextstrain builds typically require the following steps: diff --git a/docs/tutorials/tutorials.rst b/docs/tutorials/tutorials.rst deleted file mode 100644 index 831ee9a90..000000000 --- a/docs/tutorials/tutorials.rst +++ /dev/null @@ -1,12 +0,0 @@ -========= -Tutorials -========= - -.. note:: We have just released version 6 of augur -- `check our upgrading guide <../releases/migrating-v5-v6.html>`__ - -.. toctree:: - :maxdepth: 1 - :caption: Available tutorials - - zika_tutorial - tb_tutorial diff --git a/docs/tutorials/zika_tutorial.md b/docs/tutorials/zika_tutorial.md index 69ffbdc96..f7bafb66b 100644 --- a/docs/tutorials/zika_tutorial.md +++ b/docs/tutorials/zika_tutorial.md @@ -7,7 +7,7 @@ We will work off the tutorial for Zika virus on the [nextstrain web site](https: ## Setup -To run this tutorial you'll need to [install augur](../installation/installation) and [install Snakemake](https://snakemake.readthedocs.io/en/stable/getting_started/installation.html). +To run this tutorial you'll need to [install augur](../guides/install/augur_install.md) and [install Snakemake](https://snakemake.readthedocs.io/en/stable/getting_started/installation.html). ## Augur commands diff --git a/docs/usage/cli/filter.rst b/docs/usage/cli/filter.rst index f34c0bdbe..3ebf97072 100644 --- a/docs/usage/cli/filter.rst +++ b/docs/usage/cli/filter.rst @@ -15,7 +15,7 @@ augur filter How we subsample sequences in the zika-tutoral ============================================== -As an example, we'll look that the ``filter`` command in greater detail using material form the `zika tutorial <../../tutorials/zika_tutorial.html>`__. +As an example, we'll look that the ``filter`` command in greater detail using material form the :doc:`zika tutorial `. The filter command allows you to selected various subsets of your input data for different types of analysis. A simple example use of this command would be @@ -45,7 +45,7 @@ To drop such strains, you can pass the name of this file to the augur filter com --output filtered.fasta (To improve legibility, we have wrapped the command across multiple lines.) -If you run this command (you should be able to copy-paste this into your terminal) on the data provided in the `zika tutorial `__, you should see that one of the sequences in the data set was dropped since its name was in the ``dropped_strains.txt`` file. +If you run this command (you should be able to copy-paste this into your terminal) on the data provided in the :doc:`zika tutorial `, you should see that one of the sequences in the data set was dropped since its name was in the ``dropped_strains.txt`` file. Another common filtering operation is subsetting of data to a achieve a more even spatio-temporal distribution or to cut-down data set size to more manageable numbers. The filter command allows you to select a specific number of sequences from specific groups, for example one sequence per month from each country: diff --git a/docs/usage/usage.rst b/docs/usage/usage.rst index ffe2b95e6..476305957 100644 --- a/docs/usage/usage.rst +++ b/docs/usage/usage.rst @@ -26,6 +26,5 @@ For instance, the documentation for `augur filter <./cli/filter.html>`__ shows h :maxdepth: 2 cli/cli - augur_snakemake json_format envvars