Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Developer Documentation #1639

Merged
merged 31 commits into from
May 26, 2022
Merged

Add Developer Documentation #1639

Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c673bb6
Add introduction.
thorstenhater Sep 2, 2021
cda6fd0
Add Solvers page.
thorstenhater Sep 6, 2021
9a68eb4
Shuffle files.
thorstenhater Sep 6, 2021
23219f1
Fix submodule.
thorstenhater Oct 25, 2021
f6dbe9a
add shared state
thorstenhater Oct 25, 2021
cdd0432
WIP: documentation on numerical methods
schmitts Oct 26, 2021
5657608
Merge branch 'doc_numerics' into doc/dev
schmitts Oct 26, 2021
5646129
Merge diagrams
brenthuisman Oct 29, 2021
b0e4b2d
bugfix: include css files. Add htmldiag
brenthuisman Apr 21, 2021
1e535ea
Add cell groups description.
thorstenhater Jan 18, 2022
7514b0a
Merge remote-tracking branch 'origin/master' into doc/dev
thorstenhater Feb 23, 2022
80a82d7
Merge remote-tracking branch 'hater/doc/dev' into doc/dev
thorstenhater Feb 23, 2022
4e0b790
Review, remove funniness.
thorstenhater Feb 23, 2022
bfacd9a
Diffusion begone, clean-up (?) cable eq discussion.
thorstenhater Feb 23, 2022
e63effe
Update.
thorstenhater Feb 23, 2022
e97b7ce
add to TOC
thorstenhater Feb 23, 2022
f4198e3
fix indentation
boeschf Apr 28, 2022
553f7e9
Merge master.
brenthuisman May 3, 2022
0535935
nmodl formatting correctons
brenthuisman May 3, 2022
af704b4
typo
brenthuisman May 10, 2022
c966585
shuffle recipe page, dial down margins on diagrams, added diagram to …
brenthuisman May 12, 2022
19b02aa
better diags in better places, collapsible
brenthuisman May 18, 2022
4d44ae0
merge internals into dev, some minor touchups to headings
brenthuisman May 18, 2022
824ad82
fix ring network line numbers and refs
brenthuisman May 18, 2022
9f02762
Have dependecy graph
brenthuisman May 18, 2022
fbd9d4a
Ecosystem overview
brenthuisman May 18, 2022
325b177
bring some order to the tuts as per #1369
brenthuisman May 18, 2022
c750212
update .readthedocs.yml
brenthuisman May 18, 2022
243bab9
back to the future
brenthuisman May 19, 2022
46be766
back to the future
brenthuisman May 19, 2022
739dcad
Remove graphviz generation, insert png outputs. Redo/replace some gra…
brenthuisman May 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 95 additions & 0 deletions doc/concepts/domdec-diag-1.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
<div class="diag-cont">
<div id="recipe_src">Recipe
<div>Cell descriptions
<div>gid 1
<div>type <code>lif_cell_0</code>
<div>...</div>
</div>
</div>
<div>...</div>
<div> gid 37
<div>type <code>cable_cell_A</code>
<div>Morphology</div>
<div>Decor
<div>Mechanisms</div>
<div>Connection sites</div>
</div>
<div>Label dictionary</div>
</div>
</div>
</div>
<div>Cell kinds
<div>gid 1
<div>type <code>arbor.lif_cell</code>
</div>
</div>
<div>...</div>
<div>gid 37
<div>type <code>arbor.cable_cell</code></div>
</div>
</div>
<div>...
</div>
</div>
<div class="diag-col">
<div class="diag-right">
Simulator: requests cell description of <code>gid=37</code>
</div>
<div class="diag-left">
Recipe: <code>type=cable_cell_A</code>.
</div>
<div class="diag-right">
Simulator: requests cell kind of <code>gid=37</code>
</div>
<div class="diag-left">
Recipe: <code>kind=arbor.cable_cell</code>.
</div>
<div class="diag-right">
Simulator: look up cell group implementation for kind <code>arbor.cable_cell</code>.
</div>
<div class="diag-right2">
Domain decomposition: <code>cable_cell_group_gpu</code>
</div>
<div class="diag-right">
Simulator: <code>cable_cell_group_gpu</code> construct <code>cable_cell_A</code> object.
</div>
<div class="diag-right3">
<code>cable_cell_group_gpu</code>: Construction complete.
</div>
<div class="diag-right">
Simulator: <code>cable_cell_group_gpu</code> simulate for <code>t .. t + dt</code>
</div>
</div>
<div>Simulation
<div id="recipe_dst">Recipe</div>
<div id="conc_src">Context
<div>
12 threads
</div>
<div>
1 GPU
</div>
</div>
<div id="domdec_src">Domain decomposition
<div>
cable_cell_group_gpu
<div>1 GPU</div>
</div>
<div>
...
</div>
</div>
</div>
</div>
<div class="diag-caption">
An illustration of the cell-specific portion of the recipe, and how it is
used during the lifetime of the simulation: the simulation object will,
depending on its configuration, query the recipe for the neuroscientific
components it describes. This demonstration also show why the recipe separates
cell descriptions from cell types. The latter is, as you might expect,
shorthand, and is used in the allocation of the cell to a particular cell group.
A cell group implementation is a handler for a certain kind of cell, and Arbor
comes with these for all it's included cell kinds. However, users can develop
their own specialized cell group implementations. More on that in the internal
developer documentation.
</div>
5 changes: 5 additions & 0 deletions doc/concepts/domdec.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@
Domain decomposition
====================

An Arbor simulation requires a :ref:`modelrecipe`, a :ref:`(hardware) context <modelhardware>`, and a domain decomposition. The Recipe contains the neuroscientific model, the hardware context describes the computational resources you are going to execute the simulation on, and the domain decomposition describes how Arbor will use the hardware. Since the context and domain decomposition may seem closely related at first, it might be instructive to see how recipes are used by Arbor:

.. raw:: html
:file: domdec-diag-1.html

A *domain decomposition* describes the distribution of the model over the available computational resources.
The description partitions the cells in the model as follows:

Expand Down
27 changes: 15 additions & 12 deletions doc/concepts/hardware.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,22 +1,25 @@
.. _modelhardware:

Hardware management
===================
Hardware context
================

An Arbor simulation requires a :ref:`modelrecipe`, a (hardware) context, and a :ref:`modeldomdec`. The Recipe contains the neuroscientific model, the hardware context describes the computational resources you are going to execute the simulation on, and the domain decomposition describes how Arbor will use the hardware. Since the context and domain decomposition may seem closely related at first, it might be instructive to see how recipes are used by Arbor:

.. raw:: html
:file: domdec-diag-1.html

*Local resources* are locally available computational resources, specifically the number of hardware threads and the number of GPUs.

An *allocation* enumerates the computational resources to be used for a simulation, typically a subset of the resources available on a physical hardware node.

.. Note::

New users can find using contexts a little verbose.
The design is very deliberate, to allow fine-grained control over which
computational resources an Arbor simulation should use.
As a result Arbor is much easier to integrate into workflows that
run multiple applications or libraries on the same node, because
Arbor has a direct API for using on node resources (threads and GPU)
and distributed resources (MPI) that have been partitioned between
applications/libraries.
New users can find using contexts a little verbose.
The design is very deliberate, to allow fine-grained control over which
computational resources an Arbor simulation should use.
As a result Arbor is much easier to integrate into workflows that
run multiple applications or libraries on the same node, because
Arbor has a direct API for using on node resources (threads and GPU)
and distributed resources (MPI) that have been partitioned between
applications/libraries.


.. _modelcontext:
Expand Down
36 changes: 36 additions & 0 deletions doc/concepts/index-diag-1.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
<div class="diag-spacer"></div>
<div class="diag-spacer"></div>
<div class="diag-cont">
<div id="recipe_src">Recipe
<div id="recipe_dsc" class="diag-note">describe the neuroscience</div>
<div>Cells
</div>
<div>Network
</div>
<div>In- & outputs
</div>
<div>...
</div>
</div>
<div id="sim_src">Simulation
<div id="sim_dsc" class="diag-note">describe the execution</div>
<div id="recipe_dst">Recipe</div>
<div class="diag-spacer"></div>
<div id="conc_src">Context
<div id="conc_dsc" class="diag-note">describe hardware</div>
</div>
<div class="diag-spacer"></div>
<div id="domdec_src">Domain decomposition
<div id="domdec_dsc" class="diag-note">describe how to use hardware</div>
</div>
</div>
</div>
<div class="diag-caption">An overview of Arbor. The two columns reflect a division that is core to Arbor: the neuroscience is described entirely separate from the execution of the simulation. The Recipe ties the neuroscience together: the user can provide any number of cells, each with a particular type, morphology and set of mechanisms; a network configuration; and in- & outputs like event generators and probes. This description, the recipe, is self-contained and can be executed by any configuration of execution. Execution is determined by the available hardware and instructions for how certain parts of a recipe (e.g. cell groups) are executed on that hardware.</div>

<connection width="1" from="#recipe_src" to="#recipe_dst" fromX="1" fromY="0" toX="0" toY="0" onlyVisible></connection>
<connection width="1" from="#recipe_src" to="#recipe_dst" fromX="1" fromY="1" toX="0" toY="1" onlyVisible></connection>

<connection from="#recipe_src" to="#recipe_dsc" color="rgb(221, 191, 146)" fromX="0.1" fromY="0" toX="0" tail onlyVisible></connection>
<connection from="#sim_src" to="#sim_dsc" color="rgb(221, 191, 146)" fromX="0.1" fromY="0" toX="0" tail onlyVisible></connection>
<connection from="#conc_src" to="#conc_dsc" color="rgb(221, 191, 146)" fromX="0.1" fromY="0" toX="0" tail onlyVisible></connection>
<connection from="#domdec_src" to="#domdec_dsc" color="rgb(221, 191, 146)" fromX="0.1" fromY="0" toX="0" tail onlyVisible></connection>
8 changes: 8 additions & 0 deletions doc/concepts/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,14 @@
Concepts overview
=================

Arbor is a library that lets you to model neural networks with morphologically
detailed cells; which it then executes the resulting simulation on a variety of
hardware. The execution can optionally be configured in high detail but comes
with sensible defaults.

.. raw:: html
:file: index-diag-1.html

To learn how to use Arbor, it is helpful to understand some of its concepts.
Arbor's design aims to enable scalability through abstraction.
To achieve this, Arbor makes a distinction between the **description** of a model, and the
Expand Down
121 changes: 121 additions & 0 deletions doc/concepts/recipe-diag-1.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
<div class="diag-cont">
<div id="recipe_src">Recipe
<div>
<input id="collapsible1" type="checkbox" checked><label for="collapsible1">Cell descriptions</label>
<div>
<input id="collapsible10" type="checkbox"><label for="collapsible10">Cell 0</label>
<div>Morphology
<div>Segment Tree
<div>Segment 0 (soma)</div>
</div>
</div>
<div>Decor
<div>Paintables
<div>hh
<code>(all)</code>
</div>
</div>
<div>Placables
<div>iclamp <code>"midsoma"</code></div>
<div>detector_0 <code>"midsoma"</code></div>
</div>
</div>
<div>Label dictionary
<div>midsoma
<code>(location 0 0.5)</code>
</div>
</div>
</div>
<div>
<input id="collapsible11" type="checkbox"><label for="collapsible11">Cell 1</label>
<div>Morphology
<div>Segment Tree
<div>Segment 0 (soma)</div>
<div>Segment 1 (dendrite)</div>
<div>Segment 2 (dendrite)</div>
<div>Segment ...</div>
</div>
</div>
<div>Decor
<div>Paintables
<div>passive
<code>(all)</code>
</div>
</div>
<div>Placables
<div>synapse_1 <code>"endpoint"</code></div>
<div>gap_junction_1 <code>"midsoma"</code></div>
</div>
</div>
<div>Label dictionary
<div>midsoma
<code>(location 0 0.5)</code>
</div>
<div>endpoint
<code>(terminal)</code>
</div>
</div>
</div>
<div>
<input id="collapsible12" type="checkbox"><label for="collapsible12">Cell 2</label>
<div>Morphology
<div>Segment Tree
<div>Segment 0 (soma)</div>
<div>Segment 1 (dendrite)</div>
<div>Segment 2 (dendrite)</div>
<div>Segment ...</div>
</div>
</div>
<div>Decor
<div>Paintables
<div>passive
<code>(all)</code>
</div>
</div>
<div>Placables
<div>gap_junction_2 <code>"midsoma"</code></div>
</div>
</div>
<div>Label dictionary
<div>midsoma
<code>(location 0 0.5)</code>
</div>
</div>
</div>
</div>
<div>
<input id="collapsible2" type="checkbox"><label for="collapsible2">Cell kinds</label>
<div>Cell 0
<div>type <code>arbor.cable_cell</code></div>
</div>
<div>Cell 1
<div>type <code>arbor.cable_cell</code></div>
</div>
<div>Cell 2
<div>type <code>arbor.cable_cell</code></div>
</div>
</div>
<div>
<input id="collapsible3" type="checkbox"><label for="collapsible3">Connections</label>
<div>Connection 1
<div>from <code>detector_0</code></div>
<div>to <code>synapse_1</code></div>
</div>
<div>Gap Junction 1
<div>from <code>gap_junction_1</code></div>
<div>to <code>gap_junction_2</code></div>
</div>
</div>
<div>
<input id="collapsible4" type="checkbox"><label for="collapsible4">Probes</label>
<div>Probe 1
<div>gid
<code>2</code>
</div>
</div>
</div>
</div>
</div>
<div class="diag-caption">
An example recipe. Click "+" and "-" to (de)collapse contents.
</div>
29 changes: 17 additions & 12 deletions doc/concepts/recipe.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,11 @@ building phase to provide information about individual cells in the model, such
* **Gap junction connections** on each cell.
* **Probes** on each cell.

Recipes are structured to provide a consistent interface for describing each cell in the
network using their global identifier (`gid`).
This allows the simulator to be able to quickly look-up properties related to the connections
going in and out of a cell (think of synapses, gap junctions, but also probes and spike inputs);
which helps make Arbor fast and easily distributable over many nodes.
An example model
----------------

.. raw:: html
:file: recipe-diag-1.html

To better illustrate the content of a recipe, let's consider the following network of
three cells:
Expand All @@ -33,7 +33,7 @@ three cells:
(because cable cells allow complex dynamics such as ``hh``). This is referred to as
the **kind** of the cell.
- ``Cell 1``: Is a soma and a single dendrite, with ``passive`` dynamics everywhere.
It has a single synapse at the end of the dendrite labeled "syanpse_1" and a gap
It has a single synapse at the end of the dendrite labeled "synapse_1" and a gap
junction mechanism in the middle of the soma labeled "gap_junction_1".
This is the **description** of the cell. It's also a cable cell, which is its **cell kind**.
- ``Cell 2``: Is a soma and a single dendrite, with ``passive`` dynamics everywhere.
Expand All @@ -44,7 +44,7 @@ The total **number of cells** in the model is 3. The **kind**, and **description
is known and can be registered in the recipe. Next is the cell interaction.

The model is designed such that each cell has labeled source, target and gap junction sites.
A **network connection** can be formed from ``detector_0`` to ``synpase_1``; and a
A **network connection** can be formed from ``detector_0`` to ``synapse_1``; and a
**gap junction connection** between ``gap_junction_1`` and ``gap_junction_2``.
If ``detector_0`` spikes, a spike should be observed on ``gap_junction_2`` after some delay.
To monitor the voltage on ``gap_junction_2`` and record the spike, a **probe** can be set up
Expand All @@ -59,12 +59,17 @@ Technical details of the recipe class are presented in the :ref:`Python <pyreci
:ref:`C++ <cpprecipe>` APIs.

Are recipes always necessary?
------------------------------
-----------------------------

Yes. They are a fundamental building block in Arbor simulations. Recipes are structured to provide
a consistent interface for describing each cell in the network using their global identifier (`gid`).
This allows the simulator to be able to quickly look-up properties related to the connections
going in and out of a cell (think of synapses, gap junctions, but also probes and spike inputs);
which helps make Arbor fast and easily distributable over many nodes.

Yes. However, we provide a python :class:`single_cell_model <py_single_cell_model>`
that abstracts away the details of a recipe for simulations of single, stand-alone
:ref:`cable cells<modelcablecell>`, which absolves the users from having to create the
recipe themselves. This is possible because the number of cells is fixed and known,
For single, stand-alone :ref:`cable cells<modelcablecell>` simulations, the Python API provides
a :class:`single_cell_model <py_single_cell_model>` that abstracts away the details of a recipe.
This is possible because the number of cells is fixed and known,
and it is guaranteed that there can be no connections or gap junctions in a model of a
single cell. The single cell model is able to fill out the details of the recipe under
the hood, and the user need only provide the cell description, and any probes they wish
Expand Down
2 changes: 2 additions & 0 deletions doc/conf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@
sys.path.append(script_path)

html_static_path = ['static']
html_css_files = ['htmldiag.css']
html_js_files = ['domarrow.js']

def setup(app):
app.add_object_type('generic', 'gen', 'pair: %s; generic')
Expand Down
Loading