diff --git a/.readthedocs.yml b/.readthedocs.yml index 2baa5a9bbc..2750945e28 100644 --- a/.readthedocs.yml +++ b/.readthedocs.yml @@ -12,8 +12,12 @@ sphinx: formats: [] +build: + os: ubuntu-20.04 + tools: + python: "3.7" + python: - version: 3.7 system_packages: true install: - requirements: doc/requirements.txt \ No newline at end of file diff --git a/CMakeGraphVizOptions.cmake b/CMakeGraphVizOptions.cmake new file mode 100644 index 0000000000..d121a6090e --- /dev/null +++ b/CMakeGraphVizOptions.cmake @@ -0,0 +1 @@ +set(GRAPHVIZ_IGNORE_TARGETS "bench;brunel;drybench;dryrun;gap_junctions;generators;lfp;probe-demo;ring;single;unit") diff --git a/doc/concepts/domdec-diag-1.html b/doc/concepts/domdec-diag-1.html new file mode 100644 index 0000000000..b4c14cdb1b --- /dev/null +++ b/doc/concepts/domdec-diag-1.html @@ -0,0 +1,95 @@ +
+
Recipe +
Cell descriptions +
gid 1 +
type lif_cell_0 +
...
+
+
+
...
+
gid 37 +
type cable_cell_A +
Morphology
+
Decor +
Mechanisms
+
Connection sites
+
+
Label dictionary
+
+
+
+
Cell kinds +
gid 1 +
type arbor.lif_cell +
+
+
...
+
gid 37 +
type arbor.cable_cell
+
+
+
... +
+
+
+
+ Simulator: requests cell description of gid=37 +
+
+ Recipe: type=cable_cell_A. +
+
+ Simulator: requests cell kind of gid=37 +
+
+ Recipe: kind=arbor.cable_cell. +
+
+ Simulator: look up cell group implementation for kind arbor.cable_cell. +
+
+ Domain decomposition: cable_cell_group_gpu +
+
+ Simulator: cable_cell_group_gpu construct cable_cell_A object. +
+
+ cable_cell_group_gpu: Construction complete. +
+
+ Simulator: cable_cell_group_gpu simulate for t .. t + dt +
+
+
Simulation +
Recipe
+
Context +
+ 12 threads +
+
+ 1 GPU +
+
+
Domain decomposition +
+ cable_cell_group_gpu +
1 GPU
+
+
+ ... +
+
+
+
+
+ An illustration of the cell-specific portion of the recipe, and how it is + used during the lifetime of the simulation: the simulation object will, + depending on its configuration, query the recipe for the neuroscientific + components it describes. This demonstration also show why the recipe separates + cell descriptions from cell types. The latter is, as you might expect, + shorthand, and is used in the allocation of the cell to a particular cell group. + A cell group implementation is a handler for a certain kind of cell, and Arbor + comes with these for all it's included cell kinds. However, users can develop + their own specialized cell group implementations. More on that in the internal + developer documentation. +
diff --git a/doc/concepts/domdec.rst b/doc/concepts/domdec.rst index a95fe65c78..835152f2a9 100644 --- a/doc/concepts/domdec.rst +++ b/doc/concepts/domdec.rst @@ -3,6 +3,11 @@ Domain decomposition ==================== +An Arbor simulation requires a :ref:`modelrecipe`, a :ref:`(hardware) context `, and a domain decomposition. The Recipe contains the neuroscientific model, the hardware context describes the computational resources you are going to execute the simulation on, and the domain decomposition describes how Arbor will use the hardware. Since the context and domain decomposition may seem closely related at first, it might be instructive to see how recipes are used by Arbor: + +.. raw:: html + :file: domdec-diag-1.html + A *domain decomposition* describes the distribution of the model over the available computational resources. The description partitions the cells in the model as follows: diff --git a/doc/concepts/hardware.rst b/doc/concepts/hardware.rst index 4683207853..bb684e8ca9 100644 --- a/doc/concepts/hardware.rst +++ b/doc/concepts/hardware.rst @@ -1,22 +1,25 @@ .. _modelhardware: -Hardware management -=================== +Hardware context +================ + +An Arbor simulation requires a :ref:`modelrecipe`, a (hardware) context, and a :ref:`modeldomdec`. The Recipe contains the neuroscientific model, the hardware context describes the computational resources you are going to execute the simulation on, and the domain decomposition describes how Arbor will use the hardware. Since the context and domain decomposition may seem closely related at first, it might be instructive to see how recipes are used by Arbor: + +.. raw:: html + :file: domdec-diag-1.html *Local resources* are locally available computational resources, specifically the number of hardware threads and the number of GPUs. An *allocation* enumerates the computational resources to be used for a simulation, typically a subset of the resources available on a physical hardware node. -.. Note:: - - New users can find using contexts a little verbose. - The design is very deliberate, to allow fine-grained control over which - computational resources an Arbor simulation should use. - As a result Arbor is much easier to integrate into workflows that - run multiple applications or libraries on the same node, because - Arbor has a direct API for using on node resources (threads and GPU) - and distributed resources (MPI) that have been partitioned between - applications/libraries. +New users can find using contexts a little verbose. +The design is very deliberate, to allow fine-grained control over which +computational resources an Arbor simulation should use. +As a result Arbor is much easier to integrate into workflows that +run multiple applications or libraries on the same node, because +Arbor has a direct API for using on node resources (threads and GPU) +and distributed resources (MPI) that have been partitioned between +applications/libraries. .. _modelcontext: diff --git a/doc/concepts/index-diag-1.html b/doc/concepts/index-diag-1.html new file mode 100644 index 0000000000..09e64345da --- /dev/null +++ b/doc/concepts/index-diag-1.html @@ -0,0 +1,36 @@ +
+
+
+
Recipe +
describe the neuroscience
+
Cells +
+
Network +
+
In- & outputs +
+
... +
+
+
Simulation +
describe the execution
+
Recipe
+
+
Context +
describe hardware
+
+
+
Domain decomposition +
describe how to use hardware
+
+
+
+
An overview of Arbor. The two columns reflect a division that is core to Arbor: the neuroscience is described entirely separate from the execution of the simulation. The Recipe ties the neuroscience together: the user can provide any number of cells, each with a particular type, morphology and set of mechanisms; a network configuration; and in- & outputs like event generators and probes. This description, the recipe, is self-contained and can be executed by any configuration of execution. Execution is determined by the available hardware and instructions for how certain parts of a recipe (e.g. cell groups) are executed on that hardware.
+ + + + + + + + diff --git a/doc/concepts/index.rst b/doc/concepts/index.rst index b182428091..f1cf92b071 100644 --- a/doc/concepts/index.rst +++ b/doc/concepts/index.rst @@ -3,6 +3,14 @@ Concepts overview ================= +Arbor is a library that lets you to model neural networks with morphologically +detailed cells; which it then executes the resulting simulation on a variety of +hardware. The execution can optionally be configured in high detail but comes +with sensible defaults. + +.. raw:: html + :file: index-diag-1.html + To learn how to use Arbor, it is helpful to understand some of its concepts. Arbor's design aims to enable scalability through abstraction. To achieve this, Arbor makes a distinction between the **description** of a model, and the diff --git a/doc/concepts/recipe-diag-1.html b/doc/concepts/recipe-diag-1.html new file mode 100644 index 0000000000..b9d5b43a9c --- /dev/null +++ b/doc/concepts/recipe-diag-1.html @@ -0,0 +1,121 @@ +
+
Recipe +
+ +
+ +
Morphology +
Segment Tree +
Segment 0 (soma)
+
+
+
Decor +
Paintables +
hh + (all) +
+
+
Placables +
iclamp "midsoma"
+
detector_0 "midsoma"
+
+
+
Label dictionary +
midsoma + (location 0 0.5) +
+
+
+
+ +
Morphology +
Segment Tree +
Segment 0 (soma)
+
Segment 1 (dendrite)
+
Segment 2 (dendrite)
+
Segment ...
+
+
+
Decor +
Paintables +
passive + (all) +
+
+
Placables +
synapse_1 "endpoint"
+
gap_junction_1 "midsoma"
+
+
+
Label dictionary +
midsoma + (location 0 0.5) +
+
endpoint + (terminal) +
+
+
+
+ +
Morphology +
Segment Tree +
Segment 0 (soma)
+
Segment 1 (dendrite)
+
Segment 2 (dendrite)
+
Segment ...
+
+
+
Decor +
Paintables +
passive + (all) +
+
+
Placables +
gap_junction_2 "midsoma"
+
+
+
Label dictionary +
midsoma + (location 0 0.5) +
+
+
+
+
+ +
Cell 0 +
type arbor.cable_cell
+
+
Cell 1 +
type arbor.cable_cell
+
+
Cell 2 +
type arbor.cable_cell
+
+
+
+ +
Connection 1 +
from detector_0
+
to synapse_1
+
+
Gap Junction 1 +
from gap_junction_1
+
to gap_junction_2
+
+
+
+ +
Probe 1 +
gid + 2 +
+
+
+
+
+
+ An example recipe. Click "+" and "-" to (de)collapse contents. +
diff --git a/doc/concepts/recipe.rst b/doc/concepts/recipe.rst index 7498471cec..c3b98ecff4 100644 --- a/doc/concepts/recipe.rst +++ b/doc/concepts/recipe.rst @@ -14,11 +14,11 @@ building phase to provide information about individual cells in the model, such * **Gap junction connections** on each cell. * **Probes** on each cell. -Recipes are structured to provide a consistent interface for describing each cell in the -network using their global identifier (`gid`). -This allows the simulator to be able to quickly look-up properties related to the connections -going in and out of a cell (think of synapses, gap junctions, but also probes and spike inputs); -which helps make Arbor fast and easily distributable over many nodes. +An example model +---------------- + +.. raw:: html + :file: recipe-diag-1.html To better illustrate the content of a recipe, let's consider the following network of three cells: @@ -33,7 +33,7 @@ three cells: (because cable cells allow complex dynamics such as ``hh``). This is referred to as the **kind** of the cell. - ``Cell 1``: Is a soma and a single dendrite, with ``passive`` dynamics everywhere. - It has a single synapse at the end of the dendrite labeled "syanpse_1" and a gap + It has a single synapse at the end of the dendrite labeled "synapse_1" and a gap junction mechanism in the middle of the soma labeled "gap_junction_1". This is the **description** of the cell. It's also a cable cell, which is its **cell kind**. - ``Cell 2``: Is a soma and a single dendrite, with ``passive`` dynamics everywhere. @@ -44,7 +44,7 @@ The total **number of cells** in the model is 3. The **kind**, and **description is known and can be registered in the recipe. Next is the cell interaction. The model is designed such that each cell has labeled source, target and gap junction sites. -A **network connection** can be formed from ``detector_0`` to ``synpase_1``; and a +A **network connection** can be formed from ``detector_0`` to ``synapse_1``; and a **gap junction connection** between ``gap_junction_1`` and ``gap_junction_2``. If ``detector_0`` spikes, a spike should be observed on ``gap_junction_2`` after some delay. To monitor the voltage on ``gap_junction_2`` and record the spike, a **probe** can be set up @@ -59,12 +59,17 @@ Technical details of the recipe class are presented in the :ref:`Python ` APIs. Are recipes always necessary? ------------------------------- +----------------------------- + +Yes. They are a fundamental building block in Arbor simulations. Recipes are structured to provide +a consistent interface for describing each cell in the network using their global identifier (`gid`). +This allows the simulator to be able to quickly look-up properties related to the connections +going in and out of a cell (think of synapses, gap junctions, but also probes and spike inputs); +which helps make Arbor fast and easily distributable over many nodes. -Yes. However, we provide a python :class:`single_cell_model ` -that abstracts away the details of a recipe for simulations of single, stand-alone -:ref:`cable cells`, which absolves the users from having to create the -recipe themselves. This is possible because the number of cells is fixed and known, +For single, stand-alone :ref:`cable cells` simulations, the Python API provides +a :class:`single_cell_model ` that abstracts away the details of a recipe. +This is possible because the number of cells is fixed and known, and it is guaranteed that there can be no connections or gap junctions in a model of a single cell. The single cell model is able to fill out the details of the recipe under the hood, and the user need only provide the cell description, and any probes they wish diff --git a/doc/conf.py b/doc/conf.py index 3351d84f38..45f33b66fc 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -10,6 +10,8 @@ sys.path.append(script_path) html_static_path = ['static'] +html_css_files = ['htmldiag.css'] +html_js_files = ['domarrow.js'] def setup(app): app.add_object_type('generic', 'gen', 'pair: %s; generic') diff --git a/doc/contrib/release.rst b/doc/contrib/release.rst index 706605d40a..f961fcd2df 100644 --- a/doc/contrib/release.rst +++ b/doc/contrib/release.rst @@ -130,25 +130,28 @@ Release Post Release ------------ -0. Update and submit Zenodo release if necessary. -1. Announce on our website -2. Announce on HBP newsletter newsletter@humanbrainproject.eu, HBP Twitter/socials evan.hancock@ebrains.eu -3. [AUTOMATED] Add tagged version of docs on ReadTheDocs -4. HBP internal admin - - - Plus: https://plus.humanbrainproject.eu/components/2691/ - - TC Wiki: https://wiki.ebrains.eu/bin/view/Collabs/technical-coordination/EBRAINS%20components/Arbor/ - - KG: https://search.kg.ebrains.eu/instances/5cf4e24b-b0eb-4d05-96e5-a7751134a061 - - Update howto: https://github.com/bweyers/HBPVisCatalogue/wiki/How-to-start-software-meta-data-curation%3F#update-curated-software - - Previous update as template: https://github.com/bweyers/HBPVisCatalogue/issues/480 - - Supported file formats - - ContentTypes: https://humanbrainproject.github.io/openMINDS/v3/core/v4/data/contentType.html - - details: https://github.com/HumanBrainProject/openMINDS_core/tree/v3/instances/data/contentTypes - - Send an update to the folk in charge of HBP Twitter if we want to shout about it - -5. FZJ admin - - - https://juser.fz-juelich.de/submit +#. Update and submit Zenodo release if necessary. +#. Announce on our website +#. Announce on HBP newsletter newsletter@humanbrainproject.eu, HBP Twitter/socials evan.hancock@ebrains.eu +#. [AUTOMATED] Add tagged version of docs on ReadTheDocs +#. HBP internal admin + + - Plus: https://plus.humanbrainproject.eu/components/2691/ + - TC Wiki: https://wiki.ebrains.eu/bin/view/Collabs/technical-coordination/EBRAINS%20components/Arbor/ + - KG: https://search.kg.ebrains.eu/instances/5cf4e24b-b0eb-4d05-96e5-a7751134a061 + + - Update howto: https://github.com/bweyers/HBPVisCatalogue/wiki/How-to-start-software-meta-data-curation%3F#update-curated-software + - Previous update as template: https://github.com/bweyers/HBPVisCatalogue/issues/480 + - Supported file formats + + - ContentTypes: https://humanbrainproject.github.io/openMINDS/v3/core/v4/data/contentType.html + - details: https://github.com/HumanBrainProject/openMINDS_core/tree/v3/instances/data/contentTypes + + - Send an update to the folk in charge of HBP Twitter if we want to shout about it + +#. FZJ admin + + - https://juser.fz-juelich.de/submit .. _GH tags: https://github.com/arbor-sim/arbor/tags .. _AUTOMATED: https://github.com/arbor-sim/arbor/blob/master/.github/workflows/ebrains.yml diff --git a/doc/cpp/hardware.rst b/doc/cpp/hardware.rst index 1389039b27..d2e810b3ec 100644 --- a/doc/cpp/hardware.rst +++ b/doc/cpp/hardware.rst @@ -1,7 +1,7 @@ .. _cpphardware: -Hardware management -=================== +Hardware context +================ Arbor provides two library APIs for working with hardware resources: diff --git a/doc/dev/arbor-dep-graph.png b/doc/dev/arbor-dep-graph.png new file mode 100644 index 0000000000..f6f4a9a06a Binary files /dev/null and b/doc/dev/arbor-dep-graph.png differ diff --git a/doc/dev/cable_cell.rst b/doc/dev/cable_cell.rst new file mode 100644 index 0000000000..248c0e8d2b --- /dev/null +++ b/doc/dev/cable_cell.rst @@ -0,0 +1,115 @@ +.. _cable_cell: + +How the Cable Cell is made +========================== + +This is a short tour through the inner workings of a cable cell intended for new +Arbor developers. Cable cells are not the sole cell type supported in Arbor, but +both the most common and most complex kind. + +This is the introduction from which more detailed descriptions will branch out. +As such, we will start with a simple cable cell simulation and how the user input +is turned into an executable object. + +Terminology +----------- + +In Arbor's codebase some prefixes are used as a low-key namespacing + +- ``arb_``:: Mech ABI types, in general use through out Arbor, eg + ``arb_mechanism_type``. +- ``fvm_``:: Concerning use by the Finite Volume Method (FVM), eg + ``fvm_lowered_cell``. +- ``mc_``:: Related to Multi-Compartment (Cells), identical to cable cells the + difference is purely historical, eg ``mc_cell_group``. + +Setting up a Cable Cell simulation +---------------------------------- + +Arbor constructs a runnable simulation from three ingredients: + +- ``recipe``:: a queryable structure that collects cells, connections, gap + junctions, etc. +- ``context``:: a hardware configuration, summarising threads, GPU, and MPI + resources to be used. +- ``domain_decomposition``:: Distribution of cells over the ``context``, made + from ``context`` and ``recipe``. + +The interesting part here is the ``recipe``, which is used to lazily produce the +data required by the ``simulation`` and ``domain_decomposition``. A simple example +might be this model of a single cell + +.. code:: c++ + + struct recipe: public arb::recipe { + recipe(const arb::morphology& m, const arb::decor& d): cell{m, d} {} + + arb::cell_size_type num_cells() const override { return 1; } + + std::vector get_probes(arb::cell_gid_type) const override { return {}; } + + arb::cell_kind get_cell_kind(arb::cell_gid_type) const override { return arb::cell_kind::cable; } + + std::any get_global_properties(arb::cell_kind) const override { return gprop; } + + arb::util::unique_any get_cell_description(arb::cell_gid_type) const override { return cell; } + + arb::cable_cell cell + arb::cable_cell_global_properties gprop; + }; + +As you can see, data is produced on request by feeding the recipe a +``cell_gid_type``. Finally, we need to have a ``morphology`` and a ``decor`` to +generate a ``cable_cell``. Please refer to the documentation on how to construct +these objects. For now, it is sufficient to note that a ``cable_cell`` is a +description of a cell, consisting of a cell geometry and a mapping of +sub-geometries to properties, rather an object that can be simulated. At this point +ion channels are described by a structure ``(name, [(parameter, value)])``. + +Lowered Cells, Shared State, and the Discretisation +--------------------------------------------------- + +To obtain a simulation we need to turn the ``cable_cell`` description object +into a ``fvm_lowered_cell``. However, multiple cells are collected into a +``mc_cell_group`` and ``fvm_lowered_cell`` is the lowered representation of a +full cell group. The ``fvm_lowered_cell`` is responsible for holding the +backend-specific data of a cell group, managing sampling and stimuli, facilitate +event processing, and driving time integration. + +Discretisation splits the segments described by the morphology into *control +volumes* (CV; sometimes called *compartments*) according to a ``cv_policy``. +This allows us to construct a system of linear equations, the Hines matrix, to +describe the evolution of the CV voltages according to the cable equation. Refer +to :ref:`Discretisation ` and :ref:`Cable equation +`. + +Backend-dependent data is stored in ``shared_state`` as per-compartment data and +indices into these arrays. Upon ``fvm_lowered_cell::initialize`` these are +populated using the concrete discretisation and the ``cable_cell`` description. +Also, mechanisms are concretised using the provided ``mechanism_catalogue`` and +their internal data is set up in ``shared_state``. See :ref:`Shared state ` +for more details + +Main integration loop +--------------------- + +Now we are in a state to simulate a cell group by calling +``simulation::run(t_end, dt)``. + +The integration in Arbor proceeds in *epochs* with a length less than half a +time constant :math:`T_{min}`, which is the maximum time over which cell groups +can evolve independently. The length :math:`T_{min}` is derived as the minimum over all +axonal/synaptic delays. This works since we know that an event at time :math:`t` +can have an effect at time :math:`t + T_{min}` at the soonest. The factor of one +half stems from double-buffering to overlap communication and computation. So, +Arbor collects all events in an epoch and transmits them in bulk, see +:ref:`Communication ` for details. + +Integration in Arbor is then split into three parts: + +1. apply effect of events to mechanisms :ref:`Event Handling ` +2. evolve mechanisms and apply currents :ref:`Mechanisms ` +3. solve voltage equations, see :ref:`Solver ` + +Integration proceeds as far as possible without needing to process an event, but +at most with the given time step `dt`. diff --git a/doc/dev/cell_groups.rst b/doc/dev/cell_groups.rst new file mode 100644 index 0000000000..abcdc61b71 --- /dev/null +++ b/doc/dev/cell_groups.rst @@ -0,0 +1,84 @@ +.. _cell_groups: + +Cell groups +=========== + +Cell groups represent a union of cells of a single *kind* simulated in lockstep. +In a sense, their existence is an optimisation, since parts of the internal +state and computations can be shared between cell in single group. The currently +most complicated cell group is the one for cable cells, called ``mc_cell_group`` +(``mc`` stands for multi-compartment, used in older parts of Arbor), so we will +focus on this type here. + +Cell groups are created by domain decomposition methods on consideration of soft +(like performance optimisation) and hard (cells connected by gap junctions must +be in the same group) constraints. + +Cable Cell group ``mc_cell_group`` +---------------------------------- + +Cable cell groups have backing store in ``shared_state`` (given the +introduction, we now understand that the ``shared`` stands for 'shared' between +cell in a group). In this data set, we also collect the private data of +mechanisms. One thing to watch out for here is that instances of the same +mechanism on a cell group will be collated. + +Let us examine an example of such a cell group; assume the following + +1. the group comprises two cells with a total of 9 CVs + 1. cell 0 has 4 CVs + 2. cell 1 has 5 CVs +2. ``pas`` has been painted on three regions and collated + 1. region 0 covers CVs [0 1 2] + 2. region 1 covers CV 5 + 3. region 2 covers CV 7 +3. ``pas`` has two parameters ``g`` (conductance density) and ``E`` (resting potential) + +.. code:: + + - shared_state + - mechanisms + - id 0 + - name "pas" + - width 5 + - parameters + - id 0 + - name "g" + - values [* * * * *] + - id 1 + - name "E" + - values [* * * * *] + - index [0 1 2 5 7] + / | | | \ + / / / | | + / / / \ \ + - voltage [* * * * * * * * *] + +Mechanisms access their view of the cell group data via the +``arb_mechanism_ppack`` structure. To continue with our example, the ``pas`` +mechanism would iterate through its view on the cell group voltage to +compute the current density ``i`` like this + +.. code:: + + for ix in 0..width + # Obtain parameters + g = ppack.parameters["g"][ix] + E = ppack.parameters["E"][ix] + # Fetch voltage, note the indirection + cv = ppack.index[ix] + u = ppack.voltage[cv] + # Write outgoing current + ppack.i[ix] = g*(u - E) + +In general, cell group wide quantities (like ``voltage`` here) need to be +indexed via ``ppack.index``. Note, that the layout of parameters in ``ppack`` is +this in reality: + +.. code:: + + - ppack + - parameters [g g g g g E E E E E] + +When using NMODL, we translate names like ``g`` to offsets into the parameter array +at compile time. Handwritten mechanisms need to do this manually. diff --git a/doc/internals/export.rst b/doc/dev/export.rst similarity index 99% rename from doc/internals/export.rst rename to doc/dev/export.rst index 7e997f364d..75e6a59a5c 100644 --- a/doc/internals/export.rst +++ b/doc/dev/export.rst @@ -23,7 +23,7 @@ However, we currently do not handle this case in our build scripts as it is not .. Note:: When linking an application with **static** Arbor libraries the linker may issue warnings (particularly on macos). Thus, if you encounter problems, try building shared Arbor libraries (cmake option ``-DBUILD_SHARED_LIBS=ON``) instead. -Macro Descripiton +Macro Description ----------------- .. c:macro:: ARB_LIBNAME_API diff --git a/doc/internals/extending_catalogues.rst b/doc/dev/extending_catalogues.rst similarity index 100% rename from doc/internals/extending_catalogues.rst rename to doc/dev/extending_catalogues.rst diff --git a/doc/dev/index.rst b/doc/dev/index.rst new file mode 100644 index 0000000000..dedd2bc7b4 --- /dev/null +++ b/doc/dev/index.rst @@ -0,0 +1,26 @@ +.. _dev-overview: + +Developers Guide +================ + +Here we document internal components of Arbor. These pages can be useful if you're interested in developing on Arbor itself. + +.. figure:: arbor-dep-graph.png + :align: center + + Arbor dependency graph, generated by ``cmake --graphviz=graphviz/arbor-dep-graph.dot .`` + +.. toctree:: + :caption: Arbor Developers Guide: + :maxdepth: 2 + + cable_cell + cell_groups + matrix_solver + simd_api + shared_state + export + extending_catalogues + mechanism_abi + util +.. numerics diff --git a/doc/dev/matrix_solver.rst b/doc/dev/matrix_solver.rst new file mode 100644 index 0000000000..5733f1b273 --- /dev/null +++ b/doc/dev/matrix_solver.rst @@ -0,0 +1,134 @@ +.. _matrix_solver: + +Matrix Solvers +============== + +Cable Equation +-------------- + +At the heart of the time evolution step in Arbor we find a linear system that must +be solved once per time step. This system arises from the cable equation + +.. math:: + C \partial_t V = \frac{\sigma}{2\pi a}\partial_x(a^2\partial_x V) + I + + I: \mbox{External currents} + + V: \mbox{Membrane potential} + + a: \mbox{cable radius} + + \sigma: \mbox{conductivity} + +after discretisation into CVs, application of the FVM, and choosing an implicit +Euler time step as + +.. math:: + \left(\frac{\sigma_i C_i}{\Delta\,t} + \sum_{\delta(i, j)} a_{ij}\right)V_i^{k+1} - \sum_{\delta(i, j)} a_ij V_i^{k+1} + = \frac{\sigma_i C_i}{\Delta\,t}V_i^k + \sigma_i I_i + +where :math:`\delta(i, j)` indicates whether two CVs are adjacent. It is written +in form of a sparse matrix, symmetric by construction. + +The currents :math:`I` originate from the ion channels on the CVs in question, +see the discussion on mechanisms for further details. As :math:`I` potentially +depends on :math:`V`, the cable equation is non-linear. We model these +dependencies up to first order as :math:`I = gV + J` and collect all higher +orders into :math:`J`. This is done to improve accuracy and stability of the +solver. Finding :math:`I` requires the computation of the symbolic derivative +:math:`g = \partial_V I` during compilation of the mechanisms. At runtime +:math:`g` is updated alongside with the currents :math:`I` using that symbolic +expression. + +Each *branch* in the morphology leads to a tri-diagonal block in the matrix +describing the system, since *branches* do not contain interior branching +points. Thus, an interior CV couples to only its neighbours (and itself). +However, at branch points, we need to factor in the branch's parents, which +couple blocks via entries outside the tri-diagonal structure. To ensure +un-problematic data dependencies for use of a substitution algorithm, ie each +row depends only on those of larger indices, we enumerate CVs in breadth-first +ordering. This particular form of matrix is called a *Hines matrix*. + +CPU +--- + +.. note:: See ``arbor/backends/multicore/matrix_state.hpp``: + + * ``struct matrix_state`` + * the ``matrix_state`` constructor sets up the static parts + * the dynamic part is found in ``assemble`` + * the solver lives in ``solve``. + +The matrix solver proceeds in two phases: assembly and the actual solving. Since +we are working on cell groups, not individual cells, this is executed for each +cell's matrix. + +Assembly +^^^^^^^^ + +We store the matrix in compressed form, as its upper and main diagonals. The +static parts -- foremost the main diagonal -- are computed once at construction +time and stored. The dynamic parts of the matrix and the right-hand side of the +equation are initialised by calling ``assemble``. + +Solving +^^^^^^^ + +The CPU implementation is a straight-forward implemenation of a modified +Thomas-algorithm, using an extra input for the parent relationship. If each +parent is simply the previous CV, we recover the Thomas algorithm. + +.. code:: c++ + + void hines(const arb_value_type* diagonal, // main diagonal + const arb_value_type* upper, // upper diagonal + arb_value_type* rhs, // rhs / solution + const arb_index_type* parents, // CV's parent + int N) { + // backward substitution + for (int i = N-1; i>0; --i) { + const auto parent = parents[i]; + const auto factor = upper[parent] / diagonal[i]; + diagonal[parent] -= factor * upper[parent]; + rhs[parent] -= factor * rhs[i]; + } + // solve root + b[0] = b[0] / d[0]; + + // forward substitution + for(int i=1; i`_ and the references +therein are worthwhile further reading. diff --git a/doc/internals/mechanism_abi.rst b/doc/dev/mechanism_abi.rst similarity index 100% rename from doc/internals/mechanism_abi.rst rename to doc/dev/mechanism_abi.rst diff --git a/doc/dev/numerics.rst b/doc/dev/numerics.rst new file mode 100644 index 0000000000..1523aadda6 --- /dev/null +++ b/doc/dev/numerics.rst @@ -0,0 +1,20 @@ +.. _numerics: + +Numerical methods +=================================== + +Here we document which numerical methods are applied to solve +the differential equations of the cable cells and mechansims. + +Passive cable +------------- + +Integration of the cable equation: implicit Euler via +linearized set of equations and Thomas algorithm. + +Mechanisms +---------- + +Exponential Euler `cnexp`. + +Implicit Euler `sparse`. diff --git a/doc/dev/shared_state.rst b/doc/dev/shared_state.rst new file mode 100644 index 0000000000..5f8f1d42df --- /dev/null +++ b/doc/dev/shared_state.rst @@ -0,0 +1,45 @@ +.. _shared_state: + +Shared State +============ + +The ``shared_state`` classes are collections to store backend-specific simulator +state and can be found in the ``backend/{multicore, gpu}`` directories. +Functionality manipulating such items gets delegated to the implementations of +the ``shared_state`` interface. + +Ions +---- + +Ion state is stored as a series of arrays corresponding to NMODL variables. +These arrays have entries per CV and mechanisms need to go through the +``node_index_`` array to map the mechanism's internal CV index to the cell +group's CV index. + +======= ======= =============================== +Field NMODL Ion Property +======= ======= =============================== +``iX_`` ``ica`` Current density +``eX_`` ``eca`` Reversal potential +``Xi_`` ``cai`` Internal concentration +``Xo_`` ``cao`` External concentration +======= ======= =============================== + +This table shows the mapping between NMODL -- for the ``ca`` ion species -- and +the ``ion_state`` members. The class is responsible for reseting currents and +concentrations. + +Mechanisms +---------- + +All mechanisms' privates data is stored in ``shared_state``, which is also +responsible for managing the lifetime and initialisation of said data. This is +done to allow mechanisms to be implemented as essentially stateless bits of +C-code interacting with Arbor only through ``shared_state``. See the ABI +documentation for a discussion of the details. + +Further Functionality +--------------------- + +In addition the configuration and effect of stimuli, gap junctions, and +sampling, as well as the computation of per-CV time steps is handled here. diff --git a/doc/internals/simd_api.rst b/doc/dev/simd_api.rst similarity index 100% rename from doc/internals/simd_api.rst rename to doc/dev/simd_api.rst diff --git a/doc/internals/util.rst b/doc/dev/util.rst similarity index 96% rename from doc/internals/util.rst rename to doc/dev/util.rst index ea233ad016..e2f40d7228 100644 --- a/doc/internals/util.rst +++ b/doc/dev/util.rst @@ -1,7 +1,7 @@ .. _libref: Utility wrappers and containers -################################## +=============================== .. cpp:namespace:: arb::util diff --git a/doc/ecosystem/arbor.png b/doc/ecosystem/arbor.png new file mode 100644 index 0000000000..0d9ac3cb72 Binary files /dev/null and b/doc/ecosystem/arbor.png differ diff --git a/doc/ecosystem/index-sims.html b/doc/ecosystem/index-sims.html new file mode 100644 index 0000000000..1cdf7c7196 --- /dev/null +++ b/doc/ecosystem/index-sims.html @@ -0,0 +1,27 @@ +
+
Neural Mass Models +
The Virtual Brain
+
+
Point Neurons +
NEST
+
BRIAN
+
+
Detailed Neurons +
Arbor
+
(Core)NEURON, NeuroGPU
+
GENESIS
+
MOOSE
+
LEMS
+
jNML
+
EDEN
+
+
Molecular Dynamics +
ALMOST
+
NAMD
+
+
+
An informal hierarchy of the simulator landscape, showing Arbor's position.
+ + + + diff --git a/doc/ecosystem/index.rst b/doc/ecosystem/index.rst new file mode 100644 index 0000000000..3f9ea1f9e9 --- /dev/null +++ b/doc/ecosystem/index.rst @@ -0,0 +1,54 @@ +.. _ecosystemindex: + +Ecosystem +========= + +Arbor exists in a wider computational neuroscience software ecosystem. The Arbor simulator is also not the only component written by the Arbor developers. + +Arbor software +-------------- + +Arbor + The Arbor simulator. + +modcc + An important component of Arbor. It is the compiler for the Arbor dialect of `NMODL`, a language commonly used to describe mechanisms on morphologically detailed cable cells. `modcc` does not have its own documentation; the best resource is the file format page describing it :ref:`formatnmodl`. + +NSuite + A framework for maintaining and running benchmarks and validation tests for multi-compartment neural network simulations on HPC systems. `NSuite documentation `_ + +Arbor GUI + The Arbor GUI visualizes cable cells and can be used to decorate morphologies. Single cell simulations can be ran (using the Arbor simulator) and output plotted right from the GUI. `Code repository and Readme `_ + +nmlcc + Arbor has built-in read support for NeuroML morphologies, but not yet for other NeuroML components (mechanisms, networks). `nmlcc` is compiler/translator that aims to generate complete Arbor inputs for any NeuroML file. `Code repository and Readme `_ + +.. figure:: arbor.png + :align: center + + Arbor tools in relationship to each other and common file formats. Red outlines indicate currently unplanned format format support (figure generated from ``arbor.dot``). + +Wider ecosystem +--------------- + +Simulators +~~~~~~~~~~ + +Arbor exists in the wider computational neuroscience simulator ecosystem. Simulators exist roughly at these four levels: ranging from the lowest level (molecular dynamics) to the highest (whole brain simulation). Needless to say, each level comes with a higher level of abstraction, but attempt to describe larger structures. Co-simulation or Multiscale simulations refers to the attempt to link up the levels and simulators in an attempt to combine the low level of detail (in regions where it is desired) with the effects of the larger structures. The concrete method to interface between levels are typically the submission of spikes, ion (concentrations), field potentials. + +.. raw:: html + :file: index-sims.html + +Frameworks +~~~~~~~~~~ + +Certain tools combine various components to create integrated experimentation toolchains. They do not only include (interfaces for) simulators, but also converters for file formats, miscellaneous data processing (pre or post), plotting, miscellaneous analysis, and more. An incomplete list: + +- `NeuroMLlite `_ (NML) +- `OpenSourceBrain `_ (NML2) +- `jNeuroML, pyNeuroML, jLEMS, pyLEMS `_ (NML2) +- `BMTK / SONATA `_ +- `LFPy, LFPyKit `_ +- `NetPyne `_ +- `BluePyOpt `_ +- `Brain Scaffold Builder `_ diff --git a/doc/fileformat/nmodl.rst b/doc/fileformat/nmodl.rst index f1f795ce37..5175f0dd56 100644 --- a/doc/fileformat/nmodl.rst +++ b/doc/fileformat/nmodl.rst @@ -122,6 +122,10 @@ Unsupported features Arbor-specific features ----------------------- +* It is required to explicitly pass 'magic' variables like ``v`` into procedures. + It makes things more explicit by eliding shared and implicit global state. However, + this is only partially true, as having `PARAMETER v` brings it into scope, *but only* + in `BREAKPOINT`. * Arbor's NMODL dialect supports the most widely used features of NEURON. It also has some features unavailable in NEURON such as the ``POST_EVENT`` procedure block. This procedure has a single argument representing the time since the last spike on @@ -186,14 +190,17 @@ modifying the reversal potential (for example ``nernst``) can only be applied (f at a global level on a given cell. While in Neuron, different mechanisms can be used for calculating the reversal potential of an ion on different parts of the morphology. This is due to the different methods Arbor and NEURON use for discretising the morphology. -(A ``region`` in Arbor may include part of a CV, where as in NEURON, a ``section``can only +(A ``region`` in Arbor may include part of a CV, where as in NEURON, a ``section`` can only contain full ``segments``). Modelers are encouraged to verify the expected behavior of the reversal potentials of ions as it can lead to vastly different model behavior. Tips for Faster NMODL -====================== +--------------------- + +.. Note:: + If you are looking for help with NMODL in the context of NEURON this guide might not help. NMODL is a language without formal specification and many unexpected characteristics (many of which are not supported in Arbor), which results in @@ -214,11 +221,18 @@ compiler like GCC (or nvcc) to produce either a shared object (for external catalogues) and code directly linked into Arbor (the built-in catalogues). Now, we turn to a series of tips we found helpful in producing fast NMODL -mechanisms. Note that if you are looking for help with NMODL in the context of -NEURON this guide might not help. +mechanisms. In terms of performance of variable declaration, the hierarchy is +from slowest to fastest: + +1. ``RANGE ASSIGNED`` -- mutable array +2. ``RANGE PARAMETER`` -- configurable array +3. ``ASSIGNED`` -- mutable +4. ``PARAMETER`` -- configurable +5. ``CONSTANT`` -- inlined constant + ``RANGE`` ---------- +~~~~~~~~~ Parameters and ``ASSIGNED`` variables marked as ``RANGE`` will be stored as an array with one entry per CV in Arbor. Reading and writing these incurs a memory @@ -231,15 +245,15 @@ that in the generated code: a local variable that is likely residing in a register and used only as long as needed. ``PROCEDURE`` -------------- +~~~~~~~~~~~~~ Prefer ``FUNCTION`` over ``PROCEDURE``. The latter *require* ``ASSIGNED RANGE`` variables to return values and thus stress the memory system, which, as noted above, is not most efficient on current hardware. Also, they may not be inlined, as opposed to a ``FUNCTION``. -```PARAMETER`` --------------- +``PARAMETER`` +~~~~~~~~~~~~~ ``PARAMETER`` should only be used for values that must be set by the simulator. All fixed values should be ``CONSTANT`` instead. These will be inlined by @@ -247,7 +261,7 @@ All fixed values should be ``CONSTANT`` instead. These will be inlined by optimisation potential. Sharing Expressions Between ``INITIAL`` and ``BREAKPOINT`` or ``DERIVATIVE`` ----------------------------------------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ This is often done using a ``PROCEDURE``, which we now know is inefficient. On top, this ``PROCEDURE`` will likely compute more outputs than strictly needed to @@ -286,7 +300,7 @@ see when partially hidden in a ``PROCEDURE``. this option when compiling mechanism code. Complex Expressions in Current Computation ------------------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ``modcc``, Arbor's NMODL compiler, applies symbolic differentiation to the current expression to find the conductance as ``g = d I/d U`` which are then @@ -312,7 +326,7 @@ higher-order ODEs and thus will treat ``g`` as a constant across a single timestep even if ``g`` actually depends on ``v``. Specialised Functions ---------------------- +~~~~~~~~~~~~~~~~~~~~~ Another common pattern is the use of a guarded exponential of the form @@ -336,7 +350,7 @@ from NEURON often use this or related functions, e.g. ``vtrap(x, y) = y*exprelr(x/y)``. Small Tips and Micro-Optimisations ----------------------------------- +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - Divisions cost a bit more than multiplications and additions. - ``m * m`` is more efficient than ``m^2``. This holds for higher powers as well diff --git a/doc/index.rst b/doc/index.rst index 1dddfad9c3..6c84aced6e 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -30,7 +30,8 @@ Documentation organisation * :ref:`tutorial` contains a few ready-made examples you can use to quickly get started using Arbor. In the tutorial descriptions we link to the relevant Arbor concepts. * :ref:`modelintro` describes the design and concepts used in Arbor. The breakdown of concepts is mirrored (as much as possible) in the :ref:`pyoverview` and :ref:`cppoverview`, so you can easily switch between languages and concepts. -* The API section details our :ref:`pyoverview` and :ref:`cppoverview` API. :ref:`internals-overview` describes Arbor code that is not user-facing; convenience classes, architecture abstractions, etc. +* The API section details our :ref:`pyoverview` and :ref:`cppoverview` API. :ref:`dev-overview` describes Arbor code that is not user-facing; convenience classes, architecture abstractions, and other information that is relevant to understanding the inner workings of Arbor and the mathematical foundations underpinning the engine. +* :ref:`modelling-overview` is a collection of best practices and experiences collected from the Arbor modelling community, meant to spread information on how to solve common modelling questions in Arbor. * Contributions to Arbor are very welcome! Under :ref:`contribindex` describe conventions and procedures for all kinds of contributions. Citing Arbor @@ -102,6 +103,7 @@ A full list of our software attributions can be found `here - {%- for cssfile in extra_css_files %} - + {%- for css in css_files %} + {%- if css|attr("rel") %} + + {%- else %} + + {%- endif %} {%- endfor %} {%- block linktags %} diff --git a/doc/static/domarrow.js b/doc/static/domarrow.js new file mode 100644 index 0000000000..08494e997e --- /dev/null +++ b/doc/static/domarrow.js @@ -0,0 +1,260 @@ +(()=>{ + function getNumberOrDef(val, def) { + return typeof val === 'number' && !isNaN(val) ? val : def; + } + + function isVisible(element) { + return element && element.style.visibility !== 'hidden'; + } + + function inside(minX, minY, maxX, maxY, x1, y1) { + return minX <= x1 && x1 <= maxX && minY <= y1 && y1 <= maxY; + } + + function intersectionPoint (x1, y1, x2, y2, minX, minY, maxX, maxY) { + var min = Math.min, max = Math.max; + var good = inside.bind(null, min(x1, x2), min(y1, y2), max(x1, x2), max(y1, y2)); + + if ((x1 <= minX && x2 <= minX) || (y1 <= minY && y2 <= minY) || (x1 >= maxX && x2 >= maxX) || (y1 >= maxY && y2 >= maxY) || (inside(minX, minY, maxX, maxY, x1, y1) && inside(minX, minY, maxX, maxY, x2, y2))) + return null; + + var m = (y2 - y1) / (x2 - x1); + var y = m * (minX - x1) + y1; + if (minY < y && y < maxY && good(minX, y)) return {x: minX, y:y}; + + y = m * (maxX - x1) + y1; + if (minY < y && y < maxY && good(maxX, y)) return {x: maxX, y:y}; + + var x = (minY - y1) / m + x1; + if (minX < x && x < maxX && good(x, minY)) return {x: x, y:minY}; + + x = (maxY - y1) / m + x1; + if (minX < x && x < maxX && good(x, maxY)) return {x: x, y:maxY}; + + return null; + } + + + function adjustLine(from, to, line, trafo){ + var color = trafo && trafo.color || 'black'; + var W = trafo && trafo.width || 2; + + var fromB = parseFloat(from.style.top) ? null : from.getBoundingClientRect(); + var toB = parseFloat(to.style.top) ? null : to.getBoundingClientRect(); + var fromBStartY = (fromB ? window.scrollY + fromB.top : parseFloat(from.style.top)); + var fromBStartX = (fromB ? window.scrollX + fromB.left : parseFloat(from.style.left)); + var toBStartY = (toB ? window.scrollY + toB.top : parseFloat(to.style.top)); + var toBStartX = (toB ? window.scrollX + toB.left : parseFloat(to.style.left)); + var fromBWidth = (fromB ? fromB.width : parseFloat(from.style.width) || from.offsetWidth); + var fromBHeight = (fromB ? fromB.height : parseFloat(from.style.height) || from.offsetHeight); + var toBWidth = (toB ? toB.width : parseFloat(to.style.width) || to.offsetWidth); + var toBHeight = (toB ? toB.height : parseFloat(to.style.height) || to.offsetHeight); + + var fT = fromBStartY + fromBHeight * getNumberOrDef(trafo && trafo.fromY, getNumberOrDef(trafo, 0.5)); + var tT = toBStartY + toBHeight * getNumberOrDef(trafo && trafo.toY, getNumberOrDef(trafo, 0.5)); + var fL = fromBStartX + fromBWidth * getNumberOrDef(trafo && trafo.fromX, getNumberOrDef(trafo, 0.5)); + var tL = toBStartX + toBWidth * getNumberOrDef(trafo && trafo.toX, getNumberOrDef(trafo, 0.5)); + + var CA = Math.abs(tT - fT); + var CO = Math.abs(tL - fL); + var H = Math.sqrt(CA*CA + CO*CO); + var ANG = 180 / Math.PI * Math.acos( CO/H ); + + if((fT >= tT || fL >= tL) && (tT >= fT || tL >= fL)) { + ANG *= -1; + } + + if (trafo && trafo.onlyVisible) { + var arrangeFrom = intersectionPoint(fL, fT, tL, tT, fromBStartX, fromBStartY, fromBStartX + fromBWidth, fromBStartY + fromBHeight); + var arrangeTo = intersectionPoint(fL, fT, tL, tT, toBStartX, toBStartY, toBStartX + toBWidth, toBStartY + toBHeight); + + if (arrangeFrom) { + fL = arrangeFrom.x; + fT = arrangeFrom.y; + } + if (arrangeTo) { + tL = arrangeTo.x; + tT = arrangeTo.y; + } + CA = Math.abs(tT - fT); + CO = Math.abs(tL - fL); + H = Math.sqrt(CA*CA + CO*CO); + } + + var top = (tT+fT)/2 - W/2 - 60; + var left = (tL+fL)/2 - H/2; + + var arrows = [...line.getElementsByClassName('arrow')]; + + var needSwap = (fL > tL || (fL == tL && fT < tT)); + var arrowFw = needSwap && isVisible(arrows[0]) && arrows[0] || !needSwap && isVisible(arrows[1]) && arrows[1]; + var arrowBw = !needSwap && isVisible(arrows[0]) && arrows[0] || needSwap && isVisible(arrows[1]) && arrows[1]; + if (arrowFw) { + arrowFw.classList.remove('arrow-bw'); + arrowFw.classList.add('arrow-fw'); + arrowFw.style.borderRightColor = color; + arrowFw.style.top = W/2-6 + "px"; + } + if (arrowBw) { + arrowBw.classList.remove('arrow-fw'); + arrowBw.classList.add('arrow-bw'); + arrowBw.style.borderLeftColor = color; + arrowBw.style.top = W/2-6 + "px"; + } + line.style.display = "none"; + line.style["-webkit-transform"] = 'rotate('+ ANG +'deg)'; + line.style["-moz-transform"] = 'rotate('+ ANG +'deg)'; + line.style["-ms-transform"] = 'rotate('+ ANG +'deg)'; + line.style["-o-transform"] = 'rotate('+ ANG +'deg)'; + line.style["-transform"] = 'rotate('+ ANG +'deg)'; + line.style.top = top+'px'; + line.style.left = left+'px'; + line.style.width = H + 'px'; + line.style.height = W + 'px'; + line.style.background = "linear-gradient(to right, " + + (arrowFw ? "transparent" : color) +" 11px, " + + color + " 11px " + (H - 11) + "px, " + + (arrowBw ? "transparent" : color) + " " + (H - 11) + "px 100%)"; + line.style.display = "initial"; + } + + function repaintConnection(connectionElement) { + var fromQ = connectionElement.getAttribute('from'); + var fromE = document.querySelector(fromQ); + var toQ = connectionElement.getAttribute('to'); + var toE = document.querySelector(toQ); + connectedObserver.observe(fromE, {attributes:true}); + connectedObserver.observe(toE, {attributes:true}); + + var lineE = connectionElement.getElementsByClassName('line')[0]; + if (!lineE) { + lineE = document.createElement('div'); + lineE.classList.add('line'); + connectionElement.appendChild(lineE); + } + var needTail = connectionElement.hasAttribute('tail'); + var needHead = connectionElement.hasAttribute('head'); + var arrows = lineE.getElementsByClassName('arrow'); + var tail = arrows[0]; + var head = arrows[1]; + if (!tail && (needHead || needTail)) { + tail = document.createElement('div'); + tail.classList.add('arrow'); + lineE.appendChild(tail); + } + + if (!head && needHead) { + head = document.createElement('div'); + head.classList.add('arrow'); + lineE.appendChild(head); + } + + tail && (tail.style.visibility = needTail ? 'visible' : 'hidden'); + head && (head.style.visibility = needHead ? 'visible' : 'hidden'); + + var textE = lineE.getElementsByClassName('text')[0]; + var textMessage = connectionElement.getAttribute('text'); + + if (!textE && textMessage) { + textE = document.createElement('div'); + textE.classList.add('text'); + lineE.appendChild(textE); + } + if (textE && textE.innerText != textMessage) { + textE.innerText = textMessage; + } + + adjustLine(fromE, toE, lineE, { + color: connectionElement.getAttribute('color'), + onlyVisible: connectionElement.hasAttribute('onlyVisible'), + fromX: parseFloat(connectionElement.getAttribute('fromX')), + fromY: parseFloat(connectionElement.getAttribute('fromY')), + toX: parseFloat(connectionElement.getAttribute('toX')), + toY: parseFloat(connectionElement.getAttribute('toY')), + width: parseFloat(connectionElement.getAttribute('width')) + }); + } + + function repaintWithoutObserve(tag) { + connectionObserver.observe(tag, {attributeFilter: []}); + repaintConnection(tag); + connectionObserver.observe(tag, {attributes:true, childList:true, subtree: true}); + } + + function createOne(newElement) { + connectionElements.push(newElement); + repaintConnection(newElement); + connectionObserver.observe(newElement, {attributes:true, childList:true, subtree: true}); + } + + function create() { + bodyObserver.observe(document.body, {childList:true, subtree: true}); + [...document.body.getElementsByTagName('connection')].forEach(createOne); + } + + function removeConnection(tag) { + for(var i = connectionElements.length - 1; i >= 0; i--) + if(connectionElements[i] === tag) + connectionElements.splice(i, 1); + + connectionObserver.observe(tag, {attributeFilter: []}); + } + + function changedConnectedTag(changes) { + changes.forEach(e => { + var changedElem = e.target; + var wasConnection = false; + for (var i = 0; i < connectionElements.length; ++i) { + var fromE = document.querySelector(connectionElements[i].getAttribute('from')); + if (fromE === changedElem) { + wasConnection = true; + repaintWithoutObserve(connectionElements[i]); + continue; + } + var toE = document.querySelector(connectionElements[i].getAttribute('to')); + if (toE === changedElem) { + wasConnection = true; + repaintWithoutObserve(connectionElements[i]); + } + } + if (!wasConnection) { + connectionObserver.observe(changedElem, {attributeFilter: []}); + } + }); + } + + function changedConnectionTag(changes) { + changes.forEach(e => { + var conn = e.target; + if (conn.tagName.toLowerCase() !== 'connection' && e.attributeName === 'class') + ; + while (conn && conn.tagName.toLowerCase() !== 'connection') + conn = conn.parentElement; + if (!conn) return; + repaintWithoutObserve(conn); + }); + } + + function bodyNewElement(changes) { + changes.forEach(e => { + e.removedNodes.forEach(n => { + if (n.tagName && n.tagName.toLowerCase() === 'connection') + removeConnection(n); + }); + e.addedNodes.forEach(n => { + if (n.tagName && n.tagName.toLowerCase() === 'connection') + createOne(n); + }); + }); + } + + var connectionElements = []; + var MutationObserver = window.MutationObserver || + window.WebKitMutationObserver || window.MozMutationObserver; + var bodyObserver = new MutationObserver(bodyNewElement); + var connectionObserver = new MutationObserver(changedConnectionTag); + var connectedObserver = new MutationObserver(changedConnectedTag); + document.body && create() || window.addEventListener("load", create); +})(); + + diff --git a/doc/static/htmldiag.css b/doc/static/htmldiag.css new file mode 100644 index 0000000000..b496d36807 --- /dev/null +++ b/doc/static/htmldiag.css @@ -0,0 +1,138 @@ +.diag-cont { + display: flex; +} +.diag-cont code { + font-family: monospace; + background-color: rgb(255, 231, 196, 0.5); + border: 1px solid rgb(0, 0, 0, 0.5); +} +.diag-cont .diag-note { + background-color: rgb(221, 191, 146); + border-radius: 5px; + border: none; + position: absolute; + top: -60px; + left: 25%; +} +.diag-cont .diag-left { + border: none; + background-color: rgb(195, 221, 146); + border-radius: 10px 10px 10px 0px; + float: left; + width: 75%; +} +.diag-cont .diag-left2 { + border: none; + background-color: rgb(194, 146, 221); + border-radius: 10px 10px 10px 0px; + float: left; + width: 75%; +} +.diag-cont .diag-right { + border: none; + background-color: rgb(221, 164, 146); + border-radius: 10px 10px 0px 10px; + float: right; + width: 75%; +} +.diag-cont .diag-right2 { + border: none; + background-color: rgb(146, 151, 221); + border-radius: 10px 10px 0px 10px; + float: right; + width: 75%; +} +.diag-cont .diag-right3 { + border: none; + background-color: rgb(255, 231, 196); + border-radius: 10px 10px 0px 10px; + float: right; + width: 75%; +} +.diag-cont div { + border: 1px solid rgb(0, 0, 0, 0.5); +} +.diag-dim { + opacity: 0.3; +} +.diag-col { + border: none !important; + padding: 0 !important; + margin: 0 !important; + background: none !important; +} +.diag-cont > div { + flex: 1; + margin: 5px; + padding: 5px; + position: relative; + background-color: rgb(245, 245, 245); +} +.diag-cont div div { + margin: 5px; + padding: 5px; + position: relative; + background-color: rgb(255, 255, 255); +} +.diag-spacer { + border: none !important; + background: none !important; + height: 1em; +} +.diag-caption { + width: 90%; + text-align: justify; + font-style: italic; + font-size: 90%; + margin: 0 auto 20px; +} + +.diag-cont input[type='checkbox'] { + display: none; +} +.diag-cont label { + display: block; +} +.diag-cont label::before { + content: '+'; + display: inline-block; +} +.diag-cont input[type=checkbox]:checked+label::before { + content: '−'; +} +.diag-cont input[type=checkbox]+label~div { + max-height: 0px; + overflow: hidden; +} +.diag-cont input[type=checkbox]:checked+label~div{ + max-height: 100%; +} + +connection .line .arrow { + top: -5px; + height: 0px; + width: 0px; + position: absolute; + border-bottom: 6px solid transparent; + border-top: 6px solid transparent; + background-clip: border-box; +} +connection .line .arrow-fw { + border-right: 12px solid black; +} +connection .line .arrow-bw { + left: 100%; + border-left: 12px solid black; + transform: translateX(-12px); +} +connection .line .text { + position: absolute; + top: 100%; + left: 50%; + transform: translate(-50%, -100%); +} +connection .line { + position:absolute; + height:2px; + background-color: black; +} diff --git a/doc/tutorial/index.rst b/doc/tutorial/index.rst index 9d57b4d7db..67a468c359 100644 --- a/doc/tutorial/index.rst +++ b/doc/tutorial/index.rst @@ -3,29 +3,43 @@ Tutorials ========= -.. Note:: - You can find some examples of full Arbor simulations in the ``python/examples`` directory of the - `Arbor repository `_. +Grouped loosely by primary (but not exclusive!) focus, we have a set of tutorials to help you learn by doing. - The examples use ``pandas``, ``seaborn`` and ``LFPykit`` for analysis and plotting which are expected to be - installed independently from Arbor. +You can find some examples of full Arbor simulations in the ``python/examples`` directory of the +`Arbor repository `_. - In an interactive Python interpreter, you can use ``help()`` on any class or function to get its - documentation. (Try, ``help(arbor.simulation)``, for example). +The examples use ``pandas``, ``seaborn`` and ``LFPykit`` for analysis and plotting which are expected to be +installed independently from Arbor. -.. Todo:: - Add more in-depth tutorial-like pages building up examples here. +In an interactive Python interpreter, you can use ``help()`` on any class or function to get its +documentation. (Try, ``help(arbor.simulation)``, for example). + +Cells +----- .. toctree:: :maxdepth: 1 - :caption: Tutorials single_cell_model single_cell_recipe single_cell_detailed single_cell_detailed_recipe single_cell_cable + +Networks +-------- + +.. toctree:: + :maxdepth: 1 + network_ring network_ring_mpi two_cells_gap_junctions + +Probes +------ + +.. toctree:: + :maxdepth: 1 + tutorial_lfpykit diff --git a/doc/tutorial/network_ring.rst b/doc/tutorial/network_ring.rst index 25a53d95cc..84535f5a56 100644 --- a/doc/tutorial/network_ring.rst +++ b/doc/tutorial/network_ring.rst @@ -107,7 +107,7 @@ Step **(11)** instantiates the recipe with 4 cells. .. literalinclude:: ../../python/example/network_ring.py :language: python - :lines: 61-110 + :lines: 61-109 The execution ************* @@ -135,7 +135,7 @@ Step **(15)** executes the simulation for a duration of 100 ms. .. literalinclude:: ../../python/example/network_ring.py :language: python - :lines: 111-125 + :lines: 111-124 The results *********** @@ -144,7 +144,7 @@ Step **(16)** prints the timestamps of the spikes: .. literalinclude:: ../../python/example/network_ring.py :language: python - :lines: 126-131 + :lines: 126-129 Step **(17)** generates a plot of the sampling data. :py:func:`arbor.simulation.samples` takes a ``handle`` of the probe we wish to examine. It returns a list @@ -156,7 +156,7 @@ It could have described a :term:`locset`.) .. literalinclude:: ../../python/example/network_ring.py :language: python - :lines: 133- + :lines: 131- Since we have created ``ncells`` cells, we have ``ncells`` traces. We should be seeing phase shifted traces, as the action potential propagated through the network. diff --git a/doc/tutorial/network_ring_mpi.rst b/doc/tutorial/network_ring_mpi.rst index 91093e1004..e915c9df6f 100644 --- a/doc/tutorial/network_ring_mpi.rst +++ b/doc/tutorial/network_ring_mpi.rst @@ -21,7 +21,7 @@ Step **(11)** is changed to generate a network with five hundred cells. .. literalinclude:: ../../python/example/network_ring_mpi.py :language: python - :lines: 111-113 + :lines: 109-111 The hardware context ******************** @@ -33,7 +33,7 @@ We print both the communicator and context to observe how Arbor configures their .. literalinclude:: ../../python/example/network_ring_mpi.py :language: python - :lines: 115-120 + :lines: 113-118 The execution ************* @@ -42,7 +42,7 @@ Step **(16)** runs the simulation. Since we have more cells this time, which are .. literalinclude:: ../../python/example/network_ring_mpi.py :language: python - :lines: 133-135 + :lines: 131-133 An important change in the execution is how the script is run. Whereas normally you run the Python script by passing it as an argument to the ``python`` command, you need to use ``srun`` or ``mpirun`` (depending on your MPI @@ -65,18 +65,18 @@ After executing ``mpirun``, all nodes will run the same script. In the domain de the provided MPI communicator to divide the work. Once :py:func:`arbor.simulation.run` starts, each node wil work on their allocated cell ``gid`` s. -This is relevant for the collection of results: these are not gathered for you. Remember that in step **(14)** we +This is relevant for the collection of results: these are not gathered for you. Remember that in step **(15)** we store the handles to the probes; these referred to particular ``gid`` s. The ``gid`` s are now distributed, so on one node, the script will not find the cell referred to by the handle and therefore return an empty list (no results were found). -In step **(18)** we check, for each ``gid``, if the list returned by :py:func:`arbor.simulation.samples` has a nonzero +In step **(17)** we check, for each ``gid``, if the list returned by :py:func:`arbor.simulation.samples` has a nonzero length. The effect is that we collect the results generated on this particular node. Since we now have ``NRANKS`` instances of our script, and we can't access the results between nodes, we have to write the results to disk and analyse them later. We query :py:attr:`arbor.context.rank` to generate a unique filename for the result. .. literalinclude:: ../../python/example/network_ring_mpi.py :language: python - :lines: 137-147 + :lines: 135- In a second script, ``network_ring_mpi_plot.py``, we load the results stored to disk into a pandas table, and plot the concatenated table as before: