A repo to reproduce Explaining the Machine Learning Solution of the Ising Model by Alamino from 2024.
We explore the author's suggestions.
- simulation employs Wolff cluster updates (lazy variant) -- just call
python generate_configs.py
- the simulation generates the right number of configs; it however is not aware of the sign such that the SLNN is trained on less configs than it should -- the work however still reproduces favorably (lazy variant)
- PCA has a shortcoming here (secondary literature suggests nice separability for different projections on eigenvectors of M -- here, the particle number is not conserved so this is a little less visible)
- elegant idea of using a "SLNN" (a single layer NN) to predict the model's phase -- the SLNN is a simple logistic regression and it works nicely on data which has positive magnetization!
- SLNN args to activation function scale differently in this implementation -- might be due to other parameter samplers (maybe inherent in Pytorch as original work uses keras)
- the straightforward idea to extend the ML model by a 2-unit hidden layer can immediately predict the nice properties of the Ising model
- the graph on "inferred probabilities" is, at times, not reproducible -- we strongly depend on the seed; see below
- overall a very illuminating paper giving us some intuition on what's actually going on for simple NN architectures and we have some theory on the data-generating process
- more implementation shortcomings: initialization for the Ising model and the paper's suggestions varies; "1HLNN" has no strictly correct way of initializing (needs work on that front to be fully explainable)
Suggested way to run this repo:
micromamba env create -n repro-ising-ml
micromamba activate repro-ising-ml
micromamba install -c conda-forge numpy tqdm pytorch scikit-learn matplotlib
# e.g.
python generate_configs.py
python apply_model.py 20 500 100 20 mlnn
# args are: L num_configs num_T num_epochs model
# L -- 2d Ising model PBC grid with size L * L (fixed in `generate_configs.py` -- change at will
# num_configs -- number of configurations to generate per temperature
# num_T -- number of temperatures to generate samples for in [0.05, 5]
# num_epochs -- number of training epochs
# model -- either "slnn" or "mlnn"
apply_model.py
runs a training loop for num_epochs
on data in the following way:
- SLNN: here we only take configurations into account that have positive magnetization (see paper for reasoning on separating hyperplane)
- MLNN: takes all configurations into account
For both cases we sample from data equidistant from T_c
, ie. we only train on data in the union of the intervals
[0.05, 1.] and [4., 5.].
The "explanatory" part for the network with a hidden layer is fuzzy. Results are not consistent. Trained on the same data, same procedure, only the Pytorch-given seed differs -- the results are sometimes overwhelmingly different.
Compare
![]() |
![]() |
to
![]() |
![]() |
The test data here is now all of the data points that have been excluded from training, ie. all configs generated for temperatures between 1 and 4.
For the single-layer NN, results are remarkably similar.
![]() |
![]() |
The accompagnying presentation can be found here. (https://bit.ly/explain-ising-pres)