Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

363 example tensor for cp als needs to have cp structure #392

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

363 example tensor for cp als needs to have cp structure #392

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
836e6c3
Checkpoint
dmdunla Feb 3, 2025
090f892
Running nbstripout
dmdunla Feb 5, 2025
cf1f99e
Running ruff format
dmdunla Feb 5, 2025
42adca7
Run nbstripout
dmdunla Feb 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 79 additions & 52 deletions docs/source/tutorial/algorithm_cp_als.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,17 @@
"metadata": {},
"outputs": [],
"source": [
"# Pick the shape and rank\n",
"# Pick the rank and shape\n",
"R = 3\n",
"np.random.seed(0) # Set seed for reproducibility\n",
"X = ttb.tenrand(shape=(6, 8, 10))"
"tensor_shape = (6, 8, 10)\n",
"\n",
"# Set seed for reproducibility\n",
"np.random.seed(0)\n",
"\n",
"# Create a low-rank dense tensor from a Kruskal tensor\n",
"factor_matrices = [np.random.rand(s, R) for s in tensor_shape]\n",
"M_true = ttb.ktensor(factor_matrices)\n",
"X = M_true.to_tensor()"
]
},
{
Expand All @@ -70,19 +77,19 @@
"outputs": [],
"source": [
"# Compute a solution with final ktensor stored in M1\n",
"np.random.seed(0) # Set seed for reproducibility\n",
"short_tutorial = 10 # Cut off solve early for demo\n",
"M1 = ttb.cp_als(X, R, maxiters=short_tutorial)"
"np.random.seed(1) # Set seed for reproducibility\n",
"few_iters = 10 # Cut off solve early for demo\n",
"M1, M1_init, M1_info = ttb.cp_als(X, R, maxiters=few_iters)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Since we set only a single output, `M1` is actually a *tuple* containing:\n",
"1. `M1[0]`: the solution as a `ktensor`. \n",
"2. `M1[1]`: the initial guess as a `ktensor` that was generated at runtime since no initial guess was provided. \n",
"3. `M1[2]`: a dictionary containing runtime information with keys:\n",
"The `cp_als` method returns a the following objects:\n",
"1. `M1`: the solution as a `ktensor`. \n",
"2. `M1_init`: the initial guess as a `ktensor` that was generated at runtime since no initial guess was provided. \n",
"3. `M1_info`: a dictionary containing runtime information with keys:\n",
" * `params`: parameters used by `cp_als`\n",
" * `iters`: number of iterations performed\n",
" * `normresidual`: the norm of the residual `X.norm()**2 + M.norm()**2 - 2*<X,M>`\n",
Expand All @@ -95,17 +102,16 @@
"metadata": {},
"outputs": [],
"source": [
"print(f\"M1[2]['params']: {M1[2]['params']}\")\n",
"print(f\"M1[2]['iters']: {M1[2]['iters']}\")\n",
"print(f\"M1[2]['normresidual']: {M1[2]['normresidual']}\")\n",
"print(f\"M1[2]['fit']: {M1[2]['fit']}\")"
"print(\"M1_info:\")\n",
"for k, v in M1_info.items():\n",
" print(f\"\\t{k}: {v}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Run again with a different initial guess, output the initial guess."
"## Run again with a different initial guess, output the initial guess"
]
},
{
Expand All @@ -114,8 +120,11 @@
"metadata": {},
"outputs": [],
"source": [
"np.random.seed(1) # Set seed for reproducibility\n",
"M2bad, Minit, _ = ttb.cp_als(X, R, maxiters=short_tutorial)"
"# Compute a solution with final ktensor stored in M2\n",
"np.random.seed(2) # Set seed for reproducibility\n",
"M2, M2_init, _ = ttb.cp_als(\n",
" X, R, maxiters=few_iters\n",
") # Will not use third output, so leaving unassigned"
]
},
{
Expand All @@ -132,8 +141,8 @@
"metadata": {},
"outputs": [],
"source": [
"less_short_tutorial = 10 * short_tutorial\n",
"M2 = ttb.cp_als(X, R, maxiters=less_short_tutorial, init=Minit)"
"more_iters = 10 * few_iters\n",
"M2_better, _, _ = ttb.cp_als(X, R, maxiters=more_iters, init=M2_init)"
]
},
{
Expand All @@ -150,16 +159,15 @@
"metadata": {},
"outputs": [],
"source": [
"M1_ktns = M1[0]\n",
"M2_ktns = M2[0]\n",
"score = M1_ktns.score(M2_ktns)"
"score_M2 = M2.score(M_true)\n",
"score_M2_better = M2_better.score(M_true)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, `score()` returned a tuple `score` with the score as the first element:"
"Here, `score()` returns a tuple with the score as the first element:"
]
},
{
Expand All @@ -168,7 +176,8 @@
"metadata": {},
"outputs": [],
"source": [
"score[0]"
"print(f\"Score of M2 and M_true: {score_M2[0]}\")\n",
"print(f\"Score of M2_better and M_true: {score_M2_better[0]}\")"
]
},
{
Expand All @@ -192,10 +201,9 @@
"metadata": {},
"outputs": [],
"source": [
"M2alt = ttb.cp_als(X, R, maxiters=less_short_tutorial, init=Minit)\n",
"M2alt_ktns = M2alt[0]\n",
"score = M2_ktns.score(M2alt_ktns) # Score of 1 indicates the same solution\n",
"print(f\"Score: {score[0]}.\")"
"M2_rerun, _, _ = ttb.cp_als(X, R, maxiters=few_iters, init=M2_init)\n",
"score_M2_rerun = M2.score(M2_rerun) # Score of 1 indicates the same solution\n",
"print(f\"Score: {score_M2_rerun[0]}\")"
]
},
{
Expand All @@ -212,7 +220,7 @@
"metadata": {},
"outputs": [],
"source": [
"M2alt2 = ttb.cp_als(X, R, maxiters=less_short_tutorial, init=Minit, printitn=20)"
"M = ttb.cp_als(X, R, maxiters=few_iters, printitn=2)"
]
},
{
Expand All @@ -229,15 +237,15 @@
"metadata": {},
"outputs": [],
"source": [
"M2alt2 = ttb.cp_als(X, R, printitn=0) # No output"
"M = ttb.cp_als(X, R, maxiters=few_iters, printitn=0) # No output"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Use HOSVD initial guess\n",
"Use the `\"nvecs\"` option to use the leading mode-$n$ singular vectors as the initial guess."
"Use the `\"nvecs\"` option to use the leading mode-$n$ singular vectors as the initial guess. The initialization process will require more computation in general, but it may lead to better solutions."
]
},
{
Expand All @@ -246,16 +254,18 @@
"metadata": {},
"outputs": [],
"source": [
"M3 = ttb.cp_als(X, R, init=\"nvecs\", printitn=20)\n",
"s = M2[0].score(M3[0])\n",
"print(f\"score(M2,M3) = {s[0]}\")"
"M3, _, _ = ttb.cp_als(X, R, maxiters=few_iters, init=\"nvecs\", printitn=0)\n",
"score_M3 = M3.score(M_true)\n",
"print(f\"Score of M2 and M_true: {score_M2[0]}\")\n",
"print(f\"Score of M3 and M_true: {score_M3[0]}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Change the order of the dimensions in CP"
"## Change the order of the dimensions in CP\n",
"Changing the order of the dimensions in which `cp_als` iterates over the input tensor can lead to a different solution."
]
},
{
Expand All @@ -264,16 +274,19 @@
"metadata": {},
"outputs": [],
"source": [
"M4, _, info = ttb.cp_als(X, 3, dimorder=[1, 2, 0], init=\"nvecs\", printitn=20)\n",
"s = M2[0].score(M4)\n",
"print(f\"score(M2,M4) = {s[0]}\")"
"M4, _, M4_info = ttb.cp_als(\n",
" X, R, maxiters=few_iters, init=\"nvecs\", printitn=0, dimorder=[2, 1, 0]\n",
")\n",
"score_M4 = M4.score(M_true)\n",
"print(f\"Score of M3 and M_true: {score_M3[0]}\")\n",
"print(f\"Score of M4 and M_true: {score_M4[0]}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the last example, we also collected the third output argument `info` which has runtime information in it. The field `info[\"iters\"]` has the total number of iterations. The field `info[\"params\"]` has the information used to run the method. Unless the initialization method is `\"random\"`, passing the parameters back to the method will yield the exact same results."
"In the last example, we also collected the third output argument `M4_info` which has runtime information in it. The field `M4_info[\"iters\"]` has the total number of iterations. The field `M4_info[\"params\"]` has the information used to run the method. Unless the initialization method is `\"random\"`, passing the parameters back to the method will yield the exact same results."
]
},
{
Expand All @@ -282,17 +295,25 @@
"metadata": {},
"outputs": [],
"source": [
"M4alt, _, info = ttb.cp_als(X, 3, **info[\"params\"])\n",
"s = M4alt.score(M4)\n",
"print(f\"score(M4alt,M4) = {s[0]}\")"
"M4_rerun, _, M4_rerun_info = ttb.cp_als(X, R, init=\"nvecs\", **M4_info[\"params\"])\n",
"score_M4_rerun = M4.score(M4_rerun)\n",
"print(f\"Score of M4 and M4_rerun: {score_M4_rerun[0]}\")\n",
"\n",
"print(\"M4_info:\")\n",
"for k, v in M4_info.items():\n",
" print(f\"\\t{k}: {v}\")\n",
"\n",
"print(\"M4_rerun_info:\")\n",
"for k, v in M4_rerun_info.items():\n",
" print(f\"\\t{k}: {v}\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Change the tolerance\n",
"It's also possible to loosen or tighten the tolerance on the change in the fit. You may need to increase the number of iterations for it to converge."
"## Change the stopping tolerance\n",
"It's also possible to loosen or tighten the stopping tolerance on the change in the fit. Note that you may need to increase the number of iterations for it to converge."
]
},
{
Expand All @@ -301,7 +322,7 @@
"metadata": {},
"outputs": [],
"source": [
"M5 = ttb.cp_als(X, 3, init=\"nvecs\", stoptol=1e-12, printitn=100)"
"M5 = ttb.cp_als(X, R, init=\"nvecs\", maxiters=1000, stoptol=1e-12, printitn=100)"
]
},
{
Expand All @@ -318,17 +339,23 @@
"metadata": {},
"outputs": [],
"source": [
"X = ttb.ktensor(\n",
"# Create rank-2 tensor\n",
"X2 = ttb.ktensor(\n",
" factor_matrices=[\n",
" np.array([[1.0, 1.0], [1.0, -10.0]]),\n",
" np.array([[1.0, 1.0], [1.0, -10.0]]),\n",
" np.array([[1.0, 1.0], [-1.0, -10.0]]),\n",
" np.array([[1.0, 1.0], [-2.0, -10.0]]),\n",
" ],\n",
" weights=np.array([1.0, 1.0]),\n",
")\n",
"M1 = ttb.cp_als(X, 2, printitn=1, init=ttb.ktensor(X.factor_matrices))\n",
"print(M1[0]) # default behavior, fixsigns called\n",
"M2 = ttb.cp_als(X, 2, printitn=1, init=ttb.ktensor(X.factor_matrices), fixsigns=False)\n",
"print(M2[0]) # fixsigns not called"
"print(f\"X2=\\n{X2}\\n\")\n",
"\n",
"M_fixsigns, _, _ = ttb.cp_als(X2, 2, printitn=0, init=ttb.ktensor(X2.factor_matrices))\n",
"print(f\"M_fixsigns=\\n{M_fixsigns}\\n\") # default behavior, fixsigns called\n",
"\n",
"M_no_fixsigns, _, _ = ttb.cp_als(\n",
" X2, 2, printitn=0, init=ttb.ktensor(X2.factor_matrices), fixsigns=False\n",
")\n",
"print(f\"M_no_fixsigns=\\n{M_no_fixsigns}\\n\") # fixsigns not called"
]
},
{
Expand Down
Loading