[WIP] added definitions for lp-distances. Faster than just using norm

[ajinkya-k]

This PR adds functions to compute $L_p$ distance between two vectors. This is faster than just calling norm(x .- y)

Some notes (I am going to write up the benchmarks and post the details on my website soon):

• separate methods for integer p and real p. Integer versions (e.g. supplying p = 3) are an order of magnitude faster than the corresponding real version (eg p = 3.0)
• Special versions are provided for p = 0, 1, 2 (provided as integers) and for p = Inf.
• l1 and l2 dist can be made faster using sum(x .- y) but that allocates memory, and the allocation can be quite large for large vectors, if there is any non-allocating version that does not involve looping please let me know I can try it! (mapreduce also allocates and is slow)

TODOs:

• add compat notice (?)
• add tests
• use (x - y < tol) for $\ell_0$? or remove $\ell_0$ altogether?
• add comment to mention the StatsBase implementation which is the source for the loop sum version
  Any feedback is welcome!

[ajinkya-k]

@jishnub @StefanKarpinski please let me know what you think, and if there is any way i can make it more efficient.

[codecov]

Codecov Report

Attention: Patch coverage is 97.36842% with 1 line in your changes missing coverage. Please review.

Project coverage is 91.98%. Comparing base (8cc4216) to head (11df60f).

Files with missing lines	Patch %	Lines
src/generic.jl	97.36%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1234      +/-   ##
==========================================
- Coverage   91.99%   91.98%   -0.02%     
==========================================
  Files          34       34              
  Lines       15480    15514      +34     
==========================================
+ Hits        14241    14270      +29     
- Misses       1239     1244       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[RoyiAvital]

I'd make the last step of applying ^(1/p) with an opt out option.
In many calculations applying the power is not needed (Though then it is not a real norm / distance).
As an example, many use the Squared Euclidean distance (Not really a distance).

[jishnub]

I wonder if this should be a method of norm rather than a separate function that does something similar?

[fredrikekre]

I think all of this exist in https://github.com/JuliaStats/Distances.jl ?

[ajinkya-k]

@RoyiAvital great idea.

[stevengj]

Note that this algorithm suffers from spurious overflow if the elements of x or y are very large.

e.g. lpdist([1e300],[0.0]) should return 1e300, but this will return Inf.

[ajinkya-k]

Thanks for the catch! Any pointers on how I can fix this?

[jishnub]

Perhaps divide each vector by the max of the maximum absolute value of each vector before computing the distance, and rescale the result later?

[ajinkya-k]

Sounds good.

[stevengj]

Note that rescaling is not needed for the L1 norm or the maximum norm.

Note also that you should be using norm, not abs, everywhere, in order to handle vectors whose entries are themselves vectors. In fact, arguably you should be calling the distance function recursively for such cases.

In general, I would study the generic_norm implementation, which already deals with such issues.

[ajinkya-k]

Okay. I intended this to be an implementation of vectors of reals (and maybe complex) only. Would it be better to specialise for real/complex vectors and have a separate implementation for recursive structures?

[stevengj]

See how it's done for norm. There's no need to have two implementations, since norm_sqr just calls abs2 for scalars, and norm for scalars just calls abs.

You should really study the norm implementation and build off that, as otherwise you're re-inventing the wheel here.

[stevengj]

Has there been discussion in the past of whether Distances.jl functionality belongs in the stdlib?

[ViralBShah]

I don't recollect such a discussion.

[ajinkya-k]

This came up in a slack thread and I did mention that the l2dist was available in statsbase but there seemed to be some interest in having the l2dist in LinearAlgebra

[stevengj]

Should be norm and norm_sqr, respectively.