-
-
Notifications
You must be signed in to change notification settings - Fork 5.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC: generator expressions #14848
RFC: generator expressions #14848
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
""" | ||
Generator(f, iter) | ||
|
||
Given a function `f` and an iterator `iter`, construct an iterator that yields | ||
the values of `f` applied to the elements of `iter`. | ||
The syntax `f(x) for x in iter` is syntax for constructing an instance of this | ||
type. | ||
""" | ||
immutable Generator{I,F} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. any particular reason you chose to go with There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes, I think There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
f::F | ||
iter::I | ||
end | ||
|
||
start(g::Generator) = start(g.iter) | ||
done(g::Generator, s) = done(g.iter, s) | ||
function next(g::Generator, s) | ||
v, s2 = next(g.iter, s) | ||
g.f(v), s2 | ||
end | ||
|
||
collect(g::Generator) = map(g.f, g.iter) |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -292,3 +292,52 @@ eltype{I1,I2}(::Type{Prod{I1,I2}}) = tuple_type_cons(eltype(I1), eltype(I2)) | |
x = prod_next(p, st) | ||
((x[1][1],x[1][2]...), x[2]) | ||
end | ||
|
||
_size(p::Prod2) = (length(p.a), length(p.b)) | ||
_size(p::Prod) = (length(p.a), _size(p.b)...) | ||
|
||
""" | ||
IteratorND(iter, dims) | ||
|
||
Given an iterator `iter` and dimensions tuple `dims`, return an iterator that | ||
yields the same values as `iter`, but with the specified multi-dimensional shape. | ||
For example, this determines the shape of the array returned when `collect` is | ||
applied to this iterator. | ||
""" | ||
immutable IteratorND{I,N} | ||
iter::I | ||
dims::NTuple{N,Int} | ||
|
||
function (::Type{IteratorND}){I,N}(iter::I, shape::NTuple{N,Integer}) | ||
li = length(iter) | ||
if li != prod(shape) | ||
throw(DimensionMismatch("dimensions $shape must be consistent with iterator length $li")) | ||
end | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good catch, thanks. |
||
new{I,N}(iter, shape) | ||
end | ||
(::Type{IteratorND}){I<:AbstractProdIterator}(p::I) = IteratorND(p, _size(p)) | ||
end | ||
|
||
start(i::IteratorND) = start(i.iter) | ||
done(i::IteratorND, s) = done(i.iter, s) | ||
next(i::IteratorND, s) = next(i.iter, s) | ||
|
||
size(i::IteratorND) = i.dims | ||
length(i::IteratorND) = prod(size(i)) | ||
ndims{I,N}(::IteratorND{I,N}) = N | ||
|
||
eltype{I}(::IteratorND{I}) = eltype(I) | ||
|
||
collect(i::IteratorND) = copy!(Array(eltype(i),size(i)), i) | ||
|
||
function collect{I<:IteratorND}(g::Generator{I}) | ||
sz = size(g.iter) | ||
if length(g.iter) == 0 | ||
return Array(Union{}, sz) | ||
end | ||
st = start(g) | ||
first, st = next(g, st) | ||
dest = Array(typeof(first), sz) | ||
dest[1] = first | ||
return map_to!(g.f, 2, st, dest, g.iter) | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i think you need to add this signature to the appropriate manual section and then run genstdlib.jl to copy this text over
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It isn't exported, so it would be as
Base.Generator
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actually I wonder about the name of this type. The syntax for it is definitely called a "generator expression" (at least in python), but in python this produces a generator object, which is somewhat similar to a Task, while we return more of a MapIterator. I guess best not to export it for now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I saw this as the equivalent of python's
itertools.imap
, a lazy map (except thatimap
accepts multiple iterators).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we're planning on reorganizing a little to make a part-of-base Iterator or IterTools module, this would be a natural fit there under whichever final name we settle on.