Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

colmetadata does not read custom metadata with multiple writes #485

Open
JakeZw opened this issue Sep 12, 2023 · 0 comments
Open

colmetadata does not read custom metadata with multiple writes #485

JakeZw opened this issue Sep 12, 2023 · 0 comments

Comments

@JakeZw
Copy link

JakeZw commented Sep 12, 2023

This problem is described in more detail in discourse.

Specifically the following MWE works when the for loop executes once, but the colmetadata gives nothing when the for loop executes more than once.

`using Arrow
using Tables

go to runtest.jl in Arrow package for hints on how to use metadata and colmetadata

cd(@DIR)
filename = "test10.arrow"
meta = ["file" => "test.arrow", "when" => "now", "why" => "gain experience"]
metacol1 = ["id" => "chan1", "comment" => "Chan 1 comment"]
metacol2 = ["id" => "chan2", "comment" => "Chan 2 comment"]
writer = open(Arrow.Writer, filename, metadata=meta, colmetadata = Dict(:Column1 => metacol1, :Column2 => metacol2))

for i in 1:1
result = rand(2, 2) .+ i
Arrow.write(writer, Tables.table(result)) #, header=[:a,:b])
end
close(writer)

tbl = Arrow.Table(filename)

tblmeta = Arrow.getmetadata(tbl)
tblmeta
tblmeta["why"]
tblmeta["when"]

tblmetachan1 = Arrow.getmetadata(tbl.Column1)
tblmetachan1
tblmetachan1 == nothing
`

For once through the for loop we get
julia> tblmetachan1 = Arrow.getmetadata(tbl.Column1) Base.ImmutableDict{String, String} with 2 entries: "comment" => "Chan 1 comment" "id" => "chan1"

When we go through the for loop twice (or more) we get
`julia> tblmetachan1 = Arrow.getmetadata(tbl.Column1)

julia> tblmetachan1

julia> tblmetachan1 == nothing
true`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant