hacking stdin

[rrbarbosa]

I would like to see some support to read from stdin instead of only files. I am by no means a go programmer, but I think a push request is a good way to get the conversation going.

I hacked in some support for JSON files. I did not worry much about error handling and your fancy progress bar won't work.

If you would like to support the feature, I can go on and start implementing it for the other importers. Any comments are welcome.

[lukasmartinelli]

This is awesome idea and great initial work!! Thanks @rrbarbosa

Tell me when to merge. I will merge it and then go over the code afterwards to make sure progress bar etc. works again.

As far as I remember in the beginning pgfutter only supported input from stdin but then I forgot about how useful that is. Especially for streaming something into the database.

[rrbarbosa]

Sure, I will work on it this week. 2 things:

• The progress bar won't work with stdin as you don't know how many bytes/records you are reading beforehand, unless I am missing something...
• Do you have any suggestions on how to deal with the table name when reading stdin?

[rrbarbosa]

Ok. I tested both json and csv and they seem to be working from stdin. For now, if no filename is present and the --table option is not set, the data is loaded in a table named stdin.

I have not tested the JSONObject option.

Personally, I don't like that I need to test filename == "" multiple times to decide if I am reading from file or stdin. Probably a better way would be to create the proper Reader object when parsing... But hey, its working.

[lukasmartinelli]

Very very nice 👍
Yes passing around a Reader object to the different protocol implementations would be much cleaner. Will try to refactor this after the merge.