incorrect JSON format

[bishopZ]

From the README
"A lot of event logs contain JSON objects nowadays (e.g. GitHub Archive). pgfutter expects each line to have a valid JSON object. Importing JSON is only supported for Postgres 9.3 and Postgres 9.4 due to the JSON type.

Create friends.json.

{"name": "Jacob", "age": 26, "friends": ["Anthony"]}
{"name": "Anthony", "age": 25, "friends": []}
{"name": "Emma", "age": 28, "friends": ["Jacob", "Anthony"]}"

This is not valid JSON. Please see http://json.org/ to see what I mean.

Any plans to support properly formatted JSON?

[lukasmartinelli]

This is called JSONLines http://jsonlines.org/. Each line contains a valid JSON object.
The reason why this format is very good is that dealing with a 30GB big JSON file would be close to impossible.

What is invalid about that JSON object?

{"name": "Jacob", "age": 26, "friends": ["Anthony"]}

At the bottom of the README you find how to import a single JSON document.
https://github.com/lukasmartinelli/pgfutter#import-single-json-object

[bishopZ]

Any ideas on how to create a JSONLines document from a valid JSON array of objects?

I guess I could manually remove the comma from the 10,000 rows I want to import.

[lukasmartinelli]

I guess I could manually remove the comma from the 10,000 rows I want to import.

We had the same problem in #9.

You have a JSON object with 10k rows? That might still fit into memory if you try pgfutter jsonobj document.json. You will then get a single object in the database and using PostgreSQL JSON operators you can then put it into a relational schema.

Or you can use jq to pull it out.

Given this JSON file.

{
  "results": [
    {
      "text": "@twitterapi  http://tinyurl.com/ctrefg",
      "to_user_id": 396524,
      "to_user": "TwitterAPI",
      "from_user": "jkoum"
    },
    {
      "text": "@twitterapi  http://tinyurl.com/jubu",
      "to_user_id": 40314,
      "to_user": "TwitterAPI",
      "from_user": "jubu"
    }
  ]
}

You can filter out the results array as JSONLines like this.

cat test.json | jq -c '.results[]'