I want to load a JSONL file as JSON objects in python. Is there an easy way to do so?
5 Answers
Setting the parameter lines to True should do the trick.
import pandas as pd
jsonObj = pd.read_json(path_or_buf=file_path, lines=True)
2 Comments
Yash Kumar Atri
This one-liner is more helpful than the traditional method, Thanks.
YoungSheldon
Data Science folks, your answer is here.
Full steps including file operations for beginners like me
Assuming you have a .jsonl file like:
{"reviewerID": "A2IBPI20UZIR0U", "asin": "1384719342", "reviewerName": "cassandra tu \"Yeah, well, that's just like, u...", "helpful": [0, 0], "reviewText": "Not much to write about here, but it does exactly what it's supposed to. filters out the pop sounds. now my recordings are much more crisp. it is one of the lowest prices pop filters on amazon so might as well buy it, they honestly work the same despite their pricing,", "overall": 5.0, "summary": "good", "unixReviewTime": 1393545600, "reviewTime": "02 28, 2014"}
{"reviewerID": "A14VAT5EAX3D9S", "asin": "1384719342", "reviewerName": "Jake", "helpful": [13, 14], "reviewText": "The product does exactly as it should and is quite affordable.I did not realized it was double screened until it arrived, so it was even better than I had expected.As an added bonus, one of the screens carries a small hint of the smell of an old grape candy I used to buy, so for reminiscent's sake, I cannot stop putting the pop filter next to my nose and smelling it after recording. :DIf you needed a pop filter, this will work just as well as the expensive ones, and it may even come with a pleasing aroma like mine did!Buy this product! :]", "overall": 5.0, "summary": "Jake", "unixReviewTime": 1363392000, "reviewTime": "03 16, 2013"}
This code should work:
import json
with open('./data/my_filename.jsonl', 'r') as json_file:
json_list = list(json_file)
for json_str in json_list:
result = json.loads(json_str)
print(f"result: {result}")
print(isinstance(result, dict))
About .jsonl files:
http://jsonlines.org/
Comments
Quick and easy native solution without using any split() functions:
import json
with open('/path/to/file.jsonl') as f:
data = [json.loads(line) for line in f]
Comments
The splitlines would address that problem for you, so In general the code below will work for you:
import json
result = [json.loads(jline) for jline in jsonl_content.splitlines()]
If that's the response object the result would be:
result = [json.loads(jline) for jline in response.read().splitlines()]
NOTE
splitlines() treats line breaks differently than JSON's escape requirements. Specifically, splitlines() splits on Unicode Line Separator (\u2028), while JSON allows this character to remain unescaped in strings. When using json.dump with ensure_ascii=False, Python will emit \u2028 unescaped in strings.
9 Comments
CMCDragonkai
What happens if the JSON internally as a newline?
CMCDragonkai
How does
splitlines cover that? If the JSON object internally has a newline, then it would be split at that point.CMCDragonkai
Yea and what if the JSON object has linebreaks internally?
Andriy Ivaneyko
@CMCDragonkai the new line within string is serialiazed from
\n -> \\n which isn't considered to be a line break. So line breaks of object which is represented by json string is retained.
Steve Jessop
Thanks.
\u2028 was an example, and splitlines() does document the full list docs.python.org/3/library/stdtypes.html#str.splitlines |
You can add more keys, but this should work. Say, each line is of the following format. Basically, j_line is a dictionary and access each element like how you would access a dictionary. I have shared accessing nested objects as well.
{"key1":"value", "key2":{"prop_1": "value"}}
with open("foo.jsonl") as f1:
for line in f1:
j_line=json.loads(line)
key_1=j_line['key1']
prop_1=j_line['key2']['prop_2]
jq?