Qfyilyi

I have 5 json files saved in my current working directory, and the following code works fine for me to read each of them for further analysis:

import pandas as pd

import os

path=os.path.join('.') #Just want to as an example here

allfiles = os.listdir(path)

files = [files for files in allfiles if files.endswith('.json')]



for i in files:

    df=pd.read_json(i)

allfiles

But when I create a sub-folder in my current working directory, and move my 5 json files to it, then I cnanot read them:

import pandas as pd

import os

path=os.path.join('.','Result')#New Folder:Result

allfiles = os.listdir(path)

files = [files for files in allfiles if files.endswith('.json')]



for i in files:

    df=pd.read_json(i)

allfiles

The error message is: ValueError: Trailing data
I try to search it via google, but still don't know what is going on, i didn't change anything, just create a sub-folder.

When opening the file using pandas you are passing only the filename not the absolute path. so when script is running from current directory where file is present it is able to open it.
but when you have moved the files to Result directory it is still searching for file in current directory.

edited below code to have full directory path in filenames array.

cwd = os.getcwd()

path=os.path.join(cwd) #Just want to as an example here

allfiles = os.listdir(path)

files = [os.path.join(path,files) for files in allfiles if files.endswith('.json')]

print(files)



for f in files :

    df1 = pd.read_json(f)

print(df1.head(1))

['C:UsersXXXXXXXXXall_data.json']

glossary
GlossDiv {'title': 'S', 'GlossList': {'GlossEntry':
{'I...

path=os.path.join(cwd,'Result') #Just want to as an example here

allfiles = os.listdir(path)

files = [os.path.join(path,files) for files in allfiles if files.endswith('.json')]

print(files)

for f in files :

    df1 = pd.read_json(f)

print(df1.head(1))

['C:UsersXXXXXXXXXResultall_data1.json']

glossary GlossDiv {'title': 'S', 'GlossList': {'GlossEntry': {'I...

sample data in file :

 {

        "glossary": {

            "title": "example glossary",

            "GlossDiv": {

                "title": "S",

                "GlossList": {

                    "GlossEntry": {

                        "ID": "SGML",

                        "SortAs": "SGML",

                        "GlossTerm": "Standard Generalized Markup Language",

                        "Acronym": "SGML",

                        "Abbrev": "ISO 8879:1986",

                        "GlossDef": {

                            "para": "A meta-markup language, used to create markup languages such as DocBook.",

                            "GlossSeeAlso": ["GML", "XML"]

                        },

                        "GlossSee": "markup"

                    }

                }

            }

        }

    }

Try Current working directory and then join the new sub directory.

cwd = os.getcwd()

path=os.path.join(cwd,'Result')#New Folder:Result

I just tested the code on .pdf as I am having pdf in Downloads directory.

import pandas as pd

import os

cwd = os.getcwd()

path=os.path.join(cwd,"Downloads")#New Folder:Result

allfiles = os.listdir(path)

files = [files for files in allfiles if files.endswith('.pdf')]



for i in files:

    print i

It worked for me.

You could use glob.glob, which returns a list of paths that match your pathname:

import glob

pathname = 'Result/*.json'

list_of_paths_to_files = glob.glob(pathname)

If you want to read the files as many separate DataFrame:

list_of_dataframes = [pd.read_json(file_path) for file_path in list_of_paths_to_files]

If you want one DataFrame:

df = pd.concat(list_of_dataframes)