TypeError: first argument must be an iterable of pandas objects

Dung Do Tien May 30 2021 316

I have a big Dataframe, I want to load a CSV file and I try to split that and after execute run concat() method. See code below:

df2 = pd.read_csv('et_users.csv', header=None, names=names2, chunksize=100000)
for chunk in df2:
    chunk['ID'] = chunk.ID.map(rep.set_index('member_id')['panel_mm_id'])

df2 = pd.concat(chunk, ignore_index=True)

But I got an error:

 TypeError: first argument must be an iterable of pandas objects, you passed an object of type "DataFrame"

How can I resolve it?

Have 1 answer(s) found.
  • D

    Dương Trang Quốc May 30 2021

    You need to append each chunk to a list and then use concat to concatenate them all:

    df2 = pd.read_csv('et_users.csv', header=None, names=names2, chunksize=100000)
    chunks=[]
    for chunk in df2:
        chunk['ID'] = chunk.ID.map(rep.set_index('member_id')['panel_mm_id'])
        chunks.append(chunk)
    
    df2 = pd.concat(chunks, ignore_index=True)

    I hope it useful for you!!

Leave An Answer
* NOTE: You need Login before leave an answer

* Type maximum 2000 characters.

* All comments have to wait approved before display.

* Please polite comment and respect questions and answers of others.

Popular Tips

X Close