Pandas Dataframes: CSV Quoting and Escaping Strategies

Last updated:
Table of Contents

WIP Alert This is a work in progress. Current information is correct but more content may be added in the future.

Using pandas v. 1.0.3

Quote nonnumeric, backslash for escaping

This is a safe pattern for most use cases:

  • Sample CSV structure:

    "column1","column2"
    "foo bar",2
    "\"text within quotes\"", 3
    
  • Write dataframe to CSV

    import csv
    import pandas as pd
    
    df = # build dataframe here
    
    df.to_csv(
        "/path/to/output/file.csv",
        quoting=csv.QUOTE_NONNUMERIC,
        escapechar="\\",
        doublequote=False,
        index=False)
    
  • Read using the same strategy

    import pandas as pd
    
    df = pd.read_csv(
        "/path/to/output/file.csv",
        escapechar="\\")
    

Dialogue & Discussion