Pandas Dataframes: CSV Quoting and Escaping Strategies

Pandas Dataframes: CSV Quoting and Escaping Strategies

Last updated:
Table of Contents

WIP Alert This is a work in progress. Current information is correct but more content may be added in the future.

Using pandas v. 1.0.3

Idempotent read and write

Must Quote nonnumeric, backslash for escaping

This is a safe pattern for most use cases:

  • Sample CSV structure:

    "column1","column2"
    "foo bar", 2
    "text with \"text within quotes\" foo", 3,
    "text with back \ slashes in the middle",  4
    
  • TO WRITE

    import csv
    import pandas as pd
    
    df = # build dataframe here
    
    df.to_csv(
        "/path/to/output/file.csv",
        quoting=csv.QUOTE_NONNUMERIC,
        escapechar="\\",
        doublequote=False,
        index=False)
    
  • TO READ

    import pandas as pd
    
    df = pd.read_csv(
        "/path/to/output/file.csv",
        escapechar="\\")
    

Dialogue & Discussion