Working with Files & the Filesystem in Python: Reference and Examples

Last updated:
Table of Contents

All examples assume Python 3, unless otherwise stated

The full list of methods you can call on a file object can be found here: Python 3: io.IOBASE

Write string to file

write is just one method of class File. See all here

# will be overwritten if it already exists!
with open("filename.txt","w") as f:
    f.write("foo bar")

Read file into string

Heads-up: This will break if the file doesn't fit in memory

with open("filename.txt","r") as f:
    contents = f.read()

# use variable `contents`

Buffered reading

Process a large text file, but without without loading it all to memory at the same time (buffered reading)

This is easy with python:

with open("path/to/file") as f:
    for line in f:       
        # process the line

You can also use f.readline() to read a whole line (dangerous if the file has no line breaks) or f.read(size), which takes an argument indicating the maximum number of bytes to be read into memory.

Process file line by line

See above: Buffered Reading

Check if file exists

This method returns True if the given path exists and it's a file.

If you just want to test if a path exists (be it a file, directory or even link), use os.path.exists() instead.

import os.path

path_to_file = "/path/to/file"

if os.path.isfile(path_to_file):
    # it exists, and it is a regular
    # (i.e. not a directory) file

Relative to absolute path

Use os.path.abspath. But remember that the relative path is calculated with respect to your current working directory.

Note that this will not warn you if you supply a path that resolves to a file/dir that doesn't exist

If you want the relative path to be calculated with respect to the current file location, don't forget to prepend __file__+"/" to the relative path!

import os

# path is relative to the current WORKING directory!
abs_path = os.path.abspath("../../other-dir")

# path is relative to the directory where THIS file
# is located!
abs_path = os.path.abspath(__file__+"/../../other-dir")

Relative to absolute path in Jupyter

"NameError: name '__file__' is not defined"

In jupyter notebooks, you must use "__file__" (in quotes) instead:

import os

# on a jupyter notebook
abs_path = os.path.abspath("__file__"+"/../../other-dir")

File name from path

Use os.path.basename(<full-path-to-file>)

import os

print(os.path.basename("/path/to/some/file.txt"))
# "file.txt"

Path from file name

Note the trailing slash is not included.

Just use os.path.dirname(<full-path-to-file>):

import os

print(os.path.dirname("/path/to/some/file.txt"))
# "/path/to/some"

Current working directory

The current working directory (or cwd) is the directory the user that first called the script was located.

import os

os.getcwd()

Directory where current file is located

Note that this is not at all the same thing as the current working directory (seen above)

import os

# __file__ points to the where the current file is located
os.path.dirname(os.path.realpath(__file__))

List files in directory

import os

os.listdir(path_to_directory)

Loop over files in directory

import os

root_path = "/path/to/directory"

for file in os.listdir(root):

    # myfile.txt
    file_name = file

    # /path/to/directory/myfile.txt
    full_path_to_file = os.path.join(root,file)

Loop over files in directory, recursively

To loop over all files recursively under some path (i.e. including subdirectories):

import os

# you may omit trailing slashes
# i.e. '/tmp' works too
path = '/tmp/'

for root, directories, filenames in os.walk(path):
    for directory in directories:

        directory_path = os.path.join(root, directory)
        # do something with directory_path

    for filename in filenames:

        file_path = os.path.join(root,filename)
        # do something with directory_path

Delete file

Use os.remove(<path-to-file>).

import os

# raises FileNotFoundError if file doesn't exist
os.remove("/path/to/file")

Move/rename file

Use: os.replace(<source-path>, <target-path>)

import os

# raises FileNotFoundError if source file doesn't exist
os.replace('/path/to/source/file','/path/to/new/location/file')

Copy file

Use: shutil.copy(<from>,<to>) and shutil.copy2(<from>,<to>). Both methods are used the same way. See differences below.

Shutil is part of Python's standard library; there's no need to install anything.

Method Copies file data Copies file permissions Copies file creation and
modfication dates
shutil.copyYESYESNO
shutil.copy2YESYESYES

  • Target path can be the full path to the new location

    import shutil
    
    source_path = '/path/to/source/source-file'
    target_path = '/path/to/target/target-file'
    
    # raises FileNotFoundError if source file doesn't exist
    # target file is overwritten if it already exists!
    shutil.copy(source_path,target_path)
    # >>> '/path/to/target/target-file'
    
  • Target path can also be the path to a directory

    import shutil
    
    source_path = '/path/to/my-file'
    target_path = '/path/to/another/directory/'
    
    # raises FileNotFoundError if source file doesn't exist
    # target file is overwritten if it already exists!
    shutil.copy(source_path,target_path)
    # >>> '/path/to/another/directory/my-file'
    

Add directory to system PATH

That is, the PATH environment variable.

import os

# use os.pathsep so that this works under linux or windows
os.environ["PATH"] += os.pathsep + "/path/to/directory"

Dialogue & Discussion