Working with Files & Directories in Python

Working with Files & Directories in Python

Last updated:
Table of Contents

All examples assume Python 3, unless otherwise stated

Write string to file

write is a method of class File. See all here

# will be overwritten if it already exists!
with open("filename.txt","w") as f:
    f.write("foo bar")

Read file into string

Heads-up: Will crash if the file doesn't fit in memory. See Buffered Reading

with open("filename.txt","r") as f:
    contents = f.read()

# use variable `contents`

Buffered reading

Process text files line by line without loading it all to memory at once:

with open("path/to/file") as f:
    for line in f:       
        # process the line

You can also use f.readline() to read a whole line1 or f.read(size), which takes an argument indicating the maximum number of bytes to be read into memory.

Check file exists

Returns True if the given path exists and it's a file.

To test if a path exists (file, directory or even link), use os.path.exists() instead.

import os.path

path_to_file = "/path/to/file"

if os.path.isfile(path_to_file):
    # it exists, and it is a regular
    # (i.e. not a directory) file

Check directory exists

import os

# path is either absolute or relative to working dir
os.path.isdir(path_to_directory)

List files in directory

To show all files in a directory, use os.listdir(path_to_directory):

import os

os.listdir(path_to_directory)
>>> ['file1.txt', 'file2.json', 'myfile.png']

Loop over files in directory

for file in directory

import os

root_path = "/path/to/directory"

for file_name in os.listdir(root):

    # /path/to/directory/myfile.txt
    full_path_to_file = os.path.join(root, file_name)

Loop over files in directory, recursively

Loop over all files recursively under a path (including subdirectories):

import os

# you may omit trailing slashes
# i.e. '/tmp' works too
path = '/tmp/'

for root, directories, filenames in os.walk(path):
    for directory in directories:

        directory_path = os.path.join(root, directory)
        # do something with directory_path

    for filename in filenames:

        file_path = os.path.join(root, filename)
        # do something with directory_path

Delete file

Use os.remove(<path-to-file>).

import os

# raises FileNotFoundError if file doesn't exist
os.remove("/path/to/file")

Move/rename file

Use: os.replace(from, to)

import os

# raises FileNotFoundError if source file doesn't exist
os.replace('/path/to/source/file','/path/to/new/location/file')

Copy file

Use: shutil.copy(<from>,<to>) and shutil.copy2(<from>,<to>). Both methods are used the same way. See differences below.

Shutil is part of Python's standard library; there's no need to install anything.

Method Copies file data Copies file permissions Copies file creation and
modfication dates
shutil.copyYESYESNO
shutil.copy2YESYESYES

  • Example 1: Target path is the full path to the new location

    import shutil
    
    source_path = '/path/to/source/source-file'
    target_path = '/path/to/target/target-file'
    
    # raises FileNotFoundError if source file doesn't exist
    # target file is overwritten if it already exists!
    shutil.copy(source_path,target_path)
    # >>> '/path/to/target/target-file'
    
  • Example 2: Target path is the path to the new directory

    import shutil
    
    source_path = '/path/to/my-file'
    target_path = '/path/to/another/directory/'
    
    # raises FileNotFoundError if source file doesn't exist
    # target file is overwritten if it already exists!
    shutil.copy(source_path,target_path)
    # >>> '/path/to/another/directory/my-file'
    

Add directory to system PATH

That is, the PATH environment variable.

import os

os.environ["PATH"] += os.pathsep + "/path/to/directory"

1: This equates to reading the whole file into memory if the file has no line breaks

Dialogue & Discussion