Python Pickle: examples and reference

Python 3.x is used in all examples, unless explicitly noted

Pickle dict

Use a .p or .pkl extension to follow convention

import pickle

colors = { "john": "yellow", "mary": "red" }

pickle.dump(colors, open("colors.p", "wb"))

Unpickle dict

Pickles can be hacked so only read files you can trust

import pickle

# read back the file written by the method above
colors = pickle.load(open("colors.p","rb"))

Pickle Scikit-learn model

Just apply the method above after you have called fit() (or fit_transform() or partial_fit()) on your model:

import pickle
from sklearn.linear_model import LinearRegression

clf = LinearRegression()
clf.fit(<train_features>, <train_target>)

# using mode "wb" is important
pickle.dump(clf, open("/path/to/model.pkl", "wb"))

ModuleNotFoundError: No module named XYZ

This happens when you your pickled object uses imported functions and modules that are not available when you unpickle it.

To fix this, you must have the same imports available when unpickling as you did when pickling the object.

PicklingError: Can't pickle function lambda

lambda functions can't be pickled because they all have the same name.

Just define a function using def instead

AttributeError: Can't get attribute 'MyClass' on module 'main'

Code in the root scope of classes can't be unpickled by default.

While you get no error if you pickle it, you can't normally unpickle it¹

• in one file:

  import pickle
  
  # pickle class
  class MyClass:
      attr1 = "some information"
  
  pickle.dump(MyClass,open("/tmp/kls.pkl","wb"))
  

• in another file:

  import pickle
  
  unpickled_class = pickle.load(open("/tmp/kls.pkl","rb"))
  # >>> AttributeError                            Traceback (most recent call last)
  # <ipython-input-3-12b3f1c86c3a> in <module>
  #----> 1 unpickled_kls = pickle.load(open("/tmp/kls.pkl","rb"))
  
  #AttributeError: Can't get attribute 'MyClass' on <module '__main__'>
  

Heads-ups

Use wb file mode

Pickle is a binary data format so be sure to read and write files using the binary flags ("rb" and "wb", respectively).

1: There are workarounds: pickling class instances