Rare race condition in makedirs with parallel processes #5225
Description
The cbook.makedirs code does not catch all the problems with multi-threading. We are seeing a very rare failure in the LSST test suite when multiple tests are using matplotlib in parallel. I think it is because the try/catch in makedirs is assuming that only the last part of the path could have the race and only does a single catch around the entire for loop. If both processes are racing to create path A/B/C and both notice that B is missing then the second one to attempt to create B will abort when it finds that B now exists. It should just continue on with trying to make C. I think the try/catch should be in the inner part of the loop around the os.makedirs call.
This is the error we get:
import matplotlib.colorbar
File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/colorbar.py", line 36, in <module>
import matplotlib.contour as contour
File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/contour.py", line 27, in <module>
import matplotlib.texmanager as texmanager
File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/texmanager.py", line 89, in <module>
class TexManager:
File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/texmanager.py", line 124, in TexManager
mkdirs(texcache)
File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/site-packages/matplotlib/cbook.py", line 1003, in mkdirs
os.makedirs(thispart, mode)
File "/home/vagrant/stack/Linux64/anaconda/2.2.0/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 17] File exists: '/tmp/matplotlib-vagrant'with v1.4.3.
Originally filed as an addendum of #2046. @tacaswell requested I file this separately.
See the PR in #5224