• Matplotlib version 1.5.1-1ubuntu1, Python version2.7.12, Platform Ubuntu 16.04 x64
• I installed Matplotlib and Python from apt repository
• Example:

#!/usr/bin/env python
import matplotlib.pyplot as plt
plt.annotate('blah',  xy      = (1e6, 1),  xycoords  ='data')  ## 1e6 or more causes the crash.
#plt.xlim((0, 2e12))                                       ## does not help
plt.savefig("output.png", bbox_inches='tight')

Returns:

filip@comp:/tmp/$ python a.py 
Segmentation fault (core dumped)

A workaround is to establish the plot limits before the use the plt.annotate(), and call the command conditionally only if it fits into the view. But this surely is not the way Matplotlib should work (and localizing this source of crashes also took quite a long time, since the corresponding code was embedded in a 2000-line simulation script that worked until today).