Edit: After test on different machines it is Apple M1 M2 specific bug.
I am trying to run a Flask application inside a Docker container on Apple Silicon M2 (could be an issue), where I use the SentenceTransformer model to encode sentences. However, when I call the encode method on the model, the application crashes with a segmentation fault.
Here's the relevant code:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')
#Our sentences we like to encode
sentences = ['This framework generates embeddings for each input sentence',
'Sentences are passed as a list of string.',
'The quick brown fox jumps over the lazy dog.']
#Sentences are encoded by calling model.encode()
sentence_embeddings = model.encode(sentences)
Source: https://www.sbert.net/docs/quickstart.html
after using import faulthandler
and faulthandler.enable()
The error traceback is:
Fatal Python error: Segmentation fault
Thread 0x0000ffff640ff1a0 (most recent call first):
File "/usr/local/lib/python3.11/threading.py", line 331 in wait
File "/usr/local/lib/python3.11/threading.py", line 629 in wait
File "/usr/local/lib/python3.11/site-packages/tqdm/_monitor.py", line 60 in run
File "/usr/local/lib/python3.11/threading.py", line 1045 in _bootstrap_inner
File "/usr/local/lib/python3.11/threading.py", line 1002 in _bootstrap
Current thread 0x0000ffffa6814020 (most recent call first):
File "/usr/local/lib/python3.11/site-packages/transformers/activations.py", line 78 in forward
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527 in _call_impl
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518 in _wrapped_call_impl
File "/usr/local/lib/python3.11/site-packages/transformers/models/bert/modeling_bert.py", line 452 in forward
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527 in _call_impl
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518 in _wrapped_call_impl
File "/usr/local/lib/python3.11/site-packages/transformers/models/bert/modeling_bert.py", line 551 in feed_forward_chunk
File "/usr/local/lib/python3.11/site-packages/transformers/pytorch_utils.py", line 240 in apply_chunking_to_forward
File "/usr/local/lib/python3.11/site-packages/transformers/models/bert/modeling_bert.py", line 539 in forward
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527 in _call_impl
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518 in _wrapped_call_impl
File "/usr/local/lib/python3.11/site-packages/transformers/models/bert/modeling_bert.py", line 612 in forward
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527 in _call_impl
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518 in _wrapped_call_impl
File "/usr/local/lib/python3.11/site-packages/transformers/models/bert/modeling_bert.py", line 1022 in forward
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527 in _call_impl
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518 in _wrapped_call_impl
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/models/Transformer.py", line 66 in forward
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1527 in _call_impl
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1518 in _wrapped_call_impl
File "/usr/local/lib/python3.11/site-packages/torch/nn/modules/container.py", line 215 in forward
File "/usr/local/lib/python3.11/site-packages/sentence_transformers/SentenceTransformer.py", line 165 in encode
File "<stdin>", line 1 in <module>
Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, torch._C, torch._C._fft, torch._C._linalg, torch._C._nested, torch._C._nn, torch._C._sparse, torch._C._special, scipy._lib._ccallback_c, scipy.sparse._sparsetools, _csparsetools, scipy.sparse._csparsetools, scipy.sparse.linalg._isolve._iterative, scipy.linalg._fblas, scipy.linalg._flapack, scipy.linalg.cython_lapack, scipy.linalg._cythonized_array_utils, scipy.linalg._solve_toeplitz, scipy.linalg._decomp_lu_cython, scipy.linalg._matfuncs_sqrtm_triu, scipy.linalg.cython_blas, scipy.linalg._matfuncs_expm, scipy.linalg._decomp_update, scipy.linalg._flinalg, scipy.sparse.linalg._dsolve._superlu, scipy.sparse.linalg._eigen.arpack._arpack, scipy.sparse.csgraph._tools, scipy.sparse.csgraph._shortest_path, scipy.sparse.csgraph._traversal, scipy.sparse.csgraph._min_spanning_tree, scipy.sparse.csgraph._flow, scipy.sparse.csgraph._matching, scipy.sparse.csgraph._reordering, scipy.spatial._ckdtree, scipy._lib.messagestream, scipy.spatial._qhull, scipy.spatial._voronoi, scipy.spatial._distance_wrap, scipy.spatial._hausdorff, scipy.special._ufuncs_cxx, scipy.special._ufuncs, scipy.special._specfun, scipy.special._comb, scipy.special._ellip_harm_2, scipy.spatial.transform._rotation, scipy.ndimage._nd_image, _ni_label, scipy.ndimage._ni_label, scipy.optimize._minpack2, scipy.optimize._group_columns, scipy.optimize._trlib._trlib, scipy.optimize._lbfgsb, _moduleTNC, scipy.optimize._moduleTNC, scipy.optimize._cobyla, scipy.optimize._slsqp, scipy.optimize._minpack, scipy.optimize._lsq.givens_elimination, scipy.optimize._zeros, scipy.optimize.__nnls, scipy.optimize._highs.cython.src._highs_wrapper, scipy.optimize._highs._highs_wrapper, scipy.optimize._highs.cython.src._highs_constants, scipy.optimize._highs._highs_constants, scipy.linalg._interpolative, scipy.optimize._bglu_dense, scipy.optimize._lsap, scipy.optimize._direct, scipy.integrate._odepack, scipy.integrate._quadpack, scipy.integrate._vode, scipy.integrate._dop, scipy.integrate._lsoda, scipy.special.cython_special, scipy.stats._stats, scipy.stats.beta_ufunc, scipy.stats._boost.beta_ufunc, scipy.stats.binom_ufunc, scipy.stats._boost.binom_ufunc, scipy.stats.nbinom_ufunc, scipy.stats._boost.nbinom_ufunc, scipy.stats.hypergeom_ufunc, scipy.stats._boost.hypergeom_ufunc, scipy.stats.ncf_ufunc, scipy.stats._boost.ncf_ufunc, scipy.stats.ncx2_ufunc, scipy.stats._boost.ncx2_ufunc, scipy.stats.nct_ufunc, scipy.stats._boost.nct_ufunc, scipy.stats.skewnorm_ufunc, scipy.stats._boost.skewnorm_ufunc, scipy.stats.invgauss_ufunc, scipy.stats._boost.invgauss_ufunc, scipy.interpolate._fitpack, scipy.interpolate.dfitpack, scipy.interpolate._bspl, scipy.interpolate._ppoly, scipy.interpolate.interpnd, scipy.interpolate._rbfinterp_pythran, scipy.interpolate._rgi_cython, scipy.stats._biasedurn, scipy.stats._levy_stable.levyst, scipy.stats._stats_pythran, scipy._lib._uarray._uarray, scipy.stats._statlib, scipy.stats._sobol, scipy.stats._qmc_cy, scipy.stats._mvn, scipy.stats._rcont.rcont, regex._regex, sklearn.__check_build._check_build, sklearn.utils._isfinite, sklearn.utils.murmurhash, sklearn.utils._openmp_helpers, sklearn.utils._logistic_sigmoid, sklearn.utils.sparsefuncs_fast, sklearn.preprocessing._csr_polynomial_expansion, sklearn.preprocessing._target_encoder_fast, sklearn.utils._vector_sentinel, sklearn.feature_extraction._hashing_fast, sklearn.utils._random, sklearn.utils._seq_dataset, sklearn.metrics.cluster._expected_mutual_info_fast, sklearn.metrics._dist_metrics, sklearn.metrics._pairwise_distances_reduction._datasets_pair, sklearn.utils._cython_blas, sklearn.metrics._pairwise_distances_reduction._base, sklearn.metrics._pairwise_distances_reduction._middle_term_computer, sklearn.utils._heap, sklearn.utils._sorting, sklearn.metrics._pairwise_distances_reduction._argkmin, sklearn.metrics._pairwise_distances_reduction._argkmin_classmode, sklearn.metrics._pairwise_distances_reduction._radius_neighbors, sklearn.metrics._pairwise_fast, sklearn.linear_model._cd_fast, sklearn._loss._loss, sklearn.utils.arrayfuncs, sklearn.svm._liblinear, sklearn.svm._libsvm, sklearn.svm._libsvm_sparse, sklearn.utils._weight_vector, sklearn.linear_model._sgd_fast, sklearn.linear_model._sag_fast, scipy.io.matlab._mio_utils, scipy.io.matlab._streams, scipy.io.matlab._mio5_utils, sklearn.datasets._svmlight_format_fast, charset_normalizer.md, yaml._yaml, sentencepiece._sentencepiece, PIL._imaging (total: 163)
Segmentation fault
Some points:
The Docker container has ample memory allocated. I've tried updating the libraries (torch, transformers, and sentence-transformers). The same code works perfectly outside the Docker environment. How can I resolve this segmentation fault when running the code inside Docker?
Here is a pip list of actual versions.
Package Version
--------------------- ---------
blinker 1.6.3
certifi 2023.7.22
charset-normalizer 3.3.0
click 8.1.7
filelock 3.12.4
Flask 3.0.0
fsspec 2023.9.2
huggingface-hub 0.17.3
idna 3.4
itsdangerous 2.1.2
Jinja2 3.1.2
joblib 1.3.2
MarkupSafe 2.1.3
mpmath 1.3.0
networkx 3.1
nltk 3.8.1
numpy 1.26.0
packaging 23.2
Pillow 10.0.1
pip 23.2.1
PyYAML 6.0.1
regex 2023.10.3
requests 2.31.0
safetensors 0.4.0
scikit-learn 1.3.1
scipy 1.11.3
sentence-transformers 2.2.2
sentencepiece 0.1.99
setuptools 65.5.1
sympy 1.12
threadpoolctl 3.2.0
tokenizers 0.14.1
torch 2.1.0
torchvision 0.16.0
tqdm 4.66.1
transformers 4.34.0
typing_extensions 4.8.0
urllib3 2.0.6
Werkzeug 3.0.0
wheel 0.41.2
For more clarification, here is the most important part of my Dockerfile.
FROM python:3.11
RUN pip install --upgrade pip
RUN pip install Flask==3.0.0 sentence-transformers==2.2.2
Couldn't find out why newer python versions are failing, but
python:3.6.15
tag works just fine on mac arm64 apple silicon M1.Edit (better alternative based in @jackson-bierfeldt comment): Downgrading PyTorch to 2.0.1 solved it. So, put something like
torch==2.0.*
to requirements.txt, no need to downgrade Python itself.