[NLP] Sentence-transformer를 활용한 문장 임베딩

1. sentence-transformer 설치

$ pip install sentence-transformers

2. 모델 선택

- https://huggingface.co/sentence-transformers 에서 모델 선택

- 나는 이 모델 선택: https://huggingface.co/sentence-transformers/paraphrase-MiniLM-L6-v2

3. 모델 불러오기

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence-transformers/paraphrase-MiniLM-L6-v2")

4. 임베딩

my_sentence = 'I love you, darling'
response = model.encode(my_sentence)

5. 임베딩 결과 확인

response

Output exceeds the size limit. Open the full output data in a text editorarray([ 3.73888873e-02,  2.84339845e-01,  5.92863858e-01, -4.04151589e-01,
       -3.62452984e-01, -1.35070413e-01,  9.10828412e-01,  2.13023871e-01,
        8.73978212e-02,  3.87814462e-01,  1.87386442e-02, -8.04892361e-01,
       -1.97961539e-01,  2.91577708e-02,  2.97898620e-01,  1.17376382e-02,
        5.57748437e-01, -3.15046966e-01, -8.65592420e-01, -6.24333799e-01,
        8.91279355e-02, -9.49146822e-02, -3.39516938e-01, -9.46855024e-02,
       -4.84677464e-01, -2.08640546e-01,  1.27627596e-01,  3.56212914e-01,
        1.70316055e-01, -1.92422360e-01,  4.13054734e-01, -8.34294915e-01,
        5.01438737e-01,  4.00905520e-01,  2.16388777e-01,  7.73810208e-01,
       -3.08173984e-01, -3.19728523e-01, -1.36708766e-01, -3.75668436e-01,
        2.97464281e-01, -5.75280726e-01, -3.47505987e-01,  2.21415445e-01,
        3.97491962e-01, -2.88253367e-01,  1.52879342e-01, -2.01121002e-01,
        5.10286272e-01,  5.13169289e-01, -7.98814237e-01, -3.57702896e-02,
       -3.14021148e-02,  3.62671137e-01,  4.97897804e-01,  7.65251040e-01,
        3.25530440e-01,  6.36099726e-02,  2.32315615e-01, -2.28358701e-01,
        5.67676425e-01, -3.68146934e-02, -5.27653813e-01,  5.21592438e-01,
       -3.41809809e-01, -5.05130768e-01, -4.97012073e-03,  3.47984023e-02,
...

shape 확인

response.shape

(384,)

 

끝! 이제 프로젝트에 적용해보자!