michaelfeil/ct2fast-paraphrase-multilingual-MiniLM-L12-v2

Uso (Sentence-Transformers)

Usando este modelo se vuelve fácil cuando tienes sentence-transformers instalado:

pip install -U sentence-transformers

Luego puedes usar el modelo como este:

from sentence_transformers import SentenceTransformer
sentences = ["This is an example sentence", "Each sentence is converted"]
model = SentenceTransformer('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2')
embeddings = model.encode(sentences)
print(embeddings)

Uso (HuggingFace Transformers)

Sin sentence-transformers, puedes usar el modelo así: Primero, pasas tu entrada a través del modelo transformer, luego debes aplicar la operación de pooling correcta sobre los embeddings de las palabras contextualizadas.

from transformers import AutoTokenizer, AutoModel
import torch

def mean_pooling(model_output, attention_mask):
    token_embeddings = model_output[0]
    input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
    return torch.sum(token_embeddings * input_mask_expanded, 1) / torch.clamp(input_mask_expanded.sum(1), min=1e-9)

sentences = ['This is an example sentence', 'Each sentence is converted']
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2')
model = AutoModel.from_pretrained('sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2')
encoded_input = tokenizer(sentences, padding=True, truncation=True, return_tensors='pt')

with torch.no_grad():
    model_output = model(**encoded_input)

sentence_embeddings = mean_pooling(model_output, encoded_input['attention_mask'])

print("Sentence embeddings:")
print(sentence_embeddings)

Inferencia rápida con Ctranslate2

Acelera la inferencia mientras reduce la memoria 2x-4x usando inferencia int8 en C++ en CPU o GPU.

pip install hf-hub-ctranslate2>=2.12.0 ctranslate2>=3.17.1

from hf_hub_ctranslate2 import EncoderCT2fromHfHub
model = EncoderCT2fromHfHub(
model_name_or_path=model_name,
device="cuda",
compute_type="int8_float16")
outputs = model.generate(
text=["I like soccer", "I like tennis", "The Eiffel Tower is in Paris"],
max_length=64,
)
outputs["pooler_output"]
outputs["last_hidden_state"]
outputs["attention_mask"]

Alternativa, usa SentenceTransformer Mix-In para generar embeddings de oraciones de extremo a extremo (no desde este repositorio CT2fast-HF)

from hf_hub_ctranslate2 import CT2SentenceTransformer
model = CT2SentenceTransformer(
modelo_name_orig, compute_type="int8_float16", device="cuda")
embeddings = model.encode(
["I like soccer", "I like tennis", "The Eiffel Tower is in Paris"],
batch_size=32,
convert_to_numpy=True,
normalize_embeddings=True,
)
print(embeddings.shape, embeddings)
scores = (embeddings @ embeddings.T) * 100