metadata
language:
- pt
- multilingual
license: mit
tags:
- recommenders
- text-retrieval
- product-recommendation
- sentence-transformers
datasets:
- synthetic-ecommerce
metrics:
- auc
- ndcg
- mrr
foundational-model
A semantic product recommendation model that matches user profiles (free text) to products. Uses a frozen multilingual MiniLM encoder with trainable projection heads and chunk attention for user encoding.
Model description
- Architecture: Dual-encoder (user encoder + item encoder)
- Base model: paraphrase-multilingual-MiniLM-L12-v2 (frozen)
- Trainable params: ~148k (projection head + chunk attention)
- Input: User profile text + product name + description
- Output: Cosine similarity scores for ranking
Intended use
Product recommendation from user free-text profiles (e.g. "Marcos, gosto de videogames e de música, sou de Rio de janeiro"). Trained on synthetic e-commerce interactions in Portuguese.
How to use
from transformers import AutoTokenizer
import torch
from huggingface_hub import hf_hub_download
# Download checkpoint
checkpoint = hf_hub_download(repo_id="oristides/foundational-model", filename="pytorch_model.bin")
# Load model (requires model_arch1.RecSysModel - see repo for architecture)
from model.model_arch1 import RecSysModel
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2")
model = RecSysModel()
model.load_state_dict(torch.load(checkpoint, map_location="cpu"))
model.eval()
# Encode user and items, then: scores = user_emb @ item_embs.T
Or use the recommender CLI in this repo: uv run projects/reneguirecsys/model/recommender.py "your profile" -k 10
Training
- Loss: In-batch multi-negative cross-entropy
- Split: Leave-one-out per user
- Eval metrics: AUC, NDCG@10, MRR
- Max sequence length: 256 (user chunks), 128 (items)
Citation
@misc{oristides-foundational-model-2025,
author = {oristides},
title = {Foundational Model for Product Recommendation},
year = {2025},
publisher = {Hugging Face},
url = {https://huggingface.co/oristides/foundational-model}
}
License
MIT