Recommendation Systems

Recommendation systems predict user preferences to suggest relevant items. They power product suggestions on Amazon, movie recommendations on Netflix, and content feeds on social media.

Types of Recommendations

Content-Based Filtering

Recommend items similar to what user liked:

User liked: Action movies with Tom Cruise
Recommend: More action movies, more Tom Cruise movies

Based on item features.

Collaborative Filtering

Recommend what similar users liked:

Users like you also watched: ...

Based on user-item interactions.

Hybrid Systems

Combine both approaches for better results.

Collaborative Filtering

User-User CF

Find similar users, recommend their items:

User A likes: Movies 1, 2, 3
User B likes: Movies 1, 2, 4
Similarity(A, B) = high

Recommend Movie 4 to User A

Item-Item CF

Find similar items based on who likes them:

Item 1 bought by: Users A, B, C
Item 2 bought by: Users A, B, D
Similarity(1, 2) = high

If user buys Item 1, recommend Item 2

More stable than user-user (items change less than users).

Matrix Factorization

The Idea

Decompose user-item matrix into latent factors:

R ≈ U × V^T

R: User-Item rating matrix [m × n]
U: User embeddings [m × k]
V: Item embeddings [n × k]

Prediction

r̂_ui = uᵢ · vⱼ = Σ uᵢₖ × vⱼₖ

Dot product of user and item embeddings.

Training

min Σ (rᵤᵢ - uᵤ·vᵢ)² + λ(||U||² + ||V||²)

Optimize with SGD or ALS (Alternating Least Squares).

Adding Biases

r̂_ui = μ + bᵤ + bᵢ + uᵤ·vᵢ

μ: Global mean
bᵤ: User bias (some rate higher)
bᵢ: Item bias (some items rated higher)

Deep Learning Approaches

Neural Collaborative Filtering (NCF)

Replace dot product with neural network:

class NCF(nn.Module):
    def __init__(self, num_users, num_items, embed_dim):
        self.user_embed = nn.Embedding(num_users, embed_dim)
        self.item_embed = nn.Embedding(num_items, embed_dim)
        self.mlp = nn.Sequential(
            nn.Linear(embed_dim * 2, 64),
            nn.ReLU(),
            nn.Linear(64, 32),
            nn.ReLU(),
            nn.Linear(32, 1)
        )
    
    def forward(self, user_id, item_id):
        u = self.user_embed(user_id)
        v = self.item_embed(item_id)
        x = torch.cat([u, v], dim=-1)
        return self.mlp(x).squeeze()

Two-Tower Models

Separate encoders for user and item:

User features → [User Tower] → User embedding
                                      ↓
                              Dot product → Score
                                      ↑
Item features → [Item Tower] → Item embedding

Enables efficient retrieval (pre-compute item embeddings).

Sequence Models

Capture sequential patterns:

User watched: A → B → C → ?
Predict next item using RNN/Transformer

Content-Based Methods

TF-IDF for Text

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

tfidf = TfidfVectorizer()
item_vectors = tfidf.fit_transform(item_descriptions)

def recommend(item_id):
    similarities = cosine_similarity(item_vectors[item_id], item_vectors)
    return similarities.argsort()[-10:]  # Top 10 similar

Embedding-Based

item_embeddings = sentence_transformer.encode(descriptions)
similar_items = find_nearest_neighbors(item_embeddings[item_id])

Cold Start Problem

New Users

No interaction history.

Solutions:

• Ask for preferences (onboarding)
• Use demographics
• Content-based initially
• Popularity-based fallback

New Items

No one has interacted yet.

Solutions:

• Content-based features
• Exploration strategies
• Boosting for new items

Evaluation Metrics

Rating Prediction

RMSE = √(Σ(r - r̂)² / n)
MAE = Σ|r - r̂| / n

Ranking Metrics

Precision@k = relevant in top k / k
Recall@k = relevant in top k / total relevant
NDCG@k = normalized ranking quality
MRR = 1 / rank of first relevant item

Business Metrics

• Click-through rate (CTR)
• Conversion rate
• Watch time
• User engagement

Real-World Considerations

Scale

• Millions of users and items
• Approximate nearest neighbors (FAISS, Annoy)
• Two-stage: retrieval (fast) → ranking (accurate)

Implicit Feedback

Most systems use implicit signals:

• Clicks, purchases, watch time
• No explicit ratings
• Need different loss functions

Diversity and Exploration

• Don't just recommend popular items
• Explore to learn preferences
• Avoid filter bubbles

Fairness

• Item fairness: new/small items get exposure
• User fairness: equal quality across groups

Production Architecture

[User Request]
      ↓
[Candidate Generation] → 1000 items (fast, approximate)
      ↓
[Ranking Model] → Score candidates (more accurate)
      ↓
[Re-ranking] → Diversity, business rules
      ↓
[Final Recommendations] → 10 items

Key Takeaways

1. Collaborative filtering: find similar users or items
2. Matrix factorization: learn latent embeddings
3. Deep learning enables complex patterns
4. Cold start is a fundamental challenge
5. Evaluate with ranking metrics, not just RMSE
6. Production systems use candidate generation + ranking