Recommendation Systems

Recommendation systems suggest relevant items to users — powering Netflix, Spotify, and Amazon. They fall into two families: content-based filtering (recommend items similar to what you liked) and collaborative filtering (recommend what users like you enjoyed).

Fundamentals and Types

A recommender system (RS) is a set of software tools and techniques that suggests useful items to a user. The suggestions support decisions such as what to buy, what music to listen to or what news to read. Some techniques are knowledge poor: they need only very simple data, such as user ratings for items.

Fundamental Terms

• Items. The objects being recommended. Users always pay a cost: the effort of searching and, often, real money. Items range from low complexity and value (news, web pages, books, films), through mid-range (cameras, phones, PCs), to highly complex ones (insurance, investments, travel, jobs).
• Users. The people searching for items, with very diverse goals and characteristics.
• Transactions. Recorded interactions between a user and the system.
• Explicit feedback. Likes and dislikes, ratings, text comments. Simple, but it adds to the user's cognitive load.
• Implicit feedback. No active effort from the user: clicks, time spent on an item, saving an item.
• Ratings. Numerical (1 to 5 stars), ordinal ("strongly agree" to "strongly disagree") or binary (good or bad).

Types of Recommender System

1. Content-based: recommends items similar to ones the user liked before, using item features.
2. Collaborative filtering: recommends what users with similar tastes liked, using rating histories.
3. Knowledge-based: uses specific domain knowledge. Either case-based (a similarity function matches the user's needs to solutions) or constraint-based (explicit rules link requirements to item features).
4. Demographic: assumes different demographic niches need different recommendations.
5. Community-based: uses the preferences of the user's friends or community.
6. Hybrid: combinations of the above.

Why Do Providers and Users Need One?

Open Issues

Scalability on real-world datasets, proactive recommendations (offered without being asked), privacy, diversity of the recommended items, combining long-term and short-term preferences, cross-domain and distributed recommenders, optimising a sequence of recommendations, and mobile contexts. Related topics include exploration versus exploitation, guided navigation, interpreting user actions and evaluating recommenders.

Content-Based Filtering

Using the MovieLens dataset (9,742 movies), each movie is represented as a binary genre vector. Movies with similar genre vectors are considered similar.

How a Content-Based Recommender Works

The system has three components:

• Content analyser. Turns unstructured item content (text, web pages, descriptions) into a structured form, for example keyword vectors, using feature extraction. This representation feeds the next two components.
• Profile learner. Collects data on the user's preferences and generalises it, usually with machine learning, into a user profile. For a web-page recommender it might combine the vectors of liked and disliked pages into one prototype vector that represents the user.
• Filtering component. Matches the user profile against item representations, for instance with cosine similarity between the prototype and each item vector. The result is a binary or continuous relevance score, which gives a ranked list.

1. Build a training set for the active user: pairs \(\langle I_k, r_k \rangle\) of an item representation and the rating the user gave it.
2. The profile learner applies a supervised learning algorithm to produce a predictive model, the user profile, kept in a profile repository.
3. The filtering component predicts how likely each new item is to interest the user, and the top-ranked items form the recommendation list.
4. Feedback on those recommendations is saved, the model is retrained on the new data, and the profile adapts. Repeating this cycle lets the system follow changing tastes.

Common methods for learning user profiles are Naive Bayes, K-Nearest Neighbours, decision trees and K-Means.

One-Hot Encode Genres

import pandas as pd

movies = pd.read_csv('movies.csv')  # columns: movieId, title, genres

# Collect all unique genres
all_genres = set()
for g in movies['genres']:
    all_genres.update(g.split('|'))

# Create a binary column for each genre
for genre in all_genres:
    movies[genre] = movies['genres'].apply(
        lambda x: 1 if genre in x.split('|') else 0
    )

print(movies[['title', 'Action', 'Comedy', 'Drama']].head())

Recommend Similar Movies

from sklearn.metrics.pairwise import cosine_similarity

genre_cols = list(all_genres)
sim_matrix = cosine_similarity(movies[genre_cols])

def get_recommendations(title, n=5):
    idx = movies[movies['title'] == title].index[0]
    scores = sorted(enumerate(sim_matrix[idx]), key=lambda x: x[1], reverse=True)[1:n+1]
    return movies.iloc[[i for i, _ in scores]]['title'].tolist()

print(get_recommendations('Toy Story (1995)'))

Cosine similarity measures the angle between two vectors. A value of 1 = identical, 0 = no overlap, -1 = opposite.

import numpy as np

def cosine_sim(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Genre vectors: [Action, Animation, Children, Comedy, Fantasy]
toy_story = [0, 1, 1, 1, 1]
shrek     = [0, 1, 1, 1, 1]
die_hard  = [1, 0, 0, 0, 0]

print(cosine_sim(toy_story, shrek))    # 1.0 — identical genres
print(cosine_sim(toy_story, die_hard)) # 0.0 — no overlap

Collaborative Filtering

Collaborative filtering uses a user–item rating matrix. We use KNN on 100,836 ratings from 610 users with cosine distance to find similar movies.

The core assumption is simple: if person X and person Y reacted the same way to some items, they will probably share opinions on other items too. Unlike content-based filtering, CF ignores item features. It groups users with similar behaviour and recommends according to each group's preferences.

Build the Rating Matrix

from scipy.sparse import csr_matrix

ratings = pd.read_csv('ratings.csv')  # userId, movieId, rating

def create_matrix(df):
    user_mapper  = {u: i for i, u in enumerate(df['userId'].unique())}
    movie_mapper = {m: i for i, m in enumerate(df['movieId'].unique())}
    movie_inv    = {v: k for k, v in movie_mapper.items()}

    user_idx  = [user_mapper[u]  for u in df['userId']]
    movie_idx = [movie_mapper[m] for m in df['movieId']]

    X = csr_matrix((df['rating'], (movie_idx, user_idx)))
    return X, movie_mapper, movie_inv

X, movie_mapper, movie_inv = create_matrix(ratings)

KNN Movie Recommendations

from sklearn.neighbors import NearestNeighbors

knn = NearestNeighbors(n_neighbors=6, algorithm='brute', metric='cosine')
knn.fit(X)

def similar_movies(movie_id, k=5):
    idx = movie_mapper[movie_id]
    _, indices = knn.kneighbors(X[idx], n_neighbors=k+1)
    return [movie_inv[i] for i in indices.flatten()[1:]]

print(similar_movies(1))  # Movies similar to Toy Story

Attacks on Collaborative Systems

Collaborative recommenders learn from user-supplied ratings, so malicious users can bias their output to make particular items recommended or not. Each fake identity an attacker creates is an attack profile. The best attack gives the biggest impact for the least effort.

• Product push: promote an item's recommendations. Product nuke: demote them.
• Low-knowledge attacks need only public, system-independent information. High-knowledge attacks need the rating distribution, such as each item's mean and standard deviation. An informed attack exploits knowledge of the exact algorithm in use.

Countermeasures

• Use model-based or hybrid algorithms: comparable accuracy and less vulnerable.
• Raise the cost of injecting profiles, for example with captchas.
• Detect attacks statistically: look for groups of users pushing or nuking the same items, and watch changes in an item's average rating, rating entropy and bulk-rating timing. ML can also flag fake profiles.

Knowledge-Based Recommenders

Some items, such as apartments, are bought rarely, so there are few ratings for any one item, and years-old preferences would not satisfy users anyway. Knowledge-based recommenders handle this by using explicit user requirements and deep knowledge of the product domain.

They have no cold-start problem, because requirements are elicited during the session. Their weakness is the knowledge acquisition bottleneck: engineers must convert domain experts' knowledge into formal, executable rules.

Limitations

• Knowledge acquisition is costly, both from experts (rules and constraints) and from users (requirements).
• Very fine-grained preference models need many interaction cycles.
• Preferences are not always independent of each other, which challenges the usual assumption.

Constraint-Based Recommenders

These use predefined knowledge bases with explicit rules that relate customer requirements to item features. The problem is modelled as a constraint satisfaction problem, a tuple \((V, D, C)\):

• \(V\) is a set of variables: customer requirements \(V_C\) and item features \(V_{PROD}\), so \(V = V_C \cup V_{PROD}\).
• \(D\) is the finite domain of each variable.
• \(C\) is a set of constraints on which value combinations are allowed: filter constraints \(C_F\) ("if the user requires A, the item must have feature B"), compatibility constraints \(C_R\) between requirements, and product constraints \(C_{PROD}\) that define the available assortment.

The customer's query (REQ) is encoded as a constraint over these variables, and the task is to find the products (RES) that match it. Strategies include:

• If no item matches, ask the user which requirements to relax or change so that some item exists.
• Find the items that satisfy the largest weighted set of constraints.
• Rank items by the ratio of constraints they fulfil.

A conjunctive query combines the filter conditions \(C_F\) with the user requirements.

Defaults

Users are often unsure which options to pick or lack the technical detail, so the system proposes defaults:

• Static: one default per user property.
• Dependent: depends on the combination of requirements given.
• Derived: learned from earlier interactions, by 1-nearest-neighbour (copy the most similar earlier entry) or a weighted majority vote among similar items.

When Requirements Cannot Be Met

Rank items with Multi-Attribute Utility Theory (MAUT). Each item is scored on a fixed set of dimensions; for digital cameras, say, quality and economy. An item priced at or below 250 might score quality 5 and economy 10. The user's interest in each dimension weights the item's contribution:

Case-Based Recommenders

These retrieve items using similarity measures that describe how well item properties match user requirements. Early versions were query-based: the customer states requirements and the system returns similar items. The drawback is that customers often do not know what they want, which led to browsing-based navigation. The state of the art, critique-based recommenders, combines both, so users can change preferences without stating exact attribute values.

Similarity of a product \(p\) to the requirements is a weighted average over requirements \(r\), where \(w_r\) is the importance of \(r\):

The local similarity \(sim(p,r)\) depends on the attribute, where \(\phi_r(p)\) is the item's value:

• More is better (camera resolution): \(sim(p,r)=\frac{\phi_r(p)-\min(r)}{\max(r)-\min(r)}\)
• Less is better (price): \(sim(p,r)=\frac{\max(r)-\phi_r(p)}{\max(r)-\min(r)}\)
• Closest to the requested value (run time, monitor size): \(sim(p,r)=1-\frac{|\phi_r(p)-r|}{\max(r)-\min(r)}\)

Critiquing

In critiquing, customers say what is wrong with the current item ("cheaper", "more pixels"). The goal is to save time while matching the recommendation quality of query-based approaches. Critiquing several properties at once (compound critiques) shortens the dialogue and moves faster through the item space. A candidate item \(c_i\) is scored on its similarity to REQ and its compatibility with the selected compound critique \(CC_U\):

Hybrid Recommender Systems

Each base technique answers a different question. Collaborative: "tell me what's popular among my peers". Content-based: "show me more of what I've liked". Knowledge-based: "tell me what fits my needs". A good sales assistant uses all three at different stages, but each has shortcomings such as cold start. A hybrid (from the Latin hybrida, an object made by combining two different things) crosses two or more techniques to avoid some of those shortcomings and gain properties the parents lack.

There are three design families: monolithic, parallel and pipelined.

Monolithic: One Component, Several Feature Sources

A single recommendation component integrates multiple algorithms or knowledge sources as input.

• Feature combination. Mix collaborative features (movies the user liked) with content features (genres) to build hybrid features such as "user likes many mystery books". In a small example, pure collaborative filtering finds that User1 and User2 are similar to Alice. With feature combination, User2 behaves quite differently and only User1 stays similar to Alice, because both like many mystery books.
• Feature augmentation. One recommender produces features that enrich the knowledge source of the main one. In content-boosted collaborative filtering, a content-based classifier trained on the user's likes and dislikes rates unrated items, those ratings fill the user's profile, and collaborative filtering then runs again on the denser profile.

Parallel: Combine the Outputs of Several Systems

• Weighted. Combine \(n\) recommenders \(rec_k\) with weights \(\beta_k\) that sum to 1: $$rec_{weighted}(u,i)=\sum_{k=1}^{n} \beta_k \cdot rec_k(u,i), \quad \sum_{k=1}^{n}\beta_k=1$$ Weights can be found by empirical bootstrapping (try weightings on historic data and keep the best) or adjusted dynamically: start uniform, then adapt per user to minimise the prediction error (MAE). The recommenders must produce comparable scores, which may need a transformation, and stable weights need many user ratings.
• Switching. A mediator picks one recommender depending on the situation. This is dynamic weighting where all weights but one are 0. For a new user, use knowledge-based first, then collaborative; for a new item, use content-based first, then collaborative.
• Mixed. Show the results of all recommenders side by side in the interface.

Pipelined: Recommenders Build on Each Other

• Cascade. Each stage refines the list from the one before. Later stages can only reorder or exclude items, never add new ones, so results are precise but the list keeps shrinking and may end up empty. Combine it with switching to guarantee enough recommendations. The first recommender only produces a list; the next scores items or drops them.
• Meta-level. The first recommender builds a model (for example a user model) that the main recommender then uses instead of raw data to make its predictions.

Select the genres you enjoy and click Find Movies. The recommender builds your preference vector and returns the top matches using cosine similarity — exactly like a real content-based system.