Plain-English Explanation

Instead of predicting a number (regression), classification predicts a label: spam or not spam, disease or healthy, fraud or legitimate. The model learns a decision boundary that separates classes. Logistic regression is the simplest classifier — despite the name, it outputs a probability of belonging to a class.

When to Use

Key Metrics

Code Example

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, roc_auc_score
from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
X_train_sc = scaler.fit_transform(X_train)
X_test_sc  = scaler.transform(X_test)

clf = LogisticRegression(max_iter=1000)
clf.fit(X_train_sc, y_train)

y_pred  = clf.predict(X_test_sc)
y_proba = clf.predict_proba(X_test_sc)[:, 1]

print(classification_report(y_test, y_pred))
print(f"ROC-AUC: {roc_auc_score(y_test, y_proba):.3f}")

Common Mistakes

Project Idea

Interactive Notebook

Quiz

Test your understanding of Classification with 10 questions. Pass 70% to mark this topic complete.

Plain-English Explanation

A decision tree splits data by asking yes/no questions about features. At each node, it picks the split that best separates the classes (using Gini or entropy). Random Forest builds many trees on random subsets of data and features, then averages their outputs — reducing variance through ensemble averaging.

Key Metrics & Hyperparameters

Code Example

from sklearn.ensemble import RandomForestClassifier
import pandas as pd

rf = RandomForestClassifier(n_estimators=200, max_depth=8, random_state=42)
rf.fit(X_train, y_train)

# Feature importance
fi = pd.Series(rf.feature_importances_, index=feature_names)
fi.sort_values(ascending=False).plot(kind='bar')

Common Mistakes

Interactive Notebook

Quiz

Test your understanding of Tree Based Learning with 10 questions. Pass 70% to mark this topic complete.

Plain-English Explanation

Boosting trains trees sequentially. Each new tree focuses on the data points the previous ensemble got wrong. Gradient Boosting does this by fitting each tree to the residual errors. XGBoost and LightGBM add regularization, histogram-based splits, and speed optimizations on top of this idea.

Algorithm Comparison

Code Example

import xgboost as xgb
from sklearn.model_selection import cross_val_score

model = xgb.XGBClassifier(
    n_estimators=300,
    learning_rate=0.05,
    max_depth=6,
    subsample=0.8,
    colsample_bytree=0.8,
    use_label_encoder=False,
    eval_metric='logloss'
)

scores = cross_val_score(model, X, y, cv=5, scoring='roc_auc')
print(f"CV AUC: {scores.mean():.3f} ± {scores.std():.3f}")

Interactive Notebook

Quiz

Test your understanding of Boosting with 10 questions. Pass 70% to mark this topic complete.

Plain-English Explanation

SVM tries to find the widest possible "street" (margin) between two classes. The data points closest to the margin are called support vectors. The kernel trick lets SVMs work in higher-dimensional spaces without explicitly computing those dimensions — enabling non-linear decision boundaries.

Kernel Options

Code Example

from sklearn.svm import SVC
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler

pipe = Pipeline([
    ('scaler', StandardScaler()),
    ('svm', SVC(kernel='rbf', C=1.0, gamma='scale', probability=True))
])
pipe.fit(X_train, y_train)

Interactive Notebook

Quiz

Test your understanding of Svm with 10 questions. Pass 70% to mark this topic complete.

Algorithm Comparison

Code Example

from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score

# Elbow method to choose K
inertias = []
for k in range(2, 11):
    km = KMeans(n_clusters=k, random_state=42)
    km.fit(X)
    inertias.append(km.inertia_)

# Silhouette score for validation
labels = KMeans(n_clusters=4).fit_predict(X)
score  = silhouette_score(X, labels)
print(f"Silhouette: {score:.3f}")  # higher is better

Interactive Notebook

Quiz

Test your understanding of Clustering with 10 questions. Pass 70% to mark this topic complete.

Key Approaches

Code Example

from sklearn.ensemble import IsolationForest

iso = IsolationForest(contamination=0.05, random_state=42)
labels = iso.fit_predict(X)
# -1 = anomaly, 1 = normal
anomalies = X[labels == -1]

Interactive Notebook

Quiz

Test your understanding of Anomaly Detection with 10 questions. Pass 70% to mark this topic complete.

Plain-English Explanation

Naive Bayes applies Bayes' theorem with the "naive" assumption that all features are independent given the class. Despite this oversimplification, it works very well for text (bag-of-words). LDA (Linear Discriminant Analysis) finds the linear combination of features that best separates classes — it also doubles as a dimensionality reduction tool.

Variants

Interactive Notebook

Quiz

Test your understanding of Naive Bayes Lda with 10 questions. Pass 70% to mark this topic complete.

Key Concepts

Algorithm Family

Project Idea

Interactive Notebook

Quiz

Test your understanding of Time Series with 10 questions. Pass 70% to mark this topic complete.

Search Strategies

Code Example — Optuna

import optuna

def objective(trial):
    params = {
        'n_estimators': trial.suggest_int('n_estimators', 100, 500),
        'max_depth':    trial.suggest_int('max_depth', 3, 12),
        'learning_rate': trial.suggest_float('lr', 0.01, 0.3, log=True),
    }
    model = xgb.XGBClassifier(**params)
    score = cross_val_score(model, X, y, cv=3, scoring='roc_auc').mean()
    return score

study = optuna.create_study(direction='maximize')
study.optimize(objective, n_trials=50)
print(study.best_params)

Interactive Notebook

Quiz

Test your understanding of Hyperparameter Optimization with 10 questions. Pass 70% to mark this topic complete.