Log Loss (Binary Cross-Entropy) — ML Coding

Solving tips

Log loss rewards confident-and-correct and punishes confident-and-wrong; the log term blows up as a probability heads toward the wrong extreme.
Clip predictions into [eps, 1-eps] before taking any log — a raw 0.0 or 1.0 makes log(0) = -inf and the whole score becomes nan or inf.
Write it as one vectorized expression: y*log(p) + (1-y)*log(1-p), then negate the mean.

Implement binary log loss, also called binary cross-entropy, from scratch with NumPy. It is the standard loss for binary classifiers that output probabilities (logistic regression, neural nets with a sigmoid head), and interviewers use it to check that you can both write the formula and handle the numerical edge case that breaks it.

Definition

For n samples with true labels y in {0, 1} and predicted probabilities p in [0, 1]:

log_loss = -(1/n) * sum_i [ y_i * log(p_i) + (1 - y_i) * log(1 - p_i) ]

A confident correct prediction (p near 1 when y = 1) contributes near-zero loss; a confident wrong prediction (p near 0 when y = 1) contributes a large loss because log(p) diverges to -inf.

Task

Complete log_loss(y_true, y_pred, eps=1e-15) so it returns the mean log loss as a float. Before taking any logarithm, clip y_pred into the range [eps, 1 - eps] so that a predicted probability of exactly 0.0 or 1.0 does not produce log(0). Do not call sklearn or any built-in log-loss / cross-entropy helper.

Example

y_true = np.array([1, 0, 1, 1])
y_pred = np.array([0.9, 0.1, 0.8, 0.7])
log_loss(y_true, y_pred)   # -> 0.19763488164214868

The per-sample losses are [-log(0.9), -log(0.9), -log(0.8), -log(0.7)], whose mean is about 0.1976.

Constraints

1 <= n <= 10^6; use vectorized NumPy, not a Python loop.
Each y_true[i] is 0 or 1; each y_pred[i] is in [0, 1].
Values fit in float64; 0 < eps < 0.5.

Approach

Clip the predictions into [eps, 1 - eps] first so no logarithm ever sees 0 or 1, then evaluate the cross-entropy formula elementwise. The two terms y*log(p) and (1-y)*log(1-p) are complementary — for each sample exactly one of y or 1-y is nonzero — so their sum picks out the log of the probability assigned to the true class. Average over all samples and negate.

Solution

import numpy as np

def log_loss(y_true, y_pred, eps=1e-15):
    y_true = np.asarray(y_true, dtype=np.float64)
    p = np.clip(np.asarray(y_pred, dtype=np.float64), eps, 1 - eps)
    terms = y_true * np.log(p) + (1 - y_true) * np.log(1 - p)
    return float(-np.mean(terms))

Walkthrough

On the example y_true = [1, 0, 1, 1], y_pred = [0.9, 0.1, 0.8, 0.7]:

Clipping leaves every value unchanged since all are well inside [eps, 1 - eps].
For sample 0 (y = 1): 1 * log(0.9) + 0 * log(0.1) = log(0.9) ≈ -0.10536.
For sample 1 (y = 0): 0 * log(0.1) + 1 * log(0.9) = log(0.9) ≈ -0.10536.
For sample 2 (y = 1): log(0.8) ≈ -0.22314.
For sample 3 (y = 1): log(0.7) ≈ -0.35667.
terms = [-0.10536, -0.10536, -0.22314, -0.35667], mean ≈ -0.19763, negate to get 0.19763.

Complexity & notes

Time O(n), space O(n) — a clip, two logs, and a reduction, each one linear pass done in C by NumPy.
The clip is the whole point of the eps: a model that outputs a hard 1.0 while the label is 0 would otherwise give log(0) = -inf and make the mean inf or nan. Clipping caps the worst per-sample loss at -log(eps) ≈ 34.5 for eps = 1e-15.
This is the binary special case of categorical cross-entropy; the general form sums -sum_k y_k * log(p_k) over classes, which here collapses to the two-term expression.
Casting y_true to float lets the y * log(p) products vectorize cleanly even if labels arrive as ints or bools.