SpamShield ML

Overview

SpamShield treats the model, predicted probability, operating threshold, and deployment artifact as separate engineering decisions.

Problem

A good classifier can still make the wrong operating decision. A false positive can hide a legitimate message, so threshold policy should reflect an explicit cost assumption rather than defaulting silently to 0.50.

Why it matters

Classification quality alone does not define a safe operating decision. Probability quality, threshold policy, artifact integrity, and deployment behavior all affect how the model should be used.

My role

I rebuilt and evaluated the UCI classification workflow, compared models and thresholds, defined the artifact contract, and designed the FastAPI, Docker, and Streamlit delivery boundaries.

Product

The rebuild packages a reproducible classifier behind testable API and Streamlit delivery surfaces while preserving the historical public demo as a separate reference.

• FastAPI health and prediction endpoints with request-size limits, validation, stable errors, and safe logging.
• Streamlit on the same artifact-backed runtime, with guards against intentionally logging or persisting submitted messages.
• Non-root Docker runtime with pinned base-image digest and pinned release dependencies.

Data

The rebuild uses an approved UCI retrieval path with bounded downloads and archive validation before training begins.

• ZIP signature and archive-member validation.
• Decompressed-size limits before the dataset is accepted.
• Fixed stratified holdout for the reviewed rebuild evidence.

Method

TF-IDF preprocessing feeds Naive Bayes and Logistic Regression comparison. Threshold selection is then evaluated separately under the recorded false-positive cost policy.

• Historical model record: Logistic Regression, F1 0.8550, threshold 0.40.
• Rebuild selected threshold: 0.30.
• Selected simulated decision cost: 28 units versus 44 units at threshold 0.50 on 1,115 validation messages.

Results

Rebuild numbers are reported as validation evidence, not production performance.

• Validation F1: 0.9291.
• 95% bootstrap percentile interval for F1: 0.8953 to 0.9580 using 2,000 stratified resamples.
• Average precision: 0.9788.
• Brier score: 0.0347.
• Simulated cost reduction: 36.36% versus threshold 0.50 under the same policy. This is not measured financial savings.

Validation

The evaluation is reproducible, but the same fixed holdout participates in model comparison and threshold selection, so selection optimism can remain.

• Bootstrap uncertainty describes sampling variability for the fixed holdout and does not remove selection bias.
• A separate untouched test set or nested validation is the next stronger design.
• Versioned model, vectorizer, metadata, threshold, training manifest, and SHA-256 integrity checks protect artifact compatibility.

Architecture

Training, artifacts, API delivery, container runtime, and Streamlit presentation share explicit contracts rather than relying on notebook state.

• Guarded UCI retrieval and dataset validation.
• Reproducible preprocessing and model training.
• Versioned integrity-checked artifact bundle.
• FastAPI and Streamlit surfaces consuming the same artifact contract.
• Non-root Docker runtime.

1. 01Guarded UCI retrievalBounded download and archive checks
2. 02Dataset validationSchema and decompressed-size controls
3. 03Training and evaluationTF-IDF, model comparison, threshold analysis
4. 04Artifact bundleVersioned files with SHA-256 integrity checks
5. 05Delivery surfacesFastAPI, Docker, and Streamlit

Tradeoffs

The cost policy makes the operating decision interpretable, but it also means the preferred threshold depends on a stated business assumption rather than a universally optimal cutoff.

Limitations

The rebuild is not described as production performance. The historical public demo is a reference until the rebuilt artifact deployment and live privacy behavior are verified.

Live Product

A historical Streamlit demo URL is recorded, but direct availability could not be independently reverified in this environment. It is withheld from recruiter-facing links and is not evidence that the 2026 rebuilt artifacts are deployed.

Source

The rebuilt source and deployment path are not linked in this portfolio prototype until remote source access and the deployed artifact identity are reverified together.

Next milestone

Produce the intended artifact bundle, deploy the rebuilt FastAPI path, verify live health and prediction behavior, reconnect Streamlit to the same artifact contract, and verify that submitted SMS text is absent from live logs and persistent storage.