RetailFlow — Analytics, Predictions and Dashboards

An end-to-end walk through the Gold layer of the RetailFlow pipeline: the executive KPIs, one section per dashboard visual, the signal checks that justify a predictive layer, the models themselves scored against explicit baselines, and the three front-ends that serve it.

Live dashboard	https://chidex-coder.github.io/Retail-Data-Engineering-Pipeline
Source	https://github.com/chidex-coder/Retail-Data-Engineering-Pipeline
Data	data/gold/analytics_facts.parquet — the cleaned, joined fact rows the dashboards read

The pipeline is Bronze → Silver → Gold on PySpark; this notebook picks up where Gold ends. Every figure uses the same colour tokens and chart geometry as the dashboards, read from the same files, so nothing here is a re-interpretation of the data.

import json, os, sys
import numpy as np
import pandas as pd
import plotly.graph_objects as go
import plotly.io as pio
from IPython.display import display, Markdown, HTML, IFrame

# Locate the repo root whether this runs from notebooks/ or the root.
ROOT = os.getcwd()
while not os.path.exists(os.path.join(ROOT, "data", "gold")) and os.path.dirname(ROOT) != ROOT:
    ROOT = os.path.dirname(ROOT)
sys.path.insert(0, ROOT)

# Static PNG output: renders on GitHub, in nbviewer and in the HTML export.
pio.renderers.default = "png"
pio.defaults.default_width = 900
pio.defaults.default_height = 380
pio.defaults.default_scale = 2

# Colours come straight from the dashboard stylesheet; geometry from the shared theme.
from app.theme import PALETTES, CHART
P = PALETTES["light"]
BAR = CHART["bar"]

pd.set_option("display.float_format", lambda v: f"{v:,.2f}")
print("repo root located:", os.path.basename(ROOT))

repo root located: retail-data-engineering

df = pd.read_parquet(os.path.join(ROOT, "data", "gold", "analytics_facts.parquet"))
for col in df.select_dtypes("category").columns:
    df[col] = df[col].astype("object")
df["Year"] = df["Year"].astype("Int64")
df["Month"] = df["Month"].astype("Int64")
df["OrderDate"] = pd.to_datetime(dict(year=df.Year.fillna(1970).astype(int),
                                      month=df.Month.fillna(1).astype(int), day=1))
df.loc[df.Year.isna(), "OrderDate"] = pd.NaT

print(f"{len(df):,} order lines · {df.CustomerKey.nunique():,} customers · "
      f"{df.ProductKey.nunique():,} products · {df.OrderDate.min():%b %Y} → {df.OrderDate.max():%b %Y}")
df.head()

143,723 order lines · 4,100 customers · 1,190 products · Aug 2023 → Aug 2026

	Year	Month	Category	Brand	PaymentMethod	Gender	Status	City	CustomerName	CustomerKey	ProductKey	Revenue	Profit	Quantity	OrderDate
0	2026	2	Sports	AmazonBasics	NaN	Male	Delivered	London	Keith Mccall	3808	473	386.60	120.60	2	2026-02-01
1	2025	9	Home & Garden	AmazonBasics	NaN	Female	Cancelled	London	Robert Anderson	4108	566	437.28	141.78	2	2025-09-01
2	2025	4	Toys	IKEA	Apple Pay	Male	Returned	Nottingham	Jennifer Williams	4678	517	216.76	80.88	4	2025-04-01
3	2026	4	Sports	Adidas	PayPal	Female	Delivered	Birmingham	Briana Kent	4086	782	268.60	81.82	2	2026-04-01
4	2025	2	Sports	Adidas	PayPal	Male	Delivered	London	Shannon Patel	1447	1018	342.68	131.16	2	2025-02-01

Chart helpers

Two builders mirror the dashboard's horizontal bar and time-series charts: values are written on the marks, gridlines stay recessive, and each chart shows one series so no legend is needed.

def money(v):
    a = abs(v)
    if a >= 1e6: return f"£{v/1e6:.1f}M"
    if a >= 1e3: return f"£{v/1e3:.1f}K"
    return f"£{v:,.0f}"

def base_layout(**extra):
    layout = dict(
        paper_bgcolor="white", plot_bgcolor="white",
        font=dict(family=CHART["font"]["family"], size=CHART["font"]["size"], color=P["--text-secondary"]),
        margin=dict(l=8, r=16, t=40, b=28), showlegend=False,
    )
    layout.update(extra)
    return layout

def hbar(series, colour, title, fmt=money, top=None):
    """Largest at the top, value labelled on each bar, x-axis hidden."""
    s = series.sort_values(ascending=False)
    if top: s = s.head(top)
    s = s[::-1]
    fig = go.Figure(go.Bar(
        x=s.values, y=[str(i) for i in s.index], orientation="h",
        marker=dict(color=colour, cornerradius=BAR["cornerRadius"]),
        text=[fmt(v) for v in s.values], textposition="outside", cliponaxis=False,
        textfont=dict(color=P["--text-secondary"], size=CHART["font"]["labelSize"]),
    ))
    fig.update_layout(base_layout(
        title=dict(text=title, font=dict(size=15, color=P["--text-primary"]), x=0),
        bargap=BAR["gap"], margin=dict(l=8, r=70, t=44, b=10),
        xaxis=dict(visible=False, range=[0, s.max() * BAR["headroom"]]),
        yaxis=dict(automargin=True, showgrid=False, tickfont=dict(color=P["--text-secondary"])),
        height=max(260, 34 * len(s) + 90),
    ))
    return fig

def timeseries(series, colour, title, prefix="£", kind="line"):
    x = series.index
    if kind == "line":
        trace = go.Scatter(x=x, y=series.values, mode="lines+markers",
                           line=dict(color=colour, width=CHART["line"]["width"]),
                           marker=dict(size=CHART["line"]["markerSize"], color=colour),
                           fill="tozeroy", fillcolor=colour.replace(")", ",0.12)").replace("rgb(", "rgba(")
                           if colour.startswith("rgb") else None)
    else:
        trace = go.Bar(x=x, y=series.values, marker=dict(color=colour, cornerradius=4))
    fig = go.Figure(trace)
    fig.update_layout(base_layout(
        title=dict(text=title, font=dict(size=15, color=P["--text-primary"]), x=0),
        xaxis=dict(type="date", tickformat="%b %y", showgrid=False, linecolor=P["--axis"]),
        yaxis=dict(showgrid=True, gridcolor=P["--grid"], tickprefix=prefix, tickformat="~s",
                   rangemode="tozero"),
    ))
    return fig

Executive KPIs

The eight numbers on the dashboard's KPI cards. Orders counts order lines, which is the grain of fact_sales; products sold is distinct SKUs that appear in at least one order line.

revenue, profit = df.Revenue.sum(), df.Profit.sum()
orders = len(df)
kpis = pd.DataFrame({
    "KPI": ["Total revenue", "Gross profit", "Orders", "Customers", "Products sold",
            "Avg order value", "Avg profit margin", "Cities served"],
    "Value": [money(revenue), money(profit), f"{orders:,}", f"{df.CustomerKey.nunique():,}",
              f"{df.ProductKey.nunique():,}", f"£{revenue/orders:,.2f}",
              f"{profit/revenue*100:.1f}%", f"{df.City.nunique():,}"],
})
kpis.set_index("KPI")

	Value
KPI
Total revenue	£73.2M
Gross profit	£21.0M
Orders	143,723
Customers	4,100
Products sold	1,190
Avg order value	£509.45
Avg profit margin	28.6%
Cities served	40

Q1 · How has monthly revenue trended?

The first and last months are partial by construction (the window opens and closes mid-month), so they are dropped rather than left to read as a cliff.

monthly = df.dropna(subset=["OrderDate"]).groupby("OrderDate").agg(
    revenue=("Revenue", "sum"), profit=("Profit", "sum"), orders=("Revenue", "size")).iloc[1:-1]
display(monthly.tail(6).style.format({"revenue": "£{:,.0f}", "profit": "£{:,.0f}", "orders": "{:,}"}))
timeseries(monthly.revenue, P["--revenue"], "Monthly revenue")

	revenue	profit	orders
OrderDate
2026-02-01 00:00:00	£1,860,889	£531,198	3,536
2026-03-01 00:00:00	£2,516,858	£720,150	4,935
2026-04-01 00:00:00	£2,836,737	£809,617	5,510
2026-05-01 00:00:00	£3,035,186	£869,730	5,873
2026-06-01 00:00:00	£2,828,410	£809,772	5,249
2026-07-01 00:00:00	£2,690,915	£763,749	5,012

Q2 · Which brands generate the most revenue?

Brands arrive dirty (<b>Sony</b>, IKEA📱💻, N/A); the Gold layer has already standardised them. Rows whose brand never resolved are excluded from this breakdown but still count in the totals above.

by_brand = df.dropna(subset=["Brand"]).groupby("Brand").agg(
    revenue=("Revenue", "sum"), profit=("Profit", "sum"), orders=("Revenue", "size"))
by_brand["margin_%"] = by_brand.profit / by_brand.revenue * 100
display(by_brand.sort_values("revenue", ascending=False)
        .style.format({"revenue": "£{:,.0f}", "profit": "£{:,.0f}", "orders": "{:,}", "margin_%": "{:.1f}"}))
hbar(by_brand.revenue, P["--revenue"], "Revenue by brand")

	revenue	profit	orders	margin_%
Brand
AmazonBasics	£25,494,211	£7,103,405	49,260	27.9
Samsung	£14,731,238	£3,300,857	17,473	22.4
Nike	£9,184,786	£3,758,838	26,795	40.9
Sony	£8,036,455	£1,526,781	8,097	19.0
IKEA	£6,894,618	£2,063,996	20,995	29.9
Adidas	£5,957,057	£2,327,895	14,916	39.1

Q3 · How does revenue split across categories?

Category also decides margin: Books are thin, Clothing is fat. Both columns are shown because the revenue ranking and the profit ranking are not the same.

by_cat = df.dropna(subset=["Category"]).groupby("Category").agg(
    revenue=("Revenue", "sum"), profit=("Profit", "sum"), orders=("Revenue", "size"))
by_cat["margin_%"] = by_cat.profit / by_cat.revenue * 100
display(by_cat.sort_values("revenue", ascending=False)
        .style.format({"revenue": "£{:,.0f}", "profit": "£{:,.0f}", "orders": "{:,}", "margin_%": "{:.1f}"}))
hbar(by_cat.revenue, P["--revenue"], "Revenue by category")

	revenue	profit	orders	margin_%
Category
Electronics	£29,029,322	£5,410,272	27,542	18.6
Home & Garden	£16,852,065	£5,050,687	26,992	30.0
Sports	£10,823,704	£3,603,469	23,128	33.3
Clothing	£10,167,075	£4,840,098	28,962	47.6
Toys	£4,782,385	£1,811,559	19,343	37.9
Books	£1,564,656	£258,463	17,756	16.5

Q4 · Which cities drive revenue?

Customers are placed in real UK cities with population-weighted volume, so this is a genuine ranking rather than a list of thousands of one-order towns.

by_city = df.dropna(subset=["City"]).groupby("City").Revenue.sum()
share = (by_city.sort_values(ascending=False).head(10) / by_city.sum() * 100).round(1)
display(pd.DataFrame({"revenue": by_city.loc[share.index], "share_%": share})
        .style.format({"revenue": "£{:,.0f}", "share_%": "{:.1f}"}))
hbar(by_city, P["--revenue"], "Top 10 cities by revenue", top=10)

	revenue	share_%
City
London	£52,910,242	72.3
Birmingham	£6,538,608	8.9
Manchester	£1,623,822	2.2
Glasgow	£1,234,804	1.7
Liverpool	£1,091,887	1.5
Bristol	£990,969	1.4
Leeds	£980,006	1.3
Sheffield	£835,054	1.1
Edinburgh	£730,926	1.0
Nottingham	£720,383	1.0

Q5 · How does revenue split by customer gender?

by_gender = df.dropna(subset=["Gender"]).groupby("Gender").agg(
    revenue=("Revenue", "sum"), customers=("CustomerKey", "nunique"), orders=("Revenue", "size"))
by_gender["revenue_per_customer"] = by_gender.revenue / by_gender.customers
display(by_gender.style.format({"revenue": "£{:,.0f}", "customers": "{:,}", "orders": "{:,}",
                                "revenue_per_customer": "£{:,.0f}"}))
hbar(by_gender.revenue, P["--revenue"], "Revenue by gender")

	revenue	customers	orders	revenue_per_customer
Gender
Female	£35,058,192	1,857	67,282	£18,879
Male	£31,742,620	1,811	62,541	£17,528
Other	£2,770,772	167	5,672	£16,591

Q6 · Which payment methods carry the most revenue?

Payment method is missing on a share of rows — a data-quality artefact the generator injects on purpose. Those rows count toward total revenue and drop out of this chart only.

by_pay = df.dropna(subset=["PaymentMethod"]).groupby("PaymentMethod").agg(
    revenue=("Revenue", "sum"), orders=("Revenue", "size"))
by_pay["avg_order_value"] = by_pay.revenue / by_pay.orders
display(by_pay.sort_values("revenue", ascending=False)
        .style.format({"revenue": "£{:,.0f}", "orders": "{:,}", "avg_order_value": "£{:,.2f}"}))
hbar(by_pay.revenue, P["--revenue"], "Revenue by payment method")

	revenue	orders	avg_order_value
PaymentMethod
Credit Card	£27,394,335	52,151	£525.29
Bank Transfer	£19,307,008	33,703	£572.86
PayPal	£14,353,065	30,670	£467.98
Apple Pay	£6,260,551	15,644	£400.19

Q7 · How do orders end up?

A three-year window is mostly terminal states: almost everything has been delivered, returned or cancelled. Pending and Shipped are the last few days of orders still in flight. The colours are the reserved status palette — Delivered green, Cancelled red — and every bar is labelled so colour never carries meaning on its own.

status_colours = {"Delivered": P["--status-good"], "Shipped": P["--status-neutral"],
                  "Pending": P["--status-warning"], "Returned": P["--status-serious"],
                  "Cancelled": P["--status-critical"]}
by_status = df.Status.value_counts()
share = (by_status / by_status.sum() * 100).round(1)
display(pd.DataFrame({"orders": by_status, "share_%": share}).style.format({"orders": "{:,}", "share_%": "{:.1f}"}))
s = by_status[::-1]
fig = go.Figure(go.Bar(x=s.values, y=s.index, orientation="h",
                       marker=dict(color=[status_colours[k] for k in s.index], cornerradius=4),
                       text=[f"{v:,}" for v in s.values], textposition="outside", cliponaxis=False))
fig.update_layout(base_layout(title=dict(text="Order status", font=dict(size=15, color=P["--text-primary"]), x=0),
                              bargap=0.38, margin=dict(l=8, r=70, t=44, b=10),
                              xaxis=dict(visible=False, range=[0, s.max() * 1.16]), yaxis=dict(automargin=True)))
fig

	orders	share_%
Status
Delivered	101,500	70.6
Returned	22,642	15.8
Cancelled	18,525	12.9
Shipped	708	0.5
Pending	348	0.2

Q8 · Who are the top customers?

Ranked by surrogate key, then labelled by name, so two customers who share a name stay distinct.

by_cust = (df.dropna(subset=["CustomerName"]).groupby(["CustomerKey", "CustomerName"])
             .agg(revenue=("Revenue", "sum"), orders=("Revenue", "size"))
             .sort_values("revenue", ascending=False).head(10).reset_index(level=0, drop=True))
display(by_cust.style.format({"revenue": "£{:,.0f}", "orders": "{:,}"}))
hbar(by_cust.revenue, P["--revenue"], "Top 10 customers by revenue")

	revenue	orders
CustomerName
Amanda Mcdonald	£212,146	262
Holly Mitchell	£147,714	199
Jeremy Gibson	£145,741	201
Ryan Mann	£145,456	198
Odonnell	£144,867	163
Stephen Garcia	£133,742	164
Rachel Morrow	£131,477	174
Scott Frazier	£130,743	163
Erin Powers	£130,268	149
Colleen Mason	£129,898	169

Q9 · How does order volume vary month to month?

Order count rather than revenue — it separates "more orders" from "bigger orders".

timeseries(monthly.orders, P["--orders"], "Monthly orders", prefix="", kind="bar")

Is there anything to predict? Three signal checks

A model is only worth training if the data carries structure. These three checks are the same ones tests/test_ml.py enforces — if the synthetic generator were ever flattened back to independent uniform draws, they would fail before any model did.

Q10 · Does return rate depend on category?

ret = df.dropna(subset=["Category"]).assign(returned=df.Status.eq("Returned")).groupby("Category").returned.mean() * 100
display(ret.sort_values(ascending=False).to_frame("return_rate_%").style.format("{:.1f}"))
hbar(ret, P["--status-serious"], "Return rate by category (%)", fmt=lambda v: f"{v:.1f}%")

	return_rate_%
Category
Clothing	25.7
Electronics	17.6
Sports	17.2
Home & Garden	13.4
Toys	9.8
Books	4.8

Q11 · Does cancellation rate depend on payment method?

can = (df.dropna(subset=["PaymentMethod"]).assign(cancelled=df.Status.eq("Cancelled"))
         .groupby("PaymentMethod").cancelled.mean() * 100)
display(can.sort_values(ascending=False).to_frame("cancel_rate_%").style.format("{:.1f}"))
hbar(can, P["--status-critical"], "Cancellation rate by payment method (%)", fmt=lambda v: f"{v:.1f}%")

	cancel_rate_%
PaymentMethod
Bank Transfer	19.5
Credit Card	11.8
PayPal	10.2
Apple Pay	7.5

Q12 · Is demand seasonal?

idx = df.dropna(subset=["Month"]).groupby("Month").Revenue.sum()
idx = (idx / idx.mean()).rename(index=lambda m: pd.Timestamp(2000, int(m), 1).strftime("%b"))
display(idx.to_frame("seasonal_index").T.style.format("{:.2f}"))
fig = go.Figure(go.Bar(x=idx.index, y=idx.values,
                       marker=dict(color=[P["--revenue"] if v >= 1 else P["--text-muted"] for v in idx.values], cornerradius=4),
                       text=[f"{v:.2f}" for v in idx.values], textposition="outside", cliponaxis=False))
fig.add_hline(y=1.0, line=dict(color=P["--axis"], dash="dot"))
fig.update_layout(base_layout(title=dict(text="Seasonal index (1.0 = average month)", font=dict(size=15, color=P["--text-primary"]), x=0),
                              xaxis=dict(showgrid=False), yaxis=dict(showgrid=True, gridcolor=P["--grid"], rangemode="tozero")))
fig

Month	Jan	Feb	Mar	Apr	May	Jun	Jul	Aug	Sep	Oct	Nov	Dec
seasonal_index	0.73	0.71	0.95	1.01	1.10	1.03	1.00	0.95	0.88	0.97	1.23	1.43

Predictive analytics

Three supervised models and a segmentation, trained by exactly the code in ml/train.py. Every model is scored against an explicit baseline and the baseline is shown next to the result — a model that only matches its baseline is a failure, not a pass.

What stops these being fits to noise:

• customer-history features use expanding windows with a shift(), so an order's own outcome never informs its own features;
• the fulfilment split is chronological, not random;
• the repeat-purchase label is measured strictly after the feature cutoff;
• the forecast is backtested rolling-origin, with regularisation tuned inside each fold.

from ml import features as F
from ml import train as T

orders_ml = F.load_orders(os.path.join(ROOT, "data", "gold", "analytics_facts.parquet"))
print(f"{len(orders_ml):,} dated order lines available to the models")

136,678 dated order lines available to the models

Model 1 · Fulfilment risk — will this order be returned or cancelled?

risk_model, risk_metrics, risk_by_cat, risk_by_pay = T.train_fulfilment_risk(orders_ml)
pd.DataFrame({
    "metric": ["ROC AUC", "PR AUC", "Brier score"],
    "model": [risk_metrics["roc_auc"], risk_metrics["pr_auc"], risk_metrics["brier"]],
    "baseline": [risk_metrics["baseline_roc_auc"], risk_metrics["baseline_pr_auc"], risk_metrics["baseline_brier"]],
    "note": ["chance", "base rate", "constant prediction"],
}).set_index("metric").style.format({"model": "{:.3f}", "baseline": "{:.3f}"})

	model	baseline	note
metric
ROC AUC	0.634	0.500	chance
PR AUC	0.391	0.289	base rate
Brier score	0.196	0.205	constant prediction

display(risk_by_cat.set_index("Category")
        .style.format({"predicted_risk": "{:.1%}", "actual_risk": "{:.1%}", "orders": "{:,}", "revenue_at_risk": "£{:,.0f}"}))
r = risk_by_cat.iloc[::-1]
fig = go.Figure([
    go.Bar(name="Predicted", x=r.predicted_risk * 100, y=r.Category, orientation="h",
           marker=dict(color=P["--cost"], cornerradius=4)),
    go.Bar(name="Actual", x=r.actual_risk * 100, y=r.Category, orientation="h",
           marker=dict(color=P["--status-neutral"], cornerradius=4)),
])
fig.update_layout(base_layout(barmode="group", showlegend=True, legend=dict(orientation="h", y=1.1, x=0),
                              title=dict(text="Predicted vs actual fulfilment risk by category (held-out orders)",
                                         font=dict(size=15, color=P["--text-primary"]), x=0),
                              xaxis=dict(ticksuffix="%", showgrid=True, gridcolor=P["--grid"]),
                              yaxis=dict(automargin=True)))
fig

	predicted_risk	actual_risk	orders	revenue_at_risk
Category
Clothing	36.1%	38.5%	5,488	£1,978,483
Electronics	34.1%	34.2%	5,418	£5,874,320
Sports	29.7%	30.2%	4,402	£2,117,413
Home & Garden	27.9%	27.7%	5,065	£3,212,485
Toys	21.2%	20.6%	3,709	£954,634
Books	14.0%	13.2%	3,254	£295,037

Model 2 · Repeat purchase — will this customer order again within 90 days?

The honest baseline is a recency-only rule: "recent buyers come back" is what any analyst would say without a model, so the model has to beat it.

repeat_model, repeat_metrics, customer_scores = T.train_repeat_purchase(orders_ml)
display(pd.DataFrame({
    "metric": ["ROC AUC", "PR AUC"],
    "model": [repeat_metrics["roc_auc"], repeat_metrics["pr_auc"]],
    "baseline": [repeat_metrics["recency_only_roc_auc"], repeat_metrics["baseline_pr_auc"]],
    "note": ["recency-only rule", "positive rate"],
}).set_index("metric").style.format({"model": "{:.3f}", "baseline": "{:.3f}"}))

print(f"cutoff {repeat_metrics['cutoff']} · {repeat_metrics['positive_rate']:.1%} of customers returned")
display(Markdown("**Highest churn risk** — customers the model expects not to return:"))
customer_scores.sort_values("churn_risk", ascending=False).head(10).style.format(
    {"churn_risk": "{:.1%}", "recency_days": "{:.0f}", "frequency": "{:.0f}", "monetary": "£{:,.0f}"})

	model	baseline	note
metric
ROC AUC	0.965	0.947	recency-only rule
PR AUC	0.953	0.473	positive rate

cutoff 2026-05-03 · 46.5% of customers returned

Highest churn risk — customers the model expects not to return:

	CustomerKey	recency_days	tenure_days	frequency	monetary	avg_value	avg_quantity	risk_rate	orders_per_month	will_return	churn_risk
844	1034	640	1006	122	£94,326	773.161066	3.885246	0.295082	3.638171	0	100.0%
3473	4234	640	1006	122	£94,311	773.041721	3.885246	0.286885	3.638171	0	100.0%
2270	2780	884	1006	6	£987	164.545000	2.333333	0.166667	0.178926	0	100.0%
1042	1285	487	579	121	£108,619	897.677769	3.537190	0.297521	6.269430	0	100.0%
1810	2226	428	975	133	£107,118	805.400451	3.766917	0.308271	4.092308	0	100.0%
558	696	671	1006	117	£82,657	706.470513	3.529915	0.316239	3.489066	0	100.0%
74	93	518	1006	115	£87,438	760.327304	3.469565	0.269565	3.429423	0	100.0%
795	979	609	671	116	£99,507	857.821293	3.853448	0.267241	5.186289	0	100.0%
1214	1490	914	1006	137	£111,849	816.419489	3.883212	0.299270	4.085487	0	100.0%
1564	1905	609	1006	139	£111,857	804.725899	3.755396	0.352518	4.145129	0	100.0%

fig = go.Figure(go.Histogram(x=customer_scores.churn_risk, nbinsx=40,
                             marker=dict(color=P["--orders"], line=dict(width=1, color="white"))))
fig.update_layout(base_layout(title=dict(text="Distribution of churn risk across customers", font=dict(size=15, color=P["--text-primary"]), x=0),
                              xaxis=dict(tickformat=".0%", showgrid=False), yaxis=dict(showgrid=True, gridcolor=P["--grid"]),
                              bargap=0.05))
fig

Model 3 · Revenue forecast — the next three months

Ridge on a linear trend plus month dummies, backtested rolling-origin. The interval width comes from realised backtest error, not from the fit. Two baselines: last month carried forward, and the same month last year.

forecast_model, fc_metrics, fc = T.train_forecast(orders_ml)
display(pd.DataFrame({
    "MAPE": [fc_metrics["mape"], fc_metrics["naive_mape"], fc_metrics["seasonal_naive_mape"]],
}, index=["model", "naive", "seasonal naive"]).style.format("{:.1%}"))

future = fc[fc.forecast.notna()][["date", "forecast", "lower", "upper"]].set_index("date")
display(Markdown(f"**Forecast** — next {len(future)} months, total **{money(future.forecast.sum())}**:"))
future.style.format("£{:,.0f}")

	MAPE
model	12.9%
naive	20.9%
seasonal naive	35.1%

Forecast — next 3 months, total £8.3M:

	forecast	lower	upper
date
2026-08-01 00:00:00	£2,452,847	£1,831,959	£3,073,735
2026-09-01 00:00:00	£2,845,406	£2,125,150	£3,565,663
2026-10-01 00:00:00	£3,031,921	£2,264,452	£3,799,391

hist, fut = fc[fc.actual.notna()], fc[fc.forecast.notna()]
bridge = hist.tail(1)
band = pd.concat([bridge, fut])
fig = go.Figure([
    go.Scatter(x=list(band.date) + list(band.date[::-1]),
               y=list(band.upper.fillna(band.actual)) + list(band.lower.fillna(band.actual)[::-1]),
               fill="toself", fillcolor="rgba(41,98,255,0.14)", line=dict(width=0), hoverinfo="skip", showlegend=False),
    go.Scatter(name="Actual", x=hist.date, y=hist.actual, mode="lines+markers",
               line=dict(color=P["--revenue"], width=2), marker=dict(size=6)),
    go.Scatter(name="Forecast", x=pd.concat([bridge.date, fut.date]), y=pd.concat([bridge.actual, fut.forecast]),
               mode="lines+markers", line=dict(color=P["--profit"], width=2, dash="dot"), marker=dict(size=7)),
])
fig.update_layout(base_layout(showlegend=True, legend=dict(orientation="h", y=1.1, x=0),
                              title=dict(text="Monthly revenue with 3-month forecast and backtested interval",
                                         font=dict(size=15, color=P["--text-primary"]), x=0),
                              xaxis=dict(type="date", tickformat="%b %y", showgrid=False),
                              yaxis=dict(showgrid=True, gridcolor=P["--grid"], tickprefix="£", tickformat="~s", rangemode="tozero")))
fig

Model 4 · Customer segmentation — K-means on recency, frequency, monetary

Unsupervised, so there is no baseline to beat; the check is whether the clusters are distinct and ranked sensibly by value.

seg_model, seg_metrics, profile, seg_rows = T.train_segments(orders_ml)
display(profile.set_index("segment")[["customers", "recency_days", "frequency", "monetary"]]
        .style.format({"customers": "{:,}", "recency_days": "{:.0f}", "frequency": "{:.1f}", "monetary": "£{:,.0f}"}))
colours = [P["--revenue"], P["--profit"], P["--quantity"], P["--orders"]]
fig = go.Figure(go.Bar(x=profile.segment, y=profile.monetary,
                       marker=dict(color=colours[:len(profile)], cornerradius=4),
                       text=[f"{c:,} cust" for c in profile.customers], textposition="outside", cliponaxis=False))
fig.update_layout(base_layout(title=dict(text="Average spend per segment", font=dict(size=15, color=P["--text-primary"]), x=0),
                              xaxis=dict(showgrid=False), yaxis=dict(showgrid=True, gridcolor=P["--grid"], tickprefix="£", tickformat="~s"),
                              bargap=0.45))
fig

	customers	recency_days	frequency	monetary
segment
Champions	656	0	82.4	£55,325
Loyal	585	345	65.0	£37,146
Occasional	477	0	17.5	£4,686
At risk	2,382	416	15.2	£3,927

Scoreboard

Every model beside the number it had to beat. This is the same table the dashboards render from data/ml/metrics.json.

scoreboard = pd.DataFrame([
    ["Fulfilment risk", f"AUC {risk_metrics['roc_auc']:.3f}", "chance 0.500", risk_metrics["roc_auc"] > 0.55],
    ["Repeat purchase", f"AUC {repeat_metrics['roc_auc']:.3f}",
     f"recency rule {repeat_metrics['recency_only_roc_auc']:.3f}", repeat_metrics["roc_auc"] > repeat_metrics["recency_only_roc_auc"]],
    ["Revenue forecast", f"MAPE {fc_metrics['mape']:.1%}",
     f"naive {fc_metrics['naive_mape']:.1%} · seasonal {fc_metrics['seasonal_naive_mape']:.1%}",
     fc_metrics["mape"] < min(fc_metrics["naive_mape"], fc_metrics["seasonal_naive_mape"])],
    ["Segmentation", f"{seg_metrics['k']} segments", "unsupervised", True],
], columns=["model", "result", "baseline", "beats baseline"]).set_index("model")
scoreboard

	result	baseline	beats baseline
model
Fulfilment risk	AUC 0.634	chance 0.500	True
Repeat purchase	AUC 0.965	recency rule 0.947	True
Revenue forecast	MAPE 12.9%	naive 20.9% · seasonal 35.1%	True
Segmentation	4 segments	unsupervised	True

The dashboards

Everything above is served three ways from the same Gold data, the same stylesheet and the same chart geometry.

Front-end	Where	How it computes
Static report	https://chidex-coder.github.io/Retail-Data-Engineering-Pipeline	the fact rows are embedded; every filter re-aggregates in the browser (~70 ms)
Dash app	python run_dash.py · Fly.io	pandas on the server per callback (~13 ms)
Streamlit app	streamlit run streamlit_app.py	pandas, cached per filter set

All three carry a Notebook tab that opens this notebook. The predictive section on each is rendered from the artefacts ml/train.py writes to data/ml/.

display(HTML('<img src="https://github.com/chidex-coder/Retail-Data-Engineering-Pipeline/raw/main/dashboards/preview-light.png" '
             'alt="RetailFlow executive dashboard" style="max-width:100%;border:1px solid #ddd;border-radius:10px">'))

IFrame("https://chidex-coder.github.io/Retail-Data-Engineering-Pipeline/", width="100%", height=900)

---

Regenerate this notebook with python notebooks/build_notebook.py after re-running the pipeline; it executes end to end and exports docs/notebook.html for GitHub Pages.