Fix missing values that disappear from grouped reports

The overall sales total is higher than the sum of the channel breakdown. Some orders have no channel label, and the grouped report has quietly omitted that group.

Why this happens

Missing dimensions and missing measures need different treatment. An unknown marketing channel is not the same thing as zero revenue. pandas groupby excludes missing group keys by default unless you choose otherwise.

Step-by-step solution

1. Count missing values in each dimension and measure separately. Compare grouped totals with the source total before drawing conclusions.

2. Include the missing-key group with dropna=False, or assign a clearly documented Unknown label before grouping.

3. Decide how to handle missing monetary values using business rules. Keep an exception count rather than filling every empty cell with zero.

Worked example

import pandas as pd
df = pd.DataFrame({'channel':['email',None], 'sales':[100,50]})
summary = df.groupby('channel', dropna=False)['sales'].sum()
assert summary.sum() == 150
assert df['channel'].isna().sum() == 1

Verify the fix

The grouped result must reconcile to 150, including the unknown-channel sale. Run the same reconciliation for refunds and cancelled orders according to your chosen reporting definition.

Prevent a repeat

Publish completeness indicators with important dashboards. A clean-looking chart is not evidence that its source dimensions were complete.

Reference

Official documentation or standards guidance

An original Gistmax troubleshooting guide. Scenarios and sample values are illustrative; configuration depends on your software version and environment. Cover diagram: Gistmax Tutorials.