Start with the claim
What should the promotion change?
“The campaign generated revenue” can mean that recipients played after receiving an offer. It can also be used to claim that the offer caused additional activity. These are different claims and require different evidence.
Begin by choosing one decision and an outcome that matches it. A campaign review might examine defined contribution per eligible player over a fixed follow-up window. Record the incentive and other costs included in that contribution so the comparison does not count them twice.
Observed response
Participation, incentive use and activity among the people who received or used the offer.
Incremental effect
The difference attributable to the offer relative to what would have happened without it.
Economic outcome
The value of the difference under a stated accounting and cost definition.
Why totals can mislead
Recipients may already be different
A team may select players because they recently became less active, have a particular preference or are considered likely to respond. Recipients and nonrecipients then differ before the campaign starts. Comparing their subsequent totals can mix the effect of the offer with those earlier differences.
Comparing redeemers with nonredeemers adds another selection problem: redemption is a behavior after the offer. The players choosing to redeem may already have a different propensity to play. Similarly, a week after launch may differ from the week before because of seasonality, other campaigns or the mix of players.
These comparisons can be useful descriptions. A causal conclusion requires a design that addresses the alternatives.
Plan before launch
Build a credible comparison
Define eligibility and exclusions
Set the target population and apply the operator’s applicable marketing restrictions and player-protection rules before assignment.
Choose an outcome and window
Specify the primary economic measure, cost scope and observation period in advance. Keep a plan for records arriving late.
Assign a suitable comparison
Where lawful and appropriate, random assignment within the eligible population can create treatment and holdout groups that are comparable by design.
Record assignment and exposure
Keep the original assignment, actual delivery and participation as different fields. Monitor overlapping offers and deviations from the plan.
Evaluate the assigned groups
Compare the pre-defined outcome on the same basis for all assigned players. Per-assigned-player results avoid selecting only those who redeemed the offer.
Read uncertainty and cost together
Consider variability, sample adequacy, group balance and missing data. Report what the comparison supports before deciding whether to scale an offer.
- Eligible population
Define the players and outcome before assignment.
- Offer & comparison
Preserve assignment and follow the same outcome window.
- Evaluate the difference
Read the defined outcome with cost and uncertainty.
Economic interpretation
Keep the unit of comparison consistent
For comparable assigned groups, a basic estimate is the difference in their average defined outcome per eligible player. If the outcome is contribution, its formula must specify which incentive and operating costs are already included. The interpretation also depends on whether assignment, delivery and exposure followed the design.
Group totals alone can be misleading when population sizes differ. A percentage change without a baseline, window and uncertainty also leaves the decision incomplete. Keep the unit, outcome definition and comparison design next to the estimate.
Interpret with care
What common shortcuts leave unanswered
| Shortcut | What it shows | What remains unresolved |
|---|---|---|
| Revenue after launch | Observed activity in a later period. | Seasonality, other changes and what would have happened without the offer. |
| Redeemers versus nonredeemers | Differences between self-selected response groups. | The effect of the offer separate from player selection. |
| Recipient GGR | A gaming revenue measure for recipients. | Campaign cost, other costs and incremental effect. |
| Redemption rate | Use of the offer under its activation rule. | Economic value and the counterfactual outcome. |
| An estimated ROI | A ratio calculated under the chosen definitions. | Validity of attribution, cost completeness and uncertainty in the estimate. |
When the design is observational
Working without a suitable holdout
If no suitable holdout exists, begin with the descriptive result: cost, population, period and subsequent activity. An analyst may use observational methods to address differences between groups, but those methods depend on assumptions about the data and the assignment process.
Document those assumptions and the alternative explanations that remain. When the available records cannot identify incremental effect, use the review to improve the design of the next campaign. A smaller, well-defined question can be more useful than a confident causal claim from a weak comparison.
Make the result usable
A review another person can check
A concise review should contain:
- The business decision and primary outcome.
- The eligible population, exclusions and assignment process.
- The offer, timing and other campaigns affecting the same people.
- The revenue and cost definitions, including incentive treatment.
- The outcome window and data reconciliation.
- The observed response, the comparison estimate and its uncertainty.
- The decision supported by that evidence and what should be measured next.
The bonus cost implementation scenario connects this review with the operational records needed to produce it.