Skip to content

Conversation

@drbenvincent
Copy link
Collaborator

@drbenvincent drbenvincent commented Nov 3, 2025

Adds a new notebook demonstrating how to conduct lift tests using Interrupted Time Series analysis when control groups are unavailable (e.g., national-level campaigns).

Key features:

  • Simulates realistic marketing data with adstock and saturation effects
  • Shows how to run ITS analysis to measure campaign lift
  • Extracts mean and standard deviation outputs for MMM calibration
  • Calculates absolute lift, percentage lift, and ROI with uncertainty
  • Includes guidance on formatting results for PyMC-Marketing integration

This bridges the gap between experimental lift testing and MMM calibration, providing the workflow for generating experimental evidence that can improve attribution models.


📚 Documentation preview 📚: https://causalpy--547.org.readthedocs.build/en/547/

@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@drbenvincent drbenvincent marked this pull request as draft November 3, 2025 11:29
@codecov
Copy link

codecov bot commented Nov 3, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.59%. Comparing base (828ba2e) to head (3b7704d).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #547   +/-   ##
=======================================
  Coverage   95.59%   95.59%           
=======================================
  Files          29       29           
  Lines        2681     2681           
=======================================
  Hits         2563     2563           
  Misses        118      118           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@drbenvincent
Copy link
Collaborator Author

@cetagostini I think I should be adding in the channel spend variables into the model formula, right?

@review-notebook-app
Copy link

review-notebook-app bot commented Nov 5, 2025

View / edit / reply to this conversation on ReviewNB

cetagostini commented on 2025-11-05T07:17:35Z
----------------------------------------------------------------

All good, but two important points:

  1. The spend or response should not be scaled. So, you can safely pass everything in the original scale, the model will handle everything under the hood. Doc here: https://github.com/pymc-labs/pymc-marketing/blob/main/pymc_marketing/mmm/multidimensional.py#L1828
  2. The impact input should be MEAN not cumulative. Basically, this same values divide by the N length of the experiment.

@review-notebook-app
Copy link

review-notebook-app bot commented Nov 5, 2025

View / edit / reply to this conversation on ReviewNB

cetagostini commented on 2025-11-05T07:17:36Z
----------------------------------------------------------------

Good, I'll make a PR to make things clear there as well.


drbenvincent commented on 2025-11-06T09:47:21Z
----------------------------------------------------------------

I've already given that a go in a PR that I think you've seen pymc-labs/pymc-marketing#2061

@drbenvincent drbenvincent added the documentation Improvements or additions to documentation label Nov 6, 2025
Copy link
Collaborator Author

I've already given that a go in a PR that I think you've seen pymc-labs/pymc-marketing#2061


View entire conversation on ReviewNB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants