SR MKTG TESTING ANALYST
General Summary
The Measurement Senior Analyst is responsible for supporting marketing and promotion test design and
measurement for A/B tests. For each test, they own the test analytics and measurement plan where they
work closely with the test leads to ensure tests are set up to accurately assess the hypothesis and drive
key learnings. Once a test is live, they are responsible for monitoring test progress and results, and
ultimately work with test leads to both “call” tests and share results and learnings with the pod and other
stakeholders.
Reports to: Director, Marketing Analytics
Essential Duties and Responsibilities:
• Collaborate with Test Lead to understand test objective and initial target audience requirements
during initial test design phase as well as define what test analytics would be helpful for refining
test design; leverage insights from test analytics to refine test design
• Collaborate with Test Lead and Promo Lead to align on deep dive areas and scaling logic for
impact sizing during measurement period
• Run relevant test analytics to support the refinement of test design
• Make recommendations on the optimal testing approach (e.g., test cell setup)
• Identify potential measurement risks for test ideas (e.g., conflicts with other tests)
• Write measurement plans to define primary and secondary KPIs, minimum required audience
size, test audience size, and impact sizing for each test
• Provide additional data attributes for Audience Lead to pull eligible audience from RedPoint as
needed
• Ensure tests are monitored with the appropriate cadence once live and conduct any interim reads
of test results
• Recommend when to “call” a test based on measurement playbook
• Annualize impact at scale based on aligned scaling logic, audience definition and frequency;
monitor ongoing scaled campaigns and develop plan for semi-annual/annual re-tests to ensure
results continue to be as expected
• Present final test results and provide insights from deep dives in Result Reviews to support shareout with leadership and the broader organization
Required Skills:
• Proficient with reporting or query tools (Google Cloud Platform – Big Query, Phyton, Adobe)
• Strong analytical skills, including data retrieval, organization, presentation and summary
• Ability to thrive in a highly quantitative, fast-paced environment with changing business needs
• Ability to create visualizations, summaries, and present complex analytical techniques, data, and
recommendations in simple business speak
• High degree of initiative, personal responsibility and ownership
• Detail-oriented with strong organizational and project management skills
• Demonstrated critical thinking and problem-solving abilities
• High degree of analytical curiosity including ability to identify and proactively solve problems
• Ability to effectively prioritize and manage multiple projects simultaneously while meeting
deadlines; strong time management skills
• Ability to work in a multi-team environment, in which various business owners and partners
require support
• Ability to foster relationships with cross-functional counterparts and key stakeholders at all levels
of the organization
• Excellent written and verbal communication skills
• Proficient in MS Excel, Word, PowerPoint
Competencies
SETTING GOALS – Creates and follow effective plans. Anticipates risks, creates contingency plans.
Aligns plans with goals. Allocates adequate resources. Accepts and supports change. Willing to take
risks and suggests new ideas, approaches. Takes initiative. Seeks out learning activities.
WORKING WITH OTHERS – Clearly articulates own, other’s goals. Promotes a team atmosphere by
demonstrating humility and respect. Builds effective relationships, relates well to others. Delivers and
responds to feedback in a constructive manner. Considers multiple perspectives. Handles conflict,
pressure, uncertainty and adapts independently. Meets commitments. Dedicated to working with
business partners on their expectations.
GETTING RESULTS – Personally accountable for work performance targets and achieving results.
Prioritizes well. Anticipates and handles obstacles effectively. Makes good, timely decisions. Can simply
and process complex problems. Understands underlying issues and addresses root causes. Meets
deadlines, works until finished.
Qualifications
Minimum Experience:
• 5+ years’ experience in analytical role
• Experience in deep dive analysis on customer attribute and online/offline behavior
• Experience with developing and evolving A/B test measurement
Preferred Qualifications:
• Retail experience a plus
• Experience in Regression and Time Series Analysis
• Experience with Hypothesis testing and Confidence Intervals
Minimum Education:
• Bachelor's Degree, with focus in Statistics or Mathematics preferred
