Experimental A/B Test for Solutions Journalism Approaches
Project Overview
Client & Role
News Organizations
Academic Researcher
Timeline
8-16 Weeks
Methods
Experimental Methods, Evaluative Research, Survey Design
Tools
Qualtrics, Cloud Research (for recruitment)
Business Challenge
News organizations are losing audience engagement due to "doom and gloom" climate coverage. Users also tend to avoid negative climate news because of its impact on their mental well-being, which reduces readership and advertising revenue for news organizations.
Research Question
So, we had two main research questions:
- Does solutions journalism increase readers' positive emotions and their intentions to take pro-environmental action?
- What psychological factors shape this relationship between solutions journalism and intentions to take pro-environmental action
Why This Matters
This matters because:
- News organizations need engaged readers
- Solutions journalism has been proposed as an alternative approach to elicit both social and news-related engagement from news audiences.
Secondary Research
We identified two core psychological factors driving solutions and news and social engagement:
- Positive emotions
- Collective efficacy beliefs - group confidence in action
Both factors consistently predict sustained engagement and behavior change in environmental contexts.
My Role & Impact
I led a quantitative research design and analysis and managed data collection from 482 participants. Specifically, we:
- Chose experimental design to show causation
- We selected our sample after calculating the statistical power
- We then measured behavioral intentions, collective efficacy beliefs, and positive emotions using previously validated measurements
Research Approach
Method: Experimental A/B test
- Independent group post-test experiment
- Random assignment to two conditions:
- Solutions-oriented news story (n=243)
- Problem-oriented news story (n=239)
The stories matched the topic of focus (community flood management) with similar visual elements.
Research Prototypes
Why This Method
Our choice of experimental A/B testing was driven by several key advantages:
- The method enables direct comparison between different story approaches, providing clear insights into their effectiveness.
- It generates quantifiable data that can directly inform business decisions about content strategy.
- The experimental design allows us to establish clear causal relationships between content type and user engagement.
- The approach can be easily scaled across different platforms and content types.
Analysis Approach
Methodology
We conducted a comprehensive path analysis using Lavaan in R programming software. To ensure robust results, we implemented bootstrap confidence interval estimation with 5,000 resamples and controlled for political affiliation as a covariate.
Key Business Findings
Impact Comparison
+5%
Increase in Pro-environmental intentions
+12%
Increase in Positive emotions
Statistical Analysis Visualization
T-Test Showing Effects of Stories on Pro-environmental Behavioral Intentions
T-Test Showing Effects of Stories on Positive Affect (Emotions)
Engagement Pathways
Solutions journalism demonstrated positive impact on Pro-environmental Intentions through two main channels:
- Enhanced positive emotional responses from readers
- Strengthened collective efficacy beliefs (group confidence in action)
Statistical Impact
We observed a significant indirect effect on Pro-environmental Intentions when mediated by emotion and efficacy, with a total effect size of beta=0.135 (p<.05).
Business Applications
Our findings provide actionable insights for news organizations:
Content Strategy
- Balance problem coverage with solutions
- Highlight community-level responses
Impact Measurement
- Track pro-environmental intentions
- Monitor audience sentiment
Audience Understanding
Using sentiment analysis and efficacy belief surveys helps news organizations better understand their audience needs and engagement patterns.
Research Process Reflection
Key Challenge
Our main limitation was measuring behavioral intentions rather than actual behaviors.
Future Direction
To validate our findings, future studies should be conducted in natural settings on live platforms to verify if intentions translate into real actions.
Additional Information
View Resume
View my complete professional background and experience.