Learning Analytics

- How many students completed the course?
- What's the average time spent per module?
- Which topics have highest dropout?

- Why did students fail this assessment?
- What correlates with course completion?
- Which behaviors predict success?

- Which students are at risk of dropping out?
- Who will need additional support?
- What's the expected completion time?

- What intervention will help this student most?
- Which content should we recommend next?
- How should we modify the course?

Login patterns:
- Frequency of access
- Time spent on platform
- Access times (patterns)

Content interaction:
- Pages viewed
- Videos watched
- Resources downloaded
- Navigation paths

Assessments:
- Quiz attempts and scores
- Assignment submissions
- Time on assessments
- Attempt patterns

{
  user_id: "student_123",
  timestamp: "2026-01-03T14:32:15Z",
  action: "video_play",
  resource: "module_2_intro",
  duration: 237,  // seconds watched
  completion: 0.85  // 85% watched
}

Coding platforms:
- Code submissions
- Compilation errors
- Execution attempts
- Time between attempts
- Help requests

Interactive exercises:
- Clicks and inputs
- Paths through content
- Hint requests
- Error patterns

Problem: "Write function to reverse string"
Attempt 1: Syntax error (line 3)
Attempt 2: Logic error (partial solution)
Hint requested: "Consider string slicing"
Attempt 3: Correct solution
Total time: 12 minutes

Multiple-choice questions:
- Distractor analysis (wrong answers chosen)
- Time per question
- Question order effects
- Changes after review

Open-ended:
- Response length
- Vocabulary used
- Structure and coherence
- Revision patterns

Discussion forums:
- Post frequency
- Response quality
- Network position (central vs peripheral)
- Help-seeking vs help-giving

Study groups:
- Participation patterns
- Contribution balance
- Communication style
- Group dynamics

Demographics (carefully!):
- Prior education level
- Language background
- Time zone

Goals and motivation:
- Career aspirations
- Reasons for enrollment
- Time commitment

Performance:
- Employment outcomes
- Skill application
- Continued learning

Total time on platform
Active learning time (interaction vs passive)
Session duration and frequency
Time of day patterns
Consistency over time

def calculate_engagement_score(user_logs):
    # Active days in last 30
    active_days = count_unique_days(user_logs, last_n_days=30)
    
    # Average session length
    avg_session = mean(session_lengths(user_logs))
    
    # Content interaction depth
    interaction_depth = sum(clicks, video_completions, submissions)
    
    # Normalise and combine
    engagement = (
        0.4 * (active_days / 30) +
        0.3 * normalise(avg_session, min=5, max=60) +
        0.3 * normalise(interaction_depth, min=10, max=100)
    )
    
    return engagement  # 0 to 1

< 0.3: At risk (minimal engagement)
0.3-0.6: Moderate (could improve)
> 0.6: High engagement (on track)

Score distributions:
- Mean, median, mode
- Standard deviation
- Percentile rankings

Trends:
- Improvement over time
- Concept mastery progression
- Difficulty adaptation

Patterns:
- Which concepts are difficult?
- Where do students get stuck?
- What predicts success?

Concepts mastered per week
Time to competency per skill
Acceleration or deceleration
Comparison to cohort average

Content completion rate:
- Overall course
- Per module
- Per content type

Milestone achievements:
- Quizzes passed
- Projects completed
- Certificates earned

Pacing:
- Ahead of schedule
- On track
- Behind schedule
- Dropped out

Student: Jane Smith
Overall progress: 67% complete
Expected completion: March 15
Current pace: On track

Module breakdown:
✓ Module 1: Complete (95% score)
✓ Module 2: Complete (88% score)
→ Module 3: In progress (45% complete)
  Module 4: Not started
  Module 5: Not started

Declining engagement:
- Fewer logins over time
- Shorter sessions
- Incomplete content

Poor performance:
- Multiple failed assessments
- Low quiz scores
- Incomplete assignments

Behavioral patterns:
- Last-minute submissions
- No discussion participation
- Excessive hint requests
- Long gaps between sessions

def calculate_risk_score(student):
    risk_factors = {
        'low_engagement': student.avg_sessions_per_week < 2,
        'poor_performance': student.avg_score < 60,
        'declining_trend': student.recent_scores < student.early_scores,
        'no_social': student.forum_posts == 0,
        'deadline_misses': student.late_submissions > 2,
        'long_absence': days_since_last_login(student) > 7
    }
    
    weights = {
        'low_engagement': 0.25,
        'poor_performance': 0.30,
        'declining_trend': 0.20,
        'no_social': 0.10,
        'deadline_misses': 0.10,
        'long_absence': 0.05
    }
    
    risk = sum(weights[factor] for factor, present in risk_factors.items() if present)
    
    return risk  # 0 (low risk) to 1 (high risk)

Risk < 0.3: Low (automated nudges)
Risk 0.3-0.6: Medium (personalised outreach)
Risk > 0.6: High (immediate intervention)

Cohort A (Jan 2025): 
- Started: 100 students
- Week 4: 85 active (85%)
- Week 8: 67 active (67%)
- Completed: 58 (58%)

Cohort B (Feb 2025):
- Started: 120 students
- Week 4: 102 active (85%)
- Week 8: 89 active (74%) ← Better retention!
- Completed: TBD

Analysis: What changed?
- New onboarding process added
- Better pacing in Module 2
- More engaging content in Week 3

Week 0: 100% (baseline)
Week 1: 92%
Week 2: 84%
Week 3: 76%  ← Drop
Week 4: 73%
Week 8: 67%
Week 12: 58%

Investigate Week 3 content!

Common successful paths:
Path A: Lecture → Practice → Quiz → 88% pass rate
Path B: Lecture → Quiz → Practice → 76% pass rate
Path C: Practice → Lecture → Quiz → 82% pass rate

Insight: Practice before quiz improves outcomes

Module 1 → Module 2: 92% proceed
Module 2 → Module 3: 78% proceed ← Drop
Module 3 → Module 4: 95% proceed

Why does Module 3 have lower retention?
- Difficulty spike?
- Less engaging content?
- External factors (mid-semester)?

Students who succeed:
- Follow linear path: 65%
- Jump around: 35%

Students who struggle:
- Follow linear path: 45%
- Jump around: 55%

Insight: Struggling students search for help
Action: Add better navigation/search

Pattern: Typical learner
Week 1: High activity (new user excitement)
Week 2-3: Decline (novelty wears off)
Week 4: Spike (first assessment)
Week 5-7: Steady
Week 8: Decline (dropout risk)
Week 9: Push (deadline approaching)

Interventions:
- Week 3: Gamification element
- Week 8: Check-in email

Time of day usage:
Morning (6-9am): 15%
Midday (12-2pm): 25%
Evening (6-10pm): 40%  ← Peak
Late night (10pm-1am): 20%

Day of week:
Monday-Thursday: Steady
Friday: Drop
Weekend: Variable

Insight: Schedule live sessions during peak times

from sklearn.ensemble import RandomForestClassifier

# Features
features = [
    'avg_session_duration',
    'login_frequency',
    'quiz_scores_mean',
    'assignment_completion_rate',
    'forum_activity',
    'days_since_last_login',
    'video_completion_rate',
    'help_requests'
]

# Target
target = 'completed_course'  # 1 = completed, 0 = dropped

# Train model
model = RandomForestClassifier(n_estimators=100)
model.fit(X_train, y_train)

# Predict for current students
at_risk_students = model.predict_proba(X_current)[:, 0] > 0.7

# Feature importance
print(model.feature_importances_)
# → [0.25, 0.20, 0.18, 0.12, 0.08, 0.07, 0.06, 0.04]
# Top predictor: avg_session_duration

Predict final grade after Week 4:

Features:
- Week 1-4 quiz scores
- Engagement metrics
- Assignment quality
- Forum participation

Model: Linear regression
Accuracy: R² = 0.72

Use for:
- Early intervention
- Study group formation
- Personalised recommendations

Forum interaction graph:

Nodes: Students
Edges: Replies, mentions, collaborations

Metrics:
- Centrality: Who is most connected?
- Betweenness: Who bridges groups?
- Clusters: Are there isolated groups?

Insights:
- Central students → Potential mentors
- Isolated students → Need connection
- Dense clusters → Effective study groups

Who helps whom?

Helpers (high out-degree):
- Engaged, knowledgeable students
- Natural TAs/mentors
- Community builders

Help-seekers (high in-degree):
- May need additional support
- Active learners
- Engaged despite struggles

No connections:
- At-risk group
- May feel isolated
- Intervention needed

┌─ Your Progress ────────────────────┐
│ 67% Complete                       │
│ ████████████████░░░░░░░░░░        │
│                                    │
│ Modules: 3 of 5 complete           │
│ Quizzes: 8 of 12 passed            │
│ Projects: 2 of 4 submitted         │
│                                    │
│ Expected completion: March 15      │
│ Current pace: On track ✓           │
└────────────────────────────────────┘

┌─ Your Performance ─────────────────┐
│                                    │
│ Overall: 85%  ↗ +3% from last week│
│                                    │
│ Strengths:                         │
│  ⭐ Variables & Data Types: 95%   │
│  ⭐ Functions: 92%                │
│                                    │
│ Areas to improve:                  │
│  ⚠ Loops: 68%                     │
│  ⚠ Error Handling: 72%            │
│                                    │
│ 📚 Recommended: Practice loops     │
└────────────────────────────────────┘

┌─ Class Analytics ──────────────────┐
│ CS101 - Spring 2026 │ 87 students  │
├────────────────────────────────────┤
│                                    │
│ Overall Progress:                  │
│ ████████████░░░░░░░░ 62%          │
│                                    │
│ Engagement: ⚠ Below target        │
│ Performance: ✓ On track           │
│ Completion rate: 73% (vs 68% last)│
│                                    │
│ 🔴 At-risk: 12 students           │
│ 🟡 Struggling: 18 students        │
│ 🟢 On track: 57 students          │
└────────────────────────────────────┘

┌─ Student: Alice Johnson ───────────┐
│ Risk level: 🟡 Medium              │
│                                    │
│ Recent activity:                   │
│ • Last login: 6 days ago  ⚠       │
│ • Quiz 3: 62% (below avg)         │
│ • Assignment 2: Not submitted     │
│                                    │
│ Patterns:                          │
│ • Declining engagement trend      │
│ • Struggles with Module 3 content │
│ • Low forum participation         │
│                                    │
│ Recommendations:                   │
│ • Send check-in email             │
│ • Suggest office hours            │
│ • Connect with study group        │
└────────────────────────────────────┘

Module Performance:

Module 1: ████████░ 87% avg score
Module 2: ████████░ 85% avg score
Module 3: █████░░░░ 68% avg score ← Issue!
Module 4: ███████░░ 82% avg score

Video engagement:
Module 3 Video 1: 45% completion ← Drop-off point
Module 3 Video 2: 78% completion

Action items:
1. Review Module 3 Video 1 (too long? confusing?)
2. Add practice before Module 3 quiz
3. Consider splitting Module 3 into two

Quiz 3 - Question Analysis:

Q1: 92% correct ✓
Q2: 88% correct ✓
Q3: 45% correct ✗ ← Problem question
Q4: 78% correct ✓
Q5: 82% correct ✓

Q3 Distractor analysis:
- Option A: 12% (correct answer)
- Option B: 45% (common mistake: off-by-one error)
- Option C: 28% (conceptual misunderstanding)
- Option D: 15% (guess)

Action: Add example about off-by-one errors before Q3

Collect only what's necessary:
✓ Performance data
✓ Engagement metrics
✓ Anonymous usage patterns

Don't collect:
✗ Keystroke logging (too invasive)
✗ Webcam monitoring (privacy violation)
✗ Unrelated browsing behavior

For research/aggregation:
- Remove direct identifiers
- Aggregate small groups
- Use differential privacy
- K-anonymity (at least k similar records)

Tell learners:
✓ What data is collected
✓ How it's used
✓ Who has access
✓ How long it's kept
✓ How to opt out

Risks:
- Historical data reflects existing inequities
- Proxies for protected characteristics
- Feedback loops amplify bias

Example:
If model trained on data where certain groups 
historically performed poorly → Model predicts they 
will perform poorly → Creates self-fulfilling prophecy

1. Audit for bias across groups
2. Use fairness-aware ML techniques
3. Don't use sensitive attributes unless necessary
4. Regular impact assessments
5. Human oversight of algorithmic decisions

Helpful analytics ✓:
- Aggregate course improvement
- Voluntary learner progress tracking
- Opt-in performance insights

Harmful surveillance ✗:
- Constant monitoring
- Punitive use of data
- Invasion of privacy
- Pressure and stress

✓ Opt-in for detailed tracking
✓ View their own data
✓ Delete their data
✓ Understand how it's used
✓ Control who sees it

1. Define goals:
   - What questions do you need answered?
   - What decisions will be informed?
   
2. Identify data sources:
   - LMS logs
   - Assessment data
   - User surveys

3. Set up basic tracking:
   - Login frequency
   - Content completion
   - Assessment scores

4. Create simple dashboards:
   - Progress tracking
   - Engagement overview

1. Deeper metrics:
   - At-risk identification
   - Performance prediction
   - Content effectiveness

2. Visualisations:
   - Trend charts
   - Comparison views
   - Individual profiles

3. Initial interventions:
   - Automated reminders
   - Personalised recommendations

1. Advanced analytics:
   - Predictive models
   - A/B testing
   - Causal analysis

2. Automated personalisation:
   - Content recommendations
   - Difficulty adaptation
   - Pacing adjustments

3. Continuous improvement:
   - Feedback loops
   - Iterative refinement

- LAPA (Learning Analytics Platform)
- Apache Spark (big data processing)
- Jupyter notebooks (analysis)
- Tableau Public (visualisation)
- R/Python (statistical analysis)

- Brightspace Insights
- Canvas Analytics
- Blackboard Analytics
- Coursera Analytics
- EdX Insights

Stack:
- Data warehouse: PostgreSQL, MongoDB
- Processing: Python (pandas, scikit-learn)
- Visualisation: D3.js, Plotly
- Dashboards: Dash, Streamlit
- APIs: FastAPI, GraphQL

1. Validate data integrity
2. Handle missing values appropriately
3. Document data sources and definitions
4. Regular audits
5. Version control for datasets

Not just: "Engagement is down 15%"
But: "Engagement drops in Week 3. Students report 
Module 3 is too difficult. Recommend: Add scaffolding 
video and practice problems before Quiz 3."

1. Collect data
2. Analyse patterns
3. Form hypothesis
4. Test intervention
5. Measure impact
6. Refine and repeat

Move from:
- Weekly reports
- Delayed insights

To:
- Live dashboards
- Instant feedback
- Just-in-time interventions

Combine:
- Clickstream data
- Text analysis
- Video engagement
- Biometric data (ethical use)
- Social network data

- MOOCs with millions of learners
- Cross-institutional data
- Longitudinal tracking
- Career outcome analytics

1. Transparency: Clear about what's collected and why
2. Privacy: Protect learner data
3. Fairness: Monitor for and mitigate bias
4. Agency: Learner control over their data
5. Benefit: Use data to help, not harm
6. Consent: Informed opt-in
7. Purpose limitation: Use data only for stated purposes