Colin Milligan, Allison Littlejohn & Nina Hood
Learning in MOOCs: A Comparison Study
Research Track: Room Nr. HS 01.14
Date: Monday 22nd February 2016, Time: 14:30
Outline
•
•
•
•
MOOCs and Self-Regulated Learning,
Study contexts, participants and method,
Findings: Individual studies, emerging themes,
Reflection on implications, limitations and future work.
Introduction and Background
FORETHOUGHT
SELFREFLECTION
PERFORMANCE
Massive Open Online Courses
• Massively popular,
• Content-centric, pedagogically simplistic.
• Our interest: how are they being used by professionals:
• Formalising and updating knowledge,
FORETHOUGHT
• Preparing for career move,
• Networking and access to other professionals.
SELFREFLECTION
PERFORMANCE
Self-Regulated Learning
Self-regulation is the ‘self-generated thoughts, feelings and
actions that are planned and cyclically adapted to the
attainment of personal goals’ - Zimmerman, 2000.
FORETHOUGHT
FORETHOUGHT
SELFREFLECTION
PERFORMANCE
SELFREFLECTION
PERFORMANCE
Phases and Sub-Processes of SRL
Self-Regulated Learning (adapted from Zimmerman, 2000)
Phase
Forethought
Performance
Sub-processes Goal setting
Learning and Task
Self-efficacy
strategies
Task interest/value Help seeking
Interest
enhancement
Self-reflection
Self-evaluation
Self-satisfaction/affect
FORETHOUGHT
SRL is highly context dependent – an individual may be unable
to (or may choose not to self-regulate in
some contexts.
SELFREFLECTION
PERFORMANCE
SRL & Massive Open Online Courses
• Four types of Motivation/underlying goals in MOOCS:
•
•
•
•
fulfilling current needs,
preparing for the future,
satisfying curiosity,
connecting with people.
Zheng, Rosson, Shih & Carroll, 2015
FORETHOUGHT
• Higher levels of self-efficacy lead to greater persistence.
Poellhuber, Roy, Bouchoucha & Anderson, 2014, Wang & Baker, 2015
• Learners who set goals are more likely to persist.
Haug, Wodzicki, Cress & Moskaliuk, 2014
SELF-
• Forum participation linked to course
completion. PERFORMANCE
REFLECTION
Gillani & Eynon, 2014
What, Why & How: Qualitative Data
While “… quantitative studies of learner activity within MOOC
platforms provide us with a greater understanding of what
learners do within MOOCs, our understanding of why MOOC
participants learn as they do and how they actually learn is
less well developed.”
Veletsianos, CollierFORETHOUGHT
& Schneider, 2015
SELFREFLECTION
PERFORMANCE
Research Questions and Study Design
Research Questions
• RQ1 How are MOOCs currently designed to
support self-regulated learning?
• RQ2 What self-regulated learning strategies do
professionals apply in a MOOC?
• RQ3 How can MOOCs be designed to encourage
professionals to self-regulate their learning?
Littlejohn, A, & Milligan, C. (2015).
Designing MOOCs for professional learners: tools and patterns to encourage self-regulated learning.
eLearning Papers, 42, 38-45.
Contexts & Cohorts
• Fundamentals of Clinical
Trials (edX, 2013-4).
• Participants were
professionals across a
range of roles – medicine,
healthcare, statistics, bioscientists, pharmacists.
• 35 interviewees [16m,
19f], 23 countries.
• Drawn from 350 survey
respondents.
• Introduction to Data
Science (Coursera, 2014).
• Participants were software
professionals across a
range of roles – software
engineers, data analysts,
scientists.
• 32 interviewees [27m, 5f],
16 countries.
• Drawn from 768 survey
respondents.
Method & Analysis
• Administer quantitative instrument:
• to gain a measure of the extent to which they are self-regulating
in this context,
•
•
•
•
•
Recruit volunteers for interviews,
Undertake interviews,
Code interviews,
Sort by SRL score and sub-process score (high/low),
Look for patterns of SRL:
• and how it differs between the two groups.
Instrument: SRL Questionnaire
• A measure of SRL for each respondent. Items were tailored
to encourage participants to reflect specifically on their
learning practices in the MOOC,
• Adapted from SRL in non-formal contexts instrument,
previously validated (Fontana et al, 2015),
• This in turn was constructed from existing instruments:
• MSLQ (Pintrich et al, 1991); MAI (Schraw & Dennison, 1994);
OSLQ (Barnard-Brak et al, 2010); LS (Warr & Downing, 2000);
OS (Rigotti, Schyns & Mohr, 2008).
• Instrument available from figshare:
http://figshare.com/articles/SRLMQ/866774
SRL Profiles
FCT 152
FCT 213
2.00
2.00
1.50
1.50
1.00
1.00
0.50
0.50
0.00
0.00
-0.50
-0.50
-1.00
-1.00
-1.50
-1.50
-2.00
-2.00
-2.50
-2.50
-3.00
-3.00
SRL Profiles
FCT 334
FCT 394
2.00
2.00
1.50
1.50
1.00
1.00
0.50
0.50
0.00
0.00
-0.50
-0.50
-1.00
-1.00
-1.50
-1.50
-2.00
-2.00
-2.50
-2.50
-3.00
-3.00
Instrument: Semi-Structured Interview
• Explored various aspects of MOOC learning, structured
around SRL sub-processes including self-efficacy, goalsetting and learning and task strategies, as well as patterns
of help-seeking, self-reflection etc.
• Available from figshare (IDS version):
https://figshare.com/articles/Interview_Script_SRL_in_MOOCs_IDS_/1300050
What are we looking for?
• What evidence of self-regulation do we see?
• Taking the two studies together:
• what similarities and differences do we recognise?
• how do these relate to topic/course format?
Findings
Differences: Course Topic & Format
FCT
• Interdisciplinary – focused
on diverse aspects of
clinical trials: ethics,
statistics,
• Adopted a very rigid
structure
• Don’t underestimate the
brand: Harvard
IDS
• Aimed at programmers,
who wanted to gain skill in
an emerging domain.
• Included projects –
opportunities for in depth
Fundamentals of Clinical Trials
Sub-process
High group
Low group
Goal setting detailed, learning/mastery goals set, emotionally
invested, and focused on role or career,
some also mention certification.
goals, if set at all, were typically focused on completion
and certification.
Self-efficacy clear and detailed descriptions demonstrating
individual responsibility.
less detailed descriptions,
almost half indicated low self-efficacy.
Learning and note taking standard,
Task strategies active engagement,
most did not change approach (did not feel the need
to change),
minority made active decision to change based on
time pressures.
only a minority took notes,
more passive in approach,
almost half changed approach as original approach had
been ineffective,
remainder had faced challenges but not changed: citing
time pressures as a barrier (as opposed to a driver for
change).
Milligan & Littlejohn, under review
Introduction to Data Science
Sub-process
High group
Low group
Goal setting goals reference professional roles and future needs,
improving skillset, gaining content knowledge,
only a minority focused on completion.
goals more abstract, focused on love of learning
or extrinsically motivated, focused on
completion/certification.
Self-efficacy confidence in ability arising from specific factors:
- previously familiar with content knowledge,
- previous participation in a MOOC.
some participants lacked ability or confidence to
evaluate own learning (ss).
Learning and wide variety of strategies used,
Task strategies flexible in approach,
adaptive.
linear approach adopted,
highly scheduled,
inflexible.
expressed disappointment in performance but didn’t
change approach (ss).
Littlejohn, Hood, Milligan & Mustain, 2016
Similarities: Goal-setting
• Greater specificity of goals among high SRL group:
“The main aim is to become a better data analyst and get my
introduction and get the concept I need for data science, especially
data science that involves around building MapReduce programmes
and Python programmes.” (IDS 673)
• Intrinsic over extrinsic motivation among high SRL group.
“I would like to have finished the class, to get the certificate, but it
wasn't really for that. I think it’s more personal, like a personal goal,
like I just wanted to learn from the best. So it’s great that you have a
certificate, but I’m not about the piece of paper, I’m about the learning
opportunity.” (FCT334)
Similarities: Self-Efficacy
• Good self-efficacy across the board,
• Lack of confidence in a subset who had not studied online
before or who lacked background knowledge:
“I’m very familiar with the course, I already have a good background, I
have all the resources and knowledge about this issue” (FCT 152)
“I found myself lost, this is due to the background, maybe that I was
deficient” … “I always start searching on an internet engine, but it
needs some sort of assistance.” (FCT 316)
Differences: Goal-setting
• The importance of the Harvard brand:
“The goal is to have an in depth knowledge of this area from a very
prestigious university like Harvard and having it certified with a
certificate.” (FCT 152)
Differences: Learning and Task Strategies
• Note taking a standard approach for FCT while not universal
among IDS learners,
• High SRL FCT learners kept to the rigid structure of the
course, following it:
“They recommend readings /books where you can dig deeper into the
subject … I could have gone more in depth, although I chose not to
because I didn’t think I needed more in depth just now” (FCT 143).
• High SRL IDS learner were more flexible in their approach:
“I think my knowledge and my background and my work experience
was very, very helpful, because whenever I saw something I understood
I just ditched it and went to another part of the course.” (IDS 239)
Conclusions and Reflection
Conclusions
• Individuals self-regulate their learning to varying degrees,
• Other factors affect learning - motivation, prior experience
etc.
• Course topic and format can shape expectations and
learning approach adopted,
• Prestige is also a factor.
Implications for Practice
• Encourage learners to set goals,
• Get them to think about what they want
• Match content to learner expectations,
• Can one course meet diverse expectations?
• Ensure learners are confident to learn in your MOOC
environment.
Reflection: Limitations
• Small samples:
• inherent in qualitative research.
• Broad variability of key factors within sample:
•
e.g. motivation, experience of online learning.
• Limited range of SRL ability:
• all were self-regulating their learning to a significant degree.
• Lack of external measure of success:
• difficult to link learning and performance (time/access to data).
Reflection: Future Work
• Study further MOOC contexts:
• to see to what extent observations are generalisable.
• Link with completion and other quantitative data:
• to strengthen evidence and understand the impact of different
learning strategies.
• Perform longitudinal studies:
• to see the impact of MOOC learning on practice.
• Use the SRL profiles as feedback for learners:
• to encourage reflection on strengths, weaknesses.
Thank you
Dr Colin Milligan
Caledonian Academy
GLASGOW CALEDONIAN UNIVERSITY
Glasgow, SCOTLAND
Prof Allison Littlejohn
Institute of Educational Technology
OPEN UNIVERSITY
Milton Keynes, ENGLAND
Dr Nina Hood
Faculty of Education
UNIVERSITY OF AUCKLAND
Auckland, NEW ZEALAND
colin.milligan@gcu.ac.uk
@cdmilligan
allison.littlejohn@open.ac.uk
@allisonl
n.hood@auckland.ac.nz
Slides (with notes) available from: https://figshare.com/authors/Colin_Milligan/100462
This work was originally funded by the Bill & Melinda Gates Foundation. Thanks to Obiageli Ukadike,
Nabeel Gillani and Bill Howe for access and assistance, and Lou McGill for conducting interviews.
Extras
References
Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice Hall.
Bernacki, M. L., Aguilar, A. & Byrnes, J. (2011). Self-regulated learning and technology-enhanced learning environments: An
opportunity propensity analysis. In G. Dettori and D. Persico (Eds.), Fostering self-regulated learning through ICT (pp. 1-26).
Hershey, PA: IGI Global Publishers.
Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D. & Seaton, D. T. (2013). Studying learning in the worldwide
classroom: Research into edX’s first MOOC. Journal of Research & Practice in Assessment, 8, 13–25.
Fontana, R.P., Milligan, C., Littlejohn, A. & Margaryan, A. (2015). Measuring self-regulated learning in the workplace.
International Journal of Training and Development. 19(1) 32-52.
Gašević, D., Kovanović, V., Joksimović, S. & Siemens, G. (2014). Where is Research on Massive Open Online Courses
Headed? A Data Analysis of the MOOC Research Initiative. International Review of Research in Open and Distance Learning,
15(5), 134-176.
Gillani, N. & Eynon, R. (2014). Communication patterns in massively open online courses. The Internet and Higher Education, 23,
18-26.
Haug, S., Wodzicki, K., Cress, U. & Moskaliuk, J. (2014). Self-Regulated Learning in MOOCs: Do Open Badges and Certificates
of Attendance Motivate Learners to Invest More? In U. Cress & C. D. Kloos (Eds.). EMOOCs 2014 - European MOOC
Stakeholder Summit, (pp. 66-72). Lausanne, Switzerland.
Kizilcec, R. F., Piech, C. & Schneider, E. (2013). Deconstructing disengagement: Analysing learner subpopulations in massive
open online courses. In Proceedings of the 3rd International Conference on Learning Analytics and Knowledge (pp. 170–179).
New York, NY, USA: ACM. Retrieved 8 April 2015, from: http://dl.acm.org/citation.cfm?id=2460330
Littlejohn, A, & Milligan, C. (2015). Designing MOOCs for professional learners: tools and patterns to encourage self-regulated
learning. eLearning Papers, 42, 38-45.
Littlejohn, A., Hood, N., Milligan, C. & Mustain, P. (2016). Learning in MOOCs, motivations and self-regulated learning. The
Internet and Higher Education. 29, 40-48.
Littlejohn, A., Milligan, C. Fontana, R.P. & Margaryan, A. (2016). Professional learning through everyday work: how finance
professionals self-regulate their learning. Vocations and Learning.
References
Margaryan, A., Bianco, M. & Littlejohn, A. (2015). Instructional quality of Massive Open Online Courses (MOOCs). Computers &
Education, 80, 77-83.
Milligan, C., Littlejohn, A. & Margaryan, A. (2013). Patterns of engagement in connectivist MOOCs. MERLOT Journal of Online
Learning & Teaching 9(2), 149-159.
Milligan, C. & Littlejohn, A. (2014). Supporting professional learning in a massive open online course. International Review of
Research in Open and Distance Learning 15(5) 197-213.
Milligan, C. & Littlejohn. A. (under review). How health professionals regulate their learning in MOOCs.
Poellhuber, B., Roy, N. Bouchoucha, I. & Anderson, T. (2014). The relationships between the motivational profiles, engagement
profiles and persistence of MOOC participants. MOOC Research Initiative, Final Report.
Ryan, R.M. & Deci, E.L. (2000). Intrinsic and extrinsic motivations: classic definitions and new directions. Contemporary
Educational Psychology, 25, 54-67.
Veletsianos, G., Collier, A. & Schneider, E. (2015). Digging Deeper into Learners Experiences in MOOCs: Participation in social
networks outside of MOOCs, Notetaking, and contexts surrounding content consumption. British Journal of Educational
Technology, 46,570 - 587
Wang, Y. & Baker, R. (2015). Content or platform: why do students complete MOOCs. MERLOT Journal of Online Learning and
Teaching, 11(1), 17-30.
Zheng, S., Rosson, M. B., Shih, P. C. & Carroll, J. M. (2015). Understanding Student Motivation, Behaviors and Perceptions in
MOOCs. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (pp. 18821895). ACM.
Zimmerman, B. J. (2000). Attaining self-regulation: a social cognitive perspective. In M. Boekaerts, M. Zeidner, and P.R. Pintrich
(Eds.), Handbook of self-regulation (pp13-39). Academic Press, San Diego, CA.
Recommendations
1. Enable professional learners to link theory learned in the MOOC with their
work practice by setting personal goals, or personalizing course goals.
2. Help professional learners to reflect on the knowledge gained from the course
and how it may be embedded into their work practice before the end of the
course.
3. Support professional learners to continually monitor their learning to
determine its ultimate value beyond their immediate learning experience.
4. Capitalize on the diversity of motivation, expectation, and prior knowledge
and experience that is an inherent within all MOOC cohorts.
5. Encourage professional learners to discuss ideas from the course with coworkers in their external professional network as well as with other learners
on the course.
6. Utilize the existing knowledge and experience that professional learners bring
to the learning context.
https://figshare.com/articles/MOOC_Design_Recommendations/1420557