NP3 methodology

From EdFutures
Jump to navigation Jump to search
Theory Literature Methodology Frameworks Findings Significance


The project has three related strands:

  • Strand 1, mapping mobile device use, consists of two phases:
    • Phase 1, using our extensive networks and two existing questionnaires (Your Organisation’s Technology Strategy Survey (YOTSS) and Your Own Technology Survey (YOTS)) to gauge the extent to which primary schools are using mobile devices to advance teaching and learning and identify pedagogically rich sites for 20 exploratory studies;
    • Phase 2, carrying out 20 exploratory studies, which will provide a richer picture of the ways in which primary schools are using mobile devices to advance teaching and learning, and to identify suitable case study sites for further investigation in Strand 2.
  • Strand 2, the detailed case studies, will involve eight in-depth case studies that will enable us to address Research Questions 1 to 5.
  • Strand 3 involves ongoing dialogue with practitioners judged by the project team to be at the leading edge of digital technology use in education in order to extend and critique our theorising and data analysis, and maximise the reach and impact of the project.

Ethical clearance will be obtained from the Open University’s Research Ethics Committee for all aspects of this project, and BERA’s updated ethical guidelines for education research (BERA, 2011) will be adhered to throughout.

Figure 1: Project Overview
SES NP3 overview 15-06-03.jpg

Strand 1 Mapping mobile device use

This strand has four main foci:

  • To map the extent to which primary schools are using mobile devices to advance teaching and learning, resulting in a ‘Mapping report’.
  • To identify pedagogically rich settings, some of which will be used in Strand 2 for the detailed case studies. Our experiences on the Vital and Snapshot Studies make it clear that one has to take great care in selecting settings for carrying out research on pedagogical innovation – many schools that were reported by others as being at the forefront of mobile device use on closer inspection by the researchers were assimilating the technology into traditional ways of working, rather than using it to enable new (and potentially more real-world relevant) practices to develop.
  • To start to develop rich descriptions of pupils’ digital practices and teachers’ new and emerging pedagogical practices which will feed iteratively into the meta-analysis and final report. This research will thus move beyond the ICT dimensions of practice (Twining, 2014d; Twining et al., 2015) (see Figure 1). In addition these 20 exploratory studies will result in concise case study reports (similar to those from the Vital and Snapshot Studies – see
  • To develop and refine the data collection instruments to be used in Strand 2. Whilst the methodology will be based upon the one used in the Snapshot Studies (see below) we intend to introduce new data collection techniques such as live audio transcription and robot-controlled video capture.
  • To develop consistency between team members in terms of their data collection, analysis and reporting. This will help to enhance the Stage 2 research.

There are two phases of activity feeding into this strand:

  1. The initial mapping of mobile device use.
  2. Carrying out the exploratory studies.

Phase 1 – The initial mapping of mobile device use

At the beginning of Strand 1 we will work with Naace (the ICT subject association) to promote the Your Organisation’s Technology Strategy Survey (YOTSS) to primary schools in order for them to complete the survey. YOTSS is a tool that has been developed by the Open University by a team led by Peter Twining and Fiona Henry ( YOTSS asks schools about their digital technology strategies, with a specific focus on mobile devices. Having filled in the online questionnaire, schools are then provided with information about how their digital technology strategy compares with other schools’ strategies. Naace is well positioned to promote YOTSS, as its membership includes leading educational technology consultants, educational technology suppliers, and most of the schools at the forefront of ICT use (through its management of the ICT Mark, and the 3rd Millennium Schools Network).

The YOTSS works in conjunction with Your Own Technology Survey (YOTS), which is a tool to help schools audit their pupils’ access to the Internet and mobile devices ‘from home’ and specifically, whether they would be allowed (by their parents), willing, and able to bring their mobile devices into school every day. YOTS is a free, high quality, auditing tool, which supports schools in collecting data to inform their mobile device strategies. Having signed up for this free service, schools are then encouraged to also take part in the YOTSS. Whilst Naace focuses specifically on promoting YOTSS, we will promote YOTS in order to maximise the ‘cross-selling’ opportunities and thus boost the number of responses to YOTSS overall.

Where responses to the YOTSS indicates that a school has well established mobile device policies that encourage a Bring Your Own (BYO) or 1:1 strategy and the school is willing for us to do so, we will carry out follow-up telephone interviews to ascertain whether or not the school might be a productive exploratory study site.

Where possible the exploratory studies will be in settings that cater for pupils from diverse ethnic and socioeconomic backgrounds, including children from ethnic groups (e.g. Black African or Carbibbean or Pakistani) and/or who are identified as having high levels of social disadvantage (as indicated by free school meal eligibility).

At the same time as we are working to promote YOTSS and YOTS, we will be engaging with our extensive existing networks to identify potentially valuable exploratory study sites.

Phase 2 Carrying out the exploratory studies

Exploratory studies will be undertaken in 10 of the sites identified in the previous phase.

The methodology used will be based on the Vital Studies Methodology (Twining, 2014), which in turn was based on the methodology used in the original Becta tablet PC research (Twining et al., 2005) and was subsequently adapted for the Snapshot Studies (Twining, 2013b). The methodology involves seven days of researcher time per study. This includes:

  • one day for staff development to ensure that all the researchers understand the aims and foci of the project, are confident in using the data collection instruments, and are familiar with the data analysis and reporting procedures and templates.
  • half a day for preparation, including liaising with the exploratory study school prior to the researcher visit.
  • one day in the exploratory study setting, collecting data.
  • one day for reviewing and ‘reporting’ on the first day in school and to prepare for the second day in school
  • one more day in the exploratory study setting, collecting data.
  • one and a half days for data analysis and ‘reporting’ on the second day in school.
  • one day for debriefing and sharing initial data analysis

Each Exploratory Study will follow the same basic design, which involves a researcher spending two days in the school. The researcher works with a school facilitator before visiting the school to arrange the schedule for the two visits and ensure that participants are happy to take part. Data collection specifically involves:

Day 1

  • The head teacher (or a member of the senior leadership team (SLT)) takes part in a brief interview.
  • The school facilitator helps coordinate the day and organises for the researcher to go on a tour of the school (including spending a couple of minutes in a range of classes).
  • 4 pairs of pupils and their carers/parents take part in a briefing session to explain the Digital Logs that they will each keep of the pupils’ ICT use.
  • Teacher A takes part in an interview, is observed teaching and has a short debriefing discussion.
  • 6 pupils in Teacher A’s class (Focus Group A) are observed (at the same time as Teacher A) and take part in a group interview.
  • Teacher B takes part in a short briefing meeting
  • The ICT Coordinator takes part in a short briefing meeting

Between Day 1 and Day 2

  • Teacher A fills in a questionnaire.
  • Teacher B fills in a questionnaire.
  • The ICT Coordinator fills in a questionnaire.
  • 4 pairs of pupils and their carers/parents log the pupils’ use of ICT over two days.
  • 6 other teachers fill in a short questionnaire.

Day 2

  • The pupils and their carers/parents who have filled in digital logs have individual interviews (to discuss what their logs show).
  • Teacher B takes part in an interview, is observed teaching, and has a short debriefing discussion.
  • 6 pupils in Teacher B’s class (Focus Group B) are observed (at the same time as Teacher B) and take part in a group interview.
  • Other teachers who filled in the short questionnaire have 10 minute interviews about their responses.

Schools will be provided with £200 for supply cover to facilitate the data collection (e.g. to cover teachers when they are being interviewed)

Download all of the data collection guides and other documents.

Strand 2 The case studies

The focus within this strand is on setting within which new pedagogical practices are emerging that recognise and value pupils’ digital competences. This will extend the data collection and deepen the analysis from Strand 1 and enable us to answer the key research questions:

  • RQ1 What are the digital practices that pupils bring to their learning in school?
  • RQ2 Across subject domains what do teachers’ intended and enacted pedagogic practices indicate about their awareness of and the value accorded to pupils’ digital competencies, and how do pupils experience these pedagogic practices?
  • RQ3 What institutional circumstances and practices enable or undermine how pupils’ digital competencies and practices are recognised (RQ1) and integrated into teachers’ practice (RQ2)?
  • RQ4 What are the consequences of the answers to RQs 1-3 for learning in terms of social justice, and across and within subject domains?
  • RQ5 How does the research inform how to represent and model a participative pedagogy of mutuality (Bruner, 1996; Wenger, 1998, Alexander, 2000; Murphy & Wolfenden, 2013) and engage teachers with that pedagogy?

Data collection will be based around 8 case study settings (which could include classrooms, after-school clubs and/or home environments), which will have been identified through the exploratory studies in Strand 1. Whilst the focus is on new pedagogic practices in primary schools, in order to understand new pupil digital practices it is envisaged that data will need to be collected from non-formal and informal learning contexts as well as from formal ones. The specifics of where and how data will be collected may well thus vary across the eight case studies, and will be tailored to suit the particular setting and practices that are being investigated.

Given our sociocultural stance the emphasis throughout will be on constructing rich pictures that value the multiple perspectives of different actors. We recognise the critical role of dialogue and observation and wherever possible, within the constraints of our resources and ethical considerations, we will use video and audio recordings of practice, complemented by audio recordings of interviews and focus groups, and samples of work. Video and to a lesser extent audio recordings not only provide a rich record of practice, but also enable repeated re-analysis and analysis by multiple researchers, both of which help to address (unavoidable) issues of selective perception and researcher bias. We will use new data collection techniques, such as ‘live audio transcription’ and robot-controlled video/audio recording of practice, if they prove effective during the exploratory studies.

As with the exploratory studies, we anticipate collecting data from multiple sources, including senior leaders in schools, teachers, pupils, and parents.

Each of the case studies will involve 15 days of researcher time. This includes time for staff development to ensure consistency between researchers, preparation and liaison with schools, data collection, data analysis and reporting. Resource has been allocated to pay for two days of supply cover per case study, and for two nights’ accommodation for researchers per case study in order to facilitate collecting data outside school hours (for example in a pupil’s home). The central team has time allocated for supporting the individual researchers and carrying out the meta-analyses that will enable us to address the research questions. Prof. Passey and Prof. Murphy also have time allocated to support the meta-analysis.

Click here to access all the documents related to the Case Study data collection, including an overview of the data collection methods, consent forms, information forms for participants, guides for researchers, etc..

Strand 3 Dialogue and dissemination

The focus of this strand is to:

  • Extend our understanding of emerging pedagogical practices related to mobile device use in primary school communities.
  • Act as a sounding board for our work – including challenging our thinking and critiquing our emerging dimensions of pedagogical practice.
  • Maximise the reach and impact of the project.

This strand will involve dialogues with practitioners judged by the project team to be at the leading edge of digital technology use in education, taking advantage of the extensive existing networks that members of the team have.

Key activities within this strand will include:

  • Face-to-face attendance at symposia, workshops and conferences where we can share our findings.
  • Synchronous online discussions with international experts/leading practitioners (using Google Hangouts, Skype and/or the OU’s video conferencing system).
  • Asynchronous online discussions within (there is a threaded discussion linked to every page in
  • Collaborative authoring within (which is a wiki that we will encourage stakeholders to engage with).
  • A minimum of one face-to-face symposium at which leading experts in the field (including members of the team) will explore issues related to the research questions.
  • Publication of concise exploratory study and case study reports.
  • At least one open access refereed journal article (fees for which have been included in the budget), plus other refereed publications.
  • Topical articles written for a non-specialist audience in online newspapers, magazines and blogs, such as The Conversation (e.g.; MJO (e.g.; or Computer Weekly (e.g.
Theory Literature Methodology Frameworks Findings Significance
NP3 homepage NP3 findings For policy makers For teachers & school leaders For researchers Have your say

Find out about our new project and how you could take part