The problems with THAT research on mobile devices

The Stranmillis research can be found here.

If my 18 years as a teacher and 6 years as a Principal have taught me anything about the use of ICT in education and in the classroom, it is as follows:

a) Pedagogy must drive technology. For that reason I have obstinately refused as a school leader to go down the route of mass procurement and bulk purchase of I-Pads or other such devices. The quantitative evidence as to the pedagogical expediency and efficacy of such devices is - in my opinion - simply not there yet. One of my (many) concerns about the Stranmillis research is that - very early on - the authors indicate that one of their secondary objectives is 'the impact (of mobile devices) on teaching and pedagogy'. Surely, it should be the other way around.

b) ICT in the classroom is probably at its best when it is used organically from the ground up, when individual classroom teachers identify a use for ICT in their classroom which effectively meets a need in order to scaffold learning 'upwards', thus assisting as one of a range of pedagogical tools in taking pupils to the next level in their learning. This may be something as simple as a thought-provoking piece of audio or visual media, or an approach to low-stakes formative testing which consolidates knowledge and - if intertwined with spacing and interleaving - helps to embed knowledge in long-term memory. What works in one classroom may not work in the classroom two doors down, but each teacher will have their own justifiable reason for using an aspect of ICT to meet a learning and teaching need. ICT will never be a one-size-fits-all medium, and is not a panacea. To be fair to the authors of this research, they do not suggest the latter. However, too often I fear that ICT initiatives are 'shoe-horned' into classrooms without the quantitative evidence to back up their efficacy. As an example, C2K never learned from the doomed Learning NI initiative and instead went on to spend probably a small fortune on the procurement of another miserable VLE which is sold to us teachers as a wonderful pedagogical tool. Perhaps for some it is. Not for me. Indeed, it seems sometimes that teachers are  cajoled / encouraged or - dare I say it - even coerced by the powers that be if we are not using online 'rooms' and 'corridors' to share resources. I baulk at this, and indeed I know one of my teachers who gives out all the necessary resources on a memory stick to pupils. Simply because it is much quicker - and it works best.  Good for her.

Does the Stranmillis research encourage a focus on pedagogy over technology? Does the Stranmillis research include a robust analysis of how ICT can be used to stretch, challenge and instigate the necessary struggle on the part of learners to drive their learning onwards and upwards? Does the Stranmillis research encourage a cautious approach to the procurement and use of mobile devices in the classroom? I suspect not, and here is why...

My main concern with the Stranmillis research is that it is mainly qualitative. It simply astounds me that the authors of this research make sweeping statements in their key findings based on a 'mixed methods' approach, the quantitative part of which only involved the return of 27% of questionnaires from parents of children, a return which they themselves describe as 'disappointing'. Such key findings include:

'The introduction of iPads has had a positive impact on the delivery of the Foundation Stage curriculum and the development of children’s literacy and numeracy skills.'

This conclusion is based on a meagre 27% of parental returns, focus groups and 'semi-structured' (! authors' words, not mine) interviews with Principals and teachers with questions which may be accused by some of being 'leaning' in their approach (to be discussed later). I should also add that the pupils selected for the 'virtual tour' were hand-picked by the teachers - hardly the basis for a randomly controlled trial aimed at heightening unbiased findings! But anyway - back to the 27% of parent returns - the 'quantitative' evidence of this research. The authors note the following:

The response to the survey was extremely poor; yielding 32 (8%) completed and returned questionnaires. Two schools had no returns and others a low return rate which school principals described as “normal for our school”.  To increase the response rate, the survey was redistributed to schools with the lowest response rates. Again, the response rate was disappointing. In total, 110 (27%) questionnaires were returned. Whilst low, an ESRC (2008) briefing paper on survey data collection rates argues that after two distributions, “27% is a reasonable response rate.” It also notes that although there can be no claim that “the survey data is completely unbiased, the level of bias introduced can be considered to be within acceptable levels for this kind of research”

'A reasonable response rate'? Barely acceptable, in my opinion.

Which brings me neatly onto the questions used in the focus groups and interviews. Page 139 of the research findings outlines the questions used in the pupil focus groups. It includes questions such as 'What are your favourite apps...?, Are there any other apps you enjoy using?'...How does the I Pad help you learn?...Would you like to use your I-Pad again next year in school?'. Now, PLEASE, what 6 or 7 year old is not going to answer 'Yes' to the latter question?! In fact, of the fourteen questions in the pupil focus group, only one appears to ask pupils to outline what they might dislike about the I-Pad.

And then there are the teachers. Highly professional and committed, no doubt. But interestingly, one Principal describes them as follows:

'Our staff are young and keen and are very technically-minded. Each teacher was given an iPad and £10 to become familiar with the technology and different apps.'

With the greatest of respect, is this really a demographic - particularly when only based on 5 Primary Schools - with the experience and confidence to produce a valid evaluation of a project in which someone has handed their school I-Pads and funding? Which teacher or school leader wouldn't be influenced by the opportunity to obtain additional funding and resources? As a school leader myself, I totally get that. Indeed, one Principal is quoted as saying:

'We have found that it is particularly useful for literacy because we are very under resourced. You will hear the arguments and the different things in the media which we hear all the time. We do not have enough resources. There aren’t enough reading books, there aren’t enough textbooks. There aren’t enough of anything. So it has filled a huge gap for us'

Again, whilst understandable on the part of the Principal, it hardly inspires confidence that we are talking here about an unbiased, objective and quantitative piece of research.

Conversely, the opinions of Principals and teachers are subjective and based on qualitative observations:

'I think it makes the children aware of their audience and aware of how they talk to each other.'

'I think the children…. We can definitely see an improvement in their mental arithmetic, in their recalling skills.'

'We have used Google Translator and that’s proved really useful. I think that we will use it as a really good additional resource'.

Well, the bit about Google Translator says it all, really. If anything has proven not to be accurate, it is Google Translator! Which brings me to my next point. There is an implicit assumption in this research, including from the authors, that mobile devices will enhance access to information. One Principal says:

'Children really find out a lot through technology these days, much more than adults can, so it was too great an opportunity to miss.'

A further teacher states:

'I also find it really good to keep their attention. Whenever we are searching for information like ‘find me a trapezium’, they can use Google Images, and so many images come up that it really grabs
their attention, and the discussion.'

Are we not missing the point here? Has anyone taught these 6 or 7 year olds that - actually - a lot of information they find on the Internet will not be accurate? Has anyone taught these 6 or 7 year olds information literacy skills? Accuse me of being old-fashioned, but surely the best source of knowledge - and the most reliable  - is not going to be a mobile device, but rather the trusted teacher acting as 'sage on the stage', and not as 'guide on the side'.

And that brings me on to my final point. What frustrates me most about this research is the implicit denunciation of 'traditional teaching', evident in key findings such as:

'Motivation and enthusiasm amongst staff and pupils has increased significantly as a result of the initiative. Children who were previously disengaged when taught using traditional approaches appear more highly motivated when activities are presented on a digital device.'

What exactly do the authors mean here by 'traditional approaches'? We don't know, because they do not elaborate. Well, if they are referring to the emphasis on knowledge transmission over 'discovery learning' in the classroom, and pedagogical pursuits which aim to embed high-quality cultural knowledge in long-term memory so that a strong cognitive framework is established for deep lateral thinking - then give me 'traditional approaches' any day.

And then there is the attempt in the research to equate motivation, enjoyment and 'fun' with effective learning. I was, indeed, disappointed to hear a Principal on TV this week in response to this research stating that 'learning needed to be fun'. Surely, we are doing our children a disservice when we peddle this line. Learning isn't about fun. Learning is about struggle, challenge, striving upwards for the next level. When I go to the gym, I don't find it 'fun', but when after a month I see the benefits, the 'motivation' then comes from the reward of seeing myself progress. Why is there such an emphasis on learning always having to be 'enjoyable' in the first instance? If anything, we need to be instilling in our children the notion that learning is 'rewarding', but only after challenge. Asking pupils in a focus group what apps they might like to use doesn't take stock of the key point that technology has to be one of a range of complementary tools which is driven by the strong pedagogical need to bring children out of their comfort zone and to the next level in their learning.  

In conclusion, the only piece of research I will personally accept into the efficacy of mobile devices will be one which is mainly quantitative, analysing hard data and standardised scores over a significant period of time. And this research must surely by tightly controlled and comparative - analysing the performance and progress of a substantial cohort of pupils who do not have access to mobile devices in the classroom with a cohort which does. Does this research exist yet? I don't know. But what I do know is that it is NOT the Stranmillis College research. And that is where the Stranmillis research falls, because it too readily bases key findings on the qualitative assumptions and observations of practitioners, genuine though they undoubtedly are.