The paper “Turn Your Mobile Into the Ball: Rendering Live Football Game Using Vibration” is a very good example of design research which provides a full picture of how the idea could be presented and how the results should be evaluated. Researchers test their experimental findings based on vibration and video analysis in order to show how to work with new techniques. There is also an efficient work with focus groups. Personally, I believe this type of concept prototype is very effective, moreover, the paper itself provides the answers to these questions:
How can media technologies be evaluated?
Following the article we can point out some steps of evaluating of the new media technologies, such as testing its usability, by actual involvement of the potential users in the process:
1) Effectiveness, which means that the technology is working successfully with the specified system;
2) Efficiency, which means that there is not much effort needed to work with the technology;
3) User satisfaction, which means that the technology is accepted by its users as comfortable.
At the same time it is important for the researchers to organise this kind of testing carefully, to think about the motivations and the right choice of participants. In this example it was important to remember that the analysis showed that the testing had a significant effect on the effectiveness, but not on efficiency in the usability evaluation, so when the participants moved from the first experiment to the second one, they were more experienced in this task, theretofore the results were less representative.
What are characteristics and limitations of prototypes?
A prototype is not fully implemented or tested, it is sample of a product, built to test a concept or an idea, and evaluate the product throughout the testing process. This quality can be considered as both strength and limitation, as it is something to work on, which is not finished or a standard, but this is a way to prevent mistakes and work on a better design of the product.
A prototype can be very expensive to produce and still would not make a good representation of the final product, which is time and resources consuming. In the result, prototypes sometimes are quite cheap to build, but this mean a low fidelity of this prototype. It is difficult to create prototype with all the possible functionality on the early stages of work, so in some cases it is a 'time versus money' issue.
I found an article “An alternate reality game for language learning: ARGuing for multilingual motivation” in the journal “Computers & Education” (IP = 2.621).
ARGuing for multilingual motivation in Web 2.0 is a profound, high quality media technology research that discusses how the design and evaluation of an alternate reality game (ARG) can motivate and engage young people in language learning.
How is theory being used to guide the design process?
Researchers used some original theories, describing what is the ARG and discussing its educational, and even collective value. Theoretical work is based on features of ARGs: development of the game, a game “look-and-feel”, effectiveness of the usage by both students and teachers.
Which research method or methods are used in the paper?
To accompany the use of the Alternate Reality Game (ARG), a methodology has being developed within WP3 ‘Develop Methodology’. It was discussed during the conference and the evaluation within specialists and the selected users, so basically the quantitative methods used here are the case study, questionnaires and interviews, while qualitative methods were helping during the evaluation process.
Which are the benefits and limitations of using these methods?
Case study as a form of observation is well-detailed and gives a good impression of the product. As various questionnaires were used at different stages of the development, it was important to make a right sampling. Large-scale pilot of the game was tested across Europe with 328 students and 95 teachers from 28 schools across 17 European countries. The survey describes the participants and their thoughts on the product, but, in my opinion, the form of а diary would work better here. Answering questions, users give their opinion, but do not describe their full experience of a work with educational program on a daily basis.
What did you learn about conducting design research from reading the paper?
This paper gives a deep inside overview at the work of the game developers and helps to understand that the evaluation process and constant feedback are vitally important in design research. Also, to make a user analysis and evaluate results of experiments, mixed methods of research should be used: both qualitative and quantitative, in order to support concepts and confirm hypotheses.
