The paper “Turn Your Mobile Into the Ball: Rendering Live Football Game Using Vibration” is a very good example of design research which provides a full picture of how the idea could be presented and how the results should be evaluated. Researchers test their experimental findings based on vibration and video analysis in order to show how to work with new techniques. There is also an efficient work with focus groups. Personally, I believe this type of concept prototype is very effective, moreover, the paper itself provides the answers to these questions:
How can media technologies be evaluated?
Following the article we can point out some steps of evaluating of the new media technologies, such as testing its usability, by actual involvement of the potential users in the process:
1) Effectiveness, which means that the technology is working successfully with the specified system;
2) Efficiency, which means that there is not much effort needed to work with the technology;
3) User satisfaction, which means that the technology is accepted by its users as comfortable.
At the same time it is important for the researchers to organise this kind of testing carefully, to think about the motivations and the right choice of participants. In this example it was important to remember that the analysis showed that the testing had a significant effect on the effectiveness, but not on efficiency in the usability evaluation, so when the participants moved from the first experiment to the second one, they were more experienced in this task, theretofore the results were less representative.
What are characteristics and limitations of prototypes?
A prototype is not fully implemented or tested, it is sample of a product, built to test a concept or an idea, and evaluate the product throughout the testing process. This quality can be considered as both strength and limitation, as it is something to work on, which is not finished or a standard, but this is a way to prevent mistakes and work on a better design of the product.
A prototype can be very expensive to produce and still would not make a good representation of the final product, which is time and resources consuming. In the result, prototypes sometimes are quite cheap to build, but this mean a low fidelity of this prototype. It is difficult to create prototype with all the possible functionality on the early stages of work, so in some cases it is a 'time versus money' issue.
I found an article “An alternate reality game for language learning: ARGuing for multilingual motivation” in the journal “Computers & Education” (IP = 2.621).
ARGuing for multilingual motivation in Web 2.0 is a profound, high quality media technology research that discusses how the design and evaluation of an alternate reality game (ARG) can motivate and engage young people in language learning.
How is theory being used to guide the design process?
Researchers used some original theories, describing what is the ARG and discussing its educational, and even collective value. Theoretical work is based on features of ARGs: development of the game, a game “look-and-feel”, effectiveness of the usage by both students and teachers.
Which research method or methods are used in the paper?
To accompany the use of the Alternate Reality Game (ARG), a methodology has being developed within WP3 ‘Develop Methodology’. It was discussed during the conference and the evaluation within specialists and the selected users, so basically the quantitative methods used here are the case study, questionnaires and interviews, while qualitative methods were helping during the evaluation process.
Which are the benefits and limitations of using these methods?
Case study as a form of observation is well-detailed and gives a good impression of the product. As various questionnaires were used at different stages of the development, it was important to make a right sampling. Large-scale pilot of the game was tested across Europe with 328 students and 95 teachers from 28 schools across 17 European countries. The survey describes the participants and their thoughts on the product, but, in my opinion, the form of а diary would work better here. Answering questions, users give their opinion, but do not describe their full experience of a work with educational program on a daily basis.
What did you learn about conducting design research from reading the paper?
This paper gives a deep inside overview at the work of the game developers and helps to understand that the evaluation process and constant feedback are vitally important in design research. Also, to make a user analysis and evaluate results of experiments, mixed methods of research should be used: both qualitative and quantitative, in order to support concepts and confirm hypotheses.
Friday, November 23, 2012
Thursday, November 22, 2012
Fourth Week, Reflections
Theory
This week we focused on qualitative methods of research and attended a lecture by Ylva Fernaeus, during which she talked about the paper “Comics, Robots, Fashion and Programming: outlining the concept of actDresses” which was written by her and M. Jacobsson.
Basically, this article can be considered as a research in tangible interaction: it is a study about the semiotics usage. As we know, semiotic discusses signs and sign systems - in the research signs were used to personalize robots and make the work with them easier, by putting the garments on robots and analyzing how this relates to human behavior.
There was an interesting point to me during the lecture when we discussed the nature of empirical knowledge, as it is difficult to say if the survey with actDresses can be considered as a research with empirical methods or not. Authors have a hypothesis that robots can obtain some psychological characteristics, as people tend to personalize their robots (aka laptops, mobile devices, gadgets) - at the same time there is no proofs, or actual samplings, so it is a purely theoretically designed research, that should be discussed and taken further.
During the second part of the lecture we reflected on the content of this article, answering questions from the lecturer, such us “How statistics could help make this paper better?” etc. At the same time it was great to feel the difference between the research and corporate product designs, as Ylva also told us about this distinction: while the research product design means sharing the knowledge with others, creating the Grounded theory (BTW it is a very interesting site if you looking for some help in your research work: Grounded Theory Online), the corporate product design meansinventing products without any knowledge being shared.
Practice
We continued to edit the Wiki pages and study different research methods. I found our discussion during the seminar very interesting and funny, because everyone has something to reflect on in their chosen papers. This practice also helped me to categorize the methods clearly in my mind.
We also decided to look at advantages of various qualitative methods more than on its disadvantages, as the last part was done quite properly already, so we worked on the former.
Personally I found this seminar helpful also because it introduced me to the triangulation technique which I was going to use in my research without knowing a name for it (a good description is here: The use of Triangulation in Social Sciences Research, Journal of Comparative Social Work 2009/1), and also the open coding as a qualitative method which can be used to analyze textual content.
The main result of the week: I know that qualitative research methods look subjective sometimes, because the conclusions made on its basis depend on human interpretation, so this is the reason to use various types of research to prove your findings; and I also found a book which helped me to categorize all the methods and gave the tools to imply them, maybe it could be helpful for you, too, even if it says it is for librarians only: Basic Research Methods for Librarians, Powell, Ronald R. (1997).
This week we focused on qualitative methods of research and attended a lecture by Ylva Fernaeus, during which she talked about the paper “Comics, Robots, Fashion and Programming: outlining the concept of actDresses” which was written by her and M. Jacobsson.
Basically, this article can be considered as a research in tangible interaction: it is a study about the semiotics usage. As we know, semiotic discusses signs and sign systems - in the research signs were used to personalize robots and make the work with them easier, by putting the garments on robots and analyzing how this relates to human behavior.
There was an interesting point to me during the lecture when we discussed the nature of empirical knowledge, as it is difficult to say if the survey with actDresses can be considered as a research with empirical methods or not. Authors have a hypothesis that robots can obtain some psychological characteristics, as people tend to personalize their robots (aka laptops, mobile devices, gadgets) - at the same time there is no proofs, or actual samplings, so it is a purely theoretically designed research, that should be discussed and taken further.
During the second part of the lecture we reflected on the content of this article, answering questions from the lecturer, such us “How statistics could help make this paper better?” etc. At the same time it was great to feel the difference between the research and corporate product designs, as Ylva also told us about this distinction: while the research product design means sharing the knowledge with others, creating the Grounded theory (BTW it is a very interesting site if you looking for some help in your research work: Grounded Theory Online), the corporate product design meansinventing products without any knowledge being shared.
Practice
We continued to edit the Wiki pages and study different research methods. I found our discussion during the seminar very interesting and funny, because everyone has something to reflect on in their chosen papers. This practice also helped me to categorize the methods clearly in my mind.
We also decided to look at advantages of various qualitative methods more than on its disadvantages, as the last part was done quite properly already, so we worked on the former.
Personally I found this seminar helpful also because it introduced me to the triangulation technique which I was going to use in my research without knowing a name for it (a good description is here: The use of Triangulation in Social Sciences Research, Journal of Comparative Social Work 2009/1), and also the open coding as a qualitative method which can be used to analyze textual content.
The main result of the week: I know that qualitative research methods look subjective sometimes, because the conclusions made on its basis depend on human interpretation, so this is the reason to use various types of research to prove your findings; and I also found a book which helped me to categorize all the methods and gave the tools to imply them, maybe it could be helpful for you, too, even if it says it is for librarians only: Basic Research Methods for Librarians, Powell, Ronald R. (1997).
Thursday, November 15, 2012
Theme 4
I selected a media technology research paper "Social news, citizen journalism and democracy" from "New Media and Society" (2011 Impact Factor: 1.394). Its author Luke Goode uses qualitative methods in a good, and also most traditional way, I believe: in a form of the case study.
After giving the detailed information and discussing some theories on the subject of citizen journalism and new media author moves to the practical examples. He supports his theories with the cases of three chosen Web 2.0 Social News Websites. Luke Goode gives a detailed description of these websites: Digg.com, Guerilla News Network, Newsvine. He provides a historical overview for each site, describes its functions and audiences.
Personally, I find the case study very illustrative and easy to analyse. It gives comprehensive data and therefore helps to make more accurate hypotheses.
Still, the important difference from the quantitative method of research is that data here cannot be random. The researcher chooses the object, which is usually unique, or the most outstanding. It makes the observations limited. Also, qualitative method can look subjective, as the researcher works with the examples of his choice, it makes this analysis look a bit less scientific, but still qualitative method is perfect for the analysis in philosophy, psychology, sociology etc., because it helps to see the dynamics, a time-line of the event or phenomena, as in this paper.
As to the methodological problems of this study, I am positive that the author could use more examples and more types of research methods also, such as interviewing and participant observation. Nevertheless, Luke Goode proposes the usage of various methods when talking about further research, hopefully they will include more personal involvement into the subject.
The question of the article «Comics, Robots, Fashion and Programming: outlining the concept of actDresses» by Fernaeus and Jacobsson actDresses is about the usage of actual physical clothing, labels, and accessories in order to control machines. The term for this symbols is «actDresses», and it helps to personalize digital devices, as humans like to anthropomorphize objects.
ActDresses are meant to represent and produce perceivable actions in the computerised system, being shown in the immediate physical context of the object having the feature represented by the sign. There are two main examples in the paper: fashion style and comics can be used as an inspiration for the design of visual programming languages, and these clothing and labels can help to control the behaviour of digital devices.
Having a background in fashion and also in psychology, I can see the point of this innovation: it is entertaining, makes it easier to memorize and act accordingly to the functions of the machines. I also can see a potential marketing profit. Personally, I found this research quite exciting and psychologically helpful, as it is always great to have anthropomorphic devices, but I also can understand scientists who would find it a waste of time and resources.
Question: How to make qualitative methods of research maximally objective?
Question: How to make qualitative methods of research maximally objective?
Wednesday, November 14, 2012
Third Week, Reflections
Theory
It was very interesting to listen to a person whose article you have read two days ago, and understand some points that were left unclear during the reading.
Dr. Martha Cleveland-Innes told us about quantitative methods and her actual practice of using them in her research of online and distance learning. She explained how she used quantitative methods in her analysis of emotions and their impact on the learning process. She also pointed out that online education is more effective than "offline" learning, but, according to her, the most useful type of studying is a blended environment that mixes both of face to face and online activities.
Personally, I still find it impossible to research emotions using such criterias as gender and age, as it is a deeply psychological question, but now I understand how this research process works.
On example of this research, among others, Martha gave us a good picture of situations when we should use a quantitative method, and what is the difference between using a quantitative and a qualitative method.
She explained how to do sampling and made the "random sampling" principle very clear. The more random is data, the more chances to increase an experimental noise, which leads to new results of research, if correct analysis and understanding of data were applied, of course.
Mixed method of research is more demanding, but makes it possible combine data from different sources, and gives more precise results at the end, still it is difficult to analyse all the data, make correct questions and hypothesis.
It was nice to have a dialogue with a lecturer, and get a positive response, discussing such points as what questions should be asked when deciding what method to use, or logical principles of analysis, deduction or induction.
Dr. Martha Cleveland-Innes recommended us the book, "The Handbook of Emergent Technologies in Social Research" by Sharlene Nagy Hesse-Biber, which I find really helpful. It is very useful for our group project already, as it discusses the benefits and misuses of new technologies usage in the research process.
Martha shared her thoughts on this topic: she is sure that we should use modern technologies in order to get data, and I am totally agree, so it is very good to be able to see now also the disadvantages of this method of data gathering.
Practice
It was very interesting to listen to a person whose article you have read two days ago, and understand some points that were left unclear during the reading.
Dr. Martha Cleveland-Innes told us about quantitative methods and her actual practice of using them in her research of online and distance learning. She explained how she used quantitative methods in her analysis of emotions and their impact on the learning process. She also pointed out that online education is more effective than "offline" learning, but, according to her, the most useful type of studying is a blended environment that mixes both of face to face and online activities.
Personally, I still find it impossible to research emotions using such criterias as gender and age, as it is a deeply psychological question, but now I understand how this research process works.
On example of this research, among others, Martha gave us a good picture of situations when we should use a quantitative method, and what is the difference between using a quantitative and a qualitative method.
She explained how to do sampling and made the "random sampling" principle very clear. The more random is data, the more chances to increase an experimental noise, which leads to new results of research, if correct analysis and understanding of data were applied, of course.
Mixed method of research is more demanding, but makes it possible combine data from different sources, and gives more precise results at the end, still it is difficult to analyse all the data, make correct questions and hypothesis.
It was nice to have a dialogue with a lecturer, and get a positive response, discussing such points as what questions should be asked when deciding what method to use, or logical principles of analysis, deduction or induction.
Dr. Martha Cleveland-Innes recommended us the book, "The Handbook of Emergent Technologies in Social Research" by Sharlene Nagy Hesse-Biber, which I find really helpful. It is very useful for our group project already, as it discusses the benefits and misuses of new technologies usage in the research process.
Martha shared her thoughts on this topic: she is sure that we should use modern technologies in order to get data, and I am totally agree, so it is very good to be able to see now also the disadvantages of this method of data gathering.
Practice
Ester Appelgren introduced us to SPSS (Statistical Package for the Social Sciences) program, which is very useful for the survey analysis, and helps to visualize quantitative research methods. Previously I used Excel only, so the differences are obvious.
During the lab I built one multiple response variable, using the data on the subject of radio listening, another one was a split variable, differentiated between gender and age groups.
It was really easy to get the results (when SPSS was working, which was - not always) and to analyse this variable data in a graphic/table view. For example, I learned that podcast radio in Sweden is mostly popular among male teenagers - SPSS made this data literally illustrative.
Friday, November 9, 2012
Theme 3
Both articles are very helpful in understanding of how the quantitative methods of research work, and, even more interesting to me, both discuss the topic of online education, which is becoming very popular and important nowadays.
“Mixed research and online learning: Strategies for improvement” by Lowenthal, P. R. & Leech, N. (2009)
The main question of this paper is how a low quality of research on online learning can be improved. This introduces the mixed research in Data Analysis, which is a new, multiple method of collecting data.
The purpose of study is exploratory, as researchers are trying to bring up the new method into actual usage, and explain its elements.
Basically, mixed research is a research strategy that combines both quantitative and qualitative analysis in order to create a more valid way to perform a research. This may result in better ways of providing online-learning, which currently has a lot of disadvantages comparing to traditional way of learning, as the authors point out.
Speaking about qualitative analysis, it should be verified by comparative analysis with such methods as counting of the number of words, keywords, content, domain analysis, taxonomic and component analysis, social presence etc.
There are some tools, or collecting methods, that they discuss: research questions (both quantitative and qualitative), samples etc. There are also various lists of examples of how a researcher should perform the analysis. For example, the most interesting for me are Seven stages of the data analysis process: data reduction, data display, data transformation, data correlation, data consolidation, data comparison, and data integratio.
Authors say that mixed studies on the subject are difficult to perform, nevertheless, using both quantitative and qualitative methods of data analysis would be highly effective.
I agree that this way of research work can definitely increase the quality of research, and not only regarding the subject of online learning. In this particular case it would make the process of research highly productive. I think that this method is a good way to combine the best parts of qualitative and quantitative analysis. Still, sometimes it could be difficult to analyze both quantitative and qualitative data, and also the usage of multiple methods will make the research process longer, but its results will be more valuable.
“Emotional presence, learning and the online learning environment” by Cleveland-Innes, M. & Campbell P.
The main question of this paper is the presence of emotion in online learning.
The purpose of study is explanatory, as authors discuss if emotions influence on the process of online learning.
They use mixed methods of research: the quantitative data is collected from surveys and interviews (conference discussion transcripts), the qualitative data is taken from the questions in the survey (217 students of distance education programmes).
The conclusion is that emotion is a prominent experience in the online environment that may be distracting, but also may support ideas, be stimulating, if managed correctly. This is teachers’ responsibility: «Consideration of the appropriate response is needed [...] Instructors can model emotional response […] and teach how to bring emotion to consciousness and make use of the emotional state in a learning situation».
Researchers still left many questions for further discussions, like what is emotions influence on organization and direct instruction, or are where some specific emotions experienced during triggering events etc.
Nevertheless, they ended up with interesting and productive results, and, in my opinion, this goal was achieved due to the illustrative information received from the mixed methods of research.
Wednesday, November 7, 2012
Second Week, Reflections
Theory
This week we studied the theory on the subject of... theories. It was quite interesting to analyze what is theory and give it a proper definition. Doing this, we used the article "What Theory is Not" (Sutton, Staw), which reminded me of another work of this format, "The nature of explanation" (K. Craik) that has been written in 1943. Surprisingly for me, there were the same ideas in the both articles, that leads to the conclusion that this subject is still could be discussed, 60 years later.
The most important point for me was to remember that any science paper with references and diagrams, hypothesis and various data is not theoretical if there is no hypothesis that can be tested - it is an easy scheme to use while looking for the information for my master thesis and other work.
Also, theory is an important philosophical term, used since times of Plato and Aristotle. It is a human tool used in order to understand and systematize our knowledge about the world. It doesn't exist by itself, as we pointed out during our group discussion.
Five types of theories can help in describing some scientific papers due to providing structured templates of parameters for every kind of theory to include. At the same time, sometimes it is impossible to choose the only one theory, or discuss its types at all, as we learned during the seminar. Personally, I consider Gregor's work as a good mind exercise.
Practice
We discussed various theories in order to find the paper with actual theoretical concept, and have learned (and googled) a lot of theories in the end, such as Social capital theory, Game theory, Social Cognitive theory etc. It was interesting to learn how the hypothesis development works, to see the similarities in the structure of every theory, and try to think about each theory in the case of particular research, and then globally.
The editing work can also be considered as very useful. We had a difficult task: to edit a description, which was already complete. These skills sometimes can be very useful, while working on a voluminous text.
Thursday, November 1, 2012
Theme 2
1)
What is theory?
Theory IS providing explanations with clear argumentation that can be tested.
Theory IS abstract and general text, which answer the question Why? and provide links between ideas, events, structures or things.
Theory’s aims are to describe, explain and predict the future developments based on revealing of the current connections between the cause and the event.
Theory IS NOT subjective opinions, variables, diagrams, listings of references, data, hypotheses – these are not a theory, but it could be used as the kind of reasoning to support the theory.
2)
The research paper I chose is 'Online Communities: Design, Theory, and Practice' from the Journal of Computer-Mediated Communication (IF 2.172). http://jcmc.indiana.edu/vol10/issue4/preece.html
In my opinion, this paper is using theory in an excellent way, and it can be characterized as the first type of theory according Table 2 in Gregor's article: theory of Analysis.
The paper consist of the theory which provides a descriptions of the phenomena of Online Communities, used during the last decades by various researchers, so answers the question ‘what it is’.
There are also the descriptions of various current research trends, used in the field (such as Ethnography, Content and linguistic analysis, Social network analysis, History flow technique, interviews and questionnaires), with the mention that no particular theory or set of theories currently dominates research on online communities: ‘In order to study online communities, researchers have had to adapt methodologies for use online'.
The theory does not extend beyond analysis and description, but only analyses the relationships among these research trends. There are no predictions for the future, and causal relationships among phenomena are not specified.
Theory Component: Instantiation, defining the concept of community
Means of representation: Words
Primary constructs: approaches, methodologies and techniques
Statements of relationship: 'There is the relationship between community involvement and the use of the Internet to support the community'.
Scope: Methodologies that have been proposed in the scholarly literature. A procedure for including new methodologies as they arise is proposed, 'as new and novel practices emerge within the online community environments'.
3)
As to the benefits and limitations of using the selected theory, it is obvious that theory of Analysis is based on the previous research and does not demand for any hypotheses, propositions or prescriptive statements, which makes it clean and indisputable. On the other hand, there is no new knowledge or even causal explanations: it is basically a listing, which should be already familiar for specialists.
What is theory?
Theory IS providing explanations with clear argumentation that can be tested.
Theory IS abstract and general text, which answer the question Why? and provide links between ideas, events, structures or things.
Theory’s aims are to describe, explain and predict the future developments based on revealing of the current connections between the cause and the event.
Theory IS NOT subjective opinions, variables, diagrams, listings of references, data, hypotheses – these are not a theory, but it could be used as the kind of reasoning to support the theory.
2)
The research paper I chose is 'Online Communities: Design, Theory, and Practice' from the Journal of Computer-Mediated Communication (IF 2.172). http://jcmc.indiana.edu/vol10/issue4/preece.html
In my opinion, this paper is using theory in an excellent way, and it can be characterized as the first type of theory according Table 2 in Gregor's article: theory of Analysis.
The paper consist of the theory which provides a descriptions of the phenomena of Online Communities, used during the last decades by various researchers, so answers the question ‘what it is’.
There are also the descriptions of various current research trends, used in the field (such as Ethnography, Content and linguistic analysis, Social network analysis, History flow technique, interviews and questionnaires), with the mention that no particular theory or set of theories currently dominates research on online communities: ‘In order to study online communities, researchers have had to adapt methodologies for use online'.
The theory does not extend beyond analysis and description, but only analyses the relationships among these research trends. There are no predictions for the future, and causal relationships among phenomena are not specified.
Theory Component: Instantiation, defining the concept of community
Means of representation: Words
Primary constructs: approaches, methodologies and techniques
Statements of relationship: 'There is the relationship between community involvement and the use of the Internet to support the community'.
Scope: Methodologies that have been proposed in the scholarly literature. A procedure for including new methodologies as they arise is proposed, 'as new and novel practices emerge within the online community environments'.
3)
As to the benefits and limitations of using the selected theory, it is obvious that theory of Analysis is based on the previous research and does not demand for any hypotheses, propositions or prescriptive statements, which makes it clean and indisputable. On the other hand, there is no new knowledge or even causal explanations: it is basically a listing, which should be already familiar for specialists.
Subscribe to:
Posts (Atom)