Monday, 28 June 2010

Design Research Issues

There seems to be a tendency among academics to find many different ways of looking at research and thus create numerous categories and approaches to describing their research activities. At the European Conference on Research Methodology for Business and Management Studies at the IE Business School in Madrid last week a distinguished panel of academics discussed the importance of design research (DR) or design science research (DSR). All four speakers argued for the importance of design and it is not easy to disagree with such a proposition. But it was possible to interpret what they said in different ways as the term design can play more than one role in the way we describe research.

There seem to be at least 3 ways design becomes a research issue.

The first is that any research programme which is intended to lead to a degree or a publication needs to be planned and the options adopted in the plan may be referred to as the research design. If a research programme is undertaken without a carefully conceived research design then its success will be a question of serendipity – and sometimes some people are lucky. The meaning of the word design in this context is not problematic.

The second way in which design needs to be considered is described by Peffers et al. (2004) when they argue for the needs for a design science research methodology (DSRM) which they describe as an important “discipline oriented to the creation of successful artifacts (sic).” It is not difficult to see that research aimed at the creation of successful artefacts offers a different set of challenges to what researchers normally face in business and management studies. Hevner (2004) supports the view that design science research is different to information systems research and they call for collaboration between the two approaches.

This seems to be a reasonable use of the word design and it is worth noting that in certain cases the results of this type of research may not lead to a completed artefact but perhaps only to a detailed blueprint for its development. Again there is not cause for confusion here.

The third way the word design is used has roots in the work of Simon (1969) who wrote about the study of natural systems and the study of artificial ones. According to van Aken (2005) and based on the Simon distinction, there are two domains of study which are the explanatory sciences and the design sciences. Explanatory sciences include physics, biology, economics and sociology. Examples of design sciences are engineering and medicine. van Aken argues that there are differences in the core missions of these two groups of knowledge. van Aken argues that “The core mission of an explanatory science is to develop valid knowledge to understand the natural or social world, or – more specifically – to describe, explain and possibly predict. The core mission of a design science, on the other hand, is to develop knowledge that can be used by professionals in the field in question to design solutions to their field problems”.

This distinction is problematic.

I am not sure how useful this distinction between explanatory science and design science really is. Anything which bears the name science should be explanatory and therefore the expression explanatory science does not add any value. The term design science does not do much for us either mostly because the word design can be used in multiple ways as described above.

van Aken (2005) does not like the terms basic and applied sciences and I would agree that there is no room for these words, basic and applied, in the field of business and management studies. Business and management studies is primarily problem solving orientated. But there is more to this issue than just the objective of the paper. Academic research in business and management studies sometimes, if not often, written up in such a way as to be inaccessible to practitioners.

But this is not the main issue. Academic research for the purposes of degrees and/or publishing in peer reviewed journals need to make a contribution to the field of study. Some academics have trouble in defining what this means but it is relatively easily understood when it is pointed out that such research has to add something of value to the body of theoretical knowledge. In addition this research has to be conducted in a scholarly fashion which takes cognisance of what the academic community already knows about this topic. The results of this research will “describe, explain and possibly predict” as van Aken (2005) claims for explanatory research.

For many years this was all that was required of quality academic research. However in the past 10 to 15 years a new dimension has been introduced and that relates to the application of this new knowledge of management and/or business practice (Starkey 2001). To be assured of success in a degree or to have a paper published today the research ideally needs to point out how the addition to the body of theoretical knowledge can be used to solve practical problems. This does not in any way reduce the status of the research. It is not appropriate to regard this type of work as mere consulting because of its practical dimension. When developed in this way these research results “can be used by professionals in the field in question to design solutions to their field problems”. In fact the findings of the research orientated in this way will directly offer a solution to such problems. Of course not every academic in every university will agree with the need for the translation of the new theoretical contribution into practical guidelines for management but there is increasing support for this approach.

It therefore seems to me that the distinction between explanatory and design research is not of much value to researchers in business sand management studies. In fact new research distinctions, processes and methods should be named and recommended with care as they have the potential to complicate and confuse some researchers.

Any comments on the above will be gratefully received.


References

Hevner A S March J Park S Ram, March 2004, Design science in IS Research, MIS Quarterly Vol. 28 No. 1, pp. 75-105

Peffers K T Tuunanen M Rothenberger and S Chatterjee, Winter 2007–8, A Design Science Research Methodology for Information Systems Research, Journal of Management Information Systems, Vol. 24, No. 3, pp. 45–77.

Simon, H. A. (1969), The Sciences of the Artificial, MIT Press, Cambridge MA.

Starkey K, (2001), In defence of Modes One, Two and Three: A Response, British Journal of Management, Vol 12, Special Issue, S77-S80

van Aken Joan, (2005),Management Research as a Design Science: Articulating the Research Products of Mode 2 Knowledge Production in Management, British Journal of Management, Vol. 16, 19–36 (2005)

26 comments:

  1. Agree with you that the distinction between Explanatory Science and Design Science may not be very useful. My (little informed) view is that Management, Economics, Engineering and Medicine are a mixture of art and science - Practitioners use a series of recipies and operating instructions that are based on experience and that are progressively enriched by both scientific observation and practical experience that could not be considered scientific.
    The difficulty of experimental design in the complex systems under study is the main factor that prevents us from really considering the disciplnes as a "pure" science...

    ReplyDelete
  2. Have commented on Kit via a new post.

    ReplyDelete
  3. Dear Professor Remenyi,

    I read your posting and agree with your perspective. While listening to the panel, I was not sure how much consultation occurred between the presenters beforehand on the topic of Research Design, hence the disparity in the descriptions of the concept.

    To be honest, it all started to become a blur until your question arose, as the silence implied agreement with the presentations (or gave appearance as such), while I experienced some dissonance. In my opinion, it would appear that while van Aken (2005) has his heart in the right place, it comes back to the definitions, which distract from the larger concept, which has merit.

    With regards to your comments on Practical Guidelines for Management, I wonder how much of that is really lip service, as in my opinion, the gap
    between academia and management grows wider each year, as researchers tend to make theoretical and methodological choices that result in limited practical application.

    On the other hand, unless the research findings can be implemented within the short-term view of the stock market, they remain largely ignored.

    At some point, it appears that researchers are employing impression
    management, in an attempt to retain relevance in the eyes of managers,
    with limited success. What do you think about this?

    ReplyDelete
  4. There are several important points raised by Gavin’s comment, the first of which is:

    Is the claim correct that in academic research in business and management studies the emphasis on Practical Guidelines for Management is nothing more than lip service and that the gap between academia and management grows wider and wider each year?

    In answering this it is important to say that there is a large degree of variability among academics. Some are scholarly whilst others are not. Some are attentive to the needs of their research degree candidates and some aren’t. Some have accepted the need to ensure that academic research in business and management studies is relevant and some have not. There are substantial variations in approach to research between the different sides of the Atlantic. There are differences between continental Europe and the British Isles and what we call Commonwealth countries.

    My sense is that there is a growing number of academics who have realised the need for this relevance. The issue of relevance has been at least mentioned in journals across the world. Everyone who has read and has been in any way influenced by the ideas behind Mode 2 research will add to this movement. I know that not everyone is yet convinced and even some of those who are aware of the importance of relevance find it easier to direct their doctoral candidates to neatly defined research questions with little relevance. It can be an easy way out. I was more than a little surprised that a distinguished member of the Design Panel in Madrid did not know the term Mode 2. There is no value in my guessing why he didn’t know about this and all I can say that I hope he has taken the idea on board now.

    Some people say that for relevance to become the issue it should be, we have to wait for the new generation of researchers to take over the influential positions in the world of academic research in business and management studies. I am more optimistic than that. So I encourage everyone I talk to about research to emphasise the relevance of their research findings and how they can improve management performance.

    With regards the next issue Academic Panels at Conferences seldom meet before the event and thus the panel often does not have a coherent line of argument. One of the advantages of panels is to hear different and sometime conflicting views on a particular topic. The hope is that conflicting views might produce some sort of dialectic affect.

    Then I fear that what you called “impression management” is all too common in every aspect of our lives today. The politicians are probably the most active in this area but at least we all expect them to deliver their so-called spin when they make a statement.

    Finally I think that there is general agreement among those who do not work for an organisation related to the stock market that the short-termism of the market is not satisfactory. I don’t know what would have to happen for us to be able to ignore this unfortunate effect of the way business is run.

    ReplyDelete
  5. Learning about writing and editing a blog is quite a challenge in its own right.
    My next piece was too long for this system so it is being posted in 2 Parts.

    ReplyDelete
  6. I have been somewhat optimistic about the system acceting the material in 2 parts. I think that it will be in several parts

    ReplyDelete
  7. The research process - Is a generic understating or model of academic research useful?
    Part 1

    I was surprised to hear a few months ago that one of my colleagues had a very energetic debate at a doctoral examination on the question of what was the essence of positivistic research. My colleague is a well established academic but the doctoral degree candidate was no pushover and as I understand it the argument was a draw or a victory on points to the senior academic. When I reflected on this event I realised that there is quite a lot of misunderstanding about positivism and how it differs from other academic research paradigms. There was an elderly professor at the institution from which I obtained my doctorate who always maintained that he never liked the word positivism and that the term quantitative research was good enough for him. Today I am surprised that he was not challenged by his somewhat superficial understanding of what positivism is about.
    When I looked for a definition of positive I was again surprised to find a number of unsatisfactory definitions. One of the least valuable definitions was that positivism is, “a philosophical system founded by Auguste Comte, concerned with positive facts and phenomena, and excluding speculation upon ultimate causes or origins .” It is of course of no consequence who founded positivism, if it is at all possible to say that anyone founded it (and I doubt that attributing this approach to Comte alone is actually viable). When I looked further afield I found descriptions of positivism which claimed that it is a set of epistemological perspectives and philosophies of science which can be traced back to the ancient Greeks and which follow the scientific method. Oh dear! What use is this description to a researcher trying to find his or her way through the research methodological puzzle?
    So I dug a bit deeper and found an interesting presentation on positivism which said that positivism means being objective, unbiased, independent, disinterested, value free, measurable, generalisable, scientific i.e. it will occur in a laboratory. Positivists are generally seeking knowledge which will allow them to predict and control whatever the researcher is studying. Interestingly we can see from this how the elderly professor I referred to in the first paragraph was confused as he focused on the word measurable.
    In contrast to positivistic research the term interpretivistic research is sometimes used. A synonym for this is qualitative research and this is seen to be less objective, less value free, often dealing with non-measurable issues and not generalisable. Positivists can be very disparaging if not downright insulting about qualitative or interpretivistic research.

    ReplyDelete
  8. The research process - Is a generic understating or model of academic research useful?
    Part 2

    The starting point in coming to terms with the nature of positivism is to take a closer look at how we classify the different types of empirical research. In general we have distinguished between two different types of empiricism. We have said that empirical research can be either positivist or interpretivist and we have built up descriptions of these two different types of research. The description of positivism above is not atypical of what is often said about this approach. But it really isn’t satisfactory.
    Maybe the first issue to address in acquiring a fuller understanding is that taxonomies are often problematic. Once we create a framework which is designed to help us understand the issues we are working with we tend to want to fit everything we encounter in that field of study into one of the boxes in the taxonomy. The taxonomic units cry out to be used. Therefore when we look at a piece of research or a research proposal, for that matter, we are inclined to say, This is positivistic or This is interpretivist. Such a statement suggests that these concepts are entirely different and that there is nothing which is in between. This is where the problem lies. It is much more helpful to understand research as a generic process which may employ numeric data or it may employ other types of data . Research which focuses on numeric data will often use mathematical or statistical modelling while research which does not will often employ conceptual modelling. The notion of a generic set of processes which drive academic research is somewhat controversial. Some scientists believe that every research project is so intrinsically different that any attempt to formulate a generic understanding of research has to be too high level to be of any value. While I understand where such thinking comes from, I am of the view that it somewhat overstates the uniqueness of research projects. I recently found a comment by Deutscher (2006) which reinforced the need to see commonalities rather differences. He said
    …the ability to pick out patterns, to draw analogies between unequal yet similar things, in short, ‘to forget a difference’, is at the very core of our intelligence.
    This is the basis of all generic model development and it is appropriate to emphasise that (Davenport 1994) said:-
    The more complex (an information) a model, the less useful it will be
    Before exploring some of the ideas of a generic research model there is one critical point to be established. No matter what type of research is undertaken there will also come a point when the results of the experiment, the results of the survey, the interviews, the participant-observer experience etc will have to be evaluated. Data either in its original form or after it has been processed or modelled does not speak for itself. Its meaning has to be explained or interpreted. Thus all research of any type needs interpreting. Some authors speak of there being interpreting with a small “I” and Interpreting with a big “I”. The small “i” refers to the interpretation of all research findings while the big “I” refers to the category of research which is said to contrast with positivism. This way of expressing the different issues relating to interpretation is not one which I favour.

    ReplyDelete
  9. The research process - Is a generic understating or model of academic research useful?
    Part 3

    Looking at research through the generic lens I can avoid using the word quantitative or qualitative or interpretivist or positivistic research. I take the position that academic research of any kind has to comply with a number of characteristics and provided it so does so it will be acceptable and there is no need to agonise over whether it is positivistic or otherwise.
    The five generic issues in academic research are:-
    1. Establish a suitable research problem and question which is underpinned by some theory or which has being shown to have reference theory implications. The theoretic connection of the research questions is important. Failing to identify a theoretical connection, a Grounded Theory Method approach will have to be taken and this will be discussed elsewhere. It is important that the research question is relevant and this issue should be clearly demonstrable to all the stakeholders of the research. Some research questions may require a much more in depth enquiry than others and the specific question largely determines the course of the research. The research question needs to be checked to ensure that it is viable in terms of the scope of the research project especially in regards to its time frame and the resources required. It is hard to over state the importance of the research question as it is the cornerstone of any research project and thus the determinant of all the other activities in the research project. In practice there is a tendency for researchers to want to take on questions which are too difficult and there needs to be vigilance that this does not happen.
    2. Create a research design. The first step to be addressed is how can the research question be answered? The key issues here are what sort of evidence could provide a credible answer to the question. The type of evidence or data will determine many of the subsequent matters in the research design. The research design has to outline how the data required will be obtained, from whom it will be obtained and when it will be obtained. Often more than one type of data will be necessary and thought has to be given as to how different data types may be used to answer a research question. The question of from whom the data will be collected is a difficult one. Obtaining access to data is often problematic and researchers need to be prepared for the challenges ahead as the quality of the data is a fundamental concern when evaluating the rigor of the research. A clear plan is required for the capture of the data, its management and for its processing with whatever software is considered appropriate. If qualitative data is being used then there may be little or no requirement for this and if this is the case then a statement relating to whatever hermeneutic approach is to be taken is required. Of course, there are several sub-issues involved with this stage of the research. If the research is deductive and thus uses hypothesis testing then the data processing and modelling will draw on mathematical techniques. The central challenge will be to reject hypotheses and by so doing or not being able to so do, theory is developed. Underpinning this approach there will often be the assumption that a cause and effect relationship can be ascertained. If the research on the other hand is inductive then there will be a greater tendency to look for original patterns and from these to create a theoretical explanation as to how the researched issues interrelate. Some research questions require a cycle of deduction-induction as new theory is generated. This research design will need to address the issues required for an acceptable ethics protocol.

    ReplyDelete
  10. The research process - Is a generic understating or model of academic research useful?
    Part 4

    Having settled the design the next step is to conduct the research. This involves the hard work of data collection, data preparation, and data management. Different types of data will offer different challenges and the researcher has to be cognoscente of these requirements. When data of adequate quality and quantity has been obtained then the process of analysis may begin. Various analytical techniques will be used to understand the nature of the subject being studied. The purpose of the data analysis maybe to test hypotheses or it may be to find patterns or both. This phase of the research will always be technical and the type of work required will be a function of the type or types of the data collected. Data modelling will always be involved and this can range from formal statistical testing to the use of conceptual or pictorial models. The meticulous application of the required techniques is a rigour issue which requires careful attention. The result of the work from this stage of the research process is often referred to as the findings.

    As mentioned above the product of data modelling or concept presentation does not speak for itself. Therefore there needs to be some thought given as to how the results of the research will be understood. This is often thought of as the most creative part of the research as it requires the application of imagination as to what the results actually mean. This is where interpretation becomes a central issue. It should not be expected that there will only be one possible interpretation of the findings. There may be many, especially where the research has been inductive. Neither should it be thought that there will be an obviously superior interpretation. Many interpretations may be equally convincing. The task of the researcher is then to argue that the finally chosen interpretation has some comparative advantage over the others. This can be challenging and therefore considerable though/reflection should be given to this task.

    The final generic issue is that in the field of business and management studies there is a need to be able to suggest how the findings may be of some practical use or value. This is another test of relevance which is particular to this field of study.
    The five issues listed above may be regarded as a generic model of academic research in the field of business and management studies and as a model it is not intended to address every aspect of the research process. If I were to attempt to incorporate all the issues which needs to be addressed I would have to write a much more detailed account than this – possibly a book.
    Like all models the test of applicability is not its completeness but rather its usefulness. As Barney (1994) pointed out:-
    Models and metaphors are simplifications and abstractions of reality.
    But without models the world can be a very confusing place and research methodology in particular can seem very much a dark art. Using the above model/ approach to understand the processes of research throws light on what is required from the beginning to the end of any piece of academic research. The issues addressed above apply equally well to a research degree as they do to a research paper, the difference is the depth the researcher has to go.
    One of the secondary outcomes of using such a model is that it allows me to stand back from using the taxonomy of positivistic or interpretivistic research so that I don’t have to struggle or quibble about definitions such as positivism or interpretivism.
    I would of course be grateful for any feedback on the proposed generic model and the approach of finding generic patters in research in general.

    ReplyDelete
  11. The research process - Is a generic understating or model of academic research useful?
    Part 5

    References
    Barney J, (1994), Beyond Individual Metaphors in Understanding How Firms Behave: A Comment on Game Theory and Prospect Theory Models of Firm Behavior, cited in Fundamental Issues in Strategy - A research Agenda, Ed Rumelt R, Schendel, D and Teece D, HBS Press, p 55-69, Boston,MA
    Davenport T , (1997), Information Ecology. Pg. 5. Oxford University Press, NY, NY.
    Deutscher G, (2005), The Unfolding of Language: An Evolutionary Tour of Mankind's Greatest Invention, Metropolitan Books, NY
    Gould SJ, (1988), The Mismeasure of Man, p21, Penguin Books, London

    ReplyDelete
  12. a terrific topic you got..

    ReplyDelete
  13. i would really love to have this topic as my dessertation...

    bestessays.com review

    thanks,
    badloi

    ReplyDelete
  14. thanx sir this article is much more valubale for me and my research.

    ReplyDelete
  15. Well,I must agree that this seems to be a reasonable use of the word design and it is worth noting that in certain cases the results of this type of research may not lead to a completed artefact but perhaps only to a detailed blueprint for its development,so informative blog.

    legal translation services

    ReplyDelete
  16. Dear Professor Remenyi,

    Have you encountered McCloskey's brilliant and witty "Why I am no longer a positivist"? (Review of Social Economy, 47(3), 225-238, 1989).

    There is a copy at her website, to summarise it would do it an injustice. Her main point is that long after philosophy jettisoned positivism (Popper admitting fillicide) it lingered on as a reference discipline in economics and management.

    ReplyDelete
  17. Hi I am psychiatrist and I have found here very useful tips for me! thanks a lot

    ReplyDelete
  18. Having graduated from an MA program at Goldsmiths and worked as a designer I have put together a compendium of methods which can be found at http://www.designmethodsandprocesses.co.uk.

    It would be great to do a link exchange or if you can find some use for the methods on my site it would be great to have feedback.

    My experience is that there isn't enough space created for experimenting with research techniques & research is vital to the design process, but often pushed out of the process due to economic pressures. Use and development of said methods is a process of experience and iteratively testing techniques.

    ReplyDelete
  19. This comment has been removed by the author.

    ReplyDelete