1. INTRODUCTION

The informant approach is a research technique based on asking people to act in an informant role, i.e. to observe and articulate the social reality (or some aspects of it) for the researcher. It is a common practice in the analysis of policy-making processes to interview people directly involved in it. However, in this context, the distinction between “informant” and “respondent” has not been explicitly discussed neither fully exploited.

The purpose of this note is to outline some methodological issues regarding the investigation of a policy by using informants, defined, broadly speaking, as people, directly involved or not in the policy-making, asked to take the part of observer (as opposed to the role of “reacting participants“).

We shall (a) refer to the development of the informant technique (IT from now on) and to its main methodological problems; (b) discuss the adoption of such a technique in policy studies and (c) list some of the technical problems we have been confronted with in designing an exercise of cross-national investigation[1].

  1. THE INFORMANT TECHNIQUE

The informant technique (Seidler, 1974) traces back to the early stages of the anthropological research (Boas,1987; Malinowski, 1927): the informant approach was largely used as integral to participant observational studies or for collecting information “on a broken culture whose members no longer function as a living society” (Plead, 1953). In 1940 Osgood (1940:51) noted that “the basic factor in doing successful ethnography is… to find good informants”. During the 1940’s and the 1950’s the attention was centered on the scrutiny of the reliability and validity of the IT and some efforts were made in order to find out devices to cope with the technique’s bias. An important change came about in the 1960’s and the 197O’s: the use of informants emerged from the participant observation setting and was adopted in survey-based studies, in community studies, in cross-national surveys (Glaser, 1966; Hyman, 1967) and in organizational studies. More recently, a quantitative informant approach has been developed too. However simple the principle may be, the level of sophistication reached by the technique is surprising (Seidler, 1974).

The “informant” has been defined by Paul (1953:443) as “an articulate member of the studied culture who enters into a more or less personal relationship with the investigator for a relatively long period of time”. Accordingly, the IT, says Seidler (1974: 816) is “the reliance on a small number of knowledgeable participants who observe and articulate social relationship for the researcher”.

The establishment of the IT as an autonomous research device and not a component of the participant observation, was clearly operated by Zelditch (1962) who distinguished between “informant” and “respondent” in these terms: “We prefer a more restricted definition of the informant than most of the fie1dworkers use, namely that he can be called an ‘informant’ only where he is reporting information presumed factually correct about others rather than himself and his information about events is about events in their absence. Interviewing in the presence of the event is considered part of participant observation.” (1962: 569)

The technique consists of asking contacted or selected persons to act in an informant role; it can be used to generate hypotheses as well as to test them, to obtain quantitative as well as qualitative data. The reasons why the IT is used are basically: (1) it was generally accepted earlier as the most reliable way
of knowing inaccessible cultures whether present or past; (2) it is possible a better understanding that survey analysts could achieve; (3) it is normally a more economical and feasible means of data collection.

Problems posed by the IT have been frequently examined: selectivity bias, sampling and representativeness, measurement and standardization, reliability and validity (Campbell, 1955; Pennings, 1973). At the same time, techniques to cope with them have been devised. They include: recruiting
informants and standardized interview situations (Herton, 1947), adopting general principles of corroboration,  consistency and probability (Mead, 1953), testing the quality of reports and checking the informant competence (Kendall and Lazarsfeld, 1950).

  1. INFORMANT TECHNIQUE AND POLICY STUDIES

In this section we shall discuss some methodological issues concerning the transfer of the IT in the field of policy analysis by referring to the “rationale” of the study of policies, to the case study approach and to cross-national methodology. There are many criteria for evaluating the appropriateness of a technique. It is sometimes assumed that certain methods are more suitable for specific areas and less for others and it is obvious that techniques developed in one area of investigation cannot be transferred to another area. However, the history of the methods used in social science presents examples of successful adoption and transfer of techniques; sometimes this has been a source of innovation. Moreover, there are techniques that are nowadays common patrimony for social scientists. The IT has been used in very different areas and for the investigation of various themes; in these research settings the technique has remained the same, but it has been adjusted to the specific needs. In particular this development has changed the role of the IT in research design (complementary as in the participant observation, alternative to respondent-centered studies in the organizational analysis), the instrument to be used (e.g. the level of standardization), the type of data (quantitative versus qualitative).

In this note we argue that the same adjustment process may take place for using the IT in policy studies. At first sight, policy studies seem to be very far from the empiricist tradition based on the assumption of a “rational science of society” which has been dominant in social sciences. Policy studies tend to be focused on  specific processes more than guided by impelling imperatives of reaching generalizations or testing hypotheses as it is in the empiricist tradition in social research; there are few risks of methodological feticism, theories are conceived as interpretative instruments or sources of concepts, quantitative approaches are marginal (with few exceptions e.g. quantitative content analysis).

In general terms, the underlying logic of policy studies is closer to the historical method: knowledge provided by such analysis is global, contextual and interpretative. The “how questions” prevail on the “why questions”. By using the well- known dichotomy opposing “sciences of society” to “natural sciences”, we may say that the study of a policy is more based on the understanding pattern than on an explicative one. This has far-reaching implications for the techniques that are used in the investigation of policy formation and implementation. In particular it accounts for the large number of case studies. The case study approach, in the different forms in which it can be organized, still governs the literature in policy studies. It represents a way of immersion in a specific situation by using different sources of information and by trying to co-ordinate the data in a well-grounded description of the situation. In this context it is current procedure to interview people directly involved in the policy-making process; the interview does not generally take the form of a standardized interview, it tends to be a focused one (Merton, Fiske and Kendall, 1956).

Whiteley and Winyard put it in these terms: “…the best approach is to ask many questions, most of them contingent on the respondent previous answers and gradually build up a picture of the overall situation. This means a focused interview approach where the researcher takes up a series of themes probing in depth at each stage, rather than asking the same standard question to every respondent” (1983:7).

The researcher has a “creative” role: he knows the sources of information, has in mind some conceptual devices, collects pieces of information and quantitative data, tries to work out a plausible interpretation and, thus, “tells the story”. The study of the policy process has something in common with the artwork as Giddens noted: “The societal sciences are not the only field of endeavour whose object is to ‘understand’ human conduct; they share such an aim with literature and the arts” (1976: 149).

In this context, the IT has a supporting role: informants can be used as a first source of information or before compiling a questionnaire. The case study approach matches the need of a global and integrated knowledge of a set of events; it does not consent a full development of a cross-national study. A cross-national study requires some less time-consuming and workable approach than replication of national case studies.

A cross-national methodology can be seen as: “an approach to knowing social reality through the examination for similarities and differences between data gathered from more than one country” (Elder, 1976).

In its substance this method is a traditional way of approaching social reality since the inception of sociology as a science. Among the various types of cross-national methods two seem useful for our purpose: the first is the approach that focuses on national uniqueness and cross-national contrasts, the second the approach centered on cross-national similarities and cross-national comparability. The first is more likely based on the premise that social and political phenomena are discrete and ultimately idiosyncratic; the assumption of the second is that social processes cross over cultural, geographical and temporal boundaries.

In the field of vocational training policies, the first approach has been dominant and little attention has been devoted to use data gathered from more than one country in order to examine more general propositions (how does vocational training change? what is the role of ‘professionals’? how much does the implementation process account for?…) with the aim of establishing cross-national generalizations. While the first approach leads to cross-national study by summing up national case studies, the second has broader implications for the methods.

First of all, a collection of case studies may not be a comparative exercise: the comparison of ‘totalities’ is hardly possible. Any comparative effort has to limit to comparable variables or sets of variables. Unfortunately, as Almond and Verba point out: “…in the social sciences the isolation of variables is difficult...most of the variables we attempt to isolate are completely meaningful only when considered in their contexts, but to compare complete contexts is not really possible. Is there any way out of this dilemma? The answer is ‘no’ if we are looking for a perfect solution but ‘yes’ if we are looking for a reasonable solution…” (Elder, 1976: 364). Comparable variables are possible only with the definition of conceptual categories at general level according to Elder.

It is possible to counter argument for uniqueness and cross-national dissimilarity with arguments that if one uses sufficiently general level of categorization, unique phenomena can be fitted into most categories of non unique phenomena and can be dealt with accordingly” (Elder,1976:216). Moreover, a certain amount of comparability in the procedure is required: it means more standardized procedures in data gathering and data treatment.

These three requirements (comparable variables, sufficient level of categorization and standardization) provide a platform for a pattern of study of the policy process not based on the case study approach but on the investigation of a set of comparable variables defined at general level. In this perspective the IT may be exploratively seen as a viable and workable way of gathering data.

The use of informants in policy studies is not at all new, but it has not been recognized as such. Key participants, elites, leaders and experts have always been used as sources of information and asked to act in an informant role. The use of experts’ opinions and views is probably the case nearer to those of informants: they are always been considered with caution. Discussing, for instance, the role of Parliamentary Committees, S. and B. Webb drastically concluded that: “of all recognized sources of information the oral ‘evidence’ given in the course of these enquiries has proved to be the least profitable” (1932:142).

A more positive attitudes was expressed by Madge (1953) when he noted: “if properly approached…(the experts’) opinions are themselves of great interest… they are highly informed opinions. The fact that an individual is prepared to give evidence probably implies that he will have taken the trouble to collect his thoughts on the subject and thoughts on the subject support his case… Background material obtained by pumping ‘experts’ can seldom be relied on by itself, but if properly critically amassed can provide invaluable check, confirmation and correction on the results of field studies…if he really is an expert, steeper in his subject, he will probably have attained more insight into it than any outside investigator can ever hope to do” (1953:149).

In this section we have explored the implications of a cross-national study in the field of policy-analysis and proposed the use of the IT in this context.

  1. SOME PRACTICAL PROBLEMS

A “discours de la méthode” has to come to terms with reality: it implies to solve practical problems. The use of the IT in the context of a cross-national study faces four kinds of problems: the choice of “informants”, the exchange researcher-informant, the organization of the data collection and the problem of the quality of gathered data. Choosing informants is a three-step operation: (1) how to find them; (2) how to select them; (3) how to sample them. The sources for identify potential informants in a policy study may be the specialized literature, the political debate, the “invisible college” and the personal (direct or indirect) knowledge (snowball technique). For the selection, pragmatical constraints are equally decisive. It is not possible to have a random or representative sample (as is the case in a respondent-type approach): the “optimum” purposive sample based on the needs of the research (adequacy to the hypothesis, multiple informants, is a presence of alternative points of view…).

In the light of the tradition of the IT, the relationship between the researcher and the informant is one-way, it is generally understood in term of exchange: people acting as informants are not just asked to provide available information but to contribute to find out new evidence and to explore for new evidence.

There are several difficulties (access motivation, local sponsorship in the case of a cross- problem, study…) and the image of collaboration between national researcher and informant has sometimes to be replaced by one of disharmony and conflict of interest. In order to have a more standardized procedure, an effort to isolate comparable variables and define them in conceptual terms is needed. For the purpose of the data gathering at the national level a nationally-oriented list of questions may be useful. Instead of an intensive, unstructured interview, common in the case study method, a guided interview on a list of items is a good suggestion.

In policy studies the main task is to make things understandable, to facilitate the comprehension of events, to shed light upon complex processes; consequently the kind of outcome likely to be obtained by interviewing informants are: factual information (evidence), identification of supplementary sources, suggestions for answering some questions, elaborated hypotheses, speculations, reformulations of the questions and discussion of alternative views. These kinds of data cannot take a quantitative form, but there is still a problem of organizing them.

The quality of the technique is based on the quality of data collected. Validity, reliability and accuracy are the used criteria for evaluating data. The validity refers to the quality of “well-grounded, defensible data”. There is no antidote for assuring a satisfactory level of validity, but some devices can help to improve it. In particular the check of the informant’s competency (measure of knowledgeability), the test of the quality of the reports and the control of potential sources of bias.

The problem of accuracy is related to the level of specific knowledge and information provided; this was specific knowledge and information provided; this may be a main problem in a cross-national study, due to the knowledge in different levels of available knowledge in different countries.

The criteria of reliability may be dealt with in two ways. First of all, inter-informant reliability provides a useful content is of major interest for the information, but also the divergence in this investigator as it points out the less known and more conf1ictual aspects of the events under reliability points out a crucial aspect in the use of investigation. Second, the inter-method informants: the results of the documents and outcomes of the informant basic data that the researcher has to analysis of policy technique provide analye and interpret in their convergence and in their divergence. This kind of process is, perhaps, less part of the methodology as explicit statement about procedures and more embodied in the day-by-day research work.

 

 

REFERENCES

ALMOND, G. & VERBA, S.

“Some Methodological Problems in Cross-National Research”, in Etzioni, A. & Dubon, F.L. Comparative Perspective. Theories and Methods. Boston: Litt1e Brown and Co., 1970.

 

BOAS, F.

“The social organization and the secret societies of the Kwakiutl Indians”, in Report of the U.S. National Museum for 1895. Washington D-C.

CAMPBELL, D.T.

“The informant in quantitative research “, American Journal of Sociology, 60 (1955):339-342.

COHEN,  E.

“Dimensions of bias in an informant network”, Sociologia 2 (1970):159-174.

DEAN, J. & WHYTE, W.F.

“How do you know if the informant is telling the truth?” Human Organization. 17, 2 (1958):34-38.

ELDER, J.W.

“Comparative cross-national methodology”, Annual Review of Sociology”, (1976):209-230.

GIDDENS, A.

New Rules of Sociological Method, London: Hutchinson & Co. Ltd., 1976.

GLASER, W.A.

“International mail surveys of informants”, Human Organization, 25 (1966):78-86.

HOUSTON, M-J. & SUDMAN, S.

“A methodological assessment of the use of key informants”, Social Science Research. 4, 2 (1975):151-164.

HYMAN, H.H.; LEVINE, G.N. & WRIGHT, C.R.

“Studying expert informants by survey methods: a cross- national inquiry”, Public Opinion Quarterly 31 (1967): 9-26.

KENDALL, P-L. & LAZARSFELD, P.

“Problems of survey analysis” in Merton, R.K. & Lazarsfeld, P-F. (eds), Continuities in Social Research, Glencoe, Illinois:

The Free Press, 1950.

KILLWORTH, P. & BERNARD, H.R.

“Informant accuracy in social network data”, Human 0rganization, 35, 3 (1976):269-286.

JAHODA, M.; DEUTSCH, M. & COOK, S.W.

Research Methods in Social Relations. New York: Dryden 1951.

MADGE, J.

The Tools of Social Science. London: Longmans, Green and Co.,1953.

MALINOWSKI, B.

Argonauts of the Western Pacific New York: E.P. Dutton & Co, 1922.

MC GRANAHAN, D.A.

“Comments, correcting for informant bias”, American Sociological Review, 1,1 (1976): 176-178.

MEAD, M.

The study of a culture at a distance”, in M. Mead and R. Metraux, The Study of a Culture at a Distance. Chicago: University of Chicago Press.

MERTON, R.K.

“Field work in a planned community”, American Sociological Review. 2 (1947):304-12.

MERTON, R.K.; FISKE, M. & KENDALL, P.

The Focused Interview. Glencoe: The Free Press, 1956.

OSGOOD, C.

“Informants” in Osgood, C. (ed.), Ingalik Material Culture, University New Haven: Yale Publications in Anthropology, 1940 pp. 50-5.

PAUL, B.D.

“Interview techniques and field relationship “, in Kroeber, A.L. (ed.) Anthropology Today, Chicago: University of Chicago Press, 1953.

PENNINGS, J.

Measures of organizational structure: a methodological note”, American Journal of Sociology, 79 (1973):686-704.

SALAMONE, F.

The methodological significance of the lying informant”, Antropological Quarterly, 50, 3 (1977): 117-124

SEIDLER, J.

“On using informants: a technique for collecting quantitative data and controlling measurement error in organization analysis”, American Sociological Review, 38 (1974):816-831.

TREMBLAY, M.

“The key informant technique: a non-ethnographic application”, American Anthropologist, 59 (1957): 688-701.

VIDICH, A.& BENSMAN, J.

“The validity of field data”, Human Organization, 13 (1954):20-7.

WEBB, S. & WEBB, B.

Method of Social Study. London: Longmans, Green and Co., 1932.

WHITELBY, P. & WINYARD, St.

“Influencing Social Policy. The Effectiveness of the Poverty Lobby in Britain.” Unpublished paper, 1983.

ZELDITCH, M.JR.

“Some American methodological problems of field studies “, Journal of Sociology, 67 (1962):566-76.

 

 

 

[1] See Vocational Training Policies in three EC Countries, PhD Thesis, University of Edinburgh, 1988.