Methodological issue in Cross cultural research

 

Sekaran (1983) delineates methodological issues in comparative research into five broad heads: functional equivalence, instrumentation, data-collection, sampling design, and data analysis.

Data equivalence The equivalence or comparability of data collected in different cultures and countries is critical in cross-cultural research. Data equivalence or comparability refers to ‘ data that have, as far as possible, the same meaning or interpretation, and the same level of accuracy, precision of measurement, validity and reliability in all countries and cultures ’ . Two approaches to equivalence are generally identified in the literature. The first refers to those forms of equivalence that have to be addressed before and during data collection. The second approach focuses on the analysis of the data and examines the measurement equivalence or invariance of data already collected.

Problem definition Two relevant methodological issues at this stage are the selection of the unit of analysis and the relevance of the research topic. Most research on international markets involves comparisons. Therefore, defining the unit of analysis, that is, selecting the relevant contexts to be compared is a priority in cross-cultural research. Craig and Douglas propose three aspects that need to be considered in defining the unit: the geographic scope of the unit (for example, country, region, and so on); the criteria for membership in the unit (for example, demographic or socioeconomic characteristics, and so on); and the situational context (for example, specific sociocultural settings, climate context, and so on). This section will focus on geographic scope, which needs to be chosen based on the purpose of the research. Within the different geographical levels, the country level provides a practical and convenient nit for data collection. Thus, researchers mostly use this unit of analysis in their studies. However, the use of countries is criticized for several reasons. 1 First, countries are not always that relevant. Cities, regions or even the world may be more appropriate. Second, countries are not isolated or independent units. They develop and adopt similar practices and behaviors through numerous ways. Finally, the differences between countries in terms of economic, social or cultural factors, and the heterogeneity within countries can have unintended consequences. The relevance of the topic in the selected units of analysis is more difficult and important than in domestic research, due to the unfamiliarity with the countries / cultures where the research is being conducted. The research topic should be equally important and appropriate in each context, and conceptually equivalent, an issue that will be addressed in the next section. Similarly, the relevance of constructs should be carefully evaluated. 1 This issue will help to avoid pseudoetic bias (that is, to assume that a measure developed in a context is appropriate in all the contexts).

Research approach Two main alternative approaches have dominated cross-cultural research in social sciences: emic and etic.  The emic approach examines the phenomenon studied from within a specific context (for example, culture, country). It holds that theory, constructs, and so on, are specific to this context. Hence, this approach requires developing specific measures for each unit of analysis and, taken to its extreme, limited or no comparisons are possible. For instance, this approach would imply developing different measures of a construct, such as brand personality, for each context. By contrast, the etic approach examines the phenomenon analyzed from outside a specific context (for example, culture, country). It is concerned with assessing universal constructs, theories, and so on, and allows for developing universal measures that can be applied to all contexts. That is, following with the previous example, this approach would imply using the same instrument to measure a construct, such as brand personality, regardless of the context. Although the emic approach offers more reliability and internal validity, the etic approach is considered more practical, in terms of time and cost. In addition, it makes comparisons easier and increases external validity. 3 Thus, researchers often use the etic approach. Accordingly, theories, conceptual models and research designs used in one culture or country are applied in others in the same way.

Construct equivalence At the research design stage, one of the most important methodological issues is to establish construct equivalence to ensure that constructs, objects and other stimuli have the same meaning and significance in different contexts. 17 Construct equivalence is concerned with three distinct aspects: functional, conceptual and category equivalence: 19 Functional equivalence deals with whether the concepts, objects or behaviors being studied are equivalent across cultures or countries in terms of the function or the role they perform. For instance, a bicycle is considered a means of transport in the Netherlands but for recreational purposes in the United States. Conceptual equivalence is concerned with whether the same constructs, objects and other stimuli exist in different cultures or countries and are expressed in similar ways. For instance, individual values such as materialism or concepts such as ‘ the self ’may vary from one country / culture to another. Finally, category equivalence relates to the question of whether the same classification scheme of objects or other stimuli can be employed across the different contexts of analysis. It includes a wide range of issues such as product category definitions, socio-demographic classes or occupational categories. For instance, a beer can be considered as a soft drink in the south of Europe and as an alcoholic beverage in the north.

Measure equivalence At the research design stage, three additional issues need to be considered: translation, calibration and metric equivalence. Traditionally encompassed within the concept of measure equivalence, these issues are interrelated with construct equivalence as the measures involve the operational definition of the construct.  Translation equivalence is concerned with the translation of the research instrument into another language so it can be understood by respondents in different countries and has the same meaning in each context. The goal of translation equivalence is commonality in understanding the instrument. Therefore, equivalence of meaning, rather than literal translation, is most important. Translation problems may arise from different causes. For instance, sometimes terms cannot be directly translated without losing their meaning, and sometimes a term does not exist in the other language. As discussed later, different translation techniques have been proposed to deal with this. Calibration equivalence refers to the equivalence in monetary units, measures of weight, distance, volume, and other perceptual cues, such as color and shapes. For example, if the distance between two points is measured in kilometers in one country and miles in another, then questionnaire items relating to this measure should be converted. Therefore, it provides assurance that the units of measurement and other perceptual issues are comparable across populations. Finally, two aspects have to be considered when determining metric equivalence: scalar equivalence and the equivalence of the scale or scoring procedure. Scalar equivalence refers to whether a score obtained through a certain scale in one country or culture has the same meaning and interpretation in another. As such, this type of equivalence implies that two individuals from different countries or cultures with the same value for a variable (for example, the same likelihood of purchase of a product) would give the same score on the same scale (for example, a value of 4 on a 5-point Likert scale). Scaling or scoring procedures refer to the use of equivalent scales or scores procedures in different contexts. Inconsistencies in this facet may arise from different levels of familiarity with scaling and scoring formats. Category rating scales are frequently used in survey research. 34 However, respondents across different countries may interpret the scoring formats and extreme categories inconsistently. For instance, while in some countries 5- and 7-point Likert scales are common, in others 20-point scales are more often used. Similarly, the use of nonverbal scales requires detailed analysis to determine the degree of comparison between countries and cultures.

Sampling equivalence Another important issue at the design stage is the sampling design. Two main levels of sampling can be identified: sampling of cultures or countries (discussed in the problem definition section) and sampling of the individual respondents. This section will focus on the latter. Problems regarding sampling in this level fall into three areas: the choice of respondents, the conflict between comparability and representativeness and the sampling methods. The choice of relevant respondents is a key issue in sampling, since these can vary across cultures or countries. For instance, women can be suitable respondents in some countries but not in others (that is, male-dominated societies). Similarly, senior managers may play a key role in the organizational decision-making process of Asian or Latin countries, whereas middle managers may have this role in Anglo-Saxon cultures. Another key issue concerns the conflict between the need for representativeness and comparability of the samples. 38 While homogenous samples enhance comparability and are needed to ensure equivalence, they are not likely to be representative of the target population. Therefore, balancing these two extremes represents one of the most important dilemmas in cross-cultural research. Finally, the use of probabilistic methods (for example, random and stratified sampling) enhances the likelihood of obtaining a representative sample. However, they are often not a viable choice. For instance, lists or directories are not usually available in emerging country markets. Therefore, in much cross-cultural research, non-probabilistic methods, such as quota sampling and judgmental sampling, are used. Researchers use these procedures to draw matched samples, that is, samples as similar as possible in terms of some relevant variables (sex, age, education, and so on). These procedures facilitate the control of extraneous variables that could potentially confound the results.

Data collection equivalence To enhance the comparability of the data collected, attention must be paid to the following aspects: equivalence of administration, equivalence of response, status and authority of the researcher and timing of data collection. Equivalence of administration refers to the fact that the research settings and the instructions must be equivalent, not identical. Special attention should be paid to physical, technical and social administration conditions. For example, whether a survey is administered individually or in groups could affect the results. Response equivalence is concerned with the design and administration of the research in such a way that people ’ s responses to the questionnaire are equivalent on several dimensions, such as the respondent ’ s familiarity with the test instruments, their levels of anxiety and other psychological reactions. One of the major concerns related to people ’ s responses is the presence of response bias, which occurs when people ’ s responses to a questionnaire are influenced by content-irrelevant factors. In this situation, the response does not indicate what it was intended to measure, threatening the validity of the findings seriously. In interviews, the status and authority of the researchers can also influence the results. This would include factors such as the characteristics of the interviewer (for example, affiliation, origin, gender, and so on) and the respondents ’ confidence in the researcher.  For instance, the fact that the researcher is a foreigner may trigger unexpected reactions from respondents. Finally, the timing of data collection is also important. Data should be collected from different countries within acceptable time frames to enhance comparability.  Otherwise differences in factors such as the underlying economic and social situation may lead to different results.

Data analysis Post-data collection assessment of equivalence has traditionally focused on measurement equivalence or invariance, which analyzes whether or not a measurement instrument yields accurate data about a specific issue across different groups. 41 – 43 Since Douglas and Craig 44 asked for evidence of measure equivalence, in general, and metric equivalence, in particular, numerous efforts have been made to refine this issue. Measurement equivalence or invariance (ME / I) concerns ‘ whether or not, under different conditions of observing and studying phenomena, measurement operations yield measures of the same attribute ’ .Accordingly, ME / I refers to the extent to which the content of each item is being perceived and interpreted in the same way across samples.