Validity and reliability of the Indonesian version of the new brief job stress questionnaire (short version) for work-related stress screening among office workers

Nuri Purwito Adi,1 Ansgar Azurit Virenti Paska Da Lopez,1 Hervita Diatri,2 Retno Asti Werdhani,1 Dewi Sumaryani Soemarko,1 Dewi Yunia Fitriani3

Background

The short version of the new brief job stress questionnaire (SV-NBJSQ) that is originally in Japanese could evaluate work-related stress in Indonesian office workers. However, it has not been validated in Indonesian. Thus, this study aimed to test the validity and reliability of the Indonesian version of the SV-NBJSQ.

 

Methods

This study was conducted by linguistic and cultural adaptation of the original Japanese questionnaire into Indonesian and continued with exploratory factor analysis, internal consistency, and reliability test of Cronbach’s alpha. Electronic and paper-based data were collected using total sampling from office workers at two companies in Jakarta from December 2017 to August 2018.

 

Results

A total of 438 respondents filled the questionnaire. The final version of the questionnaire consisted of 63 items, screened based on a loading factor of >0.4 and 13 factors (dimensions). Each factor had a Cronbach’s alpha of 0.628–0.887, while the entire 63 had 0.904 with a total documented cumulative variation of 64.97%.

 

Conclusions

The Indonesian version of the SV-NBJSQ is valid and reliable. Thus, we may use it as a more suitable, updated, and comprehensive tool to evaluate work-related stress among office workers.

 

Keywords

Indonesia, reliability and validity, screening, translations, work-related stress, questionnaire

Work-related stress is a significant global challenge for workers in terms of health and the company/institution.¹ These impacts include dysfunctional behavior, absenteeism, presenteeism, and poor physical health such as a reduced immune system. Data in Indonesia showed that the prevalence of work-related stress among young executives in several companies in Jakarta was quite high, reaching 25%.² Meanwhile, it reached 79% among civil servants of the Ministry of Health of the Republic of Indonesia.³

Psychosocial factors at the workplace play a significant role in causing work-related stress. Thus, detecting and evaluating these psychosocial factors become necessary. One frequently used instrument for this particular purpose is the self-administered questionnaire by the employee, which is inexpensive and easily analyzable.4 Many new questionnaires are emerging every several years as the field of mental health and stress prevention is constantly evolving. One recently developed instrument is the new brief job stress questionnaire (NBJSQ) by Inoue et al5 in 2014. This questionnaire is aimed at job demands and job resources assessment as well as employee and organizational outcomes with multidimensional and comprehensive approach. Inoue’s team developed NBJSQ by adding new questions to the previously existing brief job stress questionnaire (BJSQ), which was established several years beforehand. NBJSQ had 141 questions/items (84 new items added to the BJSQ), which was not fully practical to be utilized in the common workplace. The team then created the concise/shortened version of NBJSQ (SV-NBJSQ) containing 80 questions (23 items added to the BJSQ), examined in Japanese employees aged 20–60 years who were on the payroll of a company.6 Thus, NBJSQ (and along with SV-NBJSQ) has more updated theoretical job stress models for a more holistic and detailed multidimensional assessment.

Psychosocial factors are one potential hazard for workers, including white-collar workers. Several work-related stress questionnaires have been validated in the Indonesian population, but no questionnaire has assessed workplace psychosocial factors in a multidimensional and comprehensive manner. NBJSQ (and the short version) shows this multidimensional characteristic by adding factors based on the effort-reward imbalance model, higher-level organizational factors (such as workplace social capital), factors related to positive psychology, and workplace bullying/harassment to the former BJSQ which already encompassed many factors; thus, SV-NBJSQ for evaluating Indonesian workers can be used to obtain more holistic factors that cannot be obtained by other questionnaires. SV-NBJSQ has not been officially translated into Indonesian and has not undergone validity and reliability tests. Thus, this study aimed to generate a valid and reliable Indonesian version of the SV-NBJSQ.

 

 

A cross-sectional study was conducted from December 2017 to August 2018 in Jakarta with linguistic adaptation (translations) and transcultural validity, followed by exploratory factor analysis to construct validity. The subjects were office (white-collar) workers of two companies in the oil and gas sector. The recruitment was conducted via open announcement, which was endorsed by the management. The inclusion criterion was had passed the probation period (3 months); while the exclusion criteria were had a history of psychiatric disorder, received treatment that might affect their mental state, and did not answer the questionnaire completely. Only subjects who agreed and had given an informed consent were proceeded in this study. The minimum sample size was calculated using the factor analysis method. Considering the 80-item questionnaire and the chosen respondent-to-question ratio of 5:1, the minimum sample size needed for the factor analysis was 400 subjects.

Permission to use this questionnaire in Indonesia was already granted by Inoue et al,5 followed by the forward-backward translation of the SV-NBJSQ from Japanese to Indonesian and vice versa. The translation was done by a sworn translator from Universitas Indonesia. The backward translation was provided to the representative of the original authors to discuss any identified discrepancies or unclear translations with the investigator (ADL). The result of this review and the forward-translation questionnaire were reviewed by an expert committee consisting of an occupational physician specialist (from the Department of Community Medicine, Universitas Indonesia), a psychiatrist (from a private hospital in Jakarta), a company physician (from an oil and gas company in Jakarta), a human resource manager (from a private oil and gas company), and a statistician (from the Department of Community Medicine, Universitas Indonesia). The resulting questionnaire was then tested for a cognitive debriefing phase to a small group of 28 subjects with the same characteristics as the target population. Furthermore, the final version was sent to the study subjects, bundled in a package with a document containing an explanation of the study, informed consent form, and form to capture subject characteristics. This package was delivered electronically (via the internet/email) and manually (hardcopy) upon request. After all responses were obtained, subjects fulfilling exclusion criteria were removed, and exploratory factor analysis was performed. As the initial step for factor analysis, the measurement of sampling adequacy from the total collected data was calculated based on the Kaiser-Meyer-Olkin (KMO) test and Bartlett’s test of sphericity. Factor analysis was aimed to determine the new number of dimensions (through eigenvalue >1) and the questions that would remain (items that have inter-item correlations of 0.3–0.9 with a minimum 0.4 loading factor). The expected cumulative/total variance to determine the number of dimensions was ≥60%. Values for internal consistencies were also calculated for each dimension. The reliability test was done by assessing internal consistency using a Cronbach’s alpha. The approval from the Ethics Committee of the Faculty of Medicine, Universitas Indonesia was already obtained (No: 0004/UN2.F1.ETIK/2018).

 

 

In the pre-validation phase (from the beginning until cognitive debriefing), revisions and additional sentences were applied to the initial Indonesian version based on the inputs from the expert committees and cognitive debriefing results from 28 subjects. The pre-final version was then sent to the subjects of the validation phase.

In the validation phase (post-cognitive debriefing), 438 subjects fulfilled the inclusion and exclusion criteria. Thirty-one respondents who gave incomplete answers, declined to consent, and had panic disorder were excluded; thus, only 438 were eligible for factor analysis. Table 1 shows the characteristics of the respondents.

 

 

Measurement of sampling adequacy was calculated using KMO and Bartlett’s tests of sphericity, where the scores were shown to be >0.9 and p<0.001, respectively, thus allowing for factor analysis. When the rotation of principal axis factor matrix (varimax rotation) was generated, not all 80 questions met the criteria of loading factor ≥0.4. The exclusions of such questions were done through several cycles, resulting in 63 retained questions. The calculation for eigenvalue >1 for these 63 questions resulted in 12 factors/dimensions; however, only five factors were eligible. The resulted mapping of the 63 questions into these five factors can be seen in Table 2. Based on the mapping, the questions were distributed unequally in each factor; thus, a re-arrangement of this categorization was deemed necessary.

 

 

After this re-arrangement, the investigator (ADL) established 13 new factors/dimensions and calculated the Cronbach’s alpha values for each dimension and the whole questionnaire. The cumulative variance for these 13 factors was calculated, resulting in a value of 64.966%. Each dimension was named following the principle: if it was identical to the Japanese version, the naming would mirror the original term; while if it was different from the Japanese version, a new name would be formulated. This final mapping of these 63 questions within these 13 factors, along with the factor/dimension’s names, values of item correlation, loading factor, and Cronbach’s alpha, are shown in Table 3.

 

 

As an outcome of the factor analysis process, 17 questions from the original Japanese version were excluded in the final Indonesian version such as questions related to qualitative work overload factors, co-worker support, and easiness of communication with supervisors/co-worker/others. Moreover, the questions removed in the subcategory were questions regarding physical demands, suboptimal work environments, job security, utilization of appropriate skill, job control, satisfaction with family life, and workplace harassment. The list of these questions can be seen in Table 4.

 

 

Each of the 63 questions/items selected had a minimum loading factor of 0.4 for their dimensions, indicating that they were valid for their respective dimension/factor groups. A Cronbach’s alpha (internal consistency) value of more than 0.6 for each dimension (0.628–0.887) was also obtained, confirming that the dimensions and their items were consistent or stable. All items had good validity and reliability, with item correlations ranging from 0.3 to 0.9 (more specifically between 0.431 and 0.800) and a Cronbach’s alpha of 0.904 (>0.7). The 13 factors, which encompassed these 63 questions, showed an acceptable cumulative variance of 64,966.

 

 

This Indonesian version of the SV-NBJSQ is an important tool for evaluating work-related stress among Indonesian workers. This does not only contain more updated psychosocial factors and comprehensive multidimensional assessment but is also better understood and more suitable for Indonesian workers. This questionnaire is valid and reliable, which may ultimately help companies identify any actions needed to improve the work conditions.

During the factor analysis process, the questions were not equally distributed, in which most questions were mapped into the first factor, while only a few were mapped into the other four factors. This might be because the respondents did not fully understand/comprehend the categorical differences between the questions, which required a high level of detail. However, the Cronbach’s alpha value for each created factor remained high because the division of factors/domains and sub-factors/domains mirrored the original Japanese version. This can be caused by the great reliability results from the Japanese version.

If referring to the Japanese version that has five categories (“job demands,” “job resources: task-level, workgroup-level, and organizational-level,” and “outcomes”), the existence of these five categories was still carried over to the final Indonesian version. Several subcategories in the original version that were carried over to the Indonesian version without a reduction/removal of any questions were interpersonal conflict, suitable jobs, meaningfulness at work, job satisfaction, emotional demands, role conflict, role clarity, career opportunity, monetary/status reward, esteem reward, leadership, interactional justice, workplace where people complement each other, workplace where mistakes are acceptable, trust with management, preparedness for change, respect for individuals, fair personnel evaluation, diversity, career development, work-self balance, and all subcategories under outcomes related to physical reactions and individual emotional symptoms.

There were 17 questions removed due to cultural difference between Indonesian and Japanese employees about their perceptions/knowledge of those factors; it was assumed that these factors might not be stressors for Indonesian employees and/or not fully understood or recognized. The perception of psychosocial factors might also be very different because the respondents in this study ranged from senior executives to part-time employees (even interns). Furthermore, the Japanese population generally has a higher education level, which may contribute to their ability in dealing with many questions (i.e. 80 questions).7 However, it must be noted that this study and the original Japanese study did not ask for respondents’ educational levels; thus, further insights must be obtained before drawing this conclusion.

Four questions regarding work overload in the quantitative and qualitative aspects were removed. Since there were three questions in the qualitative aspect, thus the entire qualitative aspect was removed. This phenomenon is also found in qualitative studies on work-related stress caused by heavy workloads amongst emergency unit nurses and employees of a multinational business company.8 Moreover, no studies have investigated the qualitative aspect; thus, it might not be a significant factor for evaluating work-related stress in Indonesian workers.

The exclusion of the poor physical work environment is aligned with the results of two studies. Ekawarna and Sofyan9 studied public elementary school teachers and showed that poor physical environment was correlated with anxiety that was correlated with stress levels (obtained via structural equation modeling). As the poor physical environment was not directly correlated with stress, this may explain why the factor was not included in the mapping. Meanwhile, a study in plywood company employees found that employees who were exposed to noisy environments but had good adaptability skills did not report high-stress levels, showing a similar prevalence in the high- and low-stress employees in the same environment. Moreover, the respondents were from well-established companies with good physical environments (i.e., good lighting, temperature, ventilation, noise levels, etc.).¹⁰

The exclusion of physical activity demand did not align with Ratih and Suwandi¹¹ who found a significant relationship between the high physical nature of the work and stress levels in non-workers of the production division in a company. It should also be noted that most of these respondents exhibited sedentary work lifestyles/habits. Moreover, exclusion regarding interpersonal relationship is not aligned with the results of three previous studies. A significant correlation between co-worker interactions with stress was found on studies by Dhania¹² in cigarette laborers, Lady et al¹³ in civil servants for Regional Disaster Management, as well as Sormin¹⁴ in factory workers in a palm oil factory, albeit it was not significant.

The limitation of this study was with respect to some exclusion criteria that only relied on the employees' answers (self-reported) without any verification process to the source/other documents. Hence, the provided answer might not fully accurate.

The first suggestion for future studies would be to conduct other studies which will confirm the relationship between the excluded psychosocial factors and stress among Indonesian employees. Another suggestion is to expand this study to new psychosocial factors that are not listed in the original Japanese version, with the assumption that there are specific psychosocial factors for Indonesian culture. Finally, it is suggested to create a shorter version of this Indonesian version, referring to the early study by Inoue et al6 where a short version was developed from the longer questionnaire.

In conclusion, we have developed the Indonesian SV-NBJSQ adapted from the original Japanese version by Inoue et al,6 which is the updated and more comprehensive tool to evaluate work-related stress amongst office workers. This questionnaire consisted of 63 questions with changes in several questions when compared to the Japanese version. Moreover, it is valid, reliable, and suitable for Indonesian office workers.

 

 

 

 

 

 

 

1. Leka S, Cox T, Griffiths A, World Health Organization (WHO). Work organization & stress: systematic problem approaches for employers, managers and trade union representatives. Geneva: World Health Organization; 2003.
2. Kamso S, Purwantyastuti, Lubis DU, Juwita R, Robbi YK, Besral. [Prevalency and determinant metabolic syndrome on executive group in Jakarta and nearby areas]. Kesmas Natl Public Health J. 2011;6(2):85–90. Indonesian.
3. Besral, Widiantini W. [Determinants of stress among civil servants at Health Ministry of Indonesia]. Kesmas Natl Public Health J. 2015;9(3):222–8. Indonesian.
4. Leka S, Jain A. Health impact of psychosocial hazards at work: an overview. Geneva: World Health Organization (WHO); 2010.
5. Inoue A, Kawakami N, Shimomitsu T, Tsutsumi A, Haratani T, Yoshikawa T, et al. Development of a short questionnaire to measure an extended set of job demands, job resources, and positive health outcomes: the new brief job stress questionnaire. Ind Health. 2014;52(3):175–89.
6. Inoue A, Kawakami N, Shimomitsu T, Tsutsumi A, Haratani T, Yoshikawa T, et al. Development of a short version of the new brief job stress questionnaire. Ind Health. 2014;52(6):535–40.
7. Organisation for Economic Co-operation and Development (OECD). Population with tertiary education (indicator) [Internet]. Organisation for Economic Co-operation and Development (OECD); 2018 [cited 2018 Dec 23]. Available from: http://data.oecd.org/eduatt/population-with-tertiary-education.htm.
8. Yo PMP, Surya IBK. [Influence of the workload on job satisfaction with job stress as a mediating variable]. E-J Manaj Unud. 2015;4(5):1149–65. Indonesian.
9. Ekawarna E, Sofyan H. [The physical working condition, decision making participation and anxiety as a source of work related stress on state school teachers]. Makara Seri Sos Hum. 2010;14(2):141–6. Indonesian.
10. Asmarani R. [Relationship between ability to adapt to noise and work stress amongst employees]. J Stud Insania. 2017;5(1):71–93. Indonesian.
11. Ratih YRF, Suwandi T. [Analysis of relationship between individual factors as well as physical work demands and work stress in the production department of PT. X Surabaya]. Indones J Occup Saf Health. 2013;2(2):97–105. Indonesian.
12. Dhania DR. [Work stress amongst laborers in a cigarette industry in Kudus city]. Sos Dan Budaya. 2010;3(2). Indonesian.
13. Lady L, Susihono W, Muslihati A. [Analysis of work stress levels and factors which cause work stress amongst Regional Disaster Management Agency employees in Cilegon city]. J Ind Serv. 2017;3(1b):191–7. Indonesian.
14. Sormin T. [Factors associated with work stress amongst laborers in a palm oil processing factory]. J Keperawatan. 2017;12(1):46–51. Indonesian.

Copyright @ 2022 Authors. This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original author and source are properly cited. For commercial use of this work, please see our terms at http://mji.ui.ac.id/journal/index.php/mji/copyright.