A prognostic model of excessive lung function decline among Québec apprentices: a cohort exposed to occupational sensitizing agents

Alfi Afadiyanti Parfi,1 Mahsa Taghiakbari,2 Meshack Achore,1 Denyse Gautrin,1 Gleb Bezgin,3 Eva Suarthana1,2

Background

Forced expiratory volume in 1 second (FEV1) decline as a predictor of lung-related health problems is widely observed, but not fully investigated. This study aimed to develop models to predict FEV1 decline among apprentices exposed to sensitizing agents.

 

Methods

Of 692 apprentices recruited and followed in 3.6–17.3 years, 292 were exposed to low-molecular-weight agents. The analysis was restricted to 357 apprentices with complete lung function assessment at the end of their training with a minimum of 5-year follow-up. According to the American Thoracic Society guideline, a mean FEV₁ decline >60 ml/year was defined as “accelerated.” Descriptive statistics and Cox regression analysis were utilized to determine its predictors. To develop the prognostic models, we used a logistic regression analysis adjusted for the follow-up duration. The accuracy of the models was quantified using calibration and discrimination measures.

 

Results

Of 357 subjects, 62 (17.4%) had an excessive FEV1 decline post-apprenticeship. The questionnaire model (model 1), which included male sex, wheezing, and exposure to isocyanate or animal allergens during the apprenticeship, showed a reasonable discriminative ability (area under the receiver operating characteristics curve [AUC] of 0.67, 95% CI = 0.59–0.75). Adding the percent-predicted FEV₁ value at the end of apprenticeship significantly increased the discriminative ability of the model (model 4) (AUC = 0.762, 95% CI = 0.694–0.829) with a good calibration and reasonable internal validity.

 

Conclusions

We developed a model for accelerated lung function decline with a good accuracy and internal validity. However, external validation of the model is necessary.

 

Keywords

excessive decline, lung function, model, prognosis

The importance of forced expiratory volume in 1 second (FEV₁) decline as a predictor of lung-related health problems is widely observed, but not fully explored.¹ Spirometry, a tool used to measure FEV₁, is a known simple procedure with many benefits if performed regularly in the work setting as part of medical surveillance of workers or apprentices exposed to work contaminants with potential danger to the respiratory system.²

Among large case-control studies that reinforced the potential exploration of FEV1 decline, a study in the USA found 4–9-fold adverse health conditions (e.g., chronic obstructive pulmonary disease or emphysema, medication use for respiratory diseases, and dyspnea) of chemical plant workers exposed to isocyanates compared with controls, within 10–30 years of follow-up.3 In another cohort of construction painter, electrician, insulator, and machinist apprentices in British Columbia (Canada), 9% of the subjects were reported to have experienced an increase in bronchial responsiveness (BR) within 2 years of training. Investigators found an average FEV1 decline of 65–80 ml/year,⁴ while the average rate of decline in healthy non-smoking participants was 30 ml/year.⁵ During a 13-year follow-up period post-apprenticeship, those who had an accelerated FEV₁ decline during training more likely became asthmatic and more often visited for asthma later in life.⁴ A higher than normal rate of decline, average loss >45 ml s/year (to 71 ml s/year), was also detected in workers exposed to aluminum production dust at a smelter in Australia.⁶

A prediction model could improve a secondary prevention, as they make it possible to identify individuals at a higher risk; thus, prevention or intervention strategies can be targeted to avoid adverse outcomes. Such model may also support career counseling (e.g., for apprentices) and increase the efficiency of health monitoring programs by allocating resources for advanced (and burdensome) tests in subjects with a high probability of developing the outcome. A previous study showed that early identification and removal from exposure are crucial for a better prognosis.⁴ Moreover, compensation and relocation costs saved by detecting cases at an early stage outweigh the costs of referring subjects with mild conditions. To the best of our knowledge, this study is the first to develop a prediction model by utilizing excessive lung function decline specifically among welding apprentices in Québec.

 

 

Study design and population

Longitudinal data were analyzed to develop a prognostic model for an excessive FEV₁ decline. We used data from several existing cohorts, namely, (1) 178 apprentices in pastry making, 92 in dental hygiene technology, 58 in animal health technology, and 72 in veterinary medicine that were acquired at inception, at the end of the apprenticeship period between 1994 and 1998, and at a long-term follow-up between 2003 and 2006 (working period);7,8 (2) a cohort study of 202 car-painting apprentices that were exposed to hexamethylene diisocyanate at the beginning of the apprenticeship in 1999 and the long-term follow-up was completed in 2009;⁹ and (3) a longitudinal evaluation of 90 welding apprentices, completed between 2013 and 2017.¹⁰⁻¹² In all cohorts, the evaluation was carried out at three time points: inception, end of training, and long-term follow-up.

We decided to combine several cohorts for many reasons. First, we wanted to compare the effect of lung function decline in different groups with various types of exposures, such as high-molecular-weight (HMW) versus low-molecular-weight (LMW) agents. Second, such combination provides more possibilities of different pairs of groups compared with one another to demonstrate different effects. Finally, we wanted to maximize the sample size to meet the number of events per variable included in the prediction model.¹³ Questionnaires on demographic variables, medical history, and family history; skin prick test (SPT) reactivity to common and occupational allergens; spirometry; and nonspecific bronchial provocation tests with methacholine were administered at inception and at long-term follow-up visits in all cohorts.7,9,12

For the inception cohorts, the sampling units were teaching institutions offering a career program in animal health technology, pastry making, dental hygiene technology, car painting, and welding in Québec, as well as the only school of veterinary medicine in Québec, located within 300 miles from Montréal.9,12,14 Each institution provides different possible exposure agents as follows: animal health technologies (latex and animal-related proteins), pastry making (flour), veterinary medicine (latex and animal-related proteins), dental hygiene (latex), car painting (isocyanate), welding (welding gases and fumes), and plumbing (metal fumes).

Students were given an information letter and asked to participate. Subjects with a history of cumulative exposure to specific allergen(s) for ≥3 months before entering the program were excluded.¹⁴ Subjects who completed evaluations at inception and at the end of the apprenticeship period were eligible for the follow-up study.7,9,12 Then, we sent the letter of study explanation and visit schedule by mail to the addresses of the participants and of a parent and/or friend who were available. For those who cannot be located, we asked the Québec Health Insurance Board for updated addresses after obtaining an authorization from the Agency of Protection of Personal Information.7,12 Fees to reimburse the transport were issued.

Ethical approval under research project, entitled “Impact of exposure to occupational sensitizers on long-term changes in respiratory symptoms and function: Prospective cohort study”, issued by Centre intégré universitaire de santé et de services sociaux (CIUSSS) Research Ethics Committee file number CER 2012-801. During the study, protective equipment was used to ensure the safety of the subjects. The study took place at CIUSSS du Nord-de-l’Ile-de Montréal; Hôpital du Sacré-Coeur de Montréal (HSCM), 5400 boul. Gouin Ouest, Montréal (Québec) H4J 1C5 Québec, Canada. Questionnaire, SPT, spirometry, and BR to methacholine tests were performed at the HSCM.14

 

Study measures

Details of the study measures can be traced accordingly in the study by Gautrin et al,⁷ Saab et al,⁹ and Achore et al¹². Our trained nurses utilized a standardized protocol for follow-up evaluation including the questionnaire used for assessing respiratory symptoms.¹⁵ SPTs were carried out by using 11 common allergens: mixed trees, mixed grass, ragweed pollens, Alternaria, Aspergillus, Hormodendrum, feathers, Dermatophagoides farina, D. pteronyssinus, and cat and dog dander (Omega, Canada).8 Standardized mite extracts and cat dander were utilized. The trained nurse assessed the largest wheal diameter within 10–15 min after introduction of the antigen throughout the study. A positive reaction was defined as a wheal >3 mm in the presence of a positive reaction to histamine phosphate and in the absence of reaction to the diluent.16 Atopy was defined as at least two positive reactions. To assess FEV₁ and forced vital capacity, spirometry was done with a Collins Survey/1 Plus apparatus (Collins, USA).17 Methacholine inhalation tests were performed by using a Wright nebulizer with a maximum concentration of 32 mg/ml.¹⁸

 

Outcomes

We defined an excessive FEV1 decline as >60 ml/year in pre-bronchodilation FEV1 between the end of the apprenticeship and long-term evaluation in subjects with ≥5 years of follow-up.19 Of the 692 available individual data, 292 subjects with missing FEV1 value at the end of the apprenticeship and long-term followups, as well as 43 subjects with follow-up duration <5 years, were excluded, leaving 357 participants for the analysis (Figure 1).

 

 

 

Predictors

First, we selected potential predictors from a literature review²⁰ to keep the event per predictor ratio close to 1:10.¹³ Age, sex, body mass index (BMI) ≥25 kg/m² (i.e., overweight), self-reported wheezing, current smoking habit, as well as the type of agents at the end of vocational training, were initially evaluated for the questionnaire model. Nonspecific bronchial hyperresponsiveness (NSBHR) was defined as the provocative concentration of histamine/methacholine, causing a 20% decrease in FEV₁ (PC₂₀) ≤8 mg/ml.18,21 Percent-predicted FEV₁ (pFEV₁) was estimated using the Knudson equation.¹⁹ Atopy was defined as at least two positive SPT reactions to common allergens.

 

Statistical analyses

Descriptive statistics of demographics, exposure, and clinical characteristics of workers were calculated. Mean and percentiles were calculated for continuous variables (such as age, BMI, pFEV1, and PC20), while chi-square analysis was used to test the association between group of subjects with and without lung function decline with the prevalence of binary variables (sex, smoking status, BMI overweight, exposures to LMW, wheezing, tightness, cough, SPT-based atopy, pFEV1<80%, PC20≤8 mg/ml). Prevalence rates are presented with 95% confidence intervals (CI), with a p-value of 0.05 or less considered as statistically significant. Cox regression analysis was used to evaluate univariate associations among all potential predictors of excessive FEV1 decline. Diagnostic parameters, namely, sensitivity, specificity, positive predictive value, negative predictive value (NPV), and positive and negative likelihood ratios of each predictor were calculated.

 

Model development and evaluation

Cox regression analysis adjusted for the follow-up duration was utilized to develop the prognostic model. First, we developed a questionnaire model (model 1) by including variables with a univariable p<0.157 (Akaike criterion).22 Then, we added atopy (model 2), NSBHR (model 3), pFEV₁ (model 4), and all variables (model 5). The accuracy of all models was quantified using calibration and discrimination measures.23 Calibration is the agreement between predicted probabilities and observed proportions of each outcome. Evaluations were carried out by using a calibration plot, comprising predicted probability on the x-axis and the observed frequency of the outcome on the y-axis. Perfect predictions should be on the 45º line. Calibration was evaluated by using Brier score,23,24 which links similarities between the predicted and actual outcomes. It could range from 0 for a perfect model to 0.25 for a non-informative model in a population with a 50% incidence of the outcome. The discriminative ability of the models to identify subjects with and without the outcome will be determined with the area under the receiver operating characteristics curve (AUC), which ranged from 1.0 for perfect discrimination (100% sensitivity and 100% specificity) to 0.5 for no discrimination. The AUC of 0.7–0.8 is considered fair, 0.8–0.9 good, and >0.9 excellent.25

 

Internal validation

All models were assessed using the standard bootstrapping procedure, which was superior to the split-sample or cross-validation methods.23 This procedure gives a correction factor for both the model’s AUC and the regression coefficients of the predictors in the final model. The regression coefficients of the predictors in the final model will be multiplied by this correction factor to prevent the model from producing overly optimistic predictions (i.e., too high or too low estimates) when applied in future (new) subjects.

 

Missing data

Atopy was missing in 3% of the subjects, and specific sensitization was missing in 10% of the subjects. Twenty multiple imputations with the linear regression method were carried out as the preferred method to complete the case analysis26,27 using R (mice package),28 and no interactions were included. Analyses were performed with SPSS version 25.0 (IBM Corp., USA), S-Plus 6 for Windows (Insightful Corp., USA), and R-Studio Version 3.4.4 (R Foundation for Statistical Computing, Austria).29,30

 

 

The number of cases with excessive FEV1 decline post-apprenticeship was observed in 62 of 357 (17.4%) subjects. The overall median and interquartile (IQR) range of follow-up time were 7.0 (5.0–8.0) years. The follow-up time was significantly longer in subjects without an excessive FEV₁ decline than those with an excessive FEV1 decline (7.0 [6.0–8.0] versus 5.5 [5.0–7.0]) years, respectively, p<0.001).

Characteristics at the end of the apprenticeship and their associations with the excessive FEV1 decline adjusted for follow-up duration are displayed in Table 1. Proportions of males and subjects exposed to animal allergens and isocyanates were significantly higher in the group with excessive FEV₁ decline than those without excessive FEV1 decline. The mean value of the baseline pFEV₁ was also significantly higher in subjects with excessive FEV₁ decline than those without excessive FEV₁ decline post-apprenticeship.

 

 

The diagnostic accuracy properties of each potential predictor are presented in Table 2. Among questionnaire items, age <30 years had the highest sensitivity (91.9%); cough has the highest specificity (95.9%), while exposure to animal or isocyanates had the highest NPV (90.3%). Among objective tests, SPT-based atopy demonstrated the highest sensitivity (85.5%), while pFEV₁ <80% had the highest specificity (89.2%).

 

 

The remaining sample (n = 357) and excluded subjects (n = 335) did not show differences in age (p = 0.34, independent t-test) and BMI (p = 0.07, independent t-test). However, the remaining sample was predominantly male (n = 206; 57.7%) as opposed to the excluded subjects (n = 111; 33.1%).

 

Model development and evaluation

Five models were developed (Table 3). Male, wheezing, and type of exposure adjusted for their follow-up duration were selected as model 1, with an AUC of 0.67. Adding SPT-based atopy or NSBHR to the questionnaire model (i.e., models 2 and 3) slightly improves the AUC (0.68 and 0.69, respectively).

 

 

As illustrated in Figure 2, the addition of the spirometry component to the questionnaire (model 4) significantly improved the AUC of the model (at 0.76). Adding all objective tests (i.e., model 5) did not increase the AUC compared with model 4 (AUC 0.76), and the calibration was poor.

 

 

Figure 3 show the calibration plots of the direct calibration in model 1 (questionnaire) and model 4 (questionnaire and pFEV1), respectively. In model 1, the calibration plot (solid line) was above the diagonal reference line for probability >0.3, which means that the model-based probabilities were higher than the observed excessive decline in lung function rates (i.e., overestimation). In model 4, underestimation was only seen at lower thresholds at approximately 0.2.

 

 

The result of the bootstrapping analysis using HMisc and Design libraries in S-Plus³¹ revealed that all models had a reasonable internal validity, and the correction factors for all models ranged from 0.84 to 0.88; the closer the correction factor is to 1, the less optimistic (provide too high or too low) the prediction is.

 

 

We developed a prognostic model for excessive FEV1 decline among former apprentices in Québec who were exposed to occupational sensitizing agents. We found that being male, having reported wheezing, and being exposed to animal or isocyanate during apprenticeship increased the risk for developing an excessive FEV1 decline later in life. Our findings agree with those of Dahlqvist et al that male sex and exposure to isocyanate increased the likelihood of developing excessive lung function decline.32,33

Our questionnaire model had a reasonable AUC of 0.67, and only the addition of the spirometry parameter significantly improved the AUC to 0.76. An AUC of 0.76 meant that by using the questionnaire model we could correctly assign a higher probability to a sensitized apprentice in 76% of the pairs of apprentices in which one apprentice will develop an excessive decline in lung function and one will not.²⁵ In our study, the addition of SPT-based atopy and/or NSBHR only slightly upgraded the questionnaire model (AUC increased from 0.67 to 0.69). The marginal contribution of SPT and NSBHR might be explained by the fact that the questionnaire includes items related to both atopic history and BR.

Pralong et al³⁴ developed similar models in a cohort of the general population from the Study on Air Pollution And Lung Disease In Adults (SAPALDIA) in Switzerland. Their median follow-up time was 10.9 (IQR = 10.8–11.0) years, and the mean age of the cohort was 45.1 (standard deviation 9.4) years. They used the same American Thoracic Society guideline to diagnose excessive lung function decline (>60 ml/year), and 99 of 593 (16.7%) subjects met the criteria. The cohort had a rather fixed follow-up timeframe; hence, Pralong et al³⁴ used logistic regression analysis. Their questionnaire model (including age, sex, smoking, BMI, pre-existing asthma, and type of exposure) showed an AUC of 0.716 (95% CI = 0.693–0.739). The addition of pFEV₁ >80% significantly increased the discriminative ability (corrected AUC of 0.756, 95% CI = 0.736–0.807).³⁴

In an additional analysis, we attempted to externally validate our models in SAPALDIA’s cohort and vice versa. Given significant differences between the two cohorts, we concluded that separate models are required for each of them.

This study has some strengths. We are using data from studies with most extensive apprentices exposed to occupational sensitizers. This study also had a wide-ranging measurement protocol, from a questionnaire involving clinical components, SPT-based atopy, spirometry test to NSBHR administered by trained staff. Characteristics of the nonparticipants (except for sex, which already reported in detail their exposures) were not different from those of the participants, which reduces the likelihood of biases in the estimates. The predictors of our models derived from questionnaires and spirometry are commonly available, which will allow occupational health practitioners to utilize our model in daily practice.

This study also has some limitations. First, given the limited resources, only a portion of apprentices exposed to HMW agents completed lung function assessment at the end of their apprenticeship.⁷ Moreover, given the nature of the occupation, we were only able to reach one-third of former apprentices exposed to welding fumes and gases for the follow-up.¹¹ Nevertheless, the characteristics of those who did and did not participate in the long-term evaluation, as well as those who had and had not have completed lung function assessment at the end of their apprenticeship, were comparable. The study sample had a slight majority of and more male participants than the excluded subjects (57.7% versus 33.1%); the reason was that most of the participants with complete follow-up data consisted of subjects from the car-painting and welding programs, which contained more male apprentices. The model was developed in subjects who were all exposed to occupational agents; thus, there was no actual control group. Although a reasonable internal validity was confirmed, external validation of the models in a new population is still necessary to evaluate their generalizability.

To answer the question on how applicable our model to other populations, our models were developed in relatively young former apprentices from North America with a high proportion of male participants who had longer exposure to animal allergens and isocyanates. Evaluation of the validity of the models in older population with other types of exposures in other geographical area is required. Places with less access to more sophisticated clinical examination (e.g., SPT and NSBHR) could benefit most from our model 4, which utilized only variables from the questionnaire and FEV₁ assessment, had relatively high AUC (0.762; 95% CI = 0.694–0.829), akin to the AUC (0.764; 95% CI = 0.698–0.830) of model 5, which used all variables from questionnaires and clinical examination.

Our prognostic model enables quantification of an individual’s probability of having excessive FEV₁ decline post-apprenticeship. Thus, it could support career counseling and generate an awareness of the potential effect of the exposure among individuals with a high probability. Apprentices with a high probability of having excessive FEV₁ decline post-apprenticeship would be expected to be more vigilant—e.g., utilize adequate protective equipment to reduce the adverse effect of exposure in the long run.

In conclusion, we developed a prognostic model for an excessive FEV₁ decline in former apprentices in Québec who were exposed to specific occupational sensitizers. The questionnaire model, with factors including male, wheezing, and isocyanate or animal allergen exposures, showed a reasonable discriminative ability. The pFEV₁ at the end of apprenticeship significantly increased the discriminative ability, and the model showed good calibration and internal validity. External validation of the model is necessary.

 

 

 

 

 

 

 

1. Redlich CA, Tarlo SM, Hankinson JL, Townsend MC, Eschenbacher WL, Von Essen SG, et al. Official American Thoracic Society technical standards: spirometry in the occupational setting. Am J Respir Crit Care Med. 2014;189(8):983–93.
2. Redlich CA, Tarlo SM. Longitudinal assessment of lung function decline in the occupational setting. Curr Opin Allergy Clin Immunol. 2015;15(2):145–9.
3. Wang ML, Avashia BH, Wood J, Petsonk EL. Excessive longitudinal FEV1 decline and risks to future health: a case-control study. Am J Ind Med. 2009;52(12):909–15.
4. Peters CE, Demers PA, Sehmer J, Karlen B, Kennedy SM. Early changes in respiratory health in trades’ apprentices and physician visits for respiratory illnesses later in life. Occup Environ Med. 2010;67:237–43.
5. Lee PN, Fry JS. Systematic review of the evidence relating FEV1 decline to giving up smoking. BMC Med. 2010;8:84.
6. Pfidze TD. The rate of decline in lung function (FEV1) in workers exposed to aluminium production dust at a smelter in Australia. Intern Med J. 2017;47(S3):5–5.
7. Gautrin D, Ghezzo H, Infante-Rivard C, Magnan M, L’archevêque J, Suarthana E, et al. Long-term outcomes in a prospective cohort of apprentices exposed to high-molecular-weight agents. Am J Respir Crit Care Med. 2008;177(8):871–9.
8. Gautrin D, Ghezzo H, Infante-Rivard C, Malo JL. Incidence and determinants of IgE-mediated sensitization in apprentices. A prospective study. Am J Respir Crit Care Med. 2000;162(4 Pt 1):1222–8.
9. Saab L, Gautrin D, Lavoué J, Suarthana E. Postapprenticeship isocyanate exposure and risk of work-related respiratory symptoms using an asthma-specific job exposure matrix, self-reported and expert-rated exposure estimates. J Occup Environ Med. 2014;56(2):125–7.
10. El-Zein M, Infante-Rivard C, Malo JL, Gautrin D. Is metal fume fever a determinant of welding related respiratory symptoms and/or increased bronchial responsiveness? A longitudinal study. Occup Environ Med. 2005;62(1):688–94.
11. Taghiakbari M, Castano R, Parfi AA, Achore M, El-Zein M, Rhazi MS, et al. A cross-sectional assessment of rhinitis symptoms and nasal patency in relation to welding exposure. Am J Respir Crit Care Med. 2018;198(7):958–61.
12. Achore M, Taghiakbari M, Parfi AA, Lemiere C, El-Zein M, Rhazi M, et al. Evaluation of long-term respiratory effects of exposure to welding fumes. Maj Kedokt UKI. 2019;35(1):2–8.
13. Austin PC, Steyerberg EW. Events per variable (EPV) and the relative performance of different strategies for estimating the out-of-sample validity of logistic regression models. Stat Methods Med Res. 2017;26(2):796–808.
14. Gautrin D, Infante-Rivard C, Dao TV, Magnan-Larose M, Desjardins D, Malo JL. Specific IgE-dependent sensitization, atopy, and bronchial hyperresponsiveness in apprentices starting exposure to protein-derived agents. Am J Respir Crit Care Med. 1997;155(6):1841–7.
15. Burney PG, Laitinen LA, Perdrizet S, Huckauf H, Tattersfield AE, Chinn S, et al. Validity and repeatability of the IUATLD (1984) Bronchial Symptoms Questionnaire: an international comparison. Eur Respir J. 1989;2(10):940–5.
16. Pepys J. Types of allergic reaction. Clin Exp Allergy. 1973;3(s1):491–506.
17. Miller MR, Hankinson J, Brusasco V, Burgos F, Casaburi R, Coates A, et al. Standardization of spirometry, 1994 Update. American Thoracic Society. Am J Respir Crit Care Med. 1995;152(3):1107–36.
18. Cockcroft DW, Killian DN, Mellon JJ, Hargreave FE. Bronchial reactivity to inhaled histamine: a method and clinical survey. Clin Allergy. 1977;7(3):235–43.
19. Townsend MC; Occupational and Environmental Lung Disorders Committee. Spirometry in the occupational health setting--2011 update. J Occup Environ Med. 2011;53(5):569–84.
20. Hnizdo E, Glindmeyer HW, Petsonk EL. Workplace spirometry monitoring for respiratory disease prevention: a methods review. Int J Tuberc Lung Dis. 2010;14(7):796–805.
21. Malo JL, Pineau L, Cartier A, Martin RR. Reference values of the provocative concentrations of methacholine that cause 6% and 20% changes in forced expiratory volume in one second in a normal population. Am Rev Respir Dis. 1983;128(1):8–11.
22. Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361–87.
23. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21(1):128–38.
24. Brier GW. Verification of forecasts expressed in terms of probability. Mon Weather Rev. 1950;78(1):1–3.
25. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36.
26. Donders AR, van der Heijden GJ, Stijnen T, Moons KG. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol. 2006;59(10):1087–91.
27. van der Heijden GJ, Donders AR, Stijnen T, Moons KG. Imputation of missing values is superior to complete case analysis and the missing-indicator method in multivariable diagnostic research: a clinical example. J Clin Epidemiol. 2006;59(10):1102–9.
28. van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate imputation by chained equations in R. J Stat Softw. 2011;45(3):1–67.
29. IBM Corp. Released 2017. IBM SPSS Statistics for Windows, version 25.0. Armonk: IBM Corp.; 2017.
30. R Core Team. R: a language and environment for statistical computing. R Foundation for Statistical Computing [Internet]. Vienna; 2014 [cited 2021 Jan 21]. Available from: http//www.r-project.org/.
31. Alzola C, Harrell F. An introduction to S and the Hmisc and design libraries [Internet]. 2006 [cited 2021 Jan 21]. Available from: https://cran.r-project.org/doc/contrib/Alzola+Harrell-Hmisc-Design-Intro.pdf.
32. Dahlqvist M, Tornling G, Plato N, Ulfvarson U. Effects within the week on forced vital capacity are correlated with long term changes in pulmonary function: reanalysis of studies on car painters exposed to isocyanate. Occup Environ Med. 1995;52(3):192–5.
33. De S. Annual change in spirometric parameters among patients affected in Bhopal gas disaster: a retrospective observational study. Lung India. 2013;30(2):103–107.
34. Pralong J, Miedinger D, Suarthana E, Keidel D, Beckmeyer-Borowko A, Schindler C, et al. Prognostic model for accelerated decline in lung function due to occupational sensitizers: SAPALDIA study. Eur Respir J. 2017;50(Suppl 61):PA1247.

Copyright @ 2021 Authors. This is an open access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original author and source are properly cited. For commercial use of this work, please see our terms at http://mji.ui.ac.id/journal/index.php/mji/copyright.