If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Department of Research Methodology, Measurement, and Data-Analysis, Behavioural, Management and Social Sciences, University of Twente, Enschede, The NetherlandsMedical School Twente, Medisch Spectrum Twente, Enschede, the Netherlands
To evaluate the short-term effects of use of the dr. Bart app, compared to usual care, on the number of secondary health care consultations and clinical outcomes in people with knee/hip OA in the Netherlands.
A randomized controlled design involving participants ≥50 years with self-reported knee and/or hip OA recruited from the community. The number of secondary health care consultations (primary outcome) and secondary outcomes were assessed at baseline, 3 and 6 months via online questionnaires. Data were analyzed using longitudinal mixed models, corrected for baseline values. Due to the design of this study, blinding of participants and researchers was not possible.
In total, 427 eligible participants were allocated to either the dr. Bart group (n = 214) or usual care (n = 213). We found no difference between groups in the number of secondary (i.e., orthopaedic surgeon, rheumatologist, or physician assistant) health care consultations (incidence rate ratio (IRR) 1.20 (95% CI: 0.67; 2.19)). We found positive treatment effects of the dr. Bart app on symptoms (2.6 (95% CI: 0.4; 4.9)), pain (3.5 (95% CI: 0.9; 6.0)), and activities of daily living (2.9 (95% CI: 0.2; 5.6)) on a 0–100 scale, higher score indicating less complaints, but not in any other secondary outcome.
The dr. Bart app did not change the number of secondary health care consultations compared to usual care. However, we found small positive effects (not clinically relevant) on pain, symptoms, and activities of daily living in people with knee/hip OA.
. In the Netherlands health care costs attributable to OA spent in secondary care (i.e., orthopaedic surgeon, rheumatologist, or physician assistant) are eight times higher than costs spent in primary care (e.g., general practitioner or physical therapist)
. Although total joint arthroplasty is considered a (cost-)effective treatment for people with OA, it should be considered only after conservative treatment (i.e., education, promotion of lifestyle changes, pain management, exercise therapy, and weight reduction in case of overweight) has failed
. Self-management interventions offer patients guidance in improving their skills regarding management of symptoms, treatment, and physical and psychological consequences, as well as to improve skills to navigate the health care system
. Thus, self-management interventions augmented with providing feedback and getting rewards may help patients to take better care of their illness and make optimal use of health care options.
Modern persuasive technologies (e.g., applications) offer the possibility to enhance goal setting and provide tailored information to people with OA that suits individual preferences. This enhances self-management at all times
. The use of modern technologies seems promising and can support patients in taking an active role in the management of their chronic condition in daily life. However, most eHealth applications have not been evaluated in clinical trials
Given the high potential of modern technologies to enhance self-management 24/7, we developed an e-self-management application that promotes self-management and supports people with OA to optimize the use of non-surgical treatment options: dr. Bart app. The dr. Bart app is a fully automated eHealth application based on the Fogg model for behavioral change
, also known as the “tiny habits method”, and augmented with reminders, rewards, and self-monitoring to reinforce app engagement and health behavior. According to the tiny habits method, people with low motivation need an easy objective and a simple trigger to incorporate the target behavior in daily life. The central feature of the app is a library of predefined “tiny habit” goals and triggers to a healthier lifestyle, for instance, “I will perform two squats after brushing my teeth” or “During my lunchbreak I eat an apple rather than an unhealthy snack”. The dr. Bart app proposes a list of five pre-formulated goals to a healthier lifestyle, based on machine learning techniques fed by data collected in a personal profile and previous choosing behavior of the app user; i.e., previously selected and discarded goals. We assumed that use of the app promotes health behaviors, better self-management, and optimal use of non-surgical treatment options, ultimately resulting in better coping with symptoms and fewer secondary health care consultations. The primary objective of this randomized controlled trial (RCT) was to evaluate the short-term effects (after 3 and 6 months) of use of the dr. Bart app (ad libitum), compared to usual care, on secondary health care in people with knee/hip OA in the Netherlands. Secondary objectives were to examine the short term effects on clinical outcomes (e.g. pain, physical functioning) attributable to the dr. Bart app in people with knee and/or hip OA.
Trial design and setting
This was a monocenter, stratified (main OA location; i.e. knee or hip), prospective, unblinded, RCT comprising one intervention group (dr. Bart app) and one control group (usual care) with an allocation ratio of 1:1, conducted in the Netherlands. We examined the effectiveness of the dr. Bart app on the number of self-reported consultations in secondary health care over half a year. Details of the development of the intervention and trial design have been published elsewhere
. This study was conducted alongside a controlled clinical trial in Germany. The results of this controlled clinical trial will be reported separately. Moreover, we will report the results regarding use, usability of the dr. Bart app and its relation with clinical outcomes as well as on the cost-effectiveness analysis in two other manuscripts. There were no changes to methods/design after the trial commenced. Ethical approval for this study was asked for and waived by the local Medical Research Ethics Committee of the Radboud University Medical Centre, Nijmegen Commissie Mensgebonden Onderzoek (CMO Arnhem-Nijmegen, Protocol Number: 2017–3625). The study fell outside the remit of the law for Medical Research Involving Human Subjects Act and was approved by the local ethical committee. This study is registered in the Dutch trial register (Trial NL 6505 (NTR6693)). This trial is reported according to the CONSORT-EHEALTH checklist.
Recruitment and screening procedure
This study was conducted at the Department of Rheumatology of the Sint Maartenskliniek Nijmegen (the Netherlands) from January 2018 to January 2019. All participants in this RCT were recruited in the community through advertisements in local newspapers (i.e. region Nijmegen, the Netherlands) and in campaigns on social media of the Sint Maartenskliniek (i.e. Facebook, twitter, LinkedIn). Individuals with OA in the community willing to participate, were invited to the website (www.drbart.eu) to complete several questions to check their eligibility for this RCT. Eligible individuals were asked to sign in for the study by providing their e-mail address on the website. Potential participants received online baseline assessment via CastorEDC (https://www.castoredc.com/). CastorEDC is an electronic software application for data collection and management. CastorEDC is approved with ISO 27001 and ISO9001 and is in line with the European Union (EU) Data Protection Directive.
Participants were included under the following circumstances: 1) having self-reported OA of the knee and/or hip (i.e. having a painful knee and/or hip, knee and/or hip pain >15 days of the past month, morning stiffness <30 min (knee) and/or < 60 min (hip)); 2) being ≥ 50 years age; 3) having an e-mail address; 4) possessing a smartphone or tablet and willing to download the dr. Bart application on one or more devices; and 5) being able to read, write, and sufficiently communicate in Dutch.
Exclusion criteria were as follows: 1) being wheelchair bound, 2) having a diagnosis of (other) inflammatory rheumatic disease, 3) having knee and/or hip replacements and 4) having scheduled for knee and/or hip joint arthroplasty in the next 6 months.
Randomization and blinding
Patients were allocated to the intervention group or control group (allocation ratio 1:1) by the researcher (TP) with CastorEDC after completing baseline assessment. CastorEDC is an electronic software application for data collection and management. Randomization on the individual level was stratified by main OA-location (knee or hip) with randomly varying block sizes of two, four, and six, performed with CastorEDC. The researcher who ascertained randomization was concealed for treatment allocation. After allocation, participants in the intervention group received an e-mail from the researcher (TP) with information to access the dr. Bart app. Participants allocated to the control group received an e-mail that they were assigned to the control group. Due to the design of this RCT, blinding of participants and researchers was not possible. Throughout the study participants were able to call and send mails to the researcher when they had questions regarding the dr. Bart app or the study.
Intervention (dr. Bart app)
The theoretical framework and development process of the dr. Bart app and its pilot test are published elsewhere
. The overall goal of the dr. Bart app is to enhance self-management and to actively involve people with OA in managing their own disease. The dr. Bart app is a standalone eHealth application which invites users to select pre-formulated goals (i.e. “tiny habits”) and triggers to a healthier lifestyle. The pre-formulated goals are based on four themes that are core elements in the (non-surgical) management of OA: education regarding OA and its treatment modalities and the benefits of a healthy lifestyle, physical activity (both generic and OA specific information), vitality, and nutrition
. In the education library users can find specific information regarding OA and its treatments modalities, as well as generic lifestyle advice. Moreover, an exercise library is incorporated in the app, containing 10 exercises that are important for the treatment of OA. Users can select or discard goals; the app will continue with proposing goals until three goals are selected by the user. Once one or more goals are completed, the app will propose new goals. Proposed goals are based on machine learning techniques fed by data collected in a personal profile and previous choosing behavior of the app user, i.e. previously selected and discarded goals. In this way, the proposed goals are tailored to the user and suit their personal preferences. To reinforce app engagement and health behavior, the dr. Bart app is augmented with motivation enhancing techniques; reminders, rewards, and self-monitoring, Fig. 1. Users receive a daily push notification to remind them of their chosen goal, combined with an interesting fact or answer to frequently asked question
Participants allocated to the intervention group had the ability to use the dr. Bart app (version 1.3.7.) “ad libitum”. The app was only accessible for users after the researcher (TP) provided access to the app. The content and functionalities of the dr. Bart app were frozen during the RCT. However, bug fixes (e.g. failure to log in) and system failures were resolved.
Half of the participants were allocated to the usual care group and received no active treatment. Usual care is defined as non-standardized care initiated by the participant (self-medication, self-referral) to non-medical professionals or initiated by the general practitioner (after consultation initiated by the participant). Health care providers of participants of neither study group were not informed about the study. Participants in the control group were also offered the dr. Bart app after they fulfilled the last follow-up questionnaire.
Participants received validated online questionnaires at baseline, after 3 and 6 months of follow-up, and, a reminder was sent after a week, where applicable. Participants did not receive (financial) incentives or other compensation for completion of the questionnaires or the study. In cases of missing data on the primary outcome participants were asked to record the number of visits to a secondary health care provider (i.e. orthopedic surgeon, rheumatologist, or physician assistant) via an additional e-mail. Demographic data was collected at baseline. A detailed description of outcome measures is given in a design article
The primary outcome was the difference in the number of self-reported consultations in secondary healthcare (i.e. orthopaedic surgeon, rheumatologist, physician assistant) due to OA in the knee/hip over the previous 6 months (assessed with a 3-month recall period) between groups, reported as an incidence rate ratio (IRR).
Secondary outcome measures
To evaluate the effectiveness of the dr. Bart app, several self-administered questionnaires were used. We assessed health care utilization with a self-developed questionnaire with a 3 month recall period. Participants were asked to record the number of visits to predefined primary (e.g. general practitioner and physical therapist) and secondary (i.e. orthopedic surgeon, rheumatologist, or physician assistant) health care providers in the preceding 3-month period because of their knee or hip symptoms. Pain, symptoms, and functional limitations were assessed with either knee/hip OA outcome Score (KOOS or HOOS), where appropriate, with a standardized score being presented per sub scale (0–100; higher scores reflecting better health status)
. Moreover, quantitative data about the use of the app was automatically extracted from the back end of the dr. Bart app to determine which participants to be included in our per protocol analysis. As described in our protocol paper, we measured costs, treatment beliefs in OA, system usability scale and quality indicators of OA care as well. These outcomes will be reported in separate manuscripts on costs and the comparison between the Netherlands and Germany.
Randomized trial of the effectiveness of a non-pharmacological multidisciplinary face-to-face treatment program on daily function compared to a telephone-based treatment program in patients with generalized osteoarthritis.
, an a priori sample size of 322 participants (161 per group) would provide 80% power at 5% level of significance (two-sided unpaired t-test) to detect a mean difference of 0.35 (standard deviation 1.00) in the number of consultations in secondary health care between groups, anticipating a maximum loss to follow-up of 20%, assuming a normal distribution.
All statistical analyses were performed using Stata 13.1 (www.stata.com). Missing data was managed according to the recommendations of the specific questionnaire. For the PAM, we also calculated a total score when a maximum of two items of the questionnaire were missing, though the PAM recommend to only calculate a total score if no single item is missing. For this, we calculated the mean score of the answered questions in the PAM questionnaire and multiplied this by 13. We assessed the percentage of missing data only in participants who were not lost to follow up. Baseline differences between groups were not statistically tested. Primary analyses were performed according to the intention-to-treat principle (ITT). Secondary analyses included per protocol analysis, including adherent participants of the dr. Bart app group (i.e. who chose at least one goal) and the entire usual care group. Descriptive statistics were used to present group characteristics. Selective attrition was checked by comparing baseline characteristics of respondents to baseline characteristics of dropouts at both follow-up points.
According to the recommendations of Twisk et al. about missing data in longitudinal mixed-model analyses, no imputation techniques were used
. Differences in the primary and secondary outcomes between the dr. Bart app group and usual care group after 3 and 6 months were assessed with either negative binominal regression (number of visits to health care professional) or linear mixed models (measures of self-management behavior, physical activity, quality of life, illness perceptions, and subscales of HOOS/KOOS), with random intercept, treatment group (dr. Bart app or usual care), baseline value, and interaction between treatment group and time as covariates, without random slope. In the mixed models analyses regarding the HOOS and KOOS, main OA-location (knee or hip) was added as covariate. To evaluate whether the proportion of secondary health care health visits (yes/no) was different between groups, we performed a generalized estimating equation using binomial distribution, reporting an odds ratio. Per-protocol analyses were adjusted in the same manner. The primary outcome measure is reported as an IRR with 95% confidence interval (CI).
In total 692 people were assessed for eligibility in this trial (Fig. 2). A total of 427 participants were allocated to either the intervention group (dr. Bart) (n = 214) or the control group (usual care) (n = 213). The response rates for the follow-up questionnaires were 75.4% (intervention group, n = 150; control group, n = 172) and 69.3% (intervention group, n = 130; control group, n = 166) at 3 and 6 months, respectively. We did not find relevant differences on baseline characteristics between those who filled out the follow-up questionnaires and those who did not fill out questionnaires at follow-up (see Appendix 1a+b). We defined adherence as choosing at least one goal in the dr. Bart app. Subsequently, we considered 151 (70.6%) participants adherent with the dr. Bart app and 63 (29.4%) participants non-adherent. The percentage of missing data on the primary outcome was 0.7%, 3.7% and 5.4% at baseline, and at 3 and 6 months of follow-up, respectively.
The mean age of participants was 62.1 years (SD 7.3), with the majority being female (71.7%) and having symptoms predominantly in the knee(s) (73.3%). Among the participants, 58% experienced their OA symptoms for less than 5 years (Table I).
Table IBaseline characteristics of participants allocated to the intervention and control group (n = 427)
Table II depicts health care use of the intervention and control group. We did not find a difference in the primary outcome measure, i.e., the number of secondary health care consultations over 6 months (IRR: 1.20 (95% CI: 0.67; 2.16), intervention group compared to control group).
Table IIHealth care use and proportion of patients who visited a health care provider at baseline, at 3 and 6 months per group, and mean treatment effect (95% CI) between groups over 6 months
Dr. Bart group (N = 214)
Control group (N = 213)
Treatment effect of dr. Bart app
Baseline N = 214
3 months N = 150
6 months N = 130
Mean number of consultations during follow-up (SD)
We did not find a difference in the proportion of participants (yes/no) that consulted a secondary health care provider over 6 months odds ratio (OR (1.24 (95% CI: 0.74; 2.06), intervention group compared to control group)). In both groups, consultations with a physical therapist were most common. We did not find differences in the number of consultations with other health care providers between groups over 6 months (Table II).
Table III shows the results of secondary outcomes at baseline and 3 and 6 months of follow-up. We found a difference in positive changes in symptoms (2.6 (95% CI: 0.4; 4.9)) over 6 months in favor of the intervention group. Moreover, we found differences in positive changes for pain (3.5 (95% CI: 0.9; 6.0)) and activities of daily living (2.9 (95% CI: 0.2; 5.6)) in favor of the intervention group over 6 months. We did not find differences between groups in other secondary outcome measures. Treatment effects between groups at 3 and 6 months are reported in Appendix 1.
Table IIIMean scores of secondary outcomes at baseline, at 3 and 6 months per group, and mean differences (95% CI) between groups over 6 months
One participant in the control group reported in the follow-up questionnaire to have rheumatoid arthritis. Therefore, this participant was considered a protocol violator and was excluded from the per-protocol analysis. In the intervention group, we considered 63 participants non-adherent and 151 participants adherent (i.e., who chose at least one goal) with the dr. Bart app and these 63 participants were excluded in the per-protocol analysis, Fig. 2.
We found comparable results in the per-protocol analyses regarding our primary and secondary outcome measures (Appendix 3).
Use of the app
Of the 214 participants allocated to the intervention group, 171 (80%) opened the app at least once. Of all participants, 151 (71%) chose at least one goal. A total of 113 (53%) participants achieved at least one goal. Altogether, more than 9,000 goals were achieved over half a year. For people active with completing goals (N = 113), the median length of use was 144 interquartile range (IQR: 63; 173) days, with a median of 33 (IQR: 16; 89) logins per user. A total of 56 (26%) of all participants allocated to the intervention still used the app (i.e. logged in) after 26 weeks.
The aim of this RCT was to study the short term effectiveness of the dr. Bart app, compared to usual care, on secondary health care, in people with knee/hip OA in the Netherlands. The results of this RCT show that use of the dr. Bart app, based on the “tiny habits” method, does not change the number of consultations for OA in secondary health care compared to usual care. Moreover, we found no significant differences in healthcare utilization (HCU), health-related quality of life, physical activity, self-management behavior or illness perceptions between the intervention and control group over half a year. However, the dr. Bart app is effective with respect to symptoms, pain, and activities of daily living, although these benefits were small.
We found that the dr. Bart app did not have an impact on HCU in patients with knee/hip OA over 6 months. A possible explanation could be that the impact of our eHealth intervention on HCU might take longer than 6 months; it is conceivable that the “tiny habits”
will be further incorporated in daily life by participants in following months, and the accumulation of these tiny habits will result in larger health benefits and changes in HCU patterns over time. Self-management aims to increase the capacity of patients to cope with symptoms and managing their care, including skills navigating the health care system. The dr. Bart app stimulates the use of conservative treatment options. We assumed that better self-management will result in change in health care utilization patterns (i.e. optimal use of primary care services and less use of secondary health care services). Previous studies show that the self-management programmes did influence HCU, i.e. postpone total joint replacement surgery in OA patients
. We chose, however, for a shorter follow-up because of the rapidly evolving nature of eHealth. Future research is needed to assess long-term effects of use of the dr. Bart app.
We found small but significant positive effects on symptoms, pain, and activities of daily living attributable to the dr. Bart app. Although the importance of self-management programs is underlined by international recommendations, there is no consensus on the size of a clinically meaningful effect of (e−)self-management programmes. Effects of the dr. Bart app on pain, function, and symptoms are comparable with more intensive (traditional) self-management programmes in OA
. Thus, considering the non-invasive character of the intervention and the unlikeness of harmful effects, we think the positive effects found in this RCT could be worthwhile for patients with knee/hip OA.
We found that 20% of participants allocated to the intervention group did not download the app, while they were willing to participate in eHealth research. Moreover, we considered 30% to be non-adherent with the app. This could indicate a lack of engagement of participants or user-friendliness for the app itself. However, these rates are in line with studies on standalone eHealth applications assisting patients with OA in their preparation for the first consult with an orthopaedic surgeon, which could be used one or 2 weeks prior to consultation
. Nevertheless, we found mean (SD) usability scores of the app at 3 and 6 months of follow-up of 68.6 (16.5) and 69.2 (16.9), meaning that it does not reach an acceptable score (i.e. 70); thus, there is room for improvement of the app, which in turn might improve usage.
Currently, RCTs are considered the gold standard for evaluating interventions. It is well-known that executing a RCT is time and money-consuming. This in contrast to the rapidly evolving nature of eHealth interventions. Logically, the design to evaluate these eHealth interventions should reflect this iterative process. Recently, several frameworks have been proposed in the evaluation of eHealth interventions, but these novel frameworks are not commonly applied in practice by researchers
. It could be valuable to explore the possibilities of the use of these novel frameworks in the continuous evaluation of eHealth interventions.
To the best of our knowledge, this is the first RCT examining the effectiveness of a standalone (self-management) eHealth application in people with knee/hip OA. Studies on stand-alone eHealth in other chronic diseases (e.g. chronic pulmonary disease, heart failure, and diabetes) showed positive results on disease management, clinical outcomes, and health behavior changes among others (e.g. reduction of hospitalization and improving quality of life)
. Nevertheless, these reviews concluded that methodological quality of included studies is low, and thus the evidence of efficacy of standalone eHealth is very limited.
The current study has several limitations that should be taken into account when interpreting the results. First, an unavoidable issue in this respect is the lack of blinding of participants. Moreover, loss to follow-up was high at 3 and 6 months in both groups, even though we sent newsletters just before participants received follow-up questionnaires to stimulate response rates. It is conceivable, that the lack of face to face contact could have resulted in the relatively high loss to follow-up
. In addition, our study might have been underpowered. We assumed a normal distribution of the number of consultations in secondary care in our sample size calculation, because to the best of our knowledge, there is no generally accepted method to calculate the sample size for negative binomial regression analysis. However, as our analyses were based on considerably more participants than anticipated, it is unlikely that we failed to detect an effect that was present (Type II error). Interestingly, loss to follow-up was higher in the intervention group than in the control group. A possible explanation could be that in the control group the participants were offered the dr. Bart app after completing the last questionnaire. However, we found no differences in baseline characteristics between respondents and dropouts. Finally, the association between intensity of use of the intervention and clinical outcomes is not taken into account in this study
, this will be further investigated in a future in-depth analysis.
Although effects of the intervention were not clinically relevant, but considering the non-invasive nature of the intervention, the low costs and the safety of the intervention, we think that implementation and further development of the dr. Bart app is worthwhile. The dr. Bart app incorporates treatments of first choice in knee/hip OA; education, lifestyle advice, and healthy behaviors
. Our baseline characteristics are similar to participants included in previous OA research. Moreover, we used a mixture of recruitment strategies, resulting in a heterogeneous group of people with OA and guaranteeing the external generalizability of this study. Taking this in mind, the app could be applied as primary approach for the treatment of OA in clinical practice as it has the potential to serve as a trustworthy tool to provide education and goal setting regarding OA and its treatment options.
We showed that use of an e-self-management intervention (dr. Bart app) did not impact HCU compared to usual care over 6 months, but did result in small, albeit not clinically relevant positive effects on pain, symptoms, and activities of daily living in people with knee/hip OA. Additional research is needed to identify possible subgroups of patients who benefit most. Future research is necessary to replicate these results and to determine whether the dr. Bart app is more effective in the long term.
Ethics approval and consent to participate
Ethical approval for this study was asked for and waived by the local Medical Research Ethics Committee of the Radboud University Medical Centre, Nijmegen (Protocol Number: 2017–3625). The study fell outside the remit of the law for Medical Research Involving Human Subjects Act and was approved by the local ethical committee. This study is registered in the Dutch trial register (trial number: NTR 6693). All participants gave digital consent to participate in the present study.
Availability of data and material
The datasets used and/or analyzed in the present study are available from the corresponding author on reasonable request.
TP, KB, JvdP, FHJvdH, and CHMvdE participated in the design of the study. TP was responsible for inclusion and data collection. TP, JvdP, and CHMvdE were responsible for data analysis, tables and figures. All authors were responsible for interpretation of data. TP and CHME were responsible for drafting the manuscript, all other authors critical reviewed the manuscript. Furthermore, all authors approved the final version of the manuscript.
This project is funded within the INTERREG-programme and receives financial support by the European Union, the Ministry of Economic Affairs, Innovation, Digitalisation and Energy of the State of North Rhine-Westphalia, the Ministry of Economic Affairs and Climate Policy of the Netherlands, and the Dutch Provinces of Gelderland and Limburg.
Conflict of interest
The authors declare that they have no competing interests.
We would like to thank all participants for their contribution to this study.
Appendix A. Supplementary data
The following is the supplementary data to this article:
Randomized trial of the effectiveness of a non-pharmacological multidisciplinary face-to-face treatment program on daily function compared to a telephone-based treatment program in patients with generalized osteoarthritis.