Designing A Pediatric Severe Sepsis Screening Tool

• Research
• Open Access
• Published: 26 October 2021

Knowledge translation following the implementation of a state-wide Paediatric Sepsis Pathway in the emergency department- a multi-centre survey study

• Luregn J. Schlapbach^2,4,
• Paula Lister^5,6,
• Debbie Massey⁷,
• Patricia Gilholm² &
• Amy N. B. Johnston^1,8

BMC Health Services Research volume 21, Article number:1161 (2021) Cite this article

• 385 Accesses

• 14 Altmetric

• Metrics details

Abstract

Background

Several health care systems internationally have implemented protocolised sepsis recognition and treatment bundles for children to improve outcomes, as recommended by the Surviving Sepsis Campaign. Successful implementation of clinical pathways is challenging and dependent on nurse engagement. There is limited data on knowledge translation during implementation of sepsis quality improvement programs.

Methods

This cross-sectional, multicentre observational survey study evaluated knowledge and perceptions of Emergency Department nurses in relation to the recognition, escalation and management of paediatric sepsis following implementation of a sepsis pathway. The study was conducted between September 2019 and March 2020 across 14 Emergency Departments in Queensland, Australia. The primary outcome was a sepsis knowledge score. An exploratory factor analysis was conducted to identify factors impacting nurses' perceptions of recognition, escalation and management of paediatric sepsis and their association with knowledge. Using a logistic mixed effects model we explored associations between knowledge, identified factors and other clinical, demographic and hospital site variables.

Results

In total, 676 nurses responded to the survey and 534 were included in the analysis. The median knowledge score was 57.1% (IQR = 46.7–66.7), with considerable variation observed between sites. The exploratory factor analysis identified five factors contributing to paediatric sepsis recognition, escalation and management, categorised as 1) knowledge and beliefs, 2) social influences, 3) beliefs about capability and skills delivering treatment, 4) beliefs about capability and behaviour and 5) environmental context. Nurses reported strong agreement with statements measuring four of the five factors, responding lowest to the factor pertaining to capability and skills delivering treatment for paediatric sepsis. The factors knowledge and beliefs, capability and skills, and environmental context were positively associated with a higher knowledge score. Years of paediatric experience and dedicated nurse funding for the sepsis quality improvement initiative were also associated with a higher knowledge score.

Conclusion

Translation of evidence to practice such as successful implementation of a sepsis care bundle, relies on effective education of staff and sustained uptake of protocols in daily practice. Our survey findings identify key elements associated with enhanced knowledge including dedicated funding for hospitals to target paediatric sepsis quality improvement projects.

Peer Review reports

Background

Sepsis, defined as a dysregulated host response to infection, remains one of the leading causes of preventable death and disability for children globally, with recent incidence estimates exceeding 25 million for the paediatric age group [1, 2]. In high-income countries, approximately one third of sepsis deaths occur in previously healthy children [3], and about one third of survivors will manifest ongoing sequelae [4] which impact families, healthcare facilities, and the community. In response to the global burden of paediatric sepsis the World Health Organisation (WHO) declared sepsis a global priority [5], prompting Australian professional bodies to develop a National Sepsis Action Plan to improve awareness, recognition and management of sepsis [6,7,8].

The paediatric Surviving Sepsis Campaign (SSC) recommends the use of a sepsis protocol and systematic screening tool to assist clinicians in the early recognition and management of sepsis in children [9]. Observational data have demonstrated benefit in a protocolised approach to sepsis in children, however significant barriers exist preventing the application of pathways in clinical practice that results in substantial variability in care both nationally and internationally [6, 10,11,12,13,14]. For example, the United Kingdom (UK) reported varied median implementation rates between 22 and 47% for components of the sepsis care bundle across 160 hospitals involved in sepsis pathway implementation [15]. The appointment of a sepsis board and a Patient Safety Facilitator resulted in improved pathway compliance in a four-year pilot program in London [16]. The largest paediatric sepsis Quality Improvement (QI) initiative in New York State revealed variability in the delivery of the mandated sepsis bundle [10]. Yet, to date there are limited data on the factors associated with success in implementation of sepsis QI programs, the processes related to successful knowledge translation (KT) and subsequent changes in behaviour [17, 18]. KT is a multidimensional process, whereby knowledge translates to evidence application, aiming to improve patient outcomes [19]. Recent studies have combined self-reported perceptions and participant knowledge as measures of KT [20]. Applying a KT methodology presents an opportunity to review, refine and improve implementation of clinical guidelines and reduce variability in care by knowledge generation, which when applied to clinical practice, may enhance uptake of evidence based practice [21, 22]. Implementation of a sepsis clinical pathway is an example of a KT intervention, which can be enhanced by evaluation [23]. Effective KT should include evaluation that assesses and explores underpinning factors to guideline use [21], and KT frameworks, such as the Theoretical Domains Framework (TDF), have been designed for this purpose [24]. The TDF identifies key factors supporting behaviour change in clinicians to inform strategies to enhance the uptake of evidence-based practice [21, 22, 25]. Once factors associated with the utilisation of evidence-based pathways are identified, interventions can be designed to increase KT [26].

Research conducted in the ED aims to advance knowledge, translating to improved practice and healthcare delivery. Despite this aim, such evidence is not always utilised [27]. Involving key knowledge users (nurses) is imperative to ensure pathways are integrated into practice [27]. Little is known about KT and sepsis pathways [28] and no known KT studies exist focusing on paediatric sepsis and nurses. Nurses' perceptions and engagement are key to successful KT [26] of clinical pathways. To advance knowledge and application of evidence-based practice, we introduced a Paediatric Sepsis Pathway (PSP), including targeted education for nurses on paediatric sepsis recognition, escalation and management, and on how to translate the PSP into practice. Nurses are the first point of patient contact in the ED, responsible for both initial and ongoing assessment [29, 30]. Nurses are optimally positioned to recognise, escalate and manage sepsis, and are essential to pathway implementation [31, 32]. The aim of this study was to evaluate the implementation of the PSP in participating Emergency Departments (ED) across the state of Queensland, Australia. This was achieved through a multi-centre survey of ED nurses' knowledge and perceptions that explored key elements of the PSP, reflecting KT. Specifically, we aimed to identify factors impacting nurses' perceptions of paediatric sepsis recognition, escalation and management and their association with knowledge, underpinned by the TDF.

Methods

Overview

We performed a multi-centre, cross-sectional survey assessing knowledge and perceptions of nurses working in 14 EDs following the implementation of a PSP in Queensland, Australia. Ethical approval was provided by Children's Health Queensland Human Research Ethical Committee (HREC/18/QRCH/167) and ratification from The University of Queensland (20190000093). This study followed the Consensus-Based Checklist for Reporting of Survey Studies (CROSS) [33].

Research in healthcare typically aims to progress knowledge, translating to improved health outcomes [27]. For this reason, the study aim was to i) determine nursing knowledge of paediatric sepsis recognition, escalation and management following implementation of a PSP, and to ii) explore the factors contributing to that knowledge, established from nurses' perceptions reflecting the key elements of the PSP. Nurse' knowledge score was generated from responses to questions formulated from content and implementation components of the PSP and was used as a proxy marker for KT, in combination with the factors reflecting clinical engagement with the PSP. Convenience sampling was used to recruit eligible participants and the large cohort, including widespread dissemination within each site and across the state of Queensland, aimed to result in a representative sample. Eligible participants were nurses working in participating EDs where the PSP had been implemented for a minimum period of 3 months and the nurses must have cared for paediatric patients in their current ED for a minimum of six months.

Fourteen hospitals across Queensland participated in a state-wide QI project that implemented a PSP in the ED from August 2018 to December 2019 and were included in the present study. The majority of Queensland EDs treat adult and paediatric populations (mixed ED) and nurses are trained to care across the lifespan [34]. Sites included four large, dedicated paediatric EDs and 10 mixed EDs, including metropolitan and regional facilities, with a combined nursing workforce of n = 1796. The funding allocated per site for the sepsis QI project was described as 'sepsis nurse funding'; this is the product of the designated nurse salary (weighting) and the full time equivalent in months spent in the role [35, 36]. 'Sepsis nurse funding' varied in amounts and duration (Supplementary Material 1).

Survey design

The survey was designed building on the Queensland state-wide PSP (Supplementary Material 2) as a framework to align with the key elements of recognition, escalation and management of paediatric sepsis [9].

The survey had three sections comprising 48 items (Supplementary Material 3). The first section contained demographic questions. The second section surveyed nurse's knowledge of paediatric sepsis acquired from the PSP, including 15 multiple choice questions (with pre-defined single or multiple correct answer options). The third section investigated factors impacting nurses' perceptions of paediatric sepsis recognition, escalation and management, contributing to evaluation of the implementation of a PSP. This section consisted of 25 statements where nurses responded to whether they agreed with the statements on a seven-point Likert scale ranging from one (strongly disagree) to seven (strongly agree) and were developed from the TDF. The TDF is a validated theoretical framework, developed from multiple behaviour change theories and provides insight into evidence for factors that influence clinical practice. The survey design was specifically informed by the Determinants of Implementation Behaviour Questionnaire (DIBQ), which is a validated TDF questionnaire targeting clinician implementation behaviour [25], providing insight into PSP utilisation and KT to practice.

Content, face, construct validity and reliability

Expert review, coupled with findings from reviewed literature, ensured the survey questions were formulated to achieve maximal authenticity (AH, LJS). Correct (knowledge) responses were formulated from the current Queensland PSP. The survey was piloted by 30 multidisciplinary content experts including medical specialists, ED nurses and medical officers, academics, those in leadership roles within the state-wide project and end users to ensure face, content and construct validity [25, 37]. Time to complete the survey was, on average, 13 min [37]. Pilot feedback identified expected responses and a high level of user acceptability, indicating a high degree of content validity. Based on the pilot, minor modifications were made to ensure questions were clear and appropriate to context, consensus reflected evidence-based practice.

The survey underwent reliability testing and was designed to permit an exploratory factor analysis (EFA) to explore the participants' perceptions section of the survey; to determine the factors that were present in relation to the recognition, escalation, and management of sepsis and their association with the survey primary outcome (knowledge score).

Survey dissemination and data collection

In each site a designated nurse was responsible for implementing the pathway including leading the QI initiative locally and participating in the broader state-wide program, provision of local education and collecting data. Survey commencement was guided by local governance approvals and eligibility criteria (Supplementary Material 1). The surveys were distributed over an approximately six-week period via an electronic platform, QR code linkage and on paper, which was led locally with support by the central research team. A standardised introduction to the survey was distributed to participants at each site prior to survey commencement to ensure consistency. Surveys were completed anonymously, containing no individual identifiers to maintain confidentiality. Where participants voluntarily provided an email address and were observed to not have completed the survey (as per RedCap) they were sent one reminder email promoting completion by the central team who were unaware of the participant or their results. Data were transcribed from paper where required and stored on a secure RedCap database. Sites were de-identified for confidentiality and are referred to as sites 1–14.

Exploratory factor analysis

Previous researchers have utilised theoretical frameworks to elucidate and assess factors that influence behaviour change [38] whereby design, application and interpretation is tailored to specific environments [39]. The EFA identified five factors which were categorised and adapted to fit the specific ED context, whereby allocation and alignment reflects the key constructs of the TDF [16, 39]. The five identified factors had good internal consistency measured by Cronbach's alpha, α > 0.7. The factor loadings for each item, corresponding questions and labelled factors, are displayed in Table 1. An individual's score on each factor was operationalised using factor scores. The factor scores were used in the main regression analyses to investigate their association with the primary outcome. Additional detail on the EFA and factor score methods is contained in Supplementary Material 4.

Table 1 Median response (IQR) and factor loadings for the five factors identified in the exploratory factor analysis for the broad nursing cohort. Only loadings greater than 0.3^a are displayed. Cronbach's alpha (α) is reported for each factor

Full size table

Statistical analyses

The primary outcome, the knowledge score, was calculated for each participant as the proportion of correct responses on the knowledge section of the survey (questions 8 to 23). More detail on scoring and processing of survey data, such as accounting for missing data, is contained in Supplementary Material 5 and 6.

All demographic details were summarised using descriptive statistics, including counts and proportions. A binomial logistic mixed effects model was used to explore the associations between the factor scores, demographics, and the hospital site variables on the knowledge scores [40]. A random intercept of site was incorporated into the model to determine if the variation in knowledge scores could be attributable to hospital (site).

The demographic variables, comprising questions 3 to 7 of the survey, were initially explored to be included as additional covariates in the model. However, paediatric experience, ED experience and nursing experience were found to be highly correlated (r = 0.71 to 0.81), so to avoid issues of multicollinearity, only paediatric experience was included in the model due to the population of interest, paediatrics [41]. Therefore, the demographic variables included in the model were age, years of paediatric experience and self-reported frequency of exposure to caring for a child with sepsis. Site variables included were 'sepsis nurse funding', (Supplementary Material 1), and the number of clinicians working within each ED. Models including and excluding the random intercept were compared and including the random intercept significantly improved the model fit. The model reported contains the demographic and site variables, the random intercept for hospital site and the five factor scores. All statistical analyses were undertaken using R (version 4.0.2) [42].

Results

Primary outcome – respondent's knowledge

In total, 1796 nurses were invited to complete the survey, 676 nurses responded, 544 answered 80% or more of the survey and 534 completed the demographic questions and were included in the final analysis. The combined response rate for all respondents was 37.6%, individual site response rates are reported in Supplementary Material 1. The majority of survey respondents reported working in nursing for 10+ years (39%, n = 207), and were aged between 26 and 30 years (28%, n = 151). 31% (n = 166) of respondents reported exposure to paediatric sepsis on a monthly basis. Respondent characteristics are described in Table 2.

Table 2 Respondent characteristics

Full size table

The median percentage of correct responses for the knowledge section of the survey was 57.1% (IQR = 46.7–66.7). Considerable site variability for knowledge scores was observed, with the median percentage of correct responses per site ranging from 46.4 to 65.8%, see Fig. 1. Overall, the median proportion of correct responses for sepsis recognition was 50.0% (IQR = 37.5–62.5), for escalation 50.0% (IQR = 25.0–75.0), compared to 62.5% (IQR = 50.0–72.2) for management (Supplementary Material 7).

Fig. 1

Distribution of knowledge scores per hospital. Sites have been arranged in order of median response per hospital. The centre line for each box plot is the median value, the upper-lower limits of the box are the 1st and 3rd quartile. The whiskers correspond to the maximum and minimum points that are 1.5* the IQR from the 1st and 3rd quartiles. Data that exceed these limits are considered outliers and plotted individually

Full size image

Self-reported pathway utilisation

The questions exploring implementation of the PSP (Supplementary Material 8) indicated that 96% (n = 513) of nurses utilised the PSP to guide sepsis management. The PSP was cited as the main resource used to guide treatment including antibiotic administration (89%, n = 475) and preparation of an adrenaline infusion (68%, n = 363).

The seven-point Likert scale scores for each of the statements comprising the five factors associated with the pathway utilisation are displayed in Fig. 2. The median Likert scale response for statements related to knowledge and beliefs (Factor one), social influences (Factor two), beliefs about capability and behaviour (Factor four) and environmental context and resources (Factor five) were very high on the scale; ranging from 6 to 7 (IQR = 5–7). Factor three, beliefs about capability and skills, scored lower with a median response of 5–6 (IQR = 3–7). The median response for each individual question is displayed in Table 1. Site variation for each of the five factors is displayed in Supplementary Material 9.

Fig. 2

Response percentage for questions that comprise each factor. Each subfigure comprises the statements that belong to each of the five factors. The bars represent response percentages with green colours indicating positive agreements responses, brown colours indicating disagreement responses and grey indicates neutral responses

Full size image

Variables associated with knowledge about sepsis

The results of the binomial mixed effects model used to investigate the effects of the five factors, demographic, clinical and site level variables on knowledge score, including a random intercept for hospital site to account for variation in knowledge scores between sites are displayed in Table 3. Three of the five factors had a significant and positive effect on knowledge scores, these included (a) knowledge and beliefs (OR = 1.09; p = 0.001), (b) beliefs about capability and skills (OR = 1.21; p < 0.001), and (c) environmental context and resources (OR = 1.05; p = 0.017). Years of paediatric experience was also shown to be a significant predictor of higher knowledge scores, where those with more experience had greater odds of correct responses compared to those with 6–11 months experience (7–9 years: OR = 1.28, p = 0.013; 10 + years: OR = 1.34, p = 0.004). Nurses performed better in hospitals with greater 'sepsis nurse funding' (OR = 1.11; p = 0.002). The variation in knowledge scores between hospital sites was predominantly explained by the factors, clinical and 'sepsis nurse funding' variables. Figure 3 displays the odds ratios and 95% confidence intervals for all fixed effects variables (Fig. 3a) and demonstrates the reduction in variance attributed to hospital site (random intercept) when these fixed effects were adjusted for in the model (Fig. 3b and c).

Table 3 Multivariate Odds ratios (OR), 95% confidence intervals (95% CI) and p-values of the effects of the included variables on sepsis knowledge scores

Full size table

Fig. 3

A Odds ratios and 95% Confidence intervals for the impacts of the fixed effects of the model on knowledge scores. Variables marked with * are statistically significant (p < 0.05; see Table 2). The reference category for age is 20–25 years. The reference category for paediatric experience is 6–11 months and the reference category for sepsis care is weekly. B Estimated unadjusted random intercept and 95% CI per hospital site. Hospitals (1–14) have been sorted by size of estimated odds of correct response. The red solid vertical line indicates the population estimated intercept. C Adjusted random intercept and 95% CI per hospital site. Hospitals (1–14) have been sorted by size of estimated odds of correct response. The red solid vertical line indicates the population estimated intercept. Panel C shows the variation between hospitals after adjusting for fixed effects including factors, sepsis nurse funding and years of paediatric experience

Full size image

Discussion

This study evaluated the implementation of a state-wide PSP by assessing the variables and factors contributing to ED nurse's knowledge of paediatric sepsis recognition, escalation and management. The study captured data from a broad range of nurses and varying health care settings participating in a sepsis QI initiative. Respondents were from 14 metropolitan and regional EDs, of which four were dedicated paediatric departments and 10 were mixed adult-paediatric EDs in QLD, Australia.

This is the first study to explore and evaluate knowledge and factors contributing to implementation of a sepsis pathway, providing insight into the complex phenomena of knowledge translation [43]. The survey tool, designed from the TDF, was validated by an EFA and can serve future sepsis QI initiatives in other states and countries to assess KT and implementation, however further validation including confirmatory factor analysis on an larger cohort would enhance its validity. The study provided the following key findings: First, nurses predominately agreed or strongly agreed with statements contained in four of the five factors that reflect elements indicative of utilisation of the sepsis pathway. Second, variation in knowledge of paediatric sepsis existed across hospitals, despite participation in a state-wide QI project and the provision and supported implementation of a standardised sepsis pathway; variation in knowledge scores could be explained by the final two key findings, paediatric experience, 'sepsis QI nurse funding' and three identified factors. Third, the experience in paediatrics and dedicated 'sepsis QI nurse funding' focused on supporting the sepsis QI initiative, were key respondent and hospital site characteristics associated with improved implementation as measured by the knowledge score reflecting the key elements of the PSP. Finally, the factors (i) knowledge and beliefs, (ii) beliefs about capability and skills and (iii) environmental context and resources emerged as the strongest factors associated with the primary outcome, knowledge score.

Translation of knowledge as a measure of sepsis pathway implementation

We used a knowledge assessment, composed of 15 questions targeting specific knowledge relevant for sepsis recognition, escalation, and management, formulated from the PSP. Time to initiation of sepsis treatment represents one of the strongest determinants of sepsis mortality in children [10, 44], yet studies have revealed major differences in compliance with sepsis bundles between sites using similar pathways, even when considering patient severity. Inter-institution variability in sepsis pathway knowledge and application can jeopardise sepsis recognition and treatment time [8, 9, 45], and contribute to suboptimal patient care [24, 46, 47]. Knowledge deficits may underpin the universal variability in sepsis pathway applicability resonating with previous literature articulating knowledge to be a key contributor to clinical pathway use [24]. Paediatric sepsis is particularly challenging because of the non-specific manifestation of sepsis and the relative rarity of sepsis compared to large numbers of febrile children being assessed in EDs, resulting in limited exposure for individual staff members [7]. In addition, a large proportion of children present to hospitals caring for adult and paediatric patients, staffed by a mixed, or primarily adult-experienced workforce. Hence, there is a need for evidence to guide improved strategies for the successful and sustainable implementation of sepsis pathways in children.

Our findings identified variability in sepsis knowledge, within and between sites [10, 11, 15], despite provision of a standardised, state-wide pathway and the related targeted education. These findings echo previous studies where average to low knowledge scores have been reported despite having a sepsis pathway and associated education in place [48,49,50,51]. Our study has identified that years of paediatric experience, 'sepsis QI nurse funding' and three identified factors associated with KT could explain a large proportion of variation in knowledge scores between sites, which can inform future sepsis initiatives.

Factors associated with knowledge translation and implementation

Successful QI initiatives within emergency medicine require consideration of the variety of elements influencing clinician behaviour that may affect KT and subsequent implementation and practice change [52]. We explored a range of factors and identified three key factors which were significantly associated with improved knowledge of the PSP. Knowledge and beliefs, capability and skills, and environmental context and resources, emerged as key predictors of nurse's knowledge scores. Furthermore, factors were independently indicative of KT to practice, by the reported high agreement scores, reflecting clinical practice insight, aligning with key elements of the PSP. KT is an underutilised phenomena in EDs [52], and focus on these three factors can inform future sepsis QI initiatives, campaigns and sepsis education to impact care globally, where previous efforts have reported limited success [14, 38, 53, 54].

Nurses in our study identified predominately positive agreement statements across four of the five factors indicating PSP utilisation, and 96% of nurses identified the pathway as a management resource, providing insight into implementation in clinical practice (Table 2; Supplementary Material 8). One factor, centred around the beliefs about capability and skills required to deliver treatment for paediatric sepsis, had comparatively lower ratings indicating that nurses need additional development to enhance confidence in skills required for managing paediatric sepsis. Similar findings have been identified in previous studies [51, 55], highlighting the importance of education initiatives focusing on kinaesthetic skill acquisition (and so, enhanced belief in capability) in training, especially where exposure to critically unwell children may otherwise be infrequent. In our study, only 35% of nurses reported caring for a child with sepsis as a weekly or fortnightly occurrence, highlighting the relatively low exposure to paediatric sepsis. The identified positive link between beliefs in capability and skills to sepsis knowledge should inform future QI and education initiatives to focus on increasing confidence in capability [51, 55,56,57] as a mechanism for enhancing knowledge translation and subsequent care for children.

Institutional variables associated with knowledge translation and implementation

The variation we observed in knowledge scores between the 14 hospital sites was largely explained by dedicated 'sepsis QI nurse funding', irrespective of site, geographic and facility differences (Fig. 3c). A lack of provision of dedicated human resources has been identified as the greatest challenge in caring for critically ill paediatric patients, impacting nurse's confidence in their capability [55]. Our identified link between confidence and knowledge informs future models, aiming to justify resources dedicated to paediatric sepsis QI initiatives. Our results emphasise the importance of specific training, education, tools and resources to increase nurse's confidence about their capabilities around managing challenging presentations, [58, 59] such as paediatric sepsis. National and international sepsis guidelines have often achieved implementation success through use of sepsis leads and champions to advocate for education uptake and broad awareness [60, 61], with higher performing sites containing these components [62]. Whilst dedicated funding is idealistic, the complexities associated with ED nursing practice results in competing education requirements and funding priorities. Furthermore, challenges with healthcare budgets exist including, reduced funding sources and balancing cost efficiency for such initiatives. However if sepsis care is suboptimal, the costs and other burdens imposed on health systems are significant [49].

Respondent characteristics associated with knowledge translation and implementation

We identified, years of paediatric experience as a significant predictor of increased knowledge. Previous study findings have also identified years of paediatric experience contributes to increased capability and confidence caring for paediatric patients [55]. While years of clinical experience cannot be replaced, we argue, based on our findings, there is a need for dedicated and ongoing resources targeting specialised paediatric sepsis education to enhance exposure; to supplement years of experience. These findings are supported in previous literature identifying that paediatric training and resources enhanced clinician's exposure to and subsequent confidence in delivering care to critically unwell paediatrics [51, 63].

Importantly, surveys of students, and of the healthcare workforce have revealed relatively low levels of agreement and knowledge about sepsis, which contrasts with the fact that sepsis represents one of the leading diseases associated with preventable deaths across all age groups [49, 64]. In Australia, despite the fact that national sepsis standards are being developed, currently there are no standards or mandates for under-graduate, post-graduate, or facility specific training for ED nurses on sepsis [28, 55]. In Australia, a generalist model in undergraduate nursing curriculum [34] contrasts with that in the United Kingdom, which offers specialised paediatric nursing degrees. This may pose Australian nurses with a challenge as they are required to provide care across the lifespan, however children have significant differences in pathophysiology that requires specialist knowledge and training which is currently limited [55]. Indeed, a recent university study concluded knowledge of paediatric sepsis was as low as 8% in graduating nurses in Queensland, Australia [64]. The importance of specified paediatric training, including post-graduate qualifications, facility mandated courses and certification, requires exploration in the future development of health services and education [55, 63].

Limitations

Knowledge of paediatric sepsis was used as the main outcome; however we did not assess whether increased knowledge was associated with improved treatment and outcomes in sepsis. In addition, we cannot conclusively attribute the results of the study to the implementation of the PSP due to the cross-sectional survey design. Furthermore, although this study identified important factors and variables associated with increased sepsis knowledge, we cannot assess whether there was an improvement in knowledge during the implementation of the PSP, as no baseline survey was conducted. A baseline survey would have provided valuable data and given greater certainty of KT, however given the complexities and challenges with ED nurse research including engagement with survey research, the multiple participating hospitals, competing priorities including the Coronavirus pandemic and funding, this was not possible. Thus, we explored the factors and variables contributing to knowledge, after the introduction of the PSP (and accompanying education), which may inform future QI initiatives. Participating hospitals may have chosen to support the sepsis QI initiative as part of regular business, resulting in voluntary contributions of time, which was not included in our calculations. We calculated 'sepsis nurse funding' as dedicated resourcing to lead the project and the predictive relationship of funding on knowledge score suggests that providing dedicated funding is of value. Two additional hospital sites were not included, as they had no sepsis-lead to run the survey. Independent data collection occurred locally, which may have contributed to response bias, unknown site confounding variables, and inconsistent instructions for completion alongside the biases that exist with self-reporting data [65]. We mitigated this risk by educating nurse leads at each site and providing an instruction script for consistent, supportive messaging. Each site was sent weekly response rates and reminders, the survey was available offline, and a strict inclusion criterion was created. A broad range of nurses were surveyed to include multiple perspectives and responses [39]. A sensitivity analysis of demographic details for partial responses that were excluded from analysis was undertaken to ensure no significant differences existed (Supplementary Material 6). Some sites had low response rates, with no ability to compare responders to non-responders, which impacts generalisability. Our sample was, however, heterogenous in nature with a large, combined site sample size representing many of the largest EDs within the state and is the largest paediatric sepsis implementation study conducted that assesses PSP implementation through KT.

Conclusion and future direction

This is the first study to explore the implementation of a state-wide PSP, revealing important factors associated with knowledge of recognition, escalation and management of paediatric sepsis, reflecting key elements of the PSP. Findings have key implications for policy development and design of future QI initiatives, in particular, the significant contribution of dedicated sepsis funding to ensure consistency of education and associated pathway implementation. We have developed an internally validated survey tool and statistical model to be used in future health service models aiming to enhance healthcare delivery, that can be adapted, tested and applied to alternative cohorts. Future initiatives can use the key factors originally developed from the TDF, to design education packages and target performance improvement interventions that ensure sustainable KT strategies. Our study identified key factors influencing knowledge, offering a novel perspective to inform future interventions and evaluation of evidence-based care, to ensure successful implementation and sustainability of a PSP where global variability continues to exist.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

References

1. 1.

   Rudd KE, Johnson SC, Agesa KM, Shackelford KA, Tsoi D, Kievlan DR, et al. Global, regional, and national sepsis incidence and mortality, 1990-2017: analysis for the global burden of disease study. Lancet. 2020;395(10219):200–11.

   PubMed  PubMed Central  Article  Google Scholar

2. 2.

   Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for Sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801–10.

   CAS  PubMed  PubMed Central  Article  Google Scholar

3. 3.

   Schlapbach LJ, Straney L, Alexander J, MacLaren G, Festa M, Schibler A, et al. Mortality related to invasive infections, sepsis, and septic shock in critically ill children in Australia and New Zealand, 2002-13: a multicentre retrospective cohort study. Lancet Infect Dis. 2015;15(1):46–54.

   PubMed  Article  PubMed Central  Google Scholar

4. 4.

   Thompson K, Taylor C, Jan S, Li Q, Hammond N, Myburgh J, et al. Health-related outcomes of critically ill patients with and without sepsis. Intensive Care Med. 2018;44(8):1249–57.

   PubMed  Article  PubMed Central  Google Scholar

5. 5.

   Reinhart K, Daniels R, Kissoon N, Machado FR, Schachter RD, Finfer S. Recognizing Sepsis as a Global Health priority - a WHO resolution. N Engl J Med. 2017;377(5):414–7.

   PubMed  Article  PubMed Central  Google Scholar

6. 6.

   Schlapbach LJ, Kissoon N, Alhawsawi A, Aljuaid MH, Daniels R, Gorordo-Delsol LA, et al. World Sepsis day: a global agenda to target a leading cause of morbidity and mortality. Am J Physiol Lung Cell Mol Physiol. 2020;319(3):L518–l22.

   CAS  PubMed  Article  PubMed Central  Google Scholar

7. 7.

   Schlapbach LJ, Thompson K, Finfer SR. The WHO resolution on sepsis: what action is needed in Australia? Med J Aust. 2019;211(9):395–7 e1.

   PubMed  Article  PubMed Central  Google Scholar

8. 8.

   Australia Sepsis Network. Stopping Sepsis: A National Action Plan: The George Institute for Global Health; 2017. Available from: https://www.australiansepsisnetwork.net.au/ssnap/stopping-sepsis-national-action-plan

   Google Scholar

9. 9.

   Weiss SL, Peters MJ, Alhazzani W, Agus MSD, Flori HR, Inwald DP, et al. Surviving sepsis campaign international guidelines for the management of septic shock and sepsis-associated organ dysfunction in children. Intensive Care Med. 2020;46(Suppl 1):10–67.

   CAS  PubMed  PubMed Central  Article  Google Scholar

10. 10.

    Evans IVR, Phillips GS, Alpern ER, Angus DC, Friedrich ME, Kissoon N, et al. Association between the New York Sepsis care mandate and in-hospital mortality for pediatric Sepsis. JAMA. 2018;320(4):358–67.

    PubMed  PubMed Central  Article  Google Scholar

11. 11.

    Seymour CW, Gesten F, Prescott HC, Friedrich ME, Iwashyna TJ, Phillips GS, et al. Time to treatment and mortality during mandated emergency Care for Sepsis. N Engl J Med. 2017;376(23):2235–44.

    PubMed  PubMed Central  Article  Google Scholar

12. 12.

    Balamuth F, Alpern ER, Abbadessa MK, Hayes K, Schast A, Lavelle J, et al. Improving recognition of pediatric severe Sepsis in the emergency department: contributions of a vital sign-based electronic Alert and bedside clinician identification. Ann Emerg Med. 2017;70(6):759–68.

    PubMed  PubMed Central  Article  Google Scholar

13. 13.

    Paul R, Melendez E, Stack A, Capraro A, Monuteaux M, Neuman MI. Improving adherence to PALS septic shock guidelines. Pediatrics. 2014;133(5):e1358–66.

    PubMed  Article  PubMed Central  Google Scholar

14. 14.

    Paul R, Neuman MI, Monuteaux MC, Melendez E. Adherence to PALS Sepsis guidelines and hospital length of stay. Pediatrics. 2012;130(2):e273–80.

    PubMed  Article  PubMed Central  Google Scholar

15. 15.

    Medicine TCoE. Severe sepsis and septic shock. London: Medicine TCoE; 2012.

    Google Scholar

16. 16.

    Steinmo SH, Michie S, Fuller C, Stanley S, Stapleton C, Stone SP. Bridging the gap between pragmatic intervention design and theory: using behavioural science tools to modify an existing quality improvement programme to implement "Sepsis six". Implement Sci. 2016;11(1):14.

    PubMed  PubMed Central  Article  Google Scholar

17. 17.

    Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A, et al. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14(1):26–33.

    CAS  PubMed  PubMed Central  Article  Google Scholar

18. 18.

    Roberts N, Hooper G, Lorencatto F, Storr W, Spivey M. Barriers and facilitators towards implementing the Sepsis six care bundle (BLISS-1): a mixed methods investigation using the theoretical domains framework. Scand J Trauma Resusc Emerg Med. 2017;25(1):96.

    PubMed  PubMed Central  Article  Google Scholar

19. 19.

    Straus SE, Tetroe J, Graham ID. Knowledge Translation in Health Care. Chichester: John Wiley & Sons, Ltd; 2013. p. 1–13.

    Book  Google Scholar

20. 20.

    Park JS, Moore JE, Sayal R, Holmes BJ, Scarrow G, Graham ID, et al. Evaluation of the "foundations in knowledge translation" training initiative: preparing end users to practice KT. Implement Sci. 2018;13(1):63.

    PubMed  PubMed Central  Article  Google Scholar

21. 21.

    Curtis K, Fry M, Shaban RZ, Considine J. Translating research findings to clinical nursing practice. J Clin Nurs. 2017;26(5–6):862–72.

    PubMed  Article  PubMed Central  Google Scholar

22. 22.

    Research CIoH. About knowledge translation and comercialization 2014. Available from: https://cihr-irsc.gc.ca/e/29418.html.

    Google Scholar

23. 23.

    Simunovic M, Fahim C, Coates A, Urbach D, Earle C, Grubac V, et al. A method to audit and score implementation of knowledge translation (KT) interventions in large health regions – an observational pilot study using rectal cancer surgery in Ontario. BMC Health Serv Res. 2020;20(1):506.

    PubMed  PubMed Central  Article  Google Scholar

24. 24.

    Sauro KM, Wiebe S, Holroyd-Leduc J, DeCoster C, Quan H, Bell M, et al. Knowledge translation of clinical practice guidelines among neurologists: a mixed-methods study. PLoS One. 2018;13(10):e0205280.

    PubMed  PubMed Central  Article  CAS  Google Scholar

25. 25.

    Huijg JM, Gebhardt WA, Dusseldorp E, Verheijden MW, van der Zouwe N, Middelkoop BJ, et al. Measuring determinants of implementation behavior: psychometric properties of a questionnaire based on the theoretical domains framework. Implement Sci. 2014;9(1):33.

    PubMed  PubMed Central  Article  Google Scholar

26. 26.

    Norton TC, Rodriguez DC, Willems S. Applying the theoretical domains framework to understand knowledge broker decisions in selecting evidence for knowledge translation in low- and middle-income countries. Health Res Policy Syst. 2019;17(1):60.

    PubMed  PubMed Central  Article  Google Scholar

27. 27.

    Graham ID, Kothari A, McCutcheon C, Angus D, Banner D, Bucknall T, et al. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation. Implement Sci. 2018;13(1):22.

    PubMed  PubMed Central  Article  Google Scholar

28. 28.

    Harley A, Schlapbach LJ, Johnston AN, Massey D. Challenges in the recognition and management of paediatric sepsis - The journey: Australas Emerg Care; 2021.

    Google Scholar

29. 29.

    Delaney MM, Friedman MI, Dolansky MA, Fitzpatrick JJ. Impact of a sepsis educational program on nurse competence. J Contin Educ Nurs. 2015;46(4):179–86.

    PubMed  Article  PubMed Central  Google Scholar

30. 30.

    Jeffery AD, Mutsch KS, Knapp L. Knowledge and recognition of SIRS and sepsis among pediatric nurses. Pediatr Nurs. 2014;40(6):271–8.

    PubMed  PubMed Central  Google Scholar

31. 31.

    Kleinpell R, Blot S, Boulanger C, Fulbrook P, Blackwood B. International critical care nursing considerations and quality indicators for the 2017 surviving sepsis campaign guidelines. Intensive Care Med. 2019;45(11):1663–6.

    PubMed  PubMed Central  Article  Google Scholar

32. 32.

    Harley A, Johnston ANB, Denny KJ, Keijzers G, Crilly J, Massey D. Emergency nurses' knowledge and understanding of their role in recognising and responding to patients with sepsis: a qualitative study. Int Emerg Nurs. 2019;43:106–12.

    CAS  PubMed  Article  PubMed Central  Google Scholar

33. 33.

    Sharma A, Minh Duc NT, Thang TL, Nam NH, Ng SJ, Abbas KS, et al. A Consensus-Based Checklist for Reporting of Survey Studies (CROSS). J Gen Intern Med. 2021;22:1–9.

    Google Scholar

34. 34.

    Schwartz S. In: Health Do, editor. Educating the nurse of the future: a report of the independent review into nursing education; 2019.

    Google Scholar

35. 35.

    Health. Q. Nursing Stream Wage 2020. Available from: https://www.health.qld.gov.au/hrpolicies/wage-rates/nursing.

    Google Scholar

36. 36.

    University. C. Calculating effort for grants 2021. Available from: https://www.creighton.edu/researchservices/grants/budget/calculatingeffortforgrants/.

    Google Scholar

37. 37.

    Bannigan K, Watson R. Reliability and validity in a nutshell. J Clin Nurs. 2009;18(23):3237–43.

    PubMed  Article  PubMed Central  Google Scholar

38. 38.

    Steinmo S, Fuller C, Stone SP, Michie S. Characterising an implementation intervention in terms of behaviour change techniques and theory: the 'Sepsis Six' clinical care bundle. Implement Sci. 2015;10(1):111.

    PubMed  PubMed Central  Article  Google Scholar

39. 39.

    Atkins L, Francis J, Islam R, O'Connor D, Patey A, Ivers N, et al. A guide to using the theoretical domains framework of behaviour change to investigate implementation problems. Implement Sci. 2017;12(1):77.

    PubMed  PubMed Central  Article  Google Scholar

40. 40.

    Larsen K, Petersen JH, Budtz-Jorgensen E, Endahl L. Interpreting parameters in the logistic regression model with random effects. Biometrics. 2000;56(3):909–14.

    CAS  PubMed  Article  PubMed Central  Google Scholar

41. 41.

    Colin L-B, Michael L-B. Applied Regression: An Introduction. 2nd ed. Thousand Oaks: SAGE Publications, Inc; 2016. p. 75.

    Google Scholar

42. 42.

    Foundation TR. R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2020. Available from: https://www.r-project.org/

    Google Scholar

43. 43.

    Reich EN, Then KL, Rankin JA. Barriers to clinical practice guideline implementation for septic patients in the emergency department. J Emerg Nurs. 2018;44(6):552–62.

    PubMed  Article  PubMed Central  Google Scholar

44. 44.

    Schlapbach LJ. Paediatric sepsis. Curr Opin Infect Dis. 2019;32(5):497–504.

    CAS  PubMed  Article  PubMed Central  Google Scholar

45. 45.

    Daniels R, Nutbeam T, McNamara G, Galvin C. The sepsis six and the severe sepsis resuscitation bundle: a prospective observational cohort study. Emerg Med J. 2011;28(6):507–12.

    PubMed  Article  PubMed Central  Google Scholar

46. 46.

    McQuillan P, Pilkington S, Allan A, Taylor B, Short A, Morgan G, et al. Confidential inquiry into quality of care before admission to intensive care. BMJ. 1998;316(7148):1853–8.

    CAS  PubMed  PubMed Central  Article  Google Scholar

47. 47.

    Smith GB, Osgood VM, Crane S. ALERT™—a multiprofessional training course in the care of the acutely ill adult patient. Resuscitation. 2002;52(3):281–6.

    PubMed  Article  PubMed Central  Google Scholar

48. 48.

    Robson W, Beavis S, Spittle N. An audit of ward nurses' knowledge of sepsis. Nurs Crit Care. 2007;12(2):86–92.

    PubMed  Article  PubMed Central  Google Scholar

49. 49.

    Storozuk SA, MacLeod MLP, Freeman S, Banner D. A survey of sepsis knowledge among Canadian emergency department registered nurses. Australas Emerg Care. 2019;22(2):119–25.

    PubMed  Article  PubMed Central  Google Scholar

50. 50.

    van den Hengel LC, Visseren T, Meima-Cramer PE, Rood PP, Schuit SC. Knowledge about systemic inflammatory response syndrome and sepsis: a survey among Dutch emergency department nurses. Int J Emerg Med. 2016;9(1):19.

    PubMed  PubMed Central  Article  Google Scholar

51. 51.

    Katznelson JH, Mills WA, Forsythe CS, Shaikh S, Tolleson-Rinehart S. Project CAPE: a high-fidelity, in situ simulation program to increase critical access hospital emergency department provider comfort with seriously ill pediatric patients. Pediatr Emerg Care. 2014;30(6):397–402.

    PubMed  Article  PubMed Central  Google Scholar

52. 52.

    Tavender EJ, Bosch M, Fiander M, Knott JC, Gruen RL, O'Connor D. Implementation research in emergency medicine: a systematic scoping review. Emerg Med J. 2016;33(9):652–9.

    PubMed  Article  PubMed Central  Google Scholar

53. 53.

    Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients' care. Lancet. 2003;362(9391):1225–30.

    PubMed  PubMed Central  Article  Google Scholar

54. 54.

    Cronshaw HL, Daniels R, Bleetman A, Joynes E, Sheils M. Impact of the surviving Sepsis campaign on the recognition and management of severe sepsis in the emergency department: are we failing? Emerg Med J. 2011;28(8):670.

    PubMed  Article  PubMed Central  Google Scholar

55. 55.

    Ilangakoon C, Jones T, Innes K, Morphet J. Caring for deteriorating paediatric patients in the emergency department: a mixed method study. Australas Emerg Care. 2020;23(4):252–8.

    PubMed  Article  PubMed Central  Google Scholar

56. 56.

    Lopreiato JO, Sawyer T. Simulation-based medical education in pediatrics. Acad Pediatr. 2015;15(2):134–42.

    PubMed  Article  PubMed Central  Google Scholar

57. 57.

    Abulebda K, Lutfi R, Whitfill T, Abu Sultaneh S, Leeper K, Weinstein E, et al. A collaborative in situ simulation-based pediatric readiness improvement program for community emergency departments. Acad Emerg Med. 2017;25.

58. 58.

    Numminen O, Meretoja R, Isoaho H, Leino-Kilpi H. Professional competence of practising nurses. J Clin Nurs. 2013;22(9–10):1411–23.

    PubMed  Article  PubMed Central  Google Scholar

59. 59.

    Hengstberger-Sims C, Cowin LS, Eagar SC, Gregory L, Andrew S, Rolley J. Relating new graduate nurse competence to frequency of use. Collegian. 2008;15(2):69–76.

    PubMed  Article  PubMed Central  Google Scholar

60. 60.

    Yealy DM, Huang DT, Delaney A, Knight M, Randolph AG, Daniels R, et al. Recognizing and managing sepsis: what needs to be done? BMC Med. 2015;13:98.

    PubMed  PubMed Central  Article  CAS  Google Scholar

61. 61.

    Burrell AR, McLaws ML, Fullick M, Sullivan RB, Sindhusake D. SEPSIS KILLS: early intervention saves lives. Med J Aust. 2016;204(2):73.

    PubMed  Article  PubMed Central  Google Scholar

62. 62.

    Paul R, Melendez E, Wathen B, Larsen G, Chapman L, Wheeler DS, et al. A quality improvement collaborative for pediatric Sepsis: lessons learned. Pediatr Qual Saf. 2018;3(1):1–8.

    Google Scholar

63. 63.

    Fowler J, Beovich B, Williams B. Improving paramedic confidence with paediatric patients: A scoping review. Austr J Paramed. 2018;15(1):1.

    Google Scholar

64. 64.

    Harley A, Massey D, Ullman AJ, Reid-Searl K, Schlapbach LJ, Takashima M, et al. Final year nursing student's exposure to education and knowledge about sepsis: A multi-university study. Nurse Educ Today. 2021;97:104703.

    PubMed  Article  PubMed Central  Google Scholar

65. 65.

    Creswell JW, Creswell JD. Research design. 5th ed. Los Angeles: Sage edge; 2018.

    Google Scholar

Download references

Acknowledgements

The authors would like to acknowledge and thank all the participating clinicians in Queensland Health, and the Clinical Excellence Queensland Sepsis Collaborative team. Kieran Owen for his assistance in central support and Donna Franklin for her valuable advice during initial survey development.

Funding

The study was supported by a grant from the Children's Hospital Foundation, Brisbane, Australia.

Author information

Affiliations

1. School of Nursing, Midwifery and Social Work, The University of Queensland, Brisbane, QLD, Australia

   Amanda Harley & Amy N. B. Johnston

2. Child Health Research Centre, The University of Queensland, and Paediatric Intensive Care Unit, Queensland Children's Hospital, QLD, Brisbane, Australia

   Amanda Harley, Luregn J. Schlapbach & Patricia Gilholm

3. Department of Emergency Medicine, Gold Coast University Hospital, Gold Coast, QLD, Australia

   Amanda Harley

4. Pediatric and Neonatal Intensive Care Unit, Children's Research Center, University Children's Hospital Zurich, Zurich, Switzerland

   Luregn J. Schlapbach

5. Paediatric Intensive Care Unit, Sunshine Coast University Hospital, Sunshine Coast, QLD, Australia

   Paula Lister

6. School of Medicine, Griffith University, Brisbane, QLD, Australia

   Paula Lister

7. School of Nursing and Midwifery, Southern Cross University, Gold Coast, QLD, Australia

   Debbie Massey

8. Department of Emergency Medicine, Princess Alexandra Hospital, Brisbane, QLD, Australia

   Amy N. B. Johnston

Contributions

AH & LJS conceptualised and designed the multi-site study. AH designed the survey, coordinated the study, assisted with data collection, analysis and interpretation of results and wrote the manuscript. AJ and DM assisted in study design. LJS, AJ & PG assisted in reviewing results. PG conducted the analysis. PG and AH prepared and collated figures, tables and supplementary material. All authors contributed to critical revision of manuscript and approved the final version.

Authors' information

AH is the state-wide Paediatric Sepsis Clinical Nurse Consultant in Queensland, Australia. AH is the nursing lead for designing, evaluating, educating clinicians and implementing a paediatric sepsis pathway in Queensland, Australia.

Corresponding author

Correspondence to Amanda Harley.

Ethics declarations

Ethics approval and consent to participate

All protocols are carried out in accordance with relevant guidelines and regulations. Ethical approval was provided by Children's Health Queensland Human Research Ethical Committee (HREC/18/QRCH/167) and ratification from The University of Queensland (20190000093). Written informed consent was obtained from all participants for the study after a written information sheet was provided.

Consent for publication

Not applicable.

Competing interests

None of the authors have declared a conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Cite this article

Harley, A., Schlapbach, L.J., Lister, P. et al. Knowledge translation following the implementation of a state-wide Paediatric Sepsis Pathway in the emergency department- a multi-centre survey study. BMC Health Serv Res 21, 1161 (2021). https://doi.org/10.1186/s12913-021-07128-2

Download citation

• Received: 22 March 2021

• Accepted: 22 September 2021

• Published: 26 October 2021

• DOI : https://doi.org/10.1186/s12913-021-07128-2

Keywords

• Clinical pathway
• Implementation
• Knowledge translation
• Management
• Nurse
• Recognition; sepsis; sepsis pathway; septic shock

Designing A Pediatric Severe Sepsis Screening Tool

Source: https://bmchealthservres.biomedcentral.com/articles/10.1186/s12913-021-07128-2

Posted by: williamsstairdle1946.blogspot.com

Share this post