Previous reviews of whole family support services have found that family members often experience multiple issues, but responses to those issues vary greatly, are often bespoke and lack a strong evidence base (Batty, 2014). To address the complex social issues which families experience, there has been an explosion of interest around ‘what works’ (Olsson, 2010) data over recent years to help us understand how best to improve outcomes for families. But while there has been a growth in evidence-based approaches (Mihalic, Irwin, Fagan, Ballard and Elliot, 2004), these are normally targeted at specific populations (Walsh and Doherty, 2016) for narrow but well-defined issues (Aarons and Palinkas, 2007).
The reality, as we know, is that families are complex systems comprising of individuals who interact in even more complex systems, and interventions are not designed in ways that acknowledge these complexities (Guastaferro et al., 2017). Practitioners understand that for many families, crises can arise at any time and often exacerbate pre-existing challenges. These might include a range of practical issues but also more complex psycho-social challenges. Financial issues can be common, but some families also experience trauma, abuse, addiction, incarceration and ill health, and often several of these at once (Lee, Anderson, Quranta and Shim, 2018). In one review from Northern Ireland, families experienced an average of four issues concurrently (Walsh and Doherty, 2016). Blended models, in which distinct but complementary interventions are available to families based on their specific needs, is one intervention model which is currently under-evaluated. In this study we examined the extent to which families engaged in a complex, early intervention programme in which they could access multiple interventions concurrently to address different issues experienced by family members; and whether it is possible to attribute improved outcomes to this engagement.
The Colin Early Intervention Community (CEIC)
The Colin Neighbourhood is located on the outskirts of greater West Belfast in County Antrim, has a population of approximately 30,000 people, and falls within the three per cent of most deprived areas of Northern Ireland. The unique demographic and deprivation features have contributed to a range of poor health, education and social outcomes for children and families on an inter-generational basis (NCB, 2012). The Colin Early Intervention Community (CEIC), which consists of a range of statutory, voluntary and community organisations, set out in 2011 with a vision that organisations would work differently together and be more coordinated in their approach. Implementing an Outcomes Based Accountability (OBA) Framework (Friedman, 2015), the CEIC provided data on outputs and outcomes for each quarter which were then converted into ‘report cards’ for dissemination. Report cards are summaries of analysed data, and report on key questions within the context of the OBA Framework, namely (1) how much did we do (2) how well did we do it and (3) is anyone better off? In many ways, this ‘practice informed by evidence’ approach was still novel during the inception of CEIC and represented a strategic shift by service providers in the voluntary, community and statutory sectors. In 2017, a team of researchers from Queen’s University Belfast undertook a secondary analysis of the report card data collected by CEIC since its inception, with support from the Children’s Research Network Prevention and Early Intervention Research Initiative Research Grant Scheme.
Aims of the study
The aim of this retrospective, secondary analysis was to identify the extent to which families engaged in multiple interventions to address, or seek support for, a range of issues or challenges (see Table 1). This article outlines the process by which these data were collected and analysed, as well as the findings that emerged from that analysis.
Secondary analysis involves the re-analysis of data that were collected by someone else for a different purpose (Boslaugh, 2007). Despite a growth in the amount of data available within the community and voluntary sector, few datasets have been made available for further or secondary analysis. The first step in the process was to review the report card data. Familiarisation with the data is a critical step in understanding the utility of the information and what subsequent steps need to be taken to answer the research question. The second step in the process was to identify a method of standardising the data so that different service users could be linked across programmes. CEIC established a coding system during 2016. This was used as the cut-off point for the data included in the analysis. It was apparent that not all interventions subsequently used this coding system and therefore, our analysis was limited to families that could be tracked across different interventions. The third step was to become familiar with the clinical measures being used. The fourth step involved recoding of the existing data to facilitate analysis. The fifth step involved analysis of that data and the findings are reported below. The details of 207 families were collated from the raw report card data. The data were analysed using SPSS Version 22. Descriptive statistics were conducted to examine the means, standard deviation and range across all variables. Independent samples t-tests were used to compare means between variables of interest. Cross-tabulations were used to compare categorical variables and Chi-square tests of independence were used to explore statistical relationships between them.
Summary of the findings
The CEIC programme comprises different, but complementary, intervention components. Because the aim of the study was to explore how individual family members access different elements of the programme concurrently, only those interventions which collated data using the same family identifiers were included in the review. Following the first review of all data, four interventions were included in the study (see Table 2). Some of these interventions focussed on the whole family, others solely on parents, and others on children or young people. The mode of delivery varied between individual and group work and was an important factor in recruitment and retention. This suggests that despite similarities, each intervention was positioned to address and respond to variability within families. While it is not clear from the report card data how families were engaged in the programme or what all those pathways were, it appears that a mixture of professional referral as well as self-referral was common. Where referral data were collected, individual projects documented basic demographic data, such as name and age, as well as more specific information, such as presenting needs. However, there were few cases of documented referral data on families, and when it was evidenced there did not appear to be a coherent or consistent way of recording pathways into the CEIC. Some documented referral data included comprehensive needs analysis, whilst others collected only contact details and a brief statement of perceived need. Interestingly, where these data were collected, these families were more likely to engage in multiple interventions, indicating that their needs may have been more fully understood. Comparison of means of the number of interventions engaged in by families differed between those with referral information (m=1.48 interventions) and those without (m=1.11 interventions) and illustrates that this was at the point of statistical significance (t, 73=3.6, p=.001). When referral information was routinely collected, families engaged in more elements of the programme.
There was evidence that a sizeable minority engaged in more than one programme. In fact, eighteen per cent (n=38) of families who engaged in the reviewed interventions within the specific time period (January 2016 to September 2017) engaged in multiple programmes. Within this group, the average number of interventions engaged in was 2.22, and ranged between two and four interventions.
Those who engaged in more than one intervention were more likely to complete the interventions. For instance, twelve per cent (n=20) of those who engaged in only one intervention left their intervention without any demonstrable evidence of partial or full completion. On the other hand, only three per cent (n=1) of those who completed more than one intervention left the intervention without any evidence of engagement.
The potential for data to inform practice is often missed. Given the potential risks, the real-time decision making, and the requirement for staff to be informed by evidence, services require data driven decision making, but the systems that facilitate this are not always in place. In this study, despite positive efforts to achieve this, we found little evidence that data were collected routinely to guide and inform practice. For example, it was difficult to track families across most of the interventions; it was difficult to establish which families came into CEIC and for what reason(s); the pathway(s) into the programme were not clear; and it was difficult to establish how decisions were made on which family members received which intervention, who made these decisions, and at which points these decisions were made. It was therefore difficult to answer our fourth research question, how are decisions taken around who engages in which intervention and when? The reasons for this appear to be related to the evaluation frameworks established by and on behalf of the commissioner of each evaluation. Data collected focussed on individual interventions as opposed to the synergy between the interventions; focussed on outputs such as the number of participants within each intervention; and focussed on outcomes such as the difference between baseline and endpoint measures. While this data is useful, particularly for funders and policy makers who appear to prefer synopsis rather than detail, it is limited. Some families living in Colin are likely to experience one or more significant social issues (NCB, 2012) and we know from other empirical studies that the cumulative effect exacerbates already complex issues (Finkelhor, Shattuck, Turner and Hamby, 2015). It has been proposed that single issue interventions (for example, those that attend to parenting practices) are not and cannot be sufficient to address other concurrent issues (Guastaferro et al., 2017). Therefore, blended models in which interventions are simultaneously implemented provide a conceptual framework to design complex services and potentially enhance outcomes that some families experience. It appears that in some cases, some families do receive multiple interventions, but this is neither strategically designed nor coherently coordinated. The CEIC programme provides some evidence that for families with chronic and complex needs, blending interventions could provide a platform for greater strategic delivery and coherent coordination.
Conclusion and recommendations
Engaging, supporting and improving outcomes for families who experience multiple adversity is complex. It is well established that single issue interventions cannot address multiple issues concurrently - they are not designed to. CEIC has demonstrated a vision of improving outcomes at the community level through coordinating services and the use of best evidence. Combining interventions is a novel approach and the mechanisms by which they are implemented are currently undervalued. This study demonstrated that there are some significant challenges associated with implementing blended approaches. Table 3 overleaf outlines the key lessons and recommendations.