Sláinte! BHCOE is proud to take part in three upcoming presentations at ABAI’s 11th International Conference.
When: September 1st–3rd, 2022
Where: The Convention Centre Dublin, Spencer Dock, N Wall Quay, North Wall, Dublin 1, D01 T1W6, Ireland
Register for the Conference
BHCOE Presentation Details
Chill for a SEQ: A Discussion on Service Efficiency and Quality
Date: Friday, September 2nd, 2022
Time: 10:30 AM–12:20 PM
Location: Meeting Level 2; Ecocem Room
Area: AUT; Domain: Basic Research
Chair: Sara Gershfeld Litvak
Discussant: Hanna Rue, Ph.D., BCBA-D
What patients, their parents/guardians, insurance providers, and the community at large have in common with direct care and supervisory-level staff and ABA service organizations is that each stakeholder cares about quality services. To assure all stakeholders that quality services are being provided, a neutral entity must set standards to determine what is considered quality. Such standards are typically based on the scientific literature, and where research may fall short, subject matter experts provide guidance based on best practice. Then, to determine if services meet these standards, an objective entity conducts thorough evaluations using reliable assessment methods. In this symposium, we will share how BHCOE has developed a quality assurance system with a focus on our accumulated data collected over the past six years. The first presenter will discuss best practices in quality assurance systems for the field of ABA and how organizations perform when they are evaluated on BHCOE’s standards. The second presenter will discuss factors that can predict changes in VB-MAPP and Vineland scores for individuals with autism spectrum disorder. The third and fourth presenters will share the results of evaluations with patients’ guardians and employees. All presenters will discuss the implications of their findings and future steps.
Instruction Level: Intermediate
First Presentation: Exploring Indicators of Healthy Business Practices and High-Quality Applied Behavior Analysis Services
Authors: Sara Litvak, Nichole Williams, and David J. Cox
Abstract: Essential components of the quality assurance process include collecting, analyzing, and reporting data. One way to evaluate quality assurance processes is establishing, monitoring, and evaluating key performance indicators developed around clinical and administrative outcome data. Over the past six years, the Behavioral Health Center of Excellence has evaluated over 500 ABA service providers serving cumulatively 42,000 patients and employing approximately 25,000 staff. In this presentation, we will provide information regarding how these organizations perform when we evaluate them on BHCOE’s standards, which are indices of their clinical and administrative outcomes. We will describe best practices in quality assurance systems for the field of ABA. For example, we share how organizations are doing with competency-based training around diversity, equity, and inclusion, and employees’ satisfaction with how diverse their organizations’ employees are. We review data surrounding supervisor caseload, percent utilization of direct therapy, percentage of supervision hours utilized, and percent utilization of parent/guardian training. Overall, attendees will learn various methods to measure and monitor quality-related KPIs, ways to make comparisons using industry-average benchmarks on quality-related metrics, and how to become involved in future areas of research and collaboration.
Second Presentation: Understanding Guardian Satisfaction with ABA Treatment
Authors: Ellie Kazemi, Micah Friddle, and David J. Cox
Abstract: Measuring patients’ and their guardians’ satisfaction with services are critical to ensure that interventions based on behavior analysis are socially valid. However, we currently do not have a thorough understanding of the variables that have the greatest impact on patient satisfaction within behavior analysis service delivery settings. The purpose of this study was to determine the most likely factors that influence the satisfaction of patients and their guardians. To help ABA organizations focus their limited time and attention on only the most important variables that predict guardian satisfaction, we analyzed data collected during a multi-modal quality assurance evaluation of over 500 ABA service organizations. This dataset contained several parent/guardian satisfaction measures as well as several objectively defined metrics of service delivery. In this presentation, we provide general quantitative descriptions of these variables and also share how well we can predict guardian Net Promoter Score when we analyze the parent/guardian satisfaction relative to various stratifications of our data such as staff’s interpersonal skills, specific components of service delivery, and severity of patient impact. These results will help organizations determine how to identify areas of need for their organization that will maintain high levels of guardian satisfaction.
Third Presentation: Predicting Employee Satisfaction from Quality Assurance Systems
Authors: Ellie Kazemi, Melissa Cottengim, and David J. Cox
Abstract: Employee satisfaction directly impacts organizational success. With the Great Resignation of 2021, many ABA service providers are prioritizing initiatives that directly address employee satisfaction and retention efforts. The purpose of this symposium is to demonstrate how those quality assurance systems might allow you to predict and improve organizational key performance indicators related to employee satisfaction and retention. Specifically, we discuss the results of analyses using a data set from over 500 organizations that: establish industry-average benchmarks for technician and supervisor-level employee satisfaction and turnover; and how prioritizing diversity, equity, & inclusion efforts, as well as retention, hiring, and recruitment efforts influence levels of employee satisfaction. We also discuss the results of analyses that evaluate the predictive relationships among employee satisfaction and turnover, patient satisfaction, and employee compensation to provide organizations with insight on what to prioritize for employee satisfaction initiatives. Finally, we close by discussing the limitations to our dataset as well as future directions for research.
Fourth Presentation: Organizational Outcome Data: What’s Ya bench…mark?
Authors: Ellie Kazemi, Scott Page, and David J. Cox
Abstract: A limited, but increasing, number of organizations have begun to collect and monitor data on patient outcomes using skill-based and adaptive assessments. These data allow organizations to track how well the services they provide translate to patient improvement. Last year, we provided descriptions about the number of organizations that collect outcome data, monitor patient progress, and how they identify areas of strengths and deficits based on patient outcomes. In this presentation, we use advanced analytic techniques to identify the factors that allow us to predict changes in VBMAPP and Vineland scores. To do this, we used data from 700+ patients and 40+ ABA organizations on factors such as patient demographics, ABA utilization percentages (e.g., direct, parent training), staff credentials, turnover rates, and characteristics of service delivery (e.g., home versus center-based settings). After reviewing the results of this analysis, we discuss how organizations can use patient outcomes to benchmark themselves against the industry, target areas of weakness and identify the organizational characteristics wherein improvement would likely have the greatest impact on patient outcomes. Finally, we discuss how transparency around patient outcomes has led some organizations to report growth in the number of patients seeking their services.
Defining and Measuring Outcomes for Applied Behavior Analytic Service Delivery for Autism Spectrum Disorder
Date: Friday, September 2nd, 2022
Time: 2:00 PM – 2:50 PM
Location: Meeting Level 1; Liffey Hall 2
Area: AUT/PCH; Domain: Service Delivery
CE Instructor: Ivy M Chong, Ph.D.
Chair: Ivy M Chong (May Institute)
Presenters: Sara Gershfeld Litvak; Mandy Ralston, NonBinary Solutions; Hanna C. Rue, LEARN Behavioral
Abstract: Documenting treatment outcomes in health care professions has become increasingly critical to both patients and stakeholders (e.g., third-party payors). With an ever-changing landscape, a shortage of qualified practitioners, and varying viewpoints, we are in danger of having our practice defined for us. Specifically, practitioners must demonstrate that their treatments are effective and weigh adverse effects to outcomes. Practitioners and stakeholders, such as third-party payors, agree that accountability is important, but the challenge lies in agreeing how to achieve this. Over the past year, there have been meaningful progressions in tackling this topic for our field. Most recently BHCOE released their ABA Outcomes Framework and ICHOM released their ASD Standard Set. This panel will discuss the development, methodology and implementation of these two approaches to measuring ABA/(ASD?) treatment outcomes and provide practical considerations about integrating these resources into daily behavioral practice.
Instruction Level: Intermediate
Target Audience: This presentation targets clinicians and practitioners working with individuals with ASD and looking to aggregate big data to determine impact at the individual and systems level.
Learning Objectives: (1) Describe the current problem of delineating outcomes and relevant measures from provider and payor perspectives. (2) Define proximal vs. distal vs. ultimate outcomes. (3) Name emerging and existing tools and/or resources for measuring Outcomes in practice and discuss limitations.
Means of Assessing Behavior Analytic Practice Sites
Date: Saturday, September 3rd, 2022
Time: 3:00 PM–3:50 PM
Location: Meeting Level 1; Liffey Meeting 3
Instruction Level: Intermediate
Chair: Harry Voulgarakis (St. Joseph’s College New York )
First Presentation: Standards of Behavior Analytic Supervised Experience for Practica Sites
Author: Ellie Kazemi
Abstract: High quality supervised experience is an essential component of behavior analytic training. Theoretically, the supervision guidelines provided by the Behavior Analyst Certification Board (BACB) protect candidates as supervisees who seek certification or licensure to practice behavior analysis. Additionally, there is hope that graduate training in behavior analysis provides some quality control regarding the practica sites at which candidates accrue supervision experience hours. However, unless the practica experience is offered at the institution where the graduate coursework is offered, many higher education training programs do not have the time or resources to review external practica sites to assess quality. Therefore, candidates enrolled in master’s degree programs are often on their own to find suitable sites that offer ethical and professional supervised experience that meets best-practice recommendations. Therefore, the quality of the supervised experiences of candidates preparing to become professional behavior analysts differ dramatically. In this presentation, I will review the process of standard development, the ABAI Accreditation standards, the empirical evidence and best-practice guidelines for supervision, and information we received from our diverse stakeholder groups about supervision.
Second Presentation: Assessing Cultural Responsiveness in Behavior Analytic Practice: A New Measure
Domain: Applied Research
Authors: Harry Voulgarakis (St. Joseph’s College New York ), Kerry Ann Conde (St. Joseph’s College, Behavior Science Technology), Jessica A Scher Lisa (St. Joseph’s College)
Abstract: Issues pertaining to diversity have been gaining attention in behavior analytic literature. Many papers have discussed aspects of diversity that need to be addressed in both didactic and clinical settings, as well as recommendations for practice to promote more culturally responsive care. Despite these efforts, there have been few data-based attempts or evaluative methods for which to assess culturally responsive care specific to the practice of behavior analysis. Authors conducted a thorough review of behavior analytic literature, literature relevant to cultural responsiveness in similar fields, and ethical practice considerations. Based on these sources, a standardized scale was developed to assess cultural responsiveness for behavior analysts. The purpose of this presentation is to discuss the importance and relevance of a new measure of assessing cultural responsiveness in higher education behavior analytic programs. Initial data from the validity testing will be presented, as well as pilot results for cultural responsiveness across a number of areas. In particular, initial results yielded variable data across some areas, while they presented with notable limitations in cultural responsiveness in other areas. Implications for training behavior analysts will be reviewed, as well as how to further research and use this measure in academic and clinical settings.