Introduction:
Health care settings are considered as complex systems because of the existence of high interaction between different departments of a setting as well as the presence of uncertainty such as demand and service time uncertainty. One of the basic objectives pursued by most countries is to improve their health system both in terms of quality services and efficiency and the extent to which its resources are put to good use (1).
Laboratory is a body that performs one or more of the following activities: Testing, calibration and sampling, associated with subsequent testing or calibration.
The evaluation of the efficiencies of laboratories is very important and beneficial since it is of benefit to the laboratory and hospital managers to facilitate an understanding of the current situation of the ongoing processes and to allow for targeted measures in their respective areas. The process can help measure the effects of strategic decisions, in hospitals and laboratories. The improvements can be used as targeted objectives for each lab and as a part of continual improvement. The efficiency scores can be used as key performance indicators for each lab.
The Data envelopment analysis (DEA) Model, developed by Charnes, Cooper and Rhodes is a very useful methodology as it measures the productivity and efficiency related to organisational units, like hospitals, particularly clinical laboratories which use numerous resources to produce multiple products/services (2,3). DEA offers the best approach to comparative performance assessment for health care managers. It is a powerful, yet easily understandable, analytic tool by which health care managers can assess relative performance in intra or inter departments venues. DEA is a model that measures efficiency and may be described as an extension of the simple input/output analysis ratio which is rigorously generalised to work with multiple inputs and outputs. It uses mathematical models (lineal programming) to calculate an efficiency limit. This limit provides a reference for judging comparatively the results of the remaining units that do not belong to the limit (4).
The study aims to analyse and measure performance efficiencies of highly specialised clinical laboratories at a University teaching hospital and to recommend ways to improve performance in the areas of inefficiencies using DEA approach.
Material and Methods
A retrospective study was done over a period of 3 months from March to May 2019. Molecular Biology lab, Molecular Oncology lab, Virology lab and Cytogenetics lab were included in the study and data was collected from January 2018 to March 2019. Input data collected were number of samples processed in each lab, tests Mix in each lab, number of staff involved in performing and reporting the tests and number of major equipment used in processing tests. Among the inputs, number of samples was taken as ‘uncontrolled input’ as it is not under our direct control. Rest all were considered to be ‘controlled input’. Revenue was taken as the output. Banxia’s Frontier Analysis software for Data Envelopment Analysis was used to calculate relative efficiency (5). We analysed using both input and output orientation approach and used the variable returns to scale (VRS) model.
Result
In the study, highest number of samples were processed with highest tests mix and revenue generated in Molecular Biology lab followed by Cytogenetics, Virology and Molecular Oncology. Number of staff involved in processing tests was highest in Molecular Biology followed by Molecular Oncology and Cytogenetics and Virology. Virology had the maximum number of major equipment while Cytogenetics had the least. (Table 1)
| Table 1: Input and Output data for each laboratory |
| Laboratory |
Input 1 |
Input 2 |
Input 3 |
Input 4 |
Output 1 |
No. of Samples |
Tests Mix |
No. of staff |
No. of major equipment |
Revenue |
Molecular biology |
4518 |
40 |
10 |
7 |
2,06,46,230 |
Molecular oncology |
213 |
10 |
6 |
8 |
12,55,450 |
Virology |
937 |
17 |
5 |
11 |
59,04,430 |
Cytogenetics |
2308 |
22 |
6 |
3 |
1,38,93,230 |
Using ‘Minimise input’ as the optimisation mode, i.e minimising the inputs to produce the same output (reducing resources used) both Molecular Biology and Cytogenetics lab were found to be relatively highly efficient (100%) followed by Virology (44.60%) and Molecular Oncology (14.30%) being least efficient. (Table 2)
| Table 2: Relative Efficiency Scores of labs (Minimise input) |
| Laboratory |
Score |
Efficiency (High/Medium/Low) |
Condition |
Cytogenetics |
100.00% |
High |
Green |
Molecular Biology |
100.00% |
High |
Green |
Molecular Oncology |
14.30% |
Low |
Red |
Virology |
44.60% |
Low |
Red |
Using ‘maximise output’ as the optimisation mode i.e maximising the output for the inputs used (getting more out of the process), Cytogenetics lab was found to be highly efficient (100%) followed by Virology (87.5%), Molecular Biology (84.3%) and Molecular Oncology (75.7%). (Table 3)
| Table 3: Relative Efficiency Scores of labs (maximise output) |
| Lab |
Score |
Efficiency (High/Medium/Low) |
Condition |
Cytogenetics |
100.00% |
HIGH |
Green |
Molecular Biology |
84.30% |
LOW |
Red |
Molecular Oncology |
75.70% |
LOW |
Red |
Virology |
87.50% |
LOW |
Red |
Discussion
Avkiran and Yang and Kuo, in two independent studies on Australian universities and on facility layout design showed that DEA has the capability of discriminating between multiple efficient units or organizations and determining the most efficient unit or organization (6,7). Additionally, DEA considers multiple inputs and outputs at the same time in the presence of complex relationships between inputs and outputs. This makes it different from several other efficiency approaches.
Zheng and his co-workers did a four stage DEA based efficiency evaluation of public hospitals in China (8). How to improve the technical efficiency of hospitals to achieve economy of scale is significant not only to hospital development, but also to better meet the increasing healthcare needs and demands (1). Health care service performance data are routinely collected and reported from each service/department/unit. The next logical step is to begin comparing and contrasting the performance of health care providers. By making these comparisons, best practice providers including laboratory services can be identified and used as benchmarks for improving the efficiency, quality, and effectiveness of similar programs/services delivered by other providers (9,10).
Health care settings are considered as complex systems because of the existence of high interaction between different departments of a settings as well as the presence of uncertainty such as demand and service time uncertainty. DEA is an approach for evaluating the relative efficiency of either different organizations or different units in one organization (1).
Data enveloping analysis (DEA) is a multiple-input multiple-output nonparametric evaluation method and has already been widely employed to estimate the relative efficiencies of public hospitals, particularly over recent years, including measures of overall efficiency and technical efficiency, pure technical efficiency and scale efficiency (8).
In the study, highest number of samples were processed with highest tests mix and revenue generated in Molecular Biology lab followed by Cytogenetics, Virology and Molecular Oncology. Number of staff involved in processing tests was highest in Molecular Biology followed by Molecular Oncology and Cytogenetics and Virology. Virology had the maximum number of major equipment while Cytogenetics had the least. (Table 1)
Using ‘Minimise input’ as the optimisation mode, i.e minimising the inputs to produce the same output (reducing resources used) both Molecular Biology and Cytogenetics lab were found to be relatively highly efficient (100%) followed by Virology (44.60%) and Molecular Oncology (14.30%) being least efficient. (Table 2)
Reference set frequency shows the number of times an efficient units’ appears in the inefficient unit’s reference set. The higher the number, the more representative of ‘Best Practice’ a reference unit is. The reference set frequency for Cytogenetics lab was 3 labs while Molecular Biology was 1 lab. Therefore, Cytogenetics lab should be considered to represent the ‘Best Practice’ available. Among the inputs, contribution of Tests mix was highest (65%), followed by number of samples (15%), number of major equipment (10%) and least by number of staff (9.9%). Using ‘maximise output’ as the optimisation mode i.e maximising the output for the inputs used (getting more out of the process), Cytogenetics lab was found to be highly efficient (100%) followed by Virology (87.5%), Molecular Biology (84.3%) and Molecular Oncology (75.7%). (Table 3) Cytogenetics lab being the highly efficient lab had the reference set frequency as 4 and should be considered to represent the ‘Best Practice’ available. DEA is a powerful analytical too that is user friendly and is used by healthcare managers to perform a comparative assessment of performance. The data generated from it can also become the health care/lab manager’s guide for laboratory and process improvement.
As per this study when minimising the inputs to produce the same output (reducing resources used) is done, suggested potential improvements for Molecular Oncology lab are to increase number of samples from existing 213 to 25542 (11891%). The test mix should be lowered from 10 to 2 (-80%), number of staff from 6 to 1 (-90%), major equipment from 8 to 1 (-96%) in order to achieve the same output. (Table 4). Similarly, the potential improvements for Virology lab suggested are to increase number of samples from existing 937 to 5431 (480%) and reduce test mix from 17 to 10 (-45%), number of staff from 5 to 3 (-49%) and major equipment from 11 to 2 (-88%) in order to achieve the same output. (Table 4)
| Table 4: Potential improvement for Molecular Oncology lab and Virology lab (minimise inputs) |
Input / Output Name |
Laboratory |
Value |
Target |
Potential Improvement |
No. of Samples |
Molecular Oncology lab |
213 |
25,541 |
11891.13% |
Virology lab |
937 |
5,431 |
479.59% |
Tests Mix |
Molecular Oncology lab |
10 |
2 |
-80.12% |
Virology lab |
17 |
10 |
-45.00% |
No. of staff |
Molecular Oncology lab |
6 |
1 |
-90.96% |
Virology lab |
5 |
3 |
-49.00% |
No. of major equipment |
Molecular Oncology lab |
8 |
1 |
-96.61% |
Virology lab |
11 |
2 |
-88.41% |
Revenue |
Molecular Oncology lab |
12,55,450 |
12,55,450 |
0.00% |
Virology lab |
59,04,430 |
59,04,430 |
0.00% |
As per this study when maximising the output for the inputs used (getting more out of the process) is done, suggested potential improvements for Molecular Oncology lab are to increase number of samples from existing 213 to 276 (29%) and Tests mix, number of staff and major equipment to be used are 3 (-73%), 1 (-88%) and 1 (-95%) respectively. If these improvements are implemented, revenue can be expected to increase from Rs 12,55,450/- to Rs 16,59,040/-. (Table 5) The potential improvements for Molecular Biology lab suggested are to lower number of samples from existing 4518 to 4071 (-9.9%), tests mix from 40 to 39 (-3%) and major equipment from 7 to 6 (-24%). Number of staff may be kept same or increase by 1 (5.82%). If these improvements are implemented, revenue can be expected to increase from Rs 2,06,46,230/- to Rs 2,45,03,744/-. (Table 5) The potential improvements for Virology lab suggested are to increase number of samples from existing 937 to 1121 (20%). Tests mix, number of staff and major equipment to be used to maximise the output suggested are 11 (-37%), 3 (-42%) and 2 (-88%) respectively. If these improvements are implemented, revenue can be expected to increase from Rs 59,04,430/- to Rs 67,46,370/-. (Table 5)
| Table 5: Potential improvements for Molecular Oncology, Molecular Biology and Virology labs (maximise output) |
| Input / Output Name |
Laboratory |
Value |
Target |
Potential Improvement |
No. of Samples |
Molecular Oncology lab |
213 |
276 |
29.39% |
Molecular Biology lab |
4518 |
4,070.66 |
-9.90% |
Virology lab |
937 |
1,120.73 |
19.61% |
Tests Mix |
Molecular Oncology lab |
10 |
3 |
-73.73% |
Molecular Biology lab |
40 |
38.8 |
-3.00% |
Virology lab |
17 |
10.68 |
-37.16% |
No. of staff |
Molecular Oncology lab |
6 |
0.72 |
-88.06% |
Molecular Biology lab |
10 |
10.58 |
5.82% |
Virology lab |
5 |
2.91 |
-41.73% |
No. of major equipment |
Molecular Oncology lab |
8 |
0.36 |
-95.52% |
Molecular Biology lab |
7 |
5.29 |
-24.41% |
Virology lab |
11 |
1.46 |
-86.76% |
Revenue |
Molecular Oncology lab |
12,55,450 |
16,59,039 |
32.15% |
Molecular Biology lab |
20646230 |
245,03,743 |
18.68% |
Virology lab |
5904430 |
67,46,369 |
14.26% |
The evaluation of the efficiencies of laboratories is very important and beneficial since it is of benefit to the laboratory and hospital managers to facilitate an understanding of the current situation of the ongoing processes and to allow for targeted measures in their respective areas. The process can help measure the effects of strategic decisions, in hospitals and laboratories. The improvements can be used as targeted objectives for each lab and as a part of continual improvement. The efficiency scores can be used as key performance indicators for each lab. An inefficient clinical laboratory would also impose either high cost or lower clinician/patient satisfaction under certain healthcare settings. For example, if the healthcare setting is involved with high volume of patients with multiple laboratory investigations, hospitals might have to put in lot of investment in the form of laboratory staff and equipment in order to cater to all the samples to be reported accurately and on time. At times, this might delay treatment related decision making by clinicians. This study highlights the use of DEA in measuring relative efficiency of clinical laboratories. The findings suggest that having a well-equipped lab in terms of staff and equipment need not contribute or correlate highly to its efficiency. The key lies in effectively utilising all the available resources (inputs) in processing the highest possible number of samples.
As health managers, one can use the Importance/Performance matrix (Figure 1) as a determinant of priority for improvement. According to the study findings Molecular Oncology lab needs to be focussed first for achieving potential improvements.
 |
Figure 1: Importance-Performance matrix |
The main benefit of this analysis lies in the fact that performance is based on taking all the available data into account, so it gives a good reflection of overall performance and because it is a peer based comparison the targets set for comparison are realistic and therefore more likely to be achievable. Ideally this type of study should be done at the beginning of an improvement process. However, even if done on a regular basis to give updates on performance, it provides key input as an action plan to achieve some of the potential gains identified. The importance of healthcare efficiency is extremely high, given the rapid growth in healthcare costs and the increasing numbers of people covered by publicly-financed programs. To identify useful healthcare productivity improvements, efficiency must be validly measured at individual departments including laboratories. On the other hand, if healthcare efficiency is incorrectly measured, then governmental policy makers and hospital managers may respond in ineffective and even counterproductive ways (11).
Hollingsworth B, and Street A pointed out in their study that DEA studies done in hospital set up need to be applied correctly for an effective practice and use by healthcare policy makers and healthcare managers (12). Al-Refaie, Fouad, Li and Shurrab applied the integration of simulation and DEA in the emergency department to find out the best distribution of nurses that helped to increase utilization and decrease waiting time of patients (13).
Conclusion
The hospital management always is in search of tools for optimizing patient care, studying the various processes in the operational areas and increasing their efficiency and effectiveness. DEA serves as a practically effective and a useful healthcare analytical tool that helps to evaluate relative efficiencies of clinical laboratories in hospitals. The evaluation of four special labs at a tertiary care hospital revealed that Cytogenetics lab was the most efficient lab followed by Molecular Biology and Virology lab. Molecular Oncology lab was found to be the most inefficient lab. These inefficiencies should be treated very carefully as it also reflect under-utilisation of available resources. The scope for relative efficiency improvement is high for both Molecular Oncology and Virology lab.
By tracking efficiency, managers and decision makers can reward service providers that are performing to the necessary standards and provide guidance to those who aren’t. Thereby, this study helps achieve increased responsiveness to performance. To conclude, it is important to pay attention to efficiency evaluation as it is necessary in solving the problem of limited and unbalanced medical resources. The idea and method for evaluation of relative efficiency should be put into practice. The ultimate aim of the present study is to put our conclusions into practice.
Ethical approval: Ethical approval for the study was obtained from the Institutional Ethics Committee.
Acknowledgement: Faculty and staff of Cytogenetics, Molecular Biology, Molecular Oncology and Virology labs
Conflict of interest: Nil
Sources of Funding: No grant was received for this study from any agency.
References
- Tarazona M, Clemente I, Consuelo D, Martínez I. A model to measure the efficiency of hospital performance. Mathematical and Computer Modelling. 2010;52:1095-1102.
- Charnes A, Cooper W, Rhodes E. Measuring the efficiency of decision making units. European Journal of Operational Research. 1978;2(6):429-444.
- Banker RD, Charnes A, Cooper W. Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis. Management Science. 1984;30: 1078-1092.
- Charnes A, Cooper W, Lewin A, Seiford L. Data Enrolment Analysis. Theory, Methodology and Applications, Kluwer Academic Publishers, 1994 (Chapters 1-3).
- Frontier Analyst:
Data Envelopment Analysis software. Available at https://banxia.com/frontier/
- Avkiran N. Investigating Technical and Scale Efficiencies of Australian Universities through Data Envelopment Analysis. Socioeconomic Planning Sciences.2001;35(1): 57-80.
- Yang T, Kuo C. A hierarchical AHP/DEA methodology for the facility layout design problem. European Journal of Operational Research. 2003;147(1):128-136.
- Zheng W, Sun H, Zhang P, Zhou G, Jin Q, Lu X. A four-stage DEA-based efficiency evaluation of public hospitals in China after the implementation of new medical reforms. PLoS ONE. 2018;13(10).
- Balf F, Rezai H, Jahanshahloo G, Lotfi F. Ranking efficient DMUs using the Tchebycheff norm. Applied Mathematical Modelling. 2012;36:46-56.
- Barros C, Managi S, Matousek R. The technical efficiency of the Japanese banks: Non-radial directional performance measurement with undesirable output. Omega. 2012;40:1-8.
- Barnum D, Walton S, Shields K, Schumock G. Measuring Hospital Efficiency with Data Envelopment Analysis: Nonsubstitutable vs. Substitutable Inputs and Outputs. Journal of Med Systems. 2011;35:1393-1401.
- Hollingsworth B, Street A. The market for efficiency analysis of health care organizations. Health Economics. 2006;15:1055-1059.
- Al-Refaie A, Fouad RH, Li MH, Shurrab M. Applying simulation and DEA to improve performance of emergency department in a Jordanian hospital. Simulation Modelling Practice and Theory. 2014;41,59-72.
|
