| Do Patterns Affect Design Undertandability? |
|
This e-mail address is being protected from spambots. You need JavaScript enabled to view it , This e-mail address is being protected from spambots. You need JavaScript enabled to view it y This e-mail address is being protected from spambots. You need JavaScript enabled to view it Abstract At the present moment in time, patterns have reached a zenith in popularity and diffusion thanks to the concern of the community to discover, classify and spread knowledge about all types of patterns. Patterns are useful elements, but there are still many features to be studied if we want to apply them in a rational manner. The improvements in quality that patterns can inject into design is a key issue to analyze, so there needs to be a complete body of empirical knowledge dealing with that. This paper tackles the question of whether design patterns can help to improve the ease with which designs are understood. An empirical study, composed of one experiment and a replica, has been conducted with the aim of validating our conjecture. The results suggest that patterns contribute to improving the understandability of the design. Keywords: Design Patterns, Understandability 1. IntroductionFor many years now, design patterns have been associated with the improvement of software maintainability. Popularly, patterns and maintainability are synonymous with well-designed software. A quality design that has a high level of maintainability could thus have a clear effect on the productivity of developers. The effectiveness of maintenance is, furthermore, greatly hindered by poor design [1]. These seemingly obvious statements are questioned by many professionals. Glass [2] states that well-designed systems will be improved more rapidly, have a longer life, and hence, paradoxically, consume more maintenance. Reibing [3]comments that if we have two similar designs, A and B, for the same problem, B using design patterns and A not using design patterns, B should have a higher quality than A, but if we apply “classic” object-oriented design metrics to both designs, the metrics tell us that design A is better. In the present state of affairs, there is a serious lack of hard scientific evidence for the usefulness of patterns in many situations [4]. The relationship between maintainability and patterns, then, is not clear, nor are the positive or negative effects of patterns. Regarding design maintainability, understandability is one of the key characteristics to consider, as it can influence the cost and reliability of software design evolution or maintenance. As a matter of fact, understandability is one of the five characteristics a “good” model must have, along with abstraction, accuracy, predictiveness and inexpensiveness [5]. There is no firm and formal theory that links patterns to these software quality concepts, however. Many authors claim that Design patterns claim to bring about understandability [6], but we need to build an empirical body of knowledge to confirm these assertions. The question, therefore, is this- what effect do design patterns have on understandability? This paper proposes to seek empirical evidence about whether using design patterns is beneficial for software design understandability. We conducted two experiments to compare performances of designs with patterns to alternatives where patterns were not used. The remainder of this article is organized as follows: Section 2 illustrates the experiment settings; Section 3 summarizes data analysis and its interpretation; Section 4 outlines the limits of the experiment. Finally, conclusions are drawn and future work is set out in section 6. 2. Description of the Experimental SettingsAs the main goal of the presented work is to ascertain if the use of design patterns can make the design easier to understand, our research question can be stated as: Does the use of patterns improve the understandability of design? From the above question, the general goal of our empirical study is derived as follows:
In Figure 1 an overview of the experimental plan is provided.
Figure 1. Overview of the Experiment Plan 2.1 SubjectsThe subjects of the first experiment were 10 students from the 3rd course of Computer Science at the ReyJuanCarlosUniversity (URJC, Madrid, Spain). To confirm the results, a replica was conducted at the Department of Technologies and Information Systems at the University of Castilla-La Mancha (UCLM, Ciudad Real, Spain) where 37 students from the final year (5th) of the MSc participated. The students in both experiments were familiar with UML and Design Patterns, as these were topics which they had been taught in their course of Software Engineering. Additionally, a short training session was provided for them, in which the design patterns and UML Class Diagram constructs were commented on and one example of the tasks to be performed by them in the experiment was explained. So we consider that the level of experience they brought to the experiment was acceptable. 2.2 MaterialThe material was prepared by the experiment designers and was composed of:
For the running of the experiment two sets of experimental material were obtained by grouping the diagrams in this way:
Figure 2 illustrates the UoD “Enrolment” in the two design versions:
Figure 2. Example of Experimental Material: UoD Enrolment (groups A and B versions) 2.3 VariablesThe dependent variable of our empirical study was the architectural design understandability and we measured this through the efficiency of the subjects in understanding the diagrams, i.e. the relationship between the number of right answers and the time they spent on answering them. Our main independent variable was the use or not of patterns in the diagrams. To provide a complete analysis of the experimental data, the UoD was also considered as an independent variable. 2.4 HypothesesThe experiments were planned with the purpose of testing mainly the following hypotheses:
In addition to the above objectives, the testing of the following set of hypotheses was a secondary goal of the empirical study:
2.5 Experimental Tasks and TreatmentWe applied an experimental design of confounded factorial with interaction (see Table 1). In each experiment, the subjects were randomly divided into two groups: Group A, whose members received the material set A; Group B, who were given the material set B. Each subject was required to answer the questionnaires of each diagram. In total, each subject received four diagrams of four different UoD (two of them designed by applying patterns and the other two without patterns).
Table 1. Experimental Design 3. Data Analysis and InterpretationWhen execution was over, we collected the forms filled out by the subjects, checking they were complete. As all the forms were indeed complete, all subject results were considered. Once the results were obtained, we first of all performed a descriptive analysis of the data. Table 2 shows the main descriptive statistics for the two experiments carried out:
Table 2. Descriptive Statistics The results of the first experiment show that the time subjects employed in understanding and answering the questions related to the diagrams where patterns were used was slightly lower than in non-pattern diagrams. Added to this, the correctness of answers was 13,5% better (0,4 positive difference) and the effectiveness was improved by 55% (0,00867 better score) with respect to non- use of pattern diagrams. The replica at UCLM also confirmed a better performance when understanding diagrams with patterns and in this case the differences were more significant. As a matter of fact, the time was improved by 37,7 seconds on average (58% less time), the correctness of answers was 0,48 ( a 15 % improvement) and the effectiveness score went up to 0,022519 (65%). The next step was to obtain insight as to whether these differences were statistically significant. We therefore performed an ANOVA statistical test (significance level α=0,05) in order to analyze the interaction between the independent variables under study when the measurement of the dependent variable is repeated. The statistical results for the hypotheses testing are summarized in Table 3:
Table 3: ANOVA results for understandability efficiency As we can observe in Table 3, we obtained evidence that:
Finally, in order to analyse the influence of the interaction of UoD and Usage of Patterns on understandability efficiency, a second and specific ANOVA analysis was performed with a significance level α fixed to 0,05. There were no significant differences of understandability efficiency found to come from the combined influence of the two independent variables (first experiment, p=0,183> α; second experiment α=0,062> α: H0PxUoD can be rejected in both experiments). As we can observe in box plots of Figures 3 and 4, the use of patterns allowed the obtaining of a better efficiency, irrespective of the UoD (except in UoD 1 in the first experiment where there is a very slight decrease in efficiency). This is a good insight into the usefulness of patterns for the improvement of understandability for all UoDs. It can also be deduced that different scores were obtained for each UoD, which indicates, furthermore, that the application domain can also affect the ease of understanding, as confirmed by statistical analysis (see Table 2).
Figure 3. Box Plot Diagram of UoD x Usage of Patterns Interaction in the First Experiment
Figure 4. Box Plot Diagram of UoD x Usage of Patterns Interaction in the Second Experiment 5. Limits of the ExperimentsIn the planning process of the experiments we analysed the various issues that could threaten the validity of the results. With regard to construct validity, since the measures of dependent variable were obtained from the questionnaires filled in by subjects, the measurement was objective. We provided subjects with an example of a completed form and we explained the process carefully during the prior training session. On the other hand, we tackled the aspects that could threaten the internal validity: the experiment was run with subjects who had never done a similar experiment so as to avoid persistence effects. The average duration of the experiments was an hour, which avoided the appearance of effects of fatigue; subjects were motivated to participate, since the experiments could bring benefits both as regards their training for the final examinations of the course (remembering that patterns and UML are topics included in their regular syllabus) and for their everyday work as future professionals. Plagiarism and influence among and between subjects were controlled by supervising the experimental runs. In relation to external validity, the materials and tasks used were designed bearing in mind the restrictions of time and were fitted to an average level of students. However, the subjects were not professional designers and obviously we would expect much better results if the subjects were more experienced. The limited difficulty of the tasks and the different UoD’s do make the students become suitable experimental subjects, however, Despite the above remarks on the students’ lack of expertise, these individuals are much easier to work with than some others. Nevertheless, further replications of these experiments using people already working in the field will be considered, the purpose of which will be to find the confirmation of the obtained results. The diagrams of the material were relatively simple ones, due to the experimental settings and time available. In future, more realistic diagrams (representative of real projects) must be used. 6. Concluding RemarksIn the quest to improve micro architectures and understand the effect of using patterns, we have presented an empirical study. Our goal was to verify if the use of design patterns really does improve the understandability of design, and hence help to improve its maintainability. We found evidence that design patterns do indeed affect design understandability, measured through efficiency. These results constitute a good starting point towards building up a complete body of knowledge about this issue, as there has not been much effort made in the direction of empirical studies about micro-architectures. Furthermore, as the MDA approach has reinforced an emphasis on the importance of models, we believe patterns can greatly contribute to building better applications from good quality models. We are currently working on the replication of the empirical study, this time using professionals as subjects and employing more realistic material. The extent to which patterns can affect the modifiability of the design will be a further focus of future research. 7. AknowledgementsThis research has been partially supported by the projects: MECENAS (Junta de Comunidades de Castilla-La-Mancha, Consejeria de Educacion y ciencia, PBI06-0024) and ESFINGE (Dirección General de Investigación of the Ministerio de Educación y Ciencia, TIN2006-15175-C05-05). 8. References[1] G. Wiederhold, "What is your software worth?” Communications of the ACM 49(9), 2006, pp. 65-75. [2] R.L. Glass, “Facts and Fallacies of Software Engineering”, Addison Wesley, 2003. [3] R. Reibing, “The impact of Pattern Use on Design Quality”, Proc. Of OOPSLA Workshop Beyond Design: Patterns (mis)used, Tampa Bay, Florida, USA. [4] P. Wendorff, “Assessment of Design Patterns during Software Reengineering: Lessons Learned from a Large Commercial Project”, Proc. of the Fifth European Conference on Software Maintenance and Reengineering (CSMR), Lisbon, Portugal, IEEE Computer Society, pp. 77-84. [5] Selic, B., “The Pragmatics of Model-Driven Development”, IEEE Software, 20 Vol. 5, 2003, pp. 19-25. [6] E. Gamma, R. Helm, R. Johnson, J. Vlissides, “Design Patterns: Elements of Reusable Object-Oriented Software”, Addison-Wesley Professional, 1995. |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
