Reviewing paper ‘Developing a Software Quality Framework for low-code Model Driven Development Platforms based on Behaviour Driven Development Methodology’ (Braams, 2017).
Application of a Behaviour Driven Development approach
A case study at one company eMagiz using the Low-Code Model Driven Development (LCMDD) platform Mendix is executed, to examine a possible fit of the Behaviour Driven Development (BDD) methodology and LCMDD for developing a testing within LCMDD. Based on a practitioner’s experience (Outsystems) the author assumes that BDD is the best fit to LCMDD for quality assurance. A literature study on BDD is presented. The maturity of testing in the LCMDD field is expressed based on Testing Maturity Model (which is part of the Capability Maturity Model) and it is concluded that LCMDD is nearing the end of level 2 and moving towards level 3. The paper concludes that “the potential for automation in the context of LCMDD is hindered by the low maturity of the entire field…”. The paper presents a brief LCMDD software quality framework, which should be validated with further empirical research.
The research and its results
The research question is not very clear from the start of the paper. In the abstract the author presents a short research question in the second half of a sentence “…., research is needed to investigate a possible quality assurance approach”. This could have been made clearer by adding a hypothesis; “would a Behaviour Driven Development (BDD) approach improve quality assurance within the LCMDD field”.
“By 2024 three-quarter of large enterprises will be using at least four low-code tools for both IT application development and citizen development initiatives”.
www.gartner.com accessed 4th of October 2019
The research question is very relevant as more and more companies are discovering the potential benefits from using a Low-Code platform. Based on Gartner’s ‘Magic Quadrant for Enteprise Low-Code Application Platforms’, by 2024 low-code application development will be responsible for more than 65% of application development activity (www.gartner.com accessed 4th of October 2019). Understanding any changes on quality assurance processes when using low-code platform is a relevant for all companies starting using them.
The research presented in the paper is original. When doing further research on this topic using key words in different combinations: no-code; low-code; testing; quality assurance; behaviour driven development; strategy, no peer-reviewed papers where found on the same or corresponding topics.
Adding a clear description of BDD with its six characteristics according to Solis & Wang (2011), would have improved understanding the BDD concept. Solis & Wang argue that BDD focuses mainly on the implementation phase of a software project and provide limited support in the analysis and planning phase. To connect quality assurance in the LCMDD situation with its context of quality assurance within a company as a whole, a connection of the proposed quality framework with the EFQM model can add further understanding of the interaction between IS quality and quality assurance within the company’s processes. Information systems dimensions should be integrated as supportive mechanisms in the EFQM for successful application of EFQM (Sadeh, Arumugam & Malarvizhi, 2013). Sadeh et. al., propose an IS-QM model where information systems’ quality processes are interlinked with the other EFQM dimensions.
“Netherlands’ largest construction company the BAM is using Mendix for their operation excellence goals”
www. mendix.com/customer-stories/ accessed 5th of October 2019
There is a literature study included on four researchers and a case study within one small company. The case study was conducted by doing three semi-structured interviews at eMagiz. The literature study includes several relevant thinkers on testing and quality. The case is of limited value as it includes only one small company, having only one client. An addition case study within a large company would have reduced the limited contribution of the findings. For instance Netherlands’ largest construction company the BAM is using Mendix for operational excellence (among others) mendix.com/customer-stories/ (accessed 5th of October 2019).
“A Resource Based View: Information Systems are not sufficient to create competitive advantage”
(Taher, 2012)
Adding a holistic view on how and why companies should use LCMDD like a Resource Based View (RBV) approach, would improve the context of LCMDD. According to Taher (2012), information systems are essential but not enough to attain competitive advantage. As part of the BDD approach – IT resource and non-IT resources are interacting in creating value and quality with LCMDD. How are these resources interacting and influencing each other? When applying a RBV, could resource orchestration (Taher, 2012) support quality assurance in LCMDD projects?
BDD, DevOps & ‘Silo Busting’
The research contains a literature study and a case study including three interviews at one small company. The literature research seems appropriate where it includes important frameworks from Cynefin, Adzic and Agile testing matrix. The case study is of limited value and does not support the finding and proof for the developed software quality framework.
“Outsystems has incorporated BDD and DevOps functionality into their low-code platform”
https://www.outsystems.com/news/outsystems-five-new-devops-features-low-code/ accessed 5th of October 2019
The value of the findings could be increased by referring to the fact that Outsystems.com (as the author is referring to this company to legitimate the linkage between BDD and LCMDD) has incorporated BDD in their quality framework in combination with a DevOps way of working (www.outsystems.com accessed 5th October 2019). Further interesting question here is to examine the connection of BDD with DevOps. For instance, to call in another practitioner IBM, they are actually linking BDD directly to a DevOps culture in ‘Build a DevOps culture and team’ (www.ibm.com accessed 5th October 2019). A DevOps culture is characterised by collaboration across roles and departments focusing on the business objectives instead of the departmental objectives (ww.ibm.com). Another important aspect on QA approach is the culture within the company. Further research on the impact of the culture on the effectiveness of the proposed quality framework would add to possible application. Further research on the question how a DevOps culture relates to the ‘integrated-independence’ as explained by Chapman & Chua (2003) would add to the topic about ‘Silo busting’ .
The paper does not include quantitative research. The qualitative analyses are more or less based on assumptions by the author, referring to the stories presented of thought leaders (according to the author). The conclusions by the author: ”fit of BDD with LCMDD, lack of established testing methodology & the proposed LCMDD Quality framework” are not supported by data and therefore lack sufficient background to support a solid and relevant contribution.
Possible improvements on presentation
The paper is well structured but lacks tables and figures. In several situations where the author discusses other frameworks, figures and tables within the body of the text would have improved the readability and understanding of the topic. In general, a larger part of the used concepts and methods are not explained, assuming the reader being familiar with them. Terms are all written out, explanation of some concepts (for instance BDD) is missing.
The result – research aim – the proposed quality framework for LCMDD is presented hardly readable in the appendix. Presenting the proposed model in readable size within the body of the text including explanation of the different stages within the model would have made the quality model much more clear.
Company wide context?
Significance of the quality model would be improved if a connection of Behaviour Driven Development with company wide IS projects was included: How does LCMDD and BDD fit into the implementation of ERP systems? Does it improve/mitigate the concern expressed by Chapman & Chua (2003) about possible increasing misconnect between the standardised and automated processes and the day-to-day business processes impacting the quality of management control.
Reference list
Braams, S. (2017). Developing a Software Quality Framework for Low-Code Model Driven Development Platforms Based on Behaviour Driven Development Methodology. University of Twente, The Netherlands.
Chapman, C. & Chua, W.F. (2003). Technology-Driven Integration, Automation and Standardization of Business Processes. Managing accounting in the digital economy. Oxford University Press. pp74-94.
Gartner.com. https://www.gartner.com/en/documents/3956079, via www.outsystems.com. Magic Quadrant for Enterprise Low-Code Application Platforms. Accessed 4th of 2019.
IBM https://www.ibm.com/garage/method/practices/code/practice_behavior_driven_development/practice_building_culture. Accessed 5th of October 2019.
Mendix. https://www.mendix.com/customer-stories/ Bam customer story. Accessed 5th of October 2019.
Sadeh, E., Arumugam, V. C., & Malarvizhi, C. A. (2013). Integration of EFQM framework and quality information systems. Total Quality Management & Business Excellence, 24(1-2), 188-209.
Solis, C., & Wang, X. (2011). A study of the characteristics of behaviour driven development. In 2011 37th EUROMICRO Conference on Software Engineering and Advanced Applications (pp. 383-387). IEEE.
Taher, M (2012). Resource-Based View Theory. In Y.K. Dwivedi et al. (eds.), Information Systems Theory: Explaining and Predicting Our Digital Society, Vol. 1, Integrated Series in Information Systems Vol. 1, pp151-161. Springer Science+Business Media, LLC 2012.
Recent Comments