LinguSQL – A Verification, Validation, and Transformation Tool
for Lingu Abstract Language
Rikky Wenang Purbojati1, Ade Azurat1, and Heru Suhartanto1
1
Fakultas Ilmu Komputer, Universitas Indonesia
Depok, Indonesia
[email protected], {ade, heru}@cs.ui.ac.id
Abstract. In this paper we introduce an experimental development tool called LinguSQL, intended to
run verification test, validation test and transformation of Lingu language. This tool feeds on a Lingubased script and is able to do a specified verification testing against HOL theorem prover and a
validation testing through scenario testing. Once all testing parameters and results are satisfied,
LinguSQL is able to transform it into other popular programming code to be compiled and executed.
The use of LinguSQL in a critical portion of a development process is expected to produce a much
safer software as a result of its comprehensive testing activity.
1. Introduction
The implementation of database application has become integrated with our everyday lives in
this beginning of 21st century. The breadth of it touches many things from the most personal aspect of
our lives such as the case of health information application until the very fundamental practice in a
society such as in election voting system. In implementing those systems, developers must carefully
design correct algorithms and fulfill the requirement that have been specified before. Mistakes in
designing and implementing an algorithm inside a publicly used information system could create chaos
and losses in terms of financial and confidence.
A broad variety of testing techniques are available to developers nowadays to ensure that the
algorithm and the code produced is correct and bug-free. Those varieties can be generalized into two
groups which are blackbox testing and whitebox testing. Blackbox testing is a technique which focuses
on checking whether the output of certain procedure is correct given a set of input. Meanwhile whitebox
testing concerns about how the algorithm will be run and if its conforming to the given specification.
Practically whitebox testing requires more resources to be done due to its complexity in comparison with
blackbox testing. That is mainly the reason why blackbox testing, such as unit testing[1], is the most
widely used testing technique in developing an application.
However problems arises in a way that blackbox testing is only effective as far as the input goes
into the system. It means that if the given set of input is correctly proven by blackbox test, it does not
means that another set will be correctly proven by the same technique. It does not guarantee completeness.
To make sure that the test covers the whole possibility, developers must provide the whole set of input
and run it through the test. This costs more time and money, and since time and money is a limitation in a
software development effort sometimes developers are just content with a limited set of inputs. In order
to achieve completeness in an application testing, blackbox and whitebox testing must be used
conjunctively.
LinguSQL[2] is a verification and validation tool to test algorithm defined in a Lingu script..
Lingu[3] language is an lightweight abstract language which focuses on data transformation operation on
a database. Lingu does not have the whole functionality such as in C++ or Java, but it has syntax and
grammar which enables developers to design a data manipulation algorithm on a certain database.The
implementation of verification and validation feature in LinguSQL could provide a measurement of
correctness level inside a given algorithm. Thus being said, the main motivation of the tool is to provide
comprehensive testing suite for an algorithm defined in simple Lingu language.
The rest of the paper is organized as follows. We start by introducing the feature and
functionality of LinguSQL in section 2. In section 3 we show how LinguSQL run through all the
verification, validation, and transformation steps against the defined algorithm. Finally, section 4
concludes the discussion and reports on some preliminary evaluation result and the future development
works.
2. The LinguSQL Tool
LinguSQL is a tool to run verification and validation (V&V) of a predefined algorithm and
specification which then can be transformed into an executable code. One has to prepare a defined
algorithm of a problem and its test specification prior to using this tool. The defined algorithm in Lingu
can utilize a set of native database transformation operator inside its method or function. Moreover it also
provide a set of logical syntax to provide precondition and postcondition specification for those methods
or functions. Consequently, verification and validation process will be run against these predefined
specification.
Fig 1. LinguSQL Main Windows.
2.1. Features
Integrated Testing Environment. One of the main feature of LinguSQL is its integrated testing
environment for the development process using Lingu abstract language. LinguSQL is intended to
produce a safe and more reliable code inside a software package by running it against both whitebox and
blackbox testing. Using this feature, defects and errors are expected to be lower compared to those
development process using only whitebox or blackbox testing.
The tool provide an integrated whitebox testing environment to developer using Higher-Order Logic
(HOL) theorem prover. HOL theorem prover is needed since most Lingu’s syntax is in the form of
higher-order logic where its operator have some kind of quantification informations on them. Using the
theorem prover, LinguSQL first break the algorithms and specifications into a set of mathematical
prepositions called verification conditions [4]. Afterwards These verification conditions are sent to a
Higher-Order Logic (HOL) theorem prover to assess its correctness property.
In addition, LinguSQL provides a validation means for developer to test their algorithm blackbox-wise.
Since validation is a process of determining the degree to which a model is an accurate representation of
the real world from the perspective of the intended uses of the model [5], LinguSQL is able to takes a
predefined scenario and executes it with a set of data which resembles real world data. This data is made
available by LinguSQL by generating it using internal random data generator. Afterwards the output of
the test can be compared with the predefined postcondition specification to assess if it is the expected
outcome of the function execution.
Code Generation. Another main feature of LinguSQL is the ability to transform given Lingu script into
another programming code. Current version of LinguSQL is able to transform Lingu script into a
compilable and executable Java code. Transformation process takes place when Lingu script has been
verified and validated. This guarantee that the transformation result will inherit the same nature as the
abstract language it is derived from. The expected result is that the domain specific code is safe and
conforms to the specification stated in Lingu script.
Modular Transformation System. Each language has its own characteristics and uniqueness. System
design that tries to handle all transformation of the language in one big application is inefficient. This
leads to the design that support plug-in style module for each different language. The desired effect of the
design is that this tool has an improved scalability against a whole array of programming language. One
can write a transformation module for C# code independently or even write a transformation module for
newer language such as Ruby.
Internal Data Generator. The execution of scenario testing demands an environment that resembles real
world. This includes the data that are being used. To accommodate the needs, LinguSQL provides an
automatic test generation. The automatic test generation is able to generate a collection of data that
resembles the actual data. This is done by using a constraint for the data that are being generated. For
instance, if we generate a salary data, then some constraints are needed to make the salary data looks
valid. Such as salary data cannot be negative or zero, or that the sum of all salary cannot exceed the
corporation salary budget. Ignorance of this issue will degrade the credibility of the verification and
potentially will create problem when it is implemented in actual use.
Interactive Proving. LinguSQL provides an interactive means to aid HOL theorem prover to prove the
verification conditions. By default HOL theorem prover tries to verify these conditions automatically.
However because of the nature of HOL is undecidable, sometimes HOL theorem prover cannot solve the
problem on its own. It needs an expert assistance to guide and tell HOL backend what tactics or lemma to
be used. This feature ensures that a human intervention can be done anytime HOL theorem prover meets
some unexpected obstacle in proving the verification conditions.
3. Demonstration Example
LinguSQL basically requires 4 main steps in its workflow to produce a compilable and executable code.
Figure 2 shows the basic workflow of using LinguSQL development tool.
Fig 2. LinguSQL workflow.
The first step of the workflow is the loading of Lingu script. This step will load the defined algorithm and
specification into LinguSQL memory space to be used on the verification, validation, and transformation
process. Figure 3 shows LinguSQL that has been loaded with a Lingu script.
Fig 3. LinguSQL loaded with Lingu script.
Box A on Figure 3 contains the whole Lingu script for the purpose of reviewing and editing. LinguSQL
parses this script and put all of the precondition and postcondition specifications onto box B. Meanwhile
scenario for validation process is put on box C. Results of every workflow steps will be shown on the box
D. Specific for the verification process, box E will be used to provide interactive proving assistance to the
HOL theorem prover.
The next step is verification process which utilize HOL theorem prover to assess its specification
correctness. Figure 4 shows the result interface after the verification process is done.
Fig 4. Verification Result of LinguSQL.
One can proves the script by selecting which specification he wants to proves by selecting the
specification listed in Figure 3 box B. After he selects the desired specification, he can utilizes a set of
tactics and proving techniques as seen on Figure 4 box A. The result of verification process for certain
specification is shown in the box C on Figure 4. Successful verification enables developer to continue to
the next step.
Fig 5. Validation Result of LinguSQL.
The validation steps uses scenario that are listed on Figure 5 box A. The validation box contains the
whole scenario specified inside the Lingu script, which also shows their parameters. The scenario
“safeness” with parameter of 5 will execute using the internal data generator to check if the scenario
meets the specification defined previously. The result of the selected scenario test can be seen on Figure 5
box B. If the whole validation tests are successfully run, one can process in generating the desired code to
be compiled or executed.
Finally the last step of this development process is code generation or transformation. Current LinguSQL
version is able to generate a Java code based on the given Lingu script. This transformation process
utilizes attribute grammar [6] technique to do the grammar tree building and parsing. The tool utilizes
UUAG compiler [7] to attach language transformation attributes into Lingu’s grammar. The
transformation or code generation process result can be seen on the output box A on Figure 6.
Fig 6. Transformation Result of LinguSQL.
4. Conclusion
Methods for producing a safe and reliable software have always been an active research topics
and discussion in the field of software development process. Naturally this is due to the fact that some
types of software requires a higher level of reliability to minimize financial or non-financial loss caused
by defects or errors. A wide array of testing techniques has been developed to counter this kind of
problem. However a better technique is still largely on demand since we can see today that financial
losses caused by software errors and defects are still a common headline news.
In this paper we introduced LinguSQL, a development tool which function is to test Lingu script
and generate a functional code based on it. It combines a whitebox and blackbox testing technique against
the script. LinguSQL uses HOL theorem prover to perform the whitebox testing against the script’s
algorithm and specification. Meanwhile, it also uses scenario testing through data generation to provide
blacbox testing method. Finally transformation process takes place when Lingu script has been verified
and validated. This guarantee that the generated code will inherit the same nature as the abstract language
it is derived from. The expected result is that the domain specific code is safe and conforms to the
specification stated in Lingu script.
However, LinguSQL itself has some limitations regarding its automated verification capability.
Because of the nature of HOL is undecidable, sometimes HOL theorem prover cannot solve the problem
on its own. It needs an expert assistance to guide and tell HOL backend what tactics or formula to be used.
However this is accommodated by providing an interactive panel in the LinguSQL to be used in the event
of human assistance is needed.
Finally, LinguSQL is still being developed to integrate further advance feature to improve the
quality of the testing process. First area of improvement that is possible to be worked on is an interface
for the data generator. Currently we generate the data based on the predefined specification on a separate
script. By integrating a user interface to compose the desired generated data, a developer can be more
productive in doing the validation process. Secondly, it is always possible to provide a more extensive
library of tactics and formulas so that HOL theorem prover will need less human assistance as the current
version.
References
1. Beck, K., Gamma, E. : Test infected: Programmers love writing tests. More Java Gems. Cambridge University
Press (2000) 357-356
2. Purbojati, R.W., Prasetya, I.S.W.B., Maizir, S., Wibowo B., Azurat, A. : LinguSQL: A Verification and
Transformation Tool for Database Application. In Proceedings of 6th National Seminar of Computer Science and
Information Technology Indonesia. SNIKTI (2005)
3. Azurat, A., Prasetya, I.S.W.B., Vos, T.E.J., Suhartanto, H., Widjaja, B., Stefanus, L.Y., Wenang R., Aminah, S.,
Bong, J. : Towards Automated Verification of Database Scripts. In Proceedings of 18th International Conference
on Theorem Proving in Higher Order Logics. Springer (2005)
4. Suhartanto, H., Prasetya, I.S.W.B., Puspa, C. D., Azurat, A. : Proses verifikasi piranti lunak basis data dengan
Lingu dan theorem prover HOL. LP FEUI (2007)
5. Coughlin, D. : An Integrated Approach to Verification, Validation, and Accreditation of Models and Simulations.
Proceedings of the 2000 Winter Simulation Conference. ACM (2000)
6. Knuth, D. E. : Semantics of context-free language. Theory of Computing Systems, Vol. 2, No. 2. (1968) 127-145
7. Swierstra, S.D., Alcocer, P.R.A., Saraiva, J. : Designing and Implementing Combinator Language. 3rd
International Summer School on Advanced Functional Programming. Lecture Notes in Computer Science.
Springer Verlag (1999)