How to test an EDNA plugin

From EdnaWiki
Jump to: navigation, search

EDNA provides a testing framework that allows a relatively simple, fast and automated way of testing a plugin thanks mainly to an assert mechanism used to validate obtained results towards expected ones and thanks to the 3 main classes of the framework organized according to the following hierarchy:

 EDTestCasePlugin (parent class)
    |
    |-----> EDTestCasePluginUnit 
    |-----> EDTestCasePluginExecute
         

We split the plugin tests into 2 categories: unit test and execution test:

  • The unit tests are used to specifically test the plugins methods (checks for returned value, expected case, unexpected case behavior, etc...). To create a unit test, one has to create a class deriving from EDTestCasePluginUnit.
  • The execution tests are able to validate the invocation of a plugin by simulating the plugin execution in an application context (EDTestCasePluginExecute is based on a plugin launcher). To create an execution test, one has to create a class deriving from EDTestCasePluginExecute.

See sections below for more details.

In both cases, the test sources should go under EDNA_HOME/prototype/plugins/<MyPlugin>/tests/testsuite, and the related data for the tests under EDNA_HOME/prototype/plugins/<MyPlugin>/tests/data

Note that the framework provides some facilities to retrieve useful information like directory locations (i.e: self.getPluginTestsDataHome()), to read/write files (EDUtilsTests), to create a plugin entity to be tested, etc... See more in the pydoc: http://www.edna-site.org/svn/trunk/edna/prototype/doc/tests/pydoc


How to develop a plugin unit test

The aim of a plugin unit test is to test the methods of a plugin independently from any execution context consideration. It is a kind of class test.

As said in the introduction, the first thing to do is to create an unit test class deriving from EDTestCasePluginUnit (in a file called something like EDNA_HOME/prototype/plugins/<MyPlugin>/tests/testsuite/EDTestCasePluginUnit<MyPlugin>.py):

class EDTestCasePluginUnitMyPluginv01( EDTestCasePluginUnit ):


Constructor

The constructor of the unit test should initialize the data (input file, reference file for the assert mechanism) if required for the test.


The preProcess method

The preProcess method is where more initializations should be done depending on the need.


The unit test methods

Implement your test methods (we generally use the prefix “test” before the method to test as a naming convention). Here is an example on how to check an expected behavior of a particular plugin method “doSomethingMethod” :

   def testDoSomethingMethod( self ):
       """
       Tests that doSomethingMethod() returns “OK”
       """
       # Creates the plugin
       edPlugin = self.createPlugin()
       # Checks that “OK” is the obtained string result of doSomethingMethod()
       EDAssert.equal( "OK", edPlugin.doSomethingMethod () )

The process method

Add your test method to your test case in the process method:

   def process( self ):
       """
       List of test methods
       """
       self.addTestMethod( self.testDoSomethingMethod )


The main

Don't forget the main to execute your test case:

   if __name__ == '__main__':
       # JIT compiler accelerator
       EDCompiler.accelerator()
       edTestCasePluginUnitMyPluginv01 = EDTestCasePluginUnitMyPluginv01( "EDTestCasePluginUnitMyPluginv01" )
       edTestCasePluginUnitMyPluginv01.execute()


How to write a plugin execution test

The aim of a plugin execution test is to validate the execution of a plugin in an application context. There are a lot of similarities with the way a unit test is constructed. The main difference resides in the execution test method

As said in the introduction, the first thing to do is to create an execution test class deriving from EDTestCasePluginExecute, and should go in a file be called EDNA_HOME/prototype/plugins/<MyPlugin>/tests/testsuite/EDTestCasePluginExecute<MyPlugin>.py

class EDTestCasePluginExecuteMyPluginv01( EDTestCasePluginExecute ):


Constructor

The constructor of the execution test should initialize the data related to the plugin execution (configuration file, data input file, expected output result file, obtained output result file):

   def __init__( self, _oalStringTestName = None):
       """
       """
       self.setConfigurationFile( self.getRefConfigFile() )
       self.setDataInputFile( EDDiskExplorer.mergePath( self.getPluginTestsDataHome(), "XSDataInput_reference.xml" ) )
       self.setReferenceDataOutputFile( EDDiskExplorer.mergePath( self.getPluginTestsDataHome(), "XSDataOutput_reference.xml"))
       self.m_oedObtainedOutputDataFile = self.getPluginName() + "_output.xml"


The preProcess method

The preProcess method is where more initializations should be done depending on the need.


The execution test methods

Implement your test methods (we generally use the prefix “test” before the method to test as a naming convention). Here is an example on how to launch the plugin and how to check the output data result stored in an xml output file.

   def testExecute( self ):
       """
       """ 
       # Runs the plugin. No needs to create a plugin entity.
       self.run()
       # Checks the obtained results towards the expected
       strExpectedOutput = EDUtilsTest.readAndParseFile ( self.getReferenceDataOutputFile() )
       strObtainedOutput = EDUtilsTest.readAndParseFile ( self.m_oedObtainedOutputDataFile )    
       EDAssert.equal( strExpectedOutput, strObtainedOutput)


The process method

Add your test method to your test case in the process method:

   def process( self ):
       """
       List of test methods
       """
       self.addTestMethod( self.testExecute )


The main

Don't forget the main to execute your test case:

   if __name__ == '__main__':
       # JIT compiler accelerator
       EDCompiler.accelerator()
       edTestCasePluginExecuteMyPluginv01 = EDTestCasePluginExecuteMyPluginv01( "EDTestCasePluginExecuteMyPluginv01" )
       edTestCasePluginExecuteMyPluginv01.execute()


How to integrate a test case in a suite

Once the unit and execution tests have been implemented, it is recommended to create 2 test suites for your plugin tests: one to encapsulate all the unit test cases and another to wrap the execution test cases. These 2 suites will be then integrated in the main edna unit and execution suite in order to automate their execution.

The 2 plugin test suites mentioned above should be located here: EDNA_HOME/prototype/plugins/<MyPlugin>/tests/testsuite/EDTestSuiteExecute<MyPlugin>.py and EDNA_HOME/prototype/plugins/<MyPlugin>/tests/testsuite/EDTestSuiteUnit<MyPlugin>.py

Here is an example for a unit test suite that wraps a few test cases:

    from EDImport        import EDCompiler
    from EDImportTest    import EDTestSuite

    class EDTestSuiteUnitMyPluginv01( EDTestSuite ):


        def process( self ):
            """
            """
            self.addTestCaseFromName( "EDTestCasePluginUnitMyPluginv01" )
    

    if __name__ == '__main__':

        # JIT compiler accelerator
        EDCompiler.accelerator()

        edTestSuiteUnitMyPluginv01 = EDTestSuiteUnitMyPluginv01( "EDTestSuiteUnitMyPluginv01" )
        edTestSuiteUnitMyPluginv01.execute()