Dynamic Rules Engine

The rules engine, in its initial version, is based on the ability to define core and variant validation rules for domain objects. It provides a way of dynamically expanding the validations to be performed on them without the need for writing Java code. Any rule can be applied to a domain object and its fields by modifying an XML descriptor that contains the rules.

In a general sense, based on the way the rules engine has been designed and developed, it is possible to apply these rules to any Java object. In fact you could create a Java bean with bounded attributes, where the bounded listener invoked the rules engine, prior to letting the setter succeed.

To take advantage of the current domain object implementation in Jaffa v1.2, you need to make sure that you have generated your domain objects using the latest domain object pattern (v1.0). This will automatically add hooks into your domain object ‘validateXxx()’ routine.

Here’s a quick overview

The Set Up

• There is a configuration file that defines all the supported rules for the rules engine.

  This default version of file is called 'validators.xml'

  The location of this file is defined in 'frameworks.properties' for example
  # This property holds the comma-separated list of the various validator.xml # urls used by the Dynamic Rules Engine. # # The default-value is 'classpath:///resources/validators.xml' # Eg: 'classpath:///resources/validators.xml,file:///C:/sandbox/validators/custom-validator.xml' framework.rules.validators.url.list=classpath:///resources/validators.xml

  Note: This is a comma separated list, so you can use the default files that comes with Jaffa that references the default rules, and create your own version for rules you write yourself. (If is a pretty useful one, why not send it to us to added to the Jaffa base line?)

• There is a configuration file that defines the core rules that should be applied.

  The default version of this file is called 'core-rules.xml'

  This configuration file is located by a definition in ‘frameworks.properties’, for example
  # This property holds the URL for the core-rules file used by the Dynamic Rules Engine. # The default-value is 'classpath:///resources/core-rules.xml' framework.rules.core-rules.url=classpath:///resources/core-rules.xml

• There can be other rules configuration files that define variations, these are only used if the context when running the rules engine has been set to a specified variation (this will be discussed in detail later).

  The directory for the variant rules of is defined in 'frameworks.properties', for example
  # This property holds the directory in which the customer variations to the core-rules # for the Dynamic Rules Engine will be located. # # This property may be left empty, if no variations are being used. # Eg: 'file:///C:/sandbox/custom-rules-dir' # NOTE: The variation files, if any, should be named as -rules.xml framework.rules.variations.directory=

• The rules engine is invoked by two main methods. The first is used for validating a given 'bean' property, and is used in the domain object 'validateXxx()' method to validate just that one property.

  Taken from a 'User' domain object, validateUserName()
  RulesEngine.doAllValidationsForDomainField("org.jaffa.example.User", "UserName", userName, this.getUOW());

  Note: The name of the bean class, the property name and the property value are passed in. Optionally, if an existing UOW is provided, and data base access will use this.

  In the event of an error a ValidationException will be thrown. An implementing validation class can throw its own custom exceptions, as long as they extend ValidationException.

• The second invocation the rules engine gets comes directly from the persistence layer. This validation is designed to validate the 'whole' bean. What this means is that as the bean is manipulated, each field is validated, but some validations may need to be done on fields that have not been updated. For example, if a validation is that a field may be mandatory, that needs to be caught at this level.

  From org.jaffa.persistence.engines.jdbcengine.datasource.PersistentTransaction
  RulesEngine.doMandatoryValidationsForDomainObject(object, object.getUOW());

  In this case the complete object is sent to the rules engine, it looks at all for all the mandatory based rules for the object and then invokes each one.

  From the persistence layer this gets called on a domain object when the uow.add() or uow.update() method is invoked on that object.

As there are core rules, variant rules, standard validations, custom validations, this section will explain in what sequence the rules will be executed, both in the single field, and compete object scenarios

1. Single Property

1.1. Execution Flow

• To execute all the rules for a domain field, invoke the doAllValidationsForDomainField() method of the RulesEngine, passing in the domainClassName, fieldName, fieldValue and an optional UOW

• The RulesEngine will acquire ClassMetaData object(s) representing the domainClass, from the RulesMetaDataService. The ClassMetaData object will be obtained from the core-rules.xml and the variant-rules.xml, if defined

• It will then get the appropriate FieldMetaData object(s) from the ClassMetaData object(s), and invoke the rules listed in the FieldMetaData object(s) in the following order
  • If not overridden in the variant file, execute the core rules for the field
  • Then execute the variant rules for the field
  • In case the field has no definition in the variant file, the core rules will be executed
  • NOTE: Before, invoking the rules for a field, a check is made to see if the field extends any other field, in which case a recursive call will be made to execute rules for the parent-field

• The following is the flow for executing a rule
  • Get the FieldValidatorMetaData object from the ValidatorMetaDataService
  • It will create an instance of the validator class, as defined in the FieldValidatorMetaData object
  • It will then invoke the init() method on the validator class
  • It will then set the value, labelToken, UOW on the validator class
  • It will then set the parameters (if any) specified in the validators.xml file
  • It will then set the parameters (if any) specified in the core|variant-rules.xml file
  • It will then invoke the validate() method of the validator class
  • Finally, the cleanup() method will be invoked, irrespective of the result of the validation

1.2. Example Core Rule

<domain class="org.jaffa.persistence.domainobjects.Item">
   <field name="Status2">
       <mandatory/>
   </field>
   <field name="Status3">
       <in-list list="P;Q;R" separator=";"/>
   </field>
</domain>

1.3. Example Variant Rule Extending The Core Rule

<domain class="org.jaffa.persistence.domainobjects.Item">
   <field name="Status2">
       <in-list list="P;Q;R" separator=";"/>
   </field>
</domain>

1.4. Example Variant Rule Replacing The Core Rule

<domain class="org.jaffa.persistence.domainobjects.Item">
   <field name="Status3" overridesDefault="true">
       <in-list list="I,J,K,L"/>
   </field>
</domain>

2. Complete Bean

2.1. Execution Flow

• To execute all the rules for a domain object, invoke the doAllValidationsForDomainObject() method of the RulesEngine, passing in the domainObject and an optional UOW

• The RulesEngine will acquire ClassMetaData object(s) representing the domainClass, from the RulesMetaDataService. The ClassMetaData object will be obtained from the core-rules.xml and the variant-rules.xml, if defined

• It will invoke the doAllValidationsForDomainField() method for all the fields defined in the variant file

• It will then invoke the doAllValidationsForDomainField() method for all the fields defined in the core file, but which do not have a definition in the variant file

3. Sequence Diagram

If your really interested in the details of this process we have provided the sequence diagram of the process.

Depending on how you are using the complete Jaffa stack this may vary, or need to be modified to fit in with your middleware, but basically we use a 'ThreadLocal' variable, that we bind to the thread and that contains the variation.

To make this as seamless as possible for the 2 1/2 tier deployment we have added a 'getter/setter' for variation to the UserSession object. If you set it here, and rules processed by a web server request, that is bound to that user session will automatically set the thread variable to the variation in the user session.

If you are developing a 3 (or n) tier solution, and depending whether you using the rules engine in the presentation tier, business tier, or both, the variation should be part of the data in the DTO (data transfer object). You will see if you use the Jaffa patterns, they use the 'HeaderDto' class (in package org.jaffa.components.dto) as one of the objects inside the root of all DTOs (as of today we don't fill this value in, this would need to change!). When the 'servlet wrapper' or 'web service wrapper' middleware is developed this will implement the receiving of the variation as part of the DTO, and then set the 'thread variable' before calling the real 'TransactionCotroller' for processing.

How and When to set the Variation?

If you use the sample application as a reference, when a user logs on it invokes the default security manager.

The security manager to be used for any application is defined by the following setting.

# The class that implements the Authentication Manager. It is used to manage system
# access and to control the initialization of the UserSession objects and related data
#
# This one (the framework default implementation) creates a dummy authenticated user
#framework.security.portlet.manager=org.jaffa.presentation.portlet.security.AuthenticationManager
# This is the default Jaffa for Web Container Authentication.
framework.security.portlet.manager=org.jaffa.presentation.portlet.security.WebContainerAuthenticationManager

If you look at the source of org.jaffa.presentation.portlet.security.WebContainerAuthenticationManager you can see a method called 'public void initUserInfo(UserSession us)'.

So to set the variation on the UserSession, you need to create your own AuthenticationManager, that extends the basic WebContainerAuthenticationManager, and provides an implementation for the 'initUserInfo(...)' method. Where you obtain the 'variation' for this user session is up to you, (database, xml, properties), and whether or not it varies per user, again is up to you based on how you set up your rules. You could make it based on the users locale if you had a specific need for rules per language!

In this example a Transaction Controller is used to read user information from the database (never access the database directly for a 3-tiered application). It also created a custom 'UserInfo' object in the user session to store extra user details.

public void initUserInfo(UserSession us) {
   try {
       // Get the Transaction Controller
       ILogon tx = (ILogon) Factory.createObject(ILogon.class);
       // Create Inbound Dto
       LogonInDto dto = new LogonInDto();
       dto.setUserId(us.getUserId());
       LogonOutDto userInfo = null;
       // Try to get the information
       try {
           userInfo = tx.getUserInfo(dto);
       } catch (Exception e) {
           log.fatal("Invoking getLogonInfo() Service Failed. User=" + us.getUserId(), e);
           return;
       }
       // If there was a valid user a Dto will be returned
       if(userInfo != null) {
           // Populate UsesSession with Valid User Id
           us.setUserId(user);
           // Populate the variation in the UserSession
           us.setVariation(userInfo.getVariation());
           // Create a UserInfo Object
           UserInfo ui = new UserInfo();
           ui.setName(userInfo.getName());
           ui.setEmail(userInfo.getEmail());
           // Store custom object in UserSession
           us.setUserData(ui);
       } else
           log.fatal("No User Information Was Available At Time Of Authentication");
   } catch(Exception e) {
       // Catch all errors
       log.fatal("Failed accessing User Info During Authentication", e);
   }
}

All validators must implement the IFieldValidator interface, or for convenience extend the AbstractFieldValidator class. This is illustrated in the diagram below

About the Interface

• Life Cycle - A Validator is created, then initialized, all the setters are then invoked, as per its definition. It is now in a state where it can perform its validation. After validation, a clean up method is called to release any resources it may have acquired.

• Initialization - After construction, the init() method will be called, you can do any initialization here, but remember that none of the setter will have been called yet. If you need to do stuff after the setter, but the code at the start of your validate() method.

• Set Up - This will call all the setters, by default there are setters for Value, UOW, and LabelToken, other setters must be defined in the 'validators.xml' file (see later)

• Execution - The rules engine will execute the validate() method, and if this method returns with no exception, then the validation was passed successfully. If the validation failed, you must throw a ValidationException. If an internal error occurred you may throw a FrameworkException. It is recommended in the case of Exceptions specific your validator, that you extend either FrameworkException or ValidationException with a specific Exception class (For example the validator 'GenericForeignKeyFieldValidator' throws 'org.jaffa.datatypes.exceptions.InvalidGenericForeignKeyException')

• Clean Up - After the validate() method is called, this method is called to do any final house keeping, releasing resources, etc, before the class is discarded. This is called even if an exception is thrown by validate(). It should be used instead of a finalize() method for cleanup.

You can define additional attributes (via getters/setters) for your specific validator, in addition to what is in the interface. This can allow for a given rule definition, additional information to be passed to the validator. This data will be 'constant' as it is defined in the XML file. Currently the only information available to a validator at the time of execution is values defined in the rules XML file, and the value of the field being validated.

Defining Your Validator

• Create your own version of validate.xml

  'my-validators.xml'

  <?xml version="1.0" encoding="ISO-8859-1"?> <!DOCTYPE validators PUBLIC "-//JAFFA//DTD Dynamic Rule Validators 1.0//EN" "http://jaffa.sourceforge.net/DTD/validators_1_0.dtd"> <validators> <field-validator> <name>my-validation</name> <description>My Custom Validator</description> <class>com.xyz.rules.MyValidator</class> <mandatory>false</mandatory> <param name="static1" value="data"/> </field-validator> </validators>

  Note: If you want to pass some static values to this validator you can define <param> tags, which will invoke the related setter (for example 'setStatic1(String s)')

• Reference this file in framework.properties

  'framework.properties' fragment
  # This property holds the comma-separated list of the various validator.xml urls used by the Dynamic Rules Engine. # The default-value is 'classpath:///resources/validators.xml' # Eg: 'classpath:///resources/validators.xml,file:///C:/sandbox/validators/custom-validator.xml' framework.rules.validators.url.list=classpath:///resources/validators.xml,classpath:///resources/my-validators.xml

• Create your own DTD for validation

  com/xyz/rules/metadata/rules_1_1.dtd

  <?xml version="1.0" encoding="US-ASCII"?> <!-- This dtd is used for validating the input xml to the RulesMetaDataService --> <!ELEMENT rules (domain*,dto*)> <!ELEMENT domain (field*)> <!ATTLIST domain class CDATA #REQUIRED> <!ELEMENT dto (field*)> <!ATTLIST dto class CDATA #REQUIRED> <!-- The new validators should be added to the field element --> <!ELEMENT field ((mandatory|foreign-key|generic-foreign-key|in-list|string|boolean|currency| dateonly|datetime|decimal|integer|my-validation)*)> ... ... ... <!ELEMENT my-validation EMPTY> <!ATTLIST my-validation param1 CDATA #REQUIRED param2 CDATA #IMPLIED>

  Note: The new validation has been added to the end of the 'field' element, and at the end a definition for it has be added. In this case we want two parameters to be passed into the validator. We've made param1 mandatory and param2 optional for this new validation.

• Use your own DTD as the DOCTYPE for your rules.xml files

  Example 'rules.xml'
  <?xml version="1.0" encoding="ISO-8859-1"?> <!DOCTYPE rules PUBLIC "-//JAFFA//DTD My Dynamic Rules 1.1//EN" "http://www.xyz.com/DTD/my-rules_1_1.dtd"> <rules> <domain class="com.myapp.users.domain.Users"> <field name="department"> <my-validator param1="10" param2="100"/> </field> </domain> </rules>

• Add a resolver for the Public Id for your DTD if required

  org/jaffa/config/dtd.properties
  org/jaffa/rules/metadata/validators_1_0.dtd=-//JAFFA//DTD Dynamic Rule Validators 1.0//EN org/jaffa/rules/metadata/rules_1_0.dtd=-//JAFFA//DTD Dynamic Rules 1.0//EN com/xyz/rules/metadata/rules_1_1.dtd=-//JAFFA//DTD My Dynamic Rules 1.1//EN

  Note: This assumes that the WAR or JAR that you deploy contains a copy of the dtd in the specified package

• Code your Validator. In this case it should have the following getter/setters
  • Internal (in the interface) - setValue(), setUow(), setLabelToken()
  • Static (value set in validators.xml) - setStatic1()
  • Dynamic (value set in rules.xml, per field usage) - setParam1(), setParam2()

Current Limitations

• Currently this has been designed to work in the business tier, so any validations that involve database access, will accesses directly. If you are going to invoke the rules engine in the presentation tier, and also have a requirement for a 3 tier deployment, you'll need to modify its current implementation.
• Currently as discussed above, the 'variation' property isn't currently being set in the HeaderDto in the default patterns, again for a 3 tier deployment this will need addressing.
• Rules Caching only uses a week hash map currently, so if the garbage collector keeps clearing the cache, this could degrade performance, as it will keep reading the rules from the source XML files.
• When rules are cached, the whole set of rules for that object is cached, not the entire XML Rules file. What this means is if you need to deploy a new/modifed rules file, you need to stop the server, deploy it, and restart the server. Updating a file while the server is running can be unpredictable, as it may be partly cached. There is a implementation for a CacheManager that monitors source files (XML in this case) and flushes the cache if these have been modified. This is not yet used by the rules engine, and more work on the CacheManager is needed for this. Once implemented a 'hot deploy' of new rules will be possible.

Future Direction

• Add validations that can be applied to DTO's in the Transaction Controllers, as sometimes deferring validations until the domain object is updated is too late in the process. If a DTO validation is performed and a DTO has collections of 'inner DTOs', then automatically validate those too. However, if we are moving to DTO's that are XML marshable, and generated from XSD's then we should let the XML Parser do most of the basic validation first. This needs enhancement will be developed in parallel with the 'web service wrapper' middleware.
• Some new cache managers are planned for development based on the feedback of the performance of the rules engine in a production environment.
• Extending the rules manager to be used in the presentation tier so we can let form beans perform some of the validations prior to even invoking the business tier. This probably only makes sense for rules that don't require database access, as these would require fine-grain calls to the business tier and invalidate the design approach of course-grain access to the business tier via the TransactionControllers and DTOs