Reference guide

Workflow4people reference guide

WARNING This document is work in progress. You are looking at an incomplete working copy of the Workflow4people reference guide.
Workflow4people is an all-in-one workflow solution.
It aims to provide a structured approach to data capture and process management.
It provides:
  • BPMN process flow
  • Forms solution
  • Form generator
  • XML data store

To do this, it stands on some very big shoulders:
  • Grails
  • Activiti
  • ActiveMQ
  • Solr

Workflow4people has a generic approach to process management. The process management is completely decoupled from the rest of the application. It uses Activiti and jBPM, and more engines could be added if needed.

Runtime

So what happens at runtime? A typical case is where a user wants to start some business process. To achieve this, the user fills out a web-based form. That form allows us to capture all data needed to process the request in a structured way. The data that is entered in the form is stored as an XML document in the database. That XML document will exist throughout (and after) the execution of the process. The XML document will be amended and modified at each step in the process.
As soon as the document is stored in the database, the process engine will be notified that a new document has arrived. It will start an appropriate business process. This will lead to a task that is generated for a user group. That task now appears in the task list of the users that can perform this task. Every task has a task form that will show (part of) the XML document stored earlier. Upon saving the document, the process engine will be notified again. This continues until the process engine decides to end the process. The XML document remains in the database.

Process engine

We have not created a process engine but have used Activiti and jBPM.

Forms and data model

To allow each process to have it's own data model a flexible way of storing and representing the process data is needed.
The process data is stored as XML documents. The structure of these documents and the associated forms is designed through a web-based user interface. It uses a model-first type of approach: you first design your model, then determine how this model will be presented to the user using forms.

Forms engine

The forms engine is the user interface of workflow4people for end users. It communicates to the backend application using web services. It presents the user with task and document overviews, forms, a search facility, and a data miner.
This application is implemented as a Grails plugin (wfp-forms). This application can be run as-is or you can create an application by including the plugin and extending and/or overriding the provided functionality.

Forms processing

The forms part of the application takes an XML document from the backend, renders it through a form definition, processe the input of the user and posts the resulting XML back to the backend. This part of the application is implemented as a Grails plugin (xml-forms) that is normally used by wfp-forms but can be run independently.

Backend

The wfp backend has a user interface that is for administrative purposes only. It consists of:
  • XML data store
  • Indexing (using Solr)
  • Message queue (using ActiveMQ)
  • Process engine (using Activiti)
  • Data model editor
  • Forms generator
 

XML data store

The XML data store is implemented as a GORM-based (hibernate) database that stores the XML documents as text blobs. Whenever a document is stored, it is indexed using the index definition of the document type. The index itself is Solr, which is updated asynchronously. This allows us to search through the XML documents even though there are many different document types in the database.

Every time a document in the data store is modified, a JMS message is sent to a JMS topic. The process engine responds to these messages.

Solr

One of the key components of wfp is the Solr index. It stores index information in 2 cores:
  • Document
  • Task
The 'Document' core stores the index data of the XML documents. The 'Task' core stores task index information. The task lists presented to the user are in fact the result of Solr queries. This structure allows us to add additional data to the task lists presented to the user, and allows for that information to be searched and used for sorting.

ActiveMQ

Another key component of wfp is the Activiti message queue. It provides the interface between the process engine(s) and the rest of the application, allows for asynchronous processing of requests (which has great scalability and performance advantages) and provides the basis for the data distribution mechanism in wfp.

Processing a document in detail

Document message flow

This is an overview of the message flow that occurs when an XML document is stored in the data store

All interaction of a user with the backend is done through storing a document. The header of the document contains identification information along with some process/task related information, such as the outcome of a task.

When a document is stored a wfp.event message is sent, which causes the index tables to be updated. As soon as that is done, the index data is picked up by the scheduled Solr job.
If a workflow process is needed, the document service will create a worfklow domain object in the database. This event is picked up by an event lister which in turn sends a JMS message to the queue for the process engine. In the case of activiti, this is wfp.engine.activiti.

The process engine receives the message and creates a new process. This will most likely lead to tasks that are created in activiti. An event listener in Activiti picks this up and sends another message to wfp.engine.activiti. Upon recepion of this message, a Task domain object is created. The Solr scheduled job picks this up after which the task can be found and processed by users.

We have achieved the following by doing this:
  • The process engines are completely decoupled from the rest of the application. They can have their own session management etc. Or run on a completely different system for that matter.
  • Second, the scalabilty is far better than any synchronous approach can ever be. The front of the application just sends messages. All heavy-lifting is asynchronous. When the load on the application increases, process times in the backend will increase but the front remains responsive.

One consequence is that the follow-up on a task completion is not instantaneous. This usually does not matter a lot as the next task is usually done by another user.
 
The form generator generates all artifacts that are necessary for wfp-forms to operate, and can be configured to generate practically anything based on the datamodel and forms model present in the wfp backend.
The behaviour of the template generator is defined by the template configuration file template.conf that is in the root of the template folder, for example:
def template = {

	// Per-workflow templates
		workflowDefinition {
			template("workflowDefinition/xsdelement.gsp","common/xsd/${workflowDefinition.name}.xsd")
            template("workflowDefinition/xsdelement.gsp","${workflowDefinition.name}/${workflowDefinition.name}.xsd")
			template("workflowDefinition/xml.gsp","${workflowDefinition.name}/${workflowDefinition.name}.xml")    
		 }
    
    // Per-form templates
    form {
        template("form/${form.template}.gsp","${workflowDefinition.name}/${form.name}.gsp")        
        if(form.template=="show" || form.template=="show-tabs") {
            template("form/pdf.gsp","${workflowDefinition.name}/pdf.gsp")
            template("form/mail.gsp","${workflowDefinition.name}/mail.gsp")
        }
        template("form/thankyou.gsp","${workflowDefinition.name}/${form.name}-thankyou.gsp")
    }  
    // Per-namespace templates
    namespace {
        template("workflowDefinition/xsdlib.gsp","common/xsd/${namespace.filename}")
    }

}

The template is run as a groovy DSL. Each section is a closure that is run against a binding that is specific for that particular section, and repeated as appropriate for that particular section.


The sections in the configuration file are:
  • workflow
  • form
  • fieldType
  • namespace

Bindings

Bindings for the workflow section
  • workflowDefinition - the current WorkflowDefinition object
Bindings for the form section
  • workflowDefinition - the current WorkflowDefinition object
  • form - the current Form object
Bindings for the fieldType section
fieldType - the current FieldType object
Bindings for the namespace section
namespace - the current Namespace object

Statements in the template config sections

You can put any Groovy statement in the template config sections. The 'template' method of the config delegate should be used for template generation:
template(templatePath,outputPath,options)

Parameters

templatePath
The path relative to the template.conf of the template file to be used
outputPath
The relative path of the file to be generated
options
A list of strings, can contain 'noPretty' to prevent XML pretty printing to occur and 'noReplace' to prevent tag substitution to occur

Tag substitution

We are using GSP based templates to generate GSP-based forms. This means that we cannot include GSP tags in our template as they will be interpreted at template generation time. To overcome this limitation some strings in the form are replaced after form generation
Original string matching Replaced by
*{ ${
<render- <
</render- </
-render> >
Templates are used to generate forms, xml files etc. A template is a GSP with the same model as the binding as the template.conf and some helper tags in the w: namespace that help iterate over the model.

w:snippet

Parameters

name
The name of the snippet
model
Optional additionalbinding for the snippet
type
The snippet type
var
Optional domain object on which the snippet is run

Description

The model attribute is optional.
First form: The var attribute is a domain object which needs to have a runSnippet method. The type of snippet is determined from the object's class name and the model is the domain class' binding, merged with the binding from the model attribute.
Second form form: The type attribute determines the snippet type. The model is the binding from the model attribute.
In this case the snippet body is run against the given model and passed on as the body binding

w:items

Parameters

section
The section that holds the items
var
The name of the iterator variable that will hold the item

Description

Iterate over items in a section

w:pages

Parameters

form
The form
var
The name of the iterator variable that will hold the page

Description

Iterate over pages in a form

w:sections

Parameters

page
The page
var
The name of the iterator variable that will hold the section

Description

Iterate over sections in a page

w:namespaceFieldTypes

Parameters

namespace
The namespace
var
The name of the iterator variable that will hold the fieldtype

Description

Iterates over the fieldtypes that are in a namespace

w:tag

Parameters

attributes
List of attributes
name
The name of the tag to be generated

Description

Generate a tag based on name and attributes
Form snippets are located in the snippets folder in the template.

Snippets are run from a template bij using the w:snippet tag. The appropriate snippet is searched for in the following locations:
- The FieldType folder
- The BaseType folder
- The Default folder
The first one found is used.

Each snipper has a snippet.conf configuration file that looks like this (this is the RepeatingList snippet):
 
name="RepeatingList"
description="Groups a repeating list of fields"
useList=false
parameters {
    cssClass {
        label="CSS class"
        defaultValue=""
        help="Extra CSS that is added to the form:input"
    }    
}
defaults {
    maxOccurs = "unbounded"
}

Snippet configuration items

name
The name of the snippet
description
The description of the snippet
useList
true|false - Determines if this snippet makes use of the item list that is associated with the FieldType
parameters
Parameter descriptions, see below
defaults
Default values used to populate field properties when fields assciated with this snippet are created

Snippet parameters

The parameters block of snippet.conf contains the definition of parameters that are defined for this snippet. These parameters can be edited in the data model editor in the snippet tab of each field. Each parameter has a label, defaultValue and help. Snippet parameters are always strings.

Field binding

Property Description
name The name of the field
namespacePrefix The namespace prefix
prefixedName The prefixed name of this field, e.g. prefix:name
label The label of this field
help The help text of this field
alert The alert text of this field
defaultValue The default value of this field
fieldTypeName The name of the FieldType of this field
contentText The content text of this field
occurrence An XSD-style minOccurs/maxOccurs attribute string
xpath The XPath of this field
gpath The GPath of this field
gpathExpr The GPath expression of this field, that has index values as expressions, e.g. they are surrounded by ${ ... }
field The Field object itself
snippetConfig The snippetConfig
parameters Map of snippet parameters

FieldType binding

Property Description
name The name of the FieldType
baseSchemaType The baseSchemaType of this fieldType
fieldType The fieldType object itself
snippetConfig The snippetConfig
parameters Map of snippet parameters

Form binding

Property Description
form The form object

Formitem binding

Property Description
formitem The formItem object
Permissions are assigned based on ACLs which refer to authority names that a user has.

Authority names

Authority names are the combination of authorities based on a user, group, role or just a static string. For example:
USER_joost
GROUP_administrators
GROUP_customer_service
ROLE_WF4P_ADMIN
ROLE_WF4P_DEVELOPER
EVERYONE


Permissions on workflows

To start a workflow the user needs the requestor rol on workflowDefinition. TODO Dit should be refactored so that the request action rather than the role is required. To be able to execute a task the user need to be assigned to perform the task TODO This should be properly verified. At the moment only the document permissions are enforced.
When a workflow is started the associated XML document is created. The 'Check permissions on create' checkbox on the document type can be switched off to prevent permission checks from occurring on creation of the XML document. This is useful for cases where the requestor should not be able to access the document once the workflow is started.
The roles that the requestor and the home group of the requestor are assigned can also be set for each document type.


Permissions on documents

Permissions on documents are an ACL which is constructed as a list of [authority]_[action] elements. This simple format allows them to be easily stored and queried in Solr.
For example:

GROUP_customer_service_write
EVERYONE_read

Feature permissions

Some features of the forms environment can be controlled by permissions. The show action is needed to allow access.
The feature names are:
lock
Controls access to the lock overview and the possibility to remove locks
miner
Controls access to the data miner
search
Controls access to the search function

 

The process API can be different per process engine. We try to make them as similar as we can.

Process API for Activiti

Expressions
Expressions are interpreted in Groovy. Any property in the BPMN model that has the form "${ ... }" is interpreted as-is; i.e. the value is returned as the return type of the given expression (so it can be an Integer, Date ...). If the format is anything different, it will be interpreted as a GString and forced to be a String after evaluation.
Scripts
Scripts are executed as Groovy scripts.
The binding has dynamic properties that are searchd for in the follwing order:
  • Local variable names
  • Global variable names
  • Bean names
Process variables
The following process variables are added to each process:
documentId
The document ID of the associated XML document
user
The name of user who created this process
group
The name of the home group of the user who created this process