Download and Installation



Table of Contents

  1. Prerequisites
  2. Download the formsflow.ai
  3. Installation
  4. Verifying the Installation Status

Prerequisites

  • Admin access to a local or remote server (can be local Windows PC or Mac provided it is 64-bit with at least 16GB RAM and 25GB HDD)
  • For docker based installation Docker needs to be installed.
    • For Mac, make sure the docker for mac memory allocation is set to at least 16GB.

Download the formsflow.ai

Installation

There are multiple options for installing formsflow.ai. They are given below

Docker Based Installation


Docker single click installation

Download the compressed bundle and follow the instructions from readme.

Docker Full Deployment

Follow the instructions on Docker installation guide

Docker installation guide

This page elaborates how to setup the overall solution using docker.

Table of Contents

  1. Application Setup
  2. Usage Instructions

Application Setup

  • The application will be installed in the following order.
  • Some of the services have dependencies, mentioned below.


    Srl No Service Name Usage Access Dependency Details
    1 Keycloak Authentication http://localhost:8080 Keycloak
    2 forms-flow-forms form.io form building. This must be started earlier for resource role id's creation http:/localhost:3001 forms-flow-forms
    3 forms-flow-analytics Redash analytics server, This must be started earlier for redash key creation http://localhost:7000 Keycloak forms-flow-analytics
    4 forms-flow-web formsflow Landing web app http://localhost:3000 Keycloak, forms-flow-forms, forms-flow-analytics forms-flow-web
    5 forms-flow-api API services http://localhost:5000 Keycloak forms-flow-api
    6 forms-flow-bpm Camunda integration http://localhost:8000/camunda Keycloak forms-flow-bpm

Installation Steps

Make sure you have a Docker machine up and running.

Keycloak Setup

Follow the instructions given here

forms-flow-analytics Setup


Start the analytics server by following the instructions given here

forms-flow-forms Setup


Follow the steps here

forms-flow-web, forms-flow-bpm & forms-flow-api Setup

Follow the steps here

Health Check


Local keycloak setup

  • This setup is preferred for local development only. A docker instance will be created as part of setup. If you have existing Keycloak instance go to Server Keycloak setup.

Prerequisites

  • For docker based installation Docker needs to be installed.

Environment Configuration

  • Make sure you have a Docker machine up and running.
  • Make sure your current working directory is forms-flow-ai/forms-flow-idm/keycloak.
    • Optional: Rename the file sample.env to .env. Skip this step if you want to use the default values as mentioned in the table below.
    • Optional: Modify the environment variables in the newly created .env file if needed. Environment variables are given in the table below,

Environment Variables

Keycloak Database Connection Details


Skip this for default setup

image

Keycloak Admin Details


Skip this for default setup

image

Starting or Stopping Keycloak server

  • Keycloak server uses port 8080, make sure the port is available.
  • cd {Your Directory}/forms-flow-ai/forms-flow-idm/keycloak

To start the keycloak server

  • Run docker-compose up -d to start.

NOTE: Use --build command with the start command to reflect any future changes eg : docker-compose up --build -d

To stop the keycloak server

  • Run docker-compose stop to stop.

Health Check

The application should be up and available for use in http://localhost:8080

Login Credentials
        -----------------
        User Name : admin
        Password  : changeme
        

Add custom login theme

  • Log in to http://localhost:8080
  • Select Realm settings>Themes>Login Theme>formsflow>Save.
  • Run docker-compose up --build -d to verify the changes.

formsflow-ai user credentials


  • Default User credentials are generated when keycloak started for the first time, you can modify the values on your keycloak service.
User Role User Name Password User Group
Designer formsflow-designer changeme formsflow-designer
Client formsflow-client changeme formsflow-client
Reviewer formsflow-reviewer changeme formsflow-reviewer
Clerk formsflow-clerk changeme formsflow-reviewer
Approver formsflow-approver changeme formsflow-reviewer

NOTE

All the default configurations are imported to keycloak during the startup, so no manual changes are required at this stage. Redirect uri's are configured as localhost in the default setup, you can configure the ip address (if required) as the redirect uri for the clients by logging into Keycloak.


Local keycloak set up is successfully completed now. You can skip the remaining sections in this page and continue with other installation steps.

Server keycloak setup

  • Make sure you downloaded and installed Keycloak.
  • To setup a remote keycloak server either download and import the formsflow-ai-realm.json to keycloak ( Applicable only for keycloak version 11.0.0 and above ) or follow the manual steps below.

Create Realm

Create a realm forms-flow-ai

  • Login to keycloak with admin privileges
  • Click the button "Create Realm" to add new realm forms-flow-ai
  • Click Create.

Create Keycloak setup for formsflow web

Create a forms-flow-web Client.

  • Login to KeyCloak Realm with admin privileges
  • Configure > Clients > Create
    • Client ID = forms-flow-web
    • Client Protocol = openid-connect
    • Click Save
    • Settings Tab
      • Name = forms-flow-web
      • Description = React based FormIO web components
      • Access Type = public
      • Valid Redirect URIs eg. http://localhost:3000/*
      • Valid Web Origins = *
      • Click Save
    • Roles Tab
      • Click Add Role
        • Role Name = formsflow-client
        • Click Save
      • Click Add Role
        • Role Name = formsflow-reviewer
        • Click Save
      • Click Add Role
        • Role Name = formsflow-designer
        • Click Save
  • Configure > Clients Scope > Roles > Mappers > Create
    • Name = Role
    • Mapper Type = User Client Role
    • Client ID = forms-flow-web
    • Token Claim Name = role
    • Click Save
  • Configure > Clients
  • Select forms-flow-web Client
  • Select Mappers tab
    • Click Create
      • Name = flowsflow-web-mapper
      • Mapper Type = Audience
        • Included Client Audience = forms-flow-web
        • Click Save
          • Click Create
          • Name = dashboard-mapper
          • Mapper Type = User Attribute
          • User Attribute = dashboards
          • Token Claim Name = dashboards
          • Add to ID Token = ON
          • Add to access token = ON
          • Add to userinfo = ON
          • Multivalued = ON
          • Aggregate attribute values = ON
          • Click Save

            Create Keycloak setup for formsflow analytics

Create a forms-flow-analytics Client

  • Login to KeyCloak Realm with admin privileges
  • Configure > Clients > Create
    • Client ID = forms-flow-analytics
    • Client Protocol = saml
    • Click Save
    • Settings Tab
      • Name = forms-flow-analytics
      • Description = Redash-Analytics
      • Sign Assertions = ON
        • Signature Algorithm = RSA_SHA256
        • SAML Signature Key Name = KEY_ID
        • Canonicalization Method = EXCLUSIVE_WITH_COMMENTS
      • Name ID Format = email
      • Valid Redirect URIs eg. http://localhost:7000/*
      • Master SAML Processing URL = http://localhost:7000/saml/callback?org_slug=default
      • Note: All other settings like Force POST BINDING, Client Signature Required, Front Channel Logout should be turned off and empty.
      • Click Save
    • Mappers Tab
      • Click Add Builtin
        • Click the Add checkbox for X500 surname and X500 givenName
        • Click Add selected
      • Go Back to Mappers
        • Click Edit on X500 surname
        • Change Friendly Name to LastName
        • Click Save
      • Go Back to Mappers
        • Click Edit on X500 givenName
        • Change Friendly Name to FirstName
        • Click Save

Create Keycloak setup for formsflow bpm

Create a forms-flow-bpm Client.

  • Login to KeyCloak Realm with admin privileges
  • Configure > Clients > Create
    • Client ID = forms-flow-bpm
    • Client Protocol = openid-connect
    • Click Save
    • Settings Tab
      • Name = forms-flow-bpm
      • Description = Camunda Process Engine Components
      • Access Type = confidential
      • Service Accounts Enabled = ON
      • Valid Redirect URIs eg. http://localhost:8000/camunda/*
      • Web Origins = *
      • Click Save
    • Mappers Tab
      • Click Create, and provide in below properties
          * Name = username  
                  * Mapper Type =User Property  
                  * Property = username  
                  * Token Claim Name = preferred_username  
                  * Claim JSON Type = String  
                  * Click Save  
                
    • Service Accounts Tab
      • Select Client roles as "realm-management"
      • Map the listed "Available Roles" to "Assigned Roles"
          a. manage-users
                  b. query-groups  
                  c. query-users  
                  d. view-users              
                
  • Configure > Client Scopes > Create
    • Name = camunda-rest-api
    • Click Save
    • Client Scopes > camunda-rest-api
      • Mappers Tab
        • Click Create
        • Name = camunda-rest-api
        • Mapper Type = Audience
        • Included Custom Audience = camunda-rest-api
        • Click Save
  • Configure > Clients > forms-flow-bpm
    • Client Scopes Tab
      • Default Client Scopes
      • Select camunda-rest-api
      • Click Add selected
  • Configure > Clients
  • Select forms-flow-bpm Client
  • Select Mappers tab
    • Click Create
      • Name = flowsflow-api-mapper
      • Mapper Type = Audience
        • Included Custom Audience = forms-flow-web
        • Click Save

Create Groups

Create groups to support operations

  • Create Main group by Clicking New
    • Name = formsflow
    • Click Save
  • Create Sub group by Clicking on Main group created on step-1 i.e. formsflow, and then click New
    • Name = formsflow-client
    • Click Save
  • Create Sub group by Clicking on Main group created on step-1 i.e. formsflow, and then click New
    • Name = formsflow-designer
    • Click Save
  • Create Sub group by Clicking on Main group created on step-1 i.e. formsflow, and then click New
    • Name = formsflow-reviewer
    • Click Save
  • Create Main group by Clicking New
    • Name = camunda-admin
    • Click Save
  • Default Groups Tab (Assign Default Group to self-registering users)
    • From available groups; map the group "formsflow-client" to "Default Groups".
  • Create Main group by Clicking New
    • Name = formsflow-analytics
    • Click Save
  • Create Sub group by Clicking on Main group created on step-1 i.e. formsflow-analytics, and then click New
    • Name = group1
    • Click Save
  • Create Sub group by Clicking on Main group created on step-1 i.e. formsflow-analytics, and then click New
    • Name = group2
    • Click Save

You can create as many sub-groups as you want for dashboard authorization feature

Map roles to group

Mapping different roles to group/subgroups:

  • Login to KeyCloak Realm with admin privileges
  • Manage > Groups > select a subgroup say "formsflow-client" from the list of groups
  • Select tab Role Mappings
    • Select forms-flow-web from the list of Client Roles selection
    • Select formsflow-client role and click add selected
    • The selected role will appear in assigned roles for that subgroup.
  • Repeat the step 2 and 3 for subgroups formsflow-designer, formsflow-reviewer and choose the respective roles for them according to the table :
Group Roles Description
camunda-admin Able to administer Camunda directly and create new workflows
formsflow-designer formsflow-client, formsflow-designer, formsflow-reviewer Able to access all elements of the formsflow UI including forms design, task list and forms access
formsflow-reviewer formsflow-reviewer Able to access task list and forms access of formsflow UI
formsflow-client formsflow-client Able to access form fill-in only

Test keycloak access in Postman

Test forms-flow-web access in Postman

  • Open Postman
  • Create new Request
    • Name = forms-flow-web-validation
    • Authorization Tab
      • Type = OAuth 2.0
      • Get New Access Token
        • Token Name = forms-flow-web-password-token
        • Grant Type = Password Credentials
        • Access Token URL (example) = {KEYCLOAK_URL}/auth/realms/{realm name}/protocol/openid-connect/token
        • Username = ?
        • Password = ?
        • Client ID = forms-flow-web
        • Scope = openid
        • Client Authentication = Send as Basic Auth header
        • Click Request Token
        • Copy Access Token
        • Paste in jwt.io, and examine token
        • Should see resource_access[] > roles[] > list of Effective Roles

Test forms-flow-bpm access in Postman

  • Open Postman
  • Create new Request
    • Name = forms-flow-bpm-validation
    • Authorization Tab
      • Type = OAuth 2.0
      • Get New Access Token
        • Token Name = forms-flow-bpm-admin-token
        • Grant Type = Client Credentials
        • Access Token URL (example) = {KEYCLOAK_URL}/auth/realms/{realm name}/protocol/openid-connect/token
        • Client ID = forms-flow-bpm
        • Client Secret = saved from Credentials Tab in Keycloak
        • Scope = openid
        • Client Authentication = Send as Basic Auth header
        • Click Request Token
        • Copy Access Token
        • Paste in jwt.io, and examine token
        • Should see resource_access[] > roles[] > list of Effective Roles

Server keycloak set up is successfully completed now. You can skip the remaining sections in this page and continue with other installation steps.

Get the Keycloak client secret

  • Go to http://localhost:8080 in the browser
  • Login to KeyCloak Realm with admin privileges
  • Configure > Clients >
    • Click on Client ID = forms-flow-bpm
    • Got to Credentials Tab
    • Copy the secret value if present else click on Regenerate secret button and copy the value

Analytics Engine

Redash

formsflow.ai leverages Redash to build interactive dashboards and gain insights. To create meaningful visualization for your use case with formsflow.ai checkout Redash Knowledge base.

Table of Content

  1. Solution Setup
  2. Get the Redash API key
  3. Redash how to use guide

Solution Setup

Installation

  • Make sure you have a Docker machine up and running.
  • Make sure your current working directory is "forms-flow-ai/forms-flow-analytics".
  • Rename the file sample.env to .env.
  • Modify the environment variables inside .env file if needed. Environment variables are given below

image

Running the application

  • Analytics service uses port 7000, make sure the port is available.
  • cd {Your Directory}/forms-flow-ai/forms-flow-analytics

The forked version of redash is being used to overcome the limited cors support in redash. The forked repo fixes the cors issues. But if the environment is setup in such a way that redash resides in the same url origin as forms web application , redash can be built from any redash images.

  • For Linux,
    • Run docker-compose -f docker-compose-linux.yml run --rm server create_db to setup database and to create tables.
    • Run docker-compose -f docker-compose-linux.yml up -d to start.
  • For Windows,
    • Run docker-compose -f docker-compose-windows.yml run --rm server create_db to setup database and to create tables.
    • Run docker-compose -f docker-compose-windows.yml up -d to start.

Health Check

click here

  • The application should be up and available for use at port defaulted to 7000 in http://localhost:7000/ and register with any valid credentials.

Configuration of Keycloak SAML Setup

  • Post registration, login to the application with admin credentials.
  • Click the menu icon to the left of the username and navigate to Edit Profile.
  • Go to tab "Settings", and then navigate to "General". Under "Authentication".
    • Check the option "Enabled(dynamic)".
    • Set the field "SAML Metadata URL" with value of Keycloak SAML descriptor URL. Example. http://{your-ip-address}:8080/auth/realms/forms-flow-ai/protocol/saml/descriptor. {your-ip-address} should be changed to your host system IP address. Please take special care to identify the correct IP address if your system has multiple network cards
    • Set the field "SAML Entity ID" value to be forms-flow-analytics.
    • Set the field "SAML NameID Format" value to be urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress.
  • Logout and login again using valid formsflow.ai keycloak user credentials. Default user credentials are provided here.

forms-flow-analytic (REDASH) setup is successfully completed now. You can skip remaining sections in this page and continue with other installation steps.

Get the Redash API Key

here

Redash how to use guide

Docker based full installation is completed now.

Docker Individual Service Deployment

Install the components in the listed order. (NOTE: Keycloak, form.io and redash dependencies are used on other components)

Openshift Based Installation


Openshift Full Deployment

Follow the instructions on openshift installation guide

formsflow.ai - Openshift Setup

This page details elaborates about deploying overall in openshift. All the applicaiton are deployed using openshift templates.

The templates are tested , ran and customised for running in BCGOV pathfinder OCP3 and Openshift 4 silver cluster , there could be particular places where the templates are opinionated. Feel free to alter them and generalise them for any kubernetes/openshift deployments

Every component has a build config [bc], Deployment config and might have a param file. The general syntax for running them is

oc process -f web_dc.yaml --param-file=web_param.yaml |oc apply -f - --ignoreunknownparams

an ideal topology will look like

Databases

Databases

The application uses Postgres and Mongo DB for persistence.The below templates can be used as a reference to spin up new databases.Alternatively new databse schema can be created with in the existing database.

how to deploy postgres

Postgres HA and Non-HA templates can be used.

A sample patroni templates can be found at patroni-build.yaml , patroni-build.yaml , patroni-deployment.yaml , patroni-deployment-prereq.yaml

To avail the latest patroni templates ,Please refer to BCDevOps/platform-services/patroni templates

A sample non-Ha template is checked in here at postgres-non-ha template

how to deploy Mongo

Mongo HA and Non-HA templates can be used.

A sample Mongo HA template can be found at mongo-ha.yaml A sample Mongo non-HA templates can be found at mongodb-nonha.yaml

Secrets and Config maps

secrets

The forms-flow-ai secret contain most of the values which are mandatory for application to work.

running the formio_secrets_param.yaml with proper values should create the required secret.

There are other secrets which are needed as well. They are mostly Mongo , Postgres,Redash secrets as in the below screenshot

config maps

the two major config which are required to run the application is the forms-flow-web-config and forms-flow-web-keycloak-config. forms-flow-web-config contains the ids and urls required for web project to work. forms-flow-web-keycloak-config has the keycloak configs.

Build Configs/Images

Each project needs a build configuration and images/image stream to work with.An ideal build config will look like below.

Deployment Configs

An example deployment config will look like below

Dockerfile openshift_Dockerfile openshift_custom_Dockerfile

Docker files

the project has different deployment topoloiges and there are different docker files for each purpose as below

filename sample reference Purpose
Dockerfile Dockerfile Dockerfile to support docker compose
openshift_Dockerfile openshift_Dockerfile Openshift based deployment
openshift_custom_Dockerfile openshift_custom_Dockerfile To allow customisation and build from this repo.Helps to copy stuff from the buidling repo and merge to the product.


Verifying the Installation status

The following applications will be started and can be accessed in your browser.

Srl No Service Name Usage Access Default credentials (userName / Password)
1 Keycloak Authentication http://localhost:8080 admin/changeme
2 forms-flow-forms form.io form building. This must be started earlier for resource role id's creation http://localhost:3001 admin@example.com/changeme
3 forms-flow-analytics Redash analytics server, This must be started earlier for redash key creation http://localhost:7000 Use the credentials used for registration / Default user credentials
4 forms-flow-web formsflow Landing web app http://localhost:3000 Default user credentials
5 forms-flow-api API services http://localhost:5000 Authorization tocken from keycloak role based user credentials
6 forms-flow-bpm Camunda integration http://localhost:8000/camunda Default user credentials

--> a