Download and Installation
Table of Contents
Prerequisites
- Admin access to a local or remote server (can be local Windows PC or Mac provided it is 64-bit with at least 16GB RAM and 25GB HDD)
- For docker based installation Docker needs to be
installed.
- For Mac, make sure the docker for mac memory allocation is set to at least 16GB.
Download the formsflow.ai
- Clone this github repo: https://github.com/AOT-Technologies/forms-flow-ai
- If deploying to a remote server, you can use nginx as a reverse proxy and SSL engine. To help you, follow the instructions in the nginx README
Installation
There are multiple options for installing formsflow.ai. They are given below
- Docker Based installation
- Openshift Based Installation
Docker Based Installation
Docker single click installation
Download the compressed bundle and follow the instructions from readme.
Docker Full Deployment
Follow the instructions on Docker installation guide
Docker installation guide
This page elaborates how to setup the overall solution using docker.
Table of Contents
Application Setup
- The application will be installed in the following order.
-
Some of the services have dependencies, mentioned below.
Srl No Service Name Usage Access Dependency Details 1 Keycloak
Authentication http://localhost:8080
Keycloak 2 forms-flow-forms
form.io form building. This must be started earlier for resource role id's creation http:/localhost:3001
forms-flow-forms 3 forms-flow-analytics
Redash analytics server, This must be started earlier for redash key creation http://localhost:7000
Keycloak
forms-flow-analytics 4 forms-flow-web
formsflow Landing web app http://localhost:3000
Keycloak
,forms-flow-forms
,forms-flow-analytics
forms-flow-web 5 forms-flow-api
API services http://localhost:5000
Keycloak
forms-flow-api 6 forms-flow-bpm
Camunda integration http://localhost:8000/camunda
Keycloak
forms-flow-bpm
Installation Steps
Make sure you have a Docker machine up and running.
Keycloak Setup
Follow the instructions given here
forms-flow-analytics Setup
Start the analytics server by following the instructions given here
forms-flow-forms Setup
Follow the steps here
forms-flow-web, forms-flow-bpm & forms-flow-api Setup
Follow the steps here
Health Check
- Analytics should be up and available for use at port defaulted to 7000 i.e. http://localhost:7000/
- Business Process Engine should be up and available for use at port defaulted to 8000 i.e. http://localhost:8000/camunda/
- FormIO should be up and available for use at port defaulted to 3001 i.e. http://localhost:3001/
- formsflow.ai Rest API should be up and available for use at port defaulted to 5000 i.e. http://localhost:5000/checkpoint
- formsflow.ai web application should be up and available for use at port defaulted to 3000 i.e. http://localhost:3000/
- Access credentials are mentioned here.
Local keycloak setup
- This setup is preferred for local development only. A docker instance will be created as part of setup. If you have existing Keycloak instance go to Server Keycloak setup.
Prerequisites
- For docker based installation Docker needs to be installed.
Environment Configuration
- Make sure you have a Docker machine up and running.
- Make sure your current working directory is forms-flow-ai/forms-flow-idm/keycloak.
- Optional: Rename the file sample.env to .env. Skip this step if you want to use the default values as mentioned in the table below.
- Optional: Modify the environment variables in the newly created .env file if needed. Environment variables are given in the table below,
Environment Variables
Keycloak Database Connection Details
Skip this for default setup
Keycloak Admin Details
Skip this for default setup
Starting or Stopping Keycloak server
- Keycloak server uses port 8080, make sure the port is available.
cd {Your Directory}/forms-flow-ai/forms-flow-idm/keycloak
To start the keycloak server
- Run
docker-compose up -d
to start.
NOTE: Use --build command with the start command to reflect any future
changes eg
: docker-compose up --build -d
To stop the keycloak server
- Run
docker-compose stop
to stop.
Health Check
The application should be up and available for use in http://localhost:8080
Login Credentials
-----------------
User Name : admin
Password : changeme
Add custom login theme
- Log in to http://localhost:8080
- Select Realm settings>Themes>Login Theme>formsflow>Save.
- Run
docker-compose up --build -d
to verify the changes.
formsflow-ai user credentials
- Default User credentials are generated when keycloak started for the first time, you can modify the values on your keycloak service.
User Role | User Name | Password | User Group |
---|---|---|---|
Designer |
formsflow-designer |
changeme |
formsflow-designer |
Client |
formsflow-client |
changeme |
formsflow-client |
Reviewer |
formsflow-reviewer |
changeme |
formsflow-reviewer |
Clerk |
formsflow-clerk |
changeme |
formsflow-reviewer |
Approver |
formsflow-approver |
changeme |
formsflow-reviewer |
NOTE
All the default configurations are imported to keycloak during the startup, so no manual changes are required at this stage. Redirect uri's are configured as localhost in the default setup, you can configure the ip address (if required) as the redirect uri for the clients by logging into Keycloak.
Local keycloak set up is successfully completed now. You can skip the remaining sections in this page and continue with other installation steps.
Server keycloak setup
- Make sure you downloaded and installed Keycloak.
- To setup a remote keycloak server either download and import the formsflow-ai-realm.json to keycloak ( Applicable only for keycloak version 11.0.0 and above ) or follow the manual steps below.
Create Realm
Create a realm forms-flow-ai
- Login to keycloak with admin privileges
- Click the button "Create Realm" to add new realm forms-flow-ai
- Click Create.
Create Keycloak setup for formsflow web
Create a forms-flow-web Client.
- Login to KeyCloak Realm with admin privileges
- Configure > Clients > Create
- Client ID = forms-flow-web
- Client Protocol = openid-connect
- Click Save
- Settings Tab
- Name = forms-flow-web
- Description = React based FormIO web components
- Access Type = public
- Valid Redirect URIs eg. http://localhost:3000/*
- Valid Web Origins = *
- Click Save
- Roles Tab
- Click Add Role
- Role Name = formsflow-client
- Click Save
- Click Add Role
- Role Name = formsflow-reviewer
- Click Save
- Click Add Role
- Role Name = formsflow-designer
- Click Save
- Click Add Role
- Configure > Clients Scope > Roles > Mappers > Create
- Name = Role
- Mapper Type = User Client Role
- Client ID = forms-flow-web
- Token Claim Name = role
- Click Save
- Configure > Clients
- Select forms-flow-web Client
- Select Mappers tab
- Click Create
- Name = flowsflow-web-mapper
- Mapper Type = Audience
- Included Client Audience = forms-flow-web
- Click Save
- Click Create
- Name = dashboard-mapper
- Mapper Type = User Attribute
- User Attribute = dashboards
- Token Claim Name = dashboards
- Add to ID Token = ON
- Add to access token = ON
- Add to userinfo = ON
- Multivalued = ON
- Aggregate attribute values = ON
- Click Save
Create Keycloak setup for formsflow analytics
- Click Create
Create a forms-flow-analytics Client
- Login to KeyCloak Realm with admin privileges
- Configure > Clients > Create
- Client ID = forms-flow-analytics
- Client Protocol = saml
- Click Save
- Settings Tab
- Name = forms-flow-analytics
- Description = Redash-Analytics
- Sign Assertions = ON
- Signature Algorithm = RSA_SHA256
- SAML Signature Key Name = KEY_ID
- Canonicalization Method = EXCLUSIVE_WITH_COMMENTS
- Name ID Format = email
- Valid Redirect URIs eg. http://localhost:7000/*
- Master SAML Processing URL = http://localhost:7000/saml/callback?org_slug=default
- Note: All other settings like Force POST BINDING, Client Signature Required, Front Channel Logout should be turned off and empty.
- Click Save
- Mappers Tab
- Click Add Builtin
- Click the Add checkbox for X500 surname and X500 givenName
- Click Add selected
- Go Back to Mappers
- Click Edit on X500 surname
- Change Friendly Name to LastName
- Click Save
- Go Back to Mappers
- Click Edit on X500 givenName
- Change Friendly Name to FirstName
- Click Save
- Click Add Builtin
Create Keycloak setup for formsflow bpm
Create a forms-flow-bpm Client.
- Login to KeyCloak Realm with admin privileges
- Configure > Clients > Create
- Client ID = forms-flow-bpm
- Client Protocol = openid-connect
- Click Save
- Settings Tab
- Name = forms-flow-bpm
- Description = Camunda Process Engine Components
- Access Type = confidential
- Service Accounts Enabled = ON
- Valid Redirect URIs eg. http://localhost:8000/camunda/*
- Web Origins = *
- Click Save
- Mappers Tab
- Click Create, and provide in below properties
* Name = username * Mapper Type =User Property * Property = username * Token Claim Name = preferred_username * Claim JSON Type = String * Click Save
- Click Create, and provide in below properties
- Service Accounts Tab
- Select Client roles as "realm-management"
- Map the listed "Available Roles" to
"Assigned
Roles"
a. manage-users b. query-groups c. query-users d. view-users
- Configure > Client Scopes > Create
- Name = camunda-rest-api
- Click Save
- Client Scopes > camunda-rest-api
- Mappers Tab
- Click Create
- Name = camunda-rest-api
- Mapper Type = Audience
- Included Custom Audience = camunda-rest-api
- Click Save
- Mappers Tab
- Configure > Clients > forms-flow-bpm
- Client Scopes Tab
- Default Client Scopes
- Select camunda-rest-api
- Click Add selected
- Client Scopes Tab
- Configure > Clients
- Select forms-flow-bpm Client
- Select Mappers tab
- Click Create
- Name = flowsflow-api-mapper
- Mapper Type = Audience
- Included Custom Audience = forms-flow-web
- Click Save
- Click Create
Create Groups
Create groups to support operations
- Create Main group by Clicking New
- Name = formsflow
- Click Save
- Create Sub group by Clicking on Main group created on step-1 i.e.
formsflow, and
then click New
- Name = formsflow-client
- Click Save
- Create Sub group by Clicking on Main group created on step-1 i.e.
formsflow, and
then click New
- Name = formsflow-designer
- Click Save
- Create Sub group by Clicking on Main group created on step-1 i.e.
formsflow, and
then click New
- Name = formsflow-reviewer
- Click Save
- Create Main group by Clicking New
- Name = camunda-admin
- Click Save
- Default Groups Tab (Assign Default Group to self-registering users)
- From available groups; map the group "formsflow-client" to "Default Groups".
- Create Main group by Clicking New
- Name = formsflow-analytics
- Click Save
- Create Sub group by Clicking on Main group created on step-1 i.e.
formsflow-analytics, and then click New
- Name = group1
- Click Save
- Create Sub group by Clicking on Main group created on step-1 i.e.
formsflow-analytics, and then click New
- Name = group2
- Click Save
You can create as many sub-groups as you want for dashboard authorization feature
Map roles to group
Mapping different roles to group/subgroups:
- Login to KeyCloak Realm with admin privileges
- Manage > Groups > select a subgroup say "formsflow-client" from the list of groups
- Select tab Role Mappings
- Select forms-flow-web from the list of Client Roles selection
- Select formsflow-client role and click add selected
- The selected role will appear in assigned roles for that subgroup.
- Repeat the step 2 and 3 for subgroups formsflow-designer, formsflow-reviewer and choose the respective roles for them according to the table :
Group | Roles | Description |
---|---|---|
camunda-admin | Able to administer Camunda directly and create new workflows | |
formsflow-designer | formsflow-client, formsflow-designer, formsflow-reviewer | Able to access all elements of the formsflow UI including forms design, task list and forms access |
formsflow-reviewer | formsflow-reviewer | Able to access task list and forms access of formsflow UI |
formsflow-client | formsflow-client | Able to access form fill-in only |
Test keycloak access in Postman
Test forms-flow-web access in Postman
- Open Postman
- Create new Request
- Name = forms-flow-web-validation
- Authorization Tab
- Type = OAuth 2.0
- Get New Access Token
- Token Name = forms-flow-web-password-token
- Grant Type = Password Credentials
- Access Token URL (example) = {KEYCLOAK_URL}/auth/realms/{realm name}/protocol/openid-connect/token
- Username = ?
- Password = ?
- Client ID = forms-flow-web
- Scope = openid
- Client Authentication = Send as Basic Auth header
- Click Request Token
- Copy Access Token
- Paste in jwt.io, and examine token
- Should see resource_access[] > roles[] > list of Effective Roles
Test forms-flow-bpm access in Postman
- Open Postman
- Create new Request
- Name = forms-flow-bpm-validation
- Authorization Tab
- Type = OAuth 2.0
- Get New Access Token
- Token Name = forms-flow-bpm-admin-token
- Grant Type = Client Credentials
- Access Token URL (example) = {KEYCLOAK_URL}/auth/realms/{realm name}/protocol/openid-connect/token
- Client ID = forms-flow-bpm
- Client Secret = saved from Credentials Tab in Keycloak
- Scope = openid
- Client Authentication = Send as Basic Auth header
- Click Request Token
- Copy Access Token
- Paste in jwt.io, and examine token
- Should see resource_access[] > roles[] > list of Effective Roles
Server keycloak set up is successfully completed now. You can skip the remaining sections in this page and continue with other installation steps.
Get the Keycloak client secret
- Go to http://localhost:8080 in the browser
- Login to KeyCloak Realm with admin privileges
- Configure > Clients >
- Click on Client ID = forms-flow-bpm
- Got to Credentials Tab
- Copy the secret value if present else click on Regenerate secret button and copy the value
Analytics Engine
formsflow.ai leverages Redash to build interactive dashboards and gain insights. To create meaningful visualization for your use case with formsflow.ai checkout Redash Knowledge base.
Table of Content
Solution Setup
Installation
- Make sure you have a Docker machine up and running.
- Make sure your current working directory is "forms-flow-ai/forms-flow-analytics".
- Rename the file sample.env to .env.
- Modify the environment variables inside .env file if needed. Environment variables are given below
Running the application
- Analytics service uses port 7000, make sure the port is available.
cd {Your Directory}/forms-flow-ai/forms-flow-analytics
The forked version of redash is being used to overcome the limited cors support in redash. The forked repo fixes the cors issues. But if the environment is setup in such a way that redash resides in the same url origin as forms web application , redash can be built from any redash images.
- For Linux,
- Run
docker-compose -f docker-compose-linux.yml run --rm server create_db
to setup database and to create tables. - Run
docker-compose -f docker-compose-linux.yml up -d
to start.
- Run
- For Windows,
- Run
docker-compose -f docker-compose-windows.yml run --rm server create_db
to setup database and to create tables. - Run
docker-compose -f docker-compose-windows.yml up -d
to start.
- Run
Health Check
click here
- The application should be up and available for use at port defaulted to 7000 in http://localhost:7000/ and register with any valid credentials.
Configuration of Keycloak SAML Setup
- Post registration, login to the application with admin credentials.
- Click the menu icon to the left of the username and navigate to Edit Profile.
- Go to tab "Settings", and then navigate to
"General". Under
"Authentication".
- Check the option "Enabled(dynamic)".
- Set the field "SAML Metadata URL" with value of Keycloak SAML descriptor URL. Example. http://{your-ip-address}:8080/auth/realms/forms-flow-ai/protocol/saml/descriptor. {your-ip-address} should be changed to your host system IP address. Please take special care to identify the correct IP address if your system has multiple network cards
- Set the field "SAML Entity ID" value to be
forms-flow-analytics
. - Set the field "SAML NameID Format" value to be
urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress
.
- Logout and login again using valid formsflow.ai keycloak user credentials. Default user credentials are provided here.
forms-flow-analytic (REDASH) setup is successfully completed now. You can skip remaining sections in this page and continue with other installation steps.
Get the Redash API Key
Redash how to use guide
-
Check our guide on how to configure Redash and come up with awesome visualization using redash.
-
If you want to visualize based on data in formsflow.ai, a few sample queries for default forms are available.
Docker Individual Service Deployment
Install the components in the listed order. (NOTE: Keycloak, form.io and redash dependencies are used on other components)
- Keycloak Identity keycloak components
- forms-flow-forms formsflow.ai integration with form.io
- forms-flow-analytics Redash analytics components
- forms-flow-bpm Camunda Workflow deployment and integration
- forms-flow-api REST API of formsflow.ai
- forms-flow-web formsflow.ai integration web UI
Openshift Based Installation
Openshift Full Deployment
Follow the instructions on openshift installation guide
formsflow.ai - Openshift Setup
This page details elaborates about deploying overall in openshift. All the applicaiton are deployed using openshift templates.
The templates are tested , ran and customised for running in BCGOV pathfinder OCP3 and Openshift 4 silver cluster , there could be particular places where the templates are opinionated. Feel free to alter them and generalise them for any kubernetes/openshift deployments
Every component has a build config [bc], Deployment config and might have a param file. The general syntax for running them is
oc process -f web_dc.yaml --param-file=web_param.yaml |oc apply -f - --ignoreunknownparams
an ideal topology will look like
Databases
Databases
The application uses Postgres and Mongo DB for persistence.The below templates can be used as a reference to spin up new databases.Alternatively new databse schema can be created with in the existing database.
how to deploy postgres
Postgres HA and Non-HA templates can be used.
A sample patroni templates can be found at patroni-build.yaml , patroni-build.yaml , patroni-deployment.yaml , patroni-deployment-prereq.yaml
To avail the latest patroni templates ,Please refer to BCDevOps/platform-services/patroni templates
A sample non-Ha template is checked in here at postgres-non-ha template
how to deploy Mongo
Mongo HA and Non-HA templates can be used.
A sample Mongo HA template can be found at mongo-ha.yaml A sample Mongo non-HA templates can be found at mongodb-nonha.yaml
Secrets and Config maps
secrets
The forms-flow-ai secret contain most of the values which are mandatory for application to work.
running the formio_secrets_param.yaml with proper values should create the required secret.
There are other secrets which are needed as well. They are mostly Mongo , Postgres,Redash secrets as in the below screenshot
config maps
the two major config which are required to run the application is the forms-flow-web-config and forms-flow-web-keycloak-config. forms-flow-web-config contains the ids and urls required for web project to work. forms-flow-web-keycloak-config has the keycloak configs.
Build Configs/Images
Each project needs a build configuration and images/image stream to work with.An ideal build config will look like below.
Deployment Configs
An example deployment config will look like below
Dockerfile openshift_Dockerfile openshift_custom_Dockerfile
Docker files
the project has different deployment topoloiges and there are different docker files for each purpose as below
filename | sample reference | Purpose |
---|---|---|
Dockerfile | Dockerfile | Dockerfile to support docker compose |
openshift_Dockerfile | openshift_Dockerfile | Openshift based deployment |
openshift_custom_Dockerfile | openshift_custom_Dockerfile | To allow customisation and build from this repo.Helps to copy stuff from the buidling repo and merge to the product. |
Verifying the Installation status
The following applications will be started and can be accessed in your browser.
Srl No | Service Name | Usage | Access | Default credentials (userName / Password) |
---|---|---|---|---|
1 | Keycloak |
Authentication | http://localhost:8080 |
admin/changeme |
2 | forms-flow-forms |
form.io form building. This must be started earlier for resource role id's creation | http://localhost:3001 |
admin@example.com/changeme |
3 | forms-flow-analytics |
Redash analytics server, This must be started earlier for redash key creation | http://localhost:7000 |
Use the credentials used for registration / Default user credentials |
4 | forms-flow-web |
formsflow Landing web app | http://localhost:3000 |
Default user credentials |
5 | forms-flow-api |
API services | http://localhost:5000 |
Authorization tocken from keycloak role based user credentials
|
6 | forms-flow-bpm |
Camunda integration | http://localhost:8000/camunda |
Default user credentials |
« Previous Next »