Input mapping

Animation showing expression editor in action in the designer

Introduction

As seen in the previous chapters, the transitions are used to represent the program flow, which is unlike most other visual programming paradigms, where the connection between the nodes is used to represent the data flow. That approach, while being a very pure form of visual programming, can lead to something equivalent of a spaghetti code, with too many connections between individual nodes that makes it difficult to understand the high-level control flow, especially when the programs become large.

Codeflow uses transitions to represent the high-level, asynchronous program flow. The actual input data mapping of individual steps is abstracted away into the configuration panel. This makes visualizing the program flow really easy, especially when working with large flows.

However, this approach opens up another challenge - the modules, being loosely coupled, outputs data in a structure totally independent of each other. How to map output from multiple such steps to feed to the input of another one?

Enter Codeflow expressions. Codeflow expressions are built with several unique, innovative features such as higher-order utility functions to provide a simple and efficient way to map and merge even complex data from various steps into the input of another one. The expressions help abstract, the often complex, input mapping logic to the configuration panel and keeps the program flow clean, that will convey the big picture.

The below image shows the example of a flow that remains clean and shows the high-level program flow, while the configuration tab of the buildFilter step showing the data mapping.

Image showing the input mapping, neatly abstracted from the program flow.

The expressions are designed to work with the visual editor Codeflow designer which comes with advanced auto-completion and validation mechanisms for creating expressions.

Types of expressions

• Field level. Field level expressions are expressions that are applied to a single field. They are written as one-liners, but powerful enough to perform complex transformations.

• Block level. forEach and if/elseif/else are the two block level expressions supported. Block expressions apply to a block of input fields.

Both types of Codeflow expressions are immutable and do not change the data.

Field level expressions

Field level expressions, or simply expressions, is a minimalistic expression language capable of performing complex transformations using a visual interface. The expressions are converted by the runtime engine into JavaScript for maximum performance.

Syntax

The expressions include only data and functions. To keep it simple and support touch-based interfaces in the future, there are no keywords or operators. Every operation is performed using utility methods, including maths, logical and other utility methods.

Few examples:

add(1, 2) //equivalent to 1+2

mul(add(1, 2), 3) //equivalent to (1+2) * 3

array(1,2,3) //equivalent to new Array(1,2,3)

Editing the expressions

The designer includes a novel, intuitive graphical interface for editing the expressions visually. The designer shows a dropdown when the expression is on focus like the image shown below:

Data

The data section in the autocomplete dropdown shows all the mappable data in a tree structure, including the output and the error data from all the connected (and completed) steps, the scope data (if any) and global variables.

Utility functions

The function section in the autocomplete dropdown shows the available utility functions for use in the expressions. The functions and arguments are self-documented with placeholders for each argument. To use a function simply expand the category and click on the function.

Higher-order functions

The utility functions include higher-order functions as well, similar to those found typically in popular functional programming stacks. The higher-order functions take an expression as one of its arguments (wrapped under f()) and apply the expression dynamically based on the input data. For example, the map function takes a collection as the first argument and another expression as the second argument. It then returns a new list with the expression evaluated for each element in the collection. The higher-order functions provide powerful data transformation capabilities while maintaining the immutability and a simplified user interface.

Animation showing the usage of the higher-order function find() to find an item from a list.

Few common higher-order functions:

• map - Return a new array by mapping each value in the input collection through the mapper function.

• reduce - Convert a collection of values into a single value.

• filter - An expression to filter the items in the collection.

• find - Find a single item in a collection.

• sort - Returns a sorted copy of the collection

• group - Splits the collection into groups.

• minOf - Returns the minimum value in the list.

• maxOf - Return the maximum value in the list.

Block level expressions

Block level expressions apply to one or more fields and provides an imperative style of operation, though it's also immutable. There are currently two set of block level expressions:

• forEach
• if/else/elseif

forEach

forEach is useful to iterate over a given collection (objects or arrays) and add items to the target collection. The below animation shows forEach in action:

Animation showing a simple forEach.

forEach is applied on repeating items, including the array or object items. To repeat an array item, simply click on the Add forEach helper button inside the array or manually highlight an array item and click on the forEach button on the toolbar. forEach keyword wraps the array item and will repeatedly generate the item for every entry in the mapped collection.

The input collection can be an array, object or even a string. The as field will let you specify the iteratee name and it defaults to 'item'. A special scope section inside the drop down will show the iteration data under the specified key. There can be multiple nested forEachs and the scope variable will include all the scope variables.

Animation showing a nested forEach.

if/else/elseIf

The if/else blocks are used for conditional processing, within or without the loop. They also wrap a field or group of fields and will generate the block only if the condition evaluates to true.

A sample usage of if/else.

The if/else block can also be nested and combined with forEach to perform powerful data manipulation operations.

Using forEach to create dynamic objects

Currently, forEach works out-of-box only on array items, as there is no visual support for iterating over object keys. However, the below method can be used to create dynamic objects by adding an additional property inside a forEach and changing the key's value to an expression:

• Click on Add new property field and add a new additional property inside the object. Give key as any value, we will change that to an expression.

• Select the newly added property and click the forEach button to wrap a forEach block around the property.

• Now if you run this, there will be only one property in the Object as the key will remain a constant during each iteration.

• To make the key dynamic, You need to manually put a Codeflow expression inside the curly braces {}.

• To do that, replace the temporary key with a Codeflow expression but placed inside curly braces { }. Since there won't be any auto-complete, you need to manually type the expression or create it in the value field and copy paste from there.

• Now set the value for the field and it should create an object with dynamic keys and values.