Tutorial2 - A simple web server, part 1
Requirements
- It is recommended to complete the hello world tutorial.
- Basic understanding of programming concepts.
Part1 - Reading a JSON file
In this section, we will focus on creating a standalone flow that reads a JSON file and applies some basic filtering on it.
Step1 - Create a JSON file
Above animation shows creating a JSON file with a list of fruits and their colors in it.
To start, create a new JSON file and name it fruits.json. The visual JSON editor got two tabs - Code view and Design view. Switch to code view and paste the following JSON data. It's a bunch of fruits and their colors.
[
{
"name": "apple",
"color": "red"
},
{
"name": "orange",
"color": "orange"
},
{
"name": "mango",
"color": "yellow"
},
{
"name": "banana",
"color": "yellow"
},
{
"name": "lemon",
"color": "yellow"
},
{
"name": "cherry",
"color": "red"
}
]
Step2 - Create a flow and read the JSON
The animation above shows how to create a flow and read the JSON file as a string.
Next, create a new flow and name it as read.flw. Now we will read the fruits.json's contents inside the flow. To do that, open the core/file package in the packages panel and drag and drop the read module in between the start and the end step. The read module reads a file and returns its contents as a string. If not already open, highlight the read step and switch to the input tab. The input tab contains two string fields, a path, and an encoding. Click on the browse button next to the field path and select the above-created fruits.json. Leave the encoding empty for now.
If you debug the flow now, you can see that the read step's output is the indeed contents of fruits.json.
Step3 - Parse the JSON contents into an Object
The animation above shows how to add a new step to parse the JSON contents into an Object.
We have seen that the read step's output data is in the form of a string. Now to manipulate the data, we need to parse it. To do that, open the core/json package in the package panel, drag the parse module and place it after the read step. And make a transition from the read step to the parse step.
Next, we will map the output from the read step into theJSON String field in the parse step's input. Leave other fields empty for now. We will use the schema field later during this tutorial. If you debug the flow now, you can see a parsed JSON object as the output of the parse step.
Step4 - Create schema & attach to the parse step
In this step, we will see how to bind a JSON schema with the output of the parse step.
The above animation shows how to create a schema & bind it to the parse step.
By default, the parse step cannot recognize the structure of the JSON data, it just shows an any type as its output. To do that it requires a JSON schema to be specified in its input field. After specifying the schema, the output of parse module will also get updated to the given schema. The parse module and many other modules in Codeflow can dynamically update the output schema depending on its input configuration. Binding a schema will help Codeflow designer to perform validations and provide autocompletion.
First right click the file tree and choose New...->Schema.. from the context menu. Name it as fruit.schema. The schema file is opened in a visual editor for creating/editing the schema files. By default, the design view tab is shown and an empty object as the schema. Let it be an object itself and create two string properties under the object named - name and color. This represents a single entry in the fruits array.
Next, create another schema and name it fruits.schema. We use this schema to create an array of the through through JSON schema reference type. Change the type to an array and select the array's content type to a reference. Choose the above-created fruit.schema from the popup dialog. References are a great way of reusing schema files. We just created a schema to represent an array of fruits data.
Next, select the parse step in the read.flw and go to the input tab and specify schema. In order to assign the schema, click the browse button close to the schema input and select the fruits.schema file. Now if you highlight output tab, you can see that the step now has fruits.schema mapped as its output. This kind of dynamic schema binding makes Codeflow be dynamic at the core and have the advantages of statically typed languages as well.
Step5 - Map the output of the parse step to the input of mapper
In this step, we will add a special core module called mapper into the flow.
The above animation shows how to map the output of the parse step to the input of mapper.
To begin, drag and drop the mapper module from the core package and place it after the parse step. The mapper is a data mapping module that lets you specify a JSON schema and apply complex transformations on it. Next, make a connection from the parse step to the mapper If you highlight mapper now, you can notice that it contains an additional tab, named Schema similar to that found in the start and end the steps.
The mapper will let you specify a schema that defines its input. In this case, we will choose fruits.schema as the input schema for mapper and simply assign the output of the parse step into the mapper's input, as shown in the below animation.
The above animation shows how to assign parse's output to mapper.
Step6 - Filter data using higher-order expressions
In this step, we will pass a color as an input to the flow and use it to filter the fruits collection.
The above animation depicts how to filter data using higher-order expressions.
To begin, we will make the flow accept an input parameter named color. Highlight the start step and select the schema tab and add a property color with type string.
Next we will use that to filter the fruits collection based on the name. We will use the standard built-in method filter() available in the expression builder for that.
Highlight the mapper step and click on the existing mapping in its input. It will open up a drop-down box. Click the Surround with function button and it will show a list of all the utility functions available. Search for filter and select it. It's found under the collection namespace.
The filter is a higher-order function with two arguments. The first argument is the collection you need to filter. The second argument is a function that returns a boolean expression that will be repeatedly evaluated for each item in the first argument. The filter method will return a new collection with only the items that the boolean expression returned true.
After mapping, the expression should look like below. A special scope variable 'item' is available in the second argument, that maps to each item of the collection. As you noticed, the item's schema is based on the first argument's item schema.
filter(parse.output, f(equals(item.color, start.output.color)))
Above animation shows adding a higher-order function filter to filter the collection.
Specifying an expression as a higher-order function. It retains the schema after transformation.
Notice that the output of the filter retains the schema. Codeflow expressions are made to work with the JSON schema and will intelligently apply a transformation on the schema whenever needed.
Now you can run the flow in the debug mode and inspect the filtered output.
Running flow in debug mode and inspecting the output of mapper.
Step7 - Map mapper's output to the end step.
In this step, we will finally map the output of mapper as the input of the end step.
Map mapper's output to the end step.
The end is the final step of a flow. Whatever is mapped in the end step will get passed to whoever invokes that flow. The end step works similar to the mapper step. We will repeat the process of specifying the schema in the Schema tab and simply assign the value from the output of mapper as the input value.
You can now run the debugger once again and see the end step now shows the same value as mapper. Whoever is calling the read.flw can pass a color and get back only the fruits with that color.
Summary
Congrats! You have successfully created a flow that can read, parse and filter a JSON file. In the next section, we will create a web server and invoke the flow file we built here.
What you have learned in this section:
- Creating & editing JSON files.
- Using core modules to read a file and parse it.
- Working with JSON schema.
- Specifying schema in
mapper,start&end. - Applying complex expressions to filter & transform data.