Automate image straightening with Blue Prism and Python

This article follows the article explaining how to use the Python deskew library to straighten images that are unfortunately not straight. So I will not explain how to use this library but rather show you how to use it in a Blue Prism PLC in order to automate this type of task.

To do this we will:

• Create a web service (REST) ​​that exposes the image straightening function.
• Declare this new service in Blue Prism
• Create a Blue Prism object (VBO) that will encapsulate this function

Once these steps have been completed, it will be very easy to create as many Blue Prism Processes as you want in order to be able to straighten images.

Web Service code

With Python and Flask we are going to create in a few lines of code a REST web service which exposes a deskew method. Here is the Python code that does the recovery:

import numpy as np
from skimage import io
from skimage.transform import rotate
from skimage.color import rgb2gray
from deskew import determine_skew
import jsonpickle
import cv2
from flask import Flask, request, Response
app = Flask(__name__)

# deskew image
def deskew_image(image):
    trace("Deskew Image")
    #image = io.imread(_img)
    grayscale = rgb2gray(image)
    angle = determine_skew(grayscale)
    rotated = rotate(image, angle, resize=True) * 255
    return rotated.astype(np.uint8)

# Check
@app.route('/check', methods=['GET'])
def check():
    output = {}
    output['status'] = "Service running"
    response_pickled = jsonpickle.encode(output)
    return Response(response=response_pickled, status=200, mimetype="application/json")

@app.route('/deskew', methods=['POST'])
def deskew():
    output = {}
    # Get destination filename
    targetfile = request.args.get("targetfile")
    # Get binary file, convert string of image data to uint8 and decode image
    data = request.data
    nparr = np.frombuffer(data, np.uint8)
    image = cv2.imdecode(nparr, cv2.IMREAD_COLOR)
    image_out = deskew_image(image)
    cv2.imwrite(targetfile, image_out)
    output['status'] = "Saved"
    # Prepare response, encode JSON to return
    response_pickled = jsonpickle.encode(output)
    return Response(response=response_pickled, status=200, mimetype="application/json")

if __name__ == '__main__':
    app.run(debug=True, host='127.0.0.1', port=8090)

Save this code in an xxx.py file and run it from the command line via:

$ python xxx.py

Normally you should get something like this:

This means that the Web service is listening. If this is not the case, you are certainly missing some Python libraries. Go through the articles mentioned above to verify that they were all imported with pip or conda.

You will notice that this web service has two methods:

• / check which will allow the Blue Prism Object (VBO) to check that the Web Service is listening and if it is not the case to launch it.
• / deskew which allows you to straighten the image. The code for this service is roughly the one described in the article on image straightening.

Declaration of service in Blue Prism

The declaration or referencing of a web service in Blue Prism is extremely simple and is done in a completely graphic way. To do this open the Blue Prism client and go to System, then click on Web API Services:

Add a service by clicking on the link on the right: “Add Service”. Now you have to configure the information and interfaces of your service with the wizard. First, specify the name and URL of the service (Cf. Flask above):

Referencing of the action / check

Then, select the actions (or methods) and add the first one which allows to verify that the service is started (check):

Add the name of the action and activate it (enabled):

This action has no parameters. Click Request to specify how to call this method and enter / check in the “URL Path” field.

Then click on the service response to specify the items returned by this action. Fill in as follows:

Referencing of the action / deskew

Now we are going to reference the straightening action in the same way. The principle is the same except that this service requests as input a file (binary) and a parameter (the name of the file in which the rectified image will be placed).

Let’s start with a new action: deskew.

In the Parameter section, we will specify a single “targetfile” parameter (the binary file is not considered as a parameter because it is sent via the HTTP POST method).

In the Request part we must specify several things:

• The HTTP method: here necessarily POST
• How the parameter is used in the http request (its syntax): / deskew? Targetfile = [targetfile]
• Specify that the body of the message contains a single file

All that remains is to specify the items returned by the service:

Here it is, it is not more complicated than that, it remains now to create an object which will use this service.

Creation of the Blue Prism Object

To do this create an Object in Blue Prism (here I name it pyImageUtils:

Once created, just add an action. I’ll call it deskew here. This Blue Prism action has as input data:

• The file in binary format to straighten (Variable Image)
• The name of the file that will contain the straightened image.

These are exactly the same entry points as the Web Service above. At the exit we will send back a simple message specifying whether everything went well or not.

Here’s what the action looks like in Blue Prism:

In the upper part, we find the start of the action with the two parameters (image and destination file). Then after starting the service (via the command line above), we ask for the execution of the deskew method of the web service. This is done via a simple configuration of the “deskew” action above:

Here is the service is now operational and maintained by a reusable Blue Prism object.

Let’s try it!

In debug mode with Blue Prism it will only take a few seconds. To do the test first, we must specify the service input parameters:

• Double Click on the “Data Item” Image and select an image. I’ll take the one below:

• Then specify a file name: for example C: \ BP Assets \ services \ imageutils \ test2.jpg

Start the process.

Look at the result file …

Here is your Robots or Blue Prism process now benefit from a new skill which allows them to straighten images.

You can download all the necessary components for this skill on Github .