ZEMOSO ENGINEERING STUDIO
June 5, 2022
6 min read

Orchestrating backend services for product development with AWS Step Functions

The challenge

In many use cases, there are processes that need to execute multiple tasks. We build microservices or server-less functions like Amazon Web Services (AWS) Lambda functions to carry out these tasks. Most of these services are stateless functions; hence there's a need for queues or databases to maintain the state of individual tasks and the process. Writing code that orchestrates these tasks can be painful and hard to debug and maintain.

AWS Step Functions

AWS Step Functions is a web service that enables you to coordinate the components of distributed applications and microservices using visual workflows. Step Functions have been around since 2016, it was announced at re:Invent 2016.

Step Functions

1. Enables us to define visual workflows for processes by writing minimal or no code

2. Scales out automatically

3. Deals with failures and timeouts of tasks

4. Maintains an auditable log of all state changes

Step Functions are based on the concept of tasks, states, and state machines. The tasks do all the work. A task can be a Lambda Function, an activity which can be fulfilled on other AWS-hosted resources, or even an activity defined on our own machines.

The state machine

A state machine is the workflow and the biggest component of Step Functions. State machines are defined using JavaScript Object Notation (JSON) text written in the syntax of the AWS States Language.

States

The states are blocks that represent some action. Here's a list of states available in the States Language:

1. Pass: does nothing. Mainly used for debugging or as a placeholder state

2. Task: execute some code or run an activity

3. Choice: adds branching logic to the state machine (if/else?)

4. Wait: waits for a specified time before moving on to the next state

5. Succeed: terminates the execution with a success

6. Fail: terminates the execution with a failure

7. Parallel: adds parallel branching to the state function

You can use the AWS Step Functions console to start making simple flows (state machines) using Step Functions. There are pre-configured examples available. We’ll go through the example — which extracts Exchangeable Image File (EXIF) metadata from an image, crops the object from the image, and saves it to S3. We'll assume that Lambda functions are deployed. We'll use Step Functions to orchestrate the Lambda functions to process the images.

Image metadata extraction example with Step Functions

Goal: Extract metadata from images, resize images to medium size, and a thumbnail. Upload the details to the database.

We have two Lambda functions already set up

1. get-exif-lambda: downloads image from S3, extracts EXIF data

   Input: S3 image URI

   Output: image metadata

2. process-image-lambda: downloads image from S3, resizes it to the desired size

    Input: S3 image URI, desired size of the output image

    Output: S3 URIs of the processed images

Creating the state machine

A state machine can be created by the AWS Command Line Interface (CLI), or the AWS Step Functions console. It’s easier to create it from the console because you can see a visual representation of it. It's also easy to set up the required Identity and Access Management (IAM) role to give the state machine permission to the required AWS resources from the console.

Open the Step Functions console and create a new state machine with the following JSON.

{

 "StartAt":  "GetExif",

 "States":  {

   "GetExif":  {

     "Type":  "Task",

     "Resource":  "<get-exif-lambda arn>",

     "Next":  "ResizeImage"

   },

   "ResizeImage":  {

     "Type":  "Parallel",

     "Next":  "WriteToDb",

     "ResultPath":  "$.resizedLinks",

     "Branches":  [

       {

        "StartAt":  "MediumSize",

        "States":  {

          "MediumSize":  {

           "Type":  "Task",

           "Resource":  "<resize-image-lambda arn>",

           "Parameters":  {

             "thumbnail":  false,

             "source.$":  "$.source",

             "maxHeight":  600,

             "maxWidth":  600

           },

            "End":  true

          }

        }

      },

      {

        "StartAt":  "Thumbnail",

        "States":  {

          "Thumbnail":  {

            "Type":  "Task",

            "Resource":  "<resize-image-lambda arn>",

            "Parameters":  {

               "thumbnail":  true,

               "source.$":  "$.source",

               "maxHeight":  128,

               "maxWidth":  128

             },

            "End":  true

          }

        }

      }

     ]

   },

   "WriteToDb": {

     "Type":  "Task",

     "Resource":  "arn:aws:states:::dynamodb:putItem",

     "Parameters":  {

       "TableName":  "image-details",

       "Item":  {

         "key":  {

           "S.$":  "$.source.key"

         },

         "exif":  {

           "S.$":  "$.exif"

         },

         "mediumURL":  {

           "S.$":  "$.resizedLinks[0]"

         },

         "thumbnailURL":  {

           "S.$":  "$.resizedLinks[1]"

         }

       }

     },

     "End":  true

   }

 }

}

In the next step, provide an IAM role for the state machine. Select “Create an IAM role,” give a valid role name, and proceed. This will create a new IAM role, which provides it access only to the resources that are used by the state machine. The state machine created will look like this

Visual representation of the state machine
Visual representation of the state machine

The flow starts at the GetExif state, which executes a Lambda function to retrieve metadata from the image.

Then it executes two image conversion tasks:MediumSize, and Thumbnail, parallelly.

This data is passed to WriteToDb, which writes the output to DynamoDB.

This is a simple example. It's very easy to design more complex workflows with other states available in the States Language.

We can also add error handling and retry functionality to the states.

Manipulating input and output of a state

The States Language allows us to manipulate and control the JSON data that flows between states.

In Amazon State Language, these fields manipulate and control the flow of data between states:

1. InputPath

2. OutputPath

3. ResultPath

4. Parameters

This diagram shows the sequence in which these fields are applied to the JSON data.

Input and output state as applied to JSON

InputPath and parameters

Let's assume the state receives the following input.

{

   "message": {

          "title": "Msg Title",

          "content": "Hello World!"

   },

   "timestamp": 12312432

}

We can add InputPath and Parameters to the state.

"InputPath": "$.message",

"Parameters": {

        "messageType": "text",

        "messageTitle.$": "$.title",

        "messageContent.$": "$.content"

}

This will give us the following as input to the worker.

{

         "messageType": "text",

         "messageTitle": "Msg Title",

         "messageContent": "Hello World!"

}

ResultPath

Assume the worker returns the following output for the input in the previous example.

   "HELLO WORLD!"

We can add ResultPath to add the output to the input.

   "ResultPath": "$.taskOutput"

This will include the result of the worker to the input.

{

        "messageType": "text",

        "messageTitle": "Msg Title",

        "messageContent": "Hello World!"

        "taskOutput": "HELLO WORLD!"

}

OutputPath

OutputPath field filters data to be sent to the next state. In this case, "OutputPath":"$.messageContent" will send "Hello World!" as input to the next task.

Read the Input and Output Processing documentation for more details on this.

Logging, debugging, and monitoring

We can run tests, debug, and monitor Step Functions executions on the Step Functions console. If you are using Lambda functions to run tasks, logs will be delivered to the Lambda’s CloudWatch log group as usual.

Listing of Step Functions executions in the console
All StepFunction executions are listed in the console with status, and we can dive into an execution to check the details about the underlying states

Execution event history
Each step of a particular execution can be viewed for debugging. The CloudWatch logs link redirects to the log stream of that AWS service.

It also has a CloudWatch metrics integration to monitor failures in production, which are available under CloudWatch > Metrics > States.

Conclusion

Step Functions is an easy-to-use service for orchestrating backend workflows. Very complex flows can be designed easily with the States Language. It maintains the state of all the tasks and orchestrates them to run when needed, and scale automatically. Step Functions are easily pluggable with existing architecture. Since an SFN can stay alive for one year, it can also be used for long running workflows using activity worker.

Step Functions is probably the coolest console among all the AWS services.

A version of this blog was published earlier on Medium.

ZEMOSO ENGINEERING STUDIO

Orchestrating backend services for product development with AWS Step Functions

April 1, 2019
6 min read

The challenge

In many use cases, there are processes that need to execute multiple tasks. We build microservices or server-less functions like Amazon Web Services (AWS) Lambda functions to carry out these tasks. Most of these services are stateless functions; hence there's a need for queues or databases to maintain the state of individual tasks and the process. Writing code that orchestrates these tasks can be painful and hard to debug and maintain.

AWS Step Functions

AWS Step Functions is a web service that enables you to coordinate the components of distributed applications and microservices using visual workflows. Step Functions have been around since 2016, it was announced at re:Invent 2016.

Step Functions

1. Enables us to define visual workflows for processes by writing minimal or no code

2. Scales out automatically

3. Deals with failures and timeouts of tasks

4. Maintains an auditable log of all state changes

Step Functions are based on the concept of tasks, states, and state machines. The tasks do all the work. A task can be a Lambda Function, an activity which can be fulfilled on other AWS-hosted resources, or even an activity defined on our own machines.

The state machine

A state machine is the workflow and the biggest component of Step Functions. State machines are defined using JavaScript Object Notation (JSON) text written in the syntax of the AWS States Language.

States

The states are blocks that represent some action. Here's a list of states available in the States Language:

1. Pass: does nothing. Mainly used for debugging or as a placeholder state

2. Task: execute some code or run an activity

3. Choice: adds branching logic to the state machine (if/else?)

4. Wait: waits for a specified time before moving on to the next state

5. Succeed: terminates the execution with a success

6. Fail: terminates the execution with a failure

7. Parallel: adds parallel branching to the state function

You can use the AWS Step Functions console to start making simple flows (state machines) using Step Functions. There are pre-configured examples available. We’ll go through the example — which extracts Exchangeable Image File (EXIF) metadata from an image, crops the object from the image, and saves it to S3. We'll assume that Lambda functions are deployed. We'll use Step Functions to orchestrate the Lambda functions to process the images.

Image metadata extraction example with Step Functions

Goal: Extract metadata from images, resize images to medium size, and a thumbnail. Upload the details to the database.

We have two Lambda functions already set up

1. get-exif-lambda: downloads image from S3, extracts EXIF data

   Input: S3 image URI

   Output: image metadata

2. process-image-lambda: downloads image from S3, resizes it to the desired size

    Input: S3 image URI, desired size of the output image

    Output: S3 URIs of the processed images

Creating the state machine

A state machine can be created by the AWS Command Line Interface (CLI), or the AWS Step Functions console. It’s easier to create it from the console because you can see a visual representation of it. It's also easy to set up the required Identity and Access Management (IAM) role to give the state machine permission to the required AWS resources from the console.

Open the Step Functions console and create a new state machine with the following JSON.

{

 "StartAt":  "GetExif",

 "States":  {

   "GetExif":  {

     "Type":  "Task",

     "Resource":  "<get-exif-lambda arn>",

     "Next":  "ResizeImage"

   },

   "ResizeImage":  {

     "Type":  "Parallel",

     "Next":  "WriteToDb",

     "ResultPath":  "$.resizedLinks",

     "Branches":  [

       {

        "StartAt":  "MediumSize",

        "States":  {

          "MediumSize":  {

           "Type":  "Task",

           "Resource":  "<resize-image-lambda arn>",

           "Parameters":  {

             "thumbnail":  false,

             "source.$":  "$.source",

             "maxHeight":  600,

             "maxWidth":  600

           },

            "End":  true

          }

        }

      },

      {

        "StartAt":  "Thumbnail",

        "States":  {

          "Thumbnail":  {

            "Type":  "Task",

            "Resource":  "<resize-image-lambda arn>",

            "Parameters":  {

               "thumbnail":  true,

               "source.$":  "$.source",

               "maxHeight":  128,

               "maxWidth":  128

             },

            "End":  true

          }

        }

      }

     ]

   },

   "WriteToDb": {

     "Type":  "Task",

     "Resource":  "arn:aws:states:::dynamodb:putItem",

     "Parameters":  {

       "TableName":  "image-details",

       "Item":  {

         "key":  {

           "S.$":  "$.source.key"

         },

         "exif":  {

           "S.$":  "$.exif"

         },

         "mediumURL":  {

           "S.$":  "$.resizedLinks[0]"

         },

         "thumbnailURL":  {

           "S.$":  "$.resizedLinks[1]"

         }

       }

     },

     "End":  true

   }

 }

}

In the next step, provide an IAM role for the state machine. Select “Create an IAM role,” give a valid role name, and proceed. This will create a new IAM role, which provides it access only to the resources that are used by the state machine. The state machine created will look like this

Visual representation of the state machine
Visual representation of the state machine

The flow starts at the GetExif state, which executes a Lambda function to retrieve metadata from the image.

Then it executes two image conversion tasks:MediumSize, and Thumbnail, parallelly.

This data is passed to WriteToDb, which writes the output to DynamoDB.

This is a simple example. It's very easy to design more complex workflows with other states available in the States Language.

We can also add error handling and retry functionality to the states.

Manipulating input and output of a state

The States Language allows us to manipulate and control the JSON data that flows between states.

In Amazon State Language, these fields manipulate and control the flow of data between states:

1. InputPath

2. OutputPath

3. ResultPath

4. Parameters

This diagram shows the sequence in which these fields are applied to the JSON data.

Input and output state as applied to JSON

InputPath and parameters

Let's assume the state receives the following input.

{

   "message": {

          "title": "Msg Title",

          "content": "Hello World!"

   },

   "timestamp": 12312432

}

We can add InputPath and Parameters to the state.

"InputPath": "$.message",

"Parameters": {

        "messageType": "text",

        "messageTitle.$": "$.title",

        "messageContent.$": "$.content"

}

This will give us the following as input to the worker.

{

         "messageType": "text",

         "messageTitle": "Msg Title",

         "messageContent": "Hello World!"

}

ResultPath

Assume the worker returns the following output for the input in the previous example.

   "HELLO WORLD!"

We can add ResultPath to add the output to the input.

   "ResultPath": "$.taskOutput"

This will include the result of the worker to the input.

{

        "messageType": "text",

        "messageTitle": "Msg Title",

        "messageContent": "Hello World!"

        "taskOutput": "HELLO WORLD!"

}

OutputPath

OutputPath field filters data to be sent to the next state. In this case, "OutputPath":"$.messageContent" will send "Hello World!" as input to the next task.

Read the Input and Output Processing documentation for more details on this.

Logging, debugging, and monitoring

We can run tests, debug, and monitor Step Functions executions on the Step Functions console. If you are using Lambda functions to run tasks, logs will be delivered to the Lambda’s CloudWatch log group as usual.

Listing of Step Functions executions in the console
All StepFunction executions are listed in the console with status, and we can dive into an execution to check the details about the underlying states

Execution event history
Each step of a particular execution can be viewed for debugging. The CloudWatch logs link redirects to the log stream of that AWS service.

It also has a CloudWatch metrics integration to monitor failures in production, which are available under CloudWatch > Metrics > States.

Conclusion

Step Functions is an easy-to-use service for orchestrating backend workflows. Very complex flows can be designed easily with the States Language. It maintains the state of all the tasks and orchestrates them to run when needed, and scale automatically. Step Functions are easily pluggable with existing architecture. Since an SFN can stay alive for one year, it can also be used for long running workflows using activity worker.

Step Functions is probably the coolest console among all the AWS services.

A version of this blog was published earlier on Medium.

Recent Publications
How to remix Amazon’s Working Backwards with Google’s Venture’s User Journey: The Dr. Strange Way
How to remix Amazon’s Working Backwards with Google’s Venture’s User Journey: The Dr. Strange Way
June 14, 2022
How we built a big data platform for a futuristic AgriTech product
How we built a big data platform for a futuristic AgriTech product
June 3, 2022
Zemoso Labs starts operations in Waterloo, Canada
Zemoso Labs starts operations in Waterloo, Canada
May 25, 2022
Deconstructing Elon Musk’s dog ate my homework answer for Twitter: More validation will be asked of startups
Deconstructing Elon Musk’s dog ate my homework answer for Twitter: More validation will be asked of startups
May 20, 2022
Real Talk: Lessons learned and evolved from 3M and Post-it®’s adoption of Crazy 8 methodology
Real Talk: Lessons learned and evolved from 3M and Post-it®’s adoption of Crazy 8 methodology
April 10, 2022
Why Realm trumps SQLite as database of choice for complex mobile apps — Part 1
Why Realm trumps SQLite as database of choice for complex mobile apps — Part 1
February 7, 2022
Understanding dynamic multi-column search with JPA Criteria for product development
Understanding dynamic multi-column search with JPA Criteria for product development
January 2, 2022
Product engineering zeitgeist: Implement OOPs in React JS using atomic design
Product engineering zeitgeist: Implement OOPs in React JS using atomic design
July 8, 2021
Honorable mention at O’Reilly’s Architectural Katas event
Honorable mention at O’Reilly’s Architectural Katas event
May 17, 2021
Product dev with testable spring boot applications, from day one
Product dev with testable spring boot applications, from day one
May 4, 2021
When not to @Autowire in Spring or Spring Boot applications
When not to @Autowire in Spring or Spring Boot applications
May 1, 2021
Efficiently handle data and integrations in Spring Boot
Efficiently handle data and integrations in Spring Boot
January 24, 2021
Unlock the power of atomic design in React and React Native using a theming library — Part 1
Unlock the power of atomic design in React and React Native using a theming library — Part 1
November 2, 2020
Our favorite CI/CD DevOps Practice: Simplify with GitHub Actions
Our favorite CI/CD DevOps Practice: Simplify with GitHub Actions
October 25, 2020
Kubernetes — What is it, what problems does it solve, and how does it compare with alternatives?
Kubernetes — What is it, what problems does it solve, and how does it compare with alternatives?
June 21, 2019
GraphQL — Why is it essential for agile product development?
GraphQL — Why is it essential for agile product development?
April 30, 2019
How To Decide When to Use Amazon Working Backwards, Business Model Canvas and Lean Canvas
How To Decide When to Use Amazon Working Backwards, Business Model Canvas and Lean Canvas
November 30, 2018
How to validate your Innovation: Mastering Experiment Design
How to validate your Innovation: Mastering Experiment Design
November 22, 2018
Working Backwards: Amazon's Culture of Innovation: My notes
Working Backwards: Amazon's Culture of Innovation: My notes
November 19, 2018
Product developer POV: Caveats when building with Spark
Product developer POV: Caveats when building with Spark
November 5, 2018
ZEMOSO ENGINEERING STUDIO
April 1, 2019
6 min read

Orchestrating backend services for product development with AWS Step Functions

The challenge

In many use cases, there are processes that need to execute multiple tasks. We build microservices or server-less functions like Amazon Web Services (AWS) Lambda functions to carry out these tasks. Most of these services are stateless functions; hence there's a need for queues or databases to maintain the state of individual tasks and the process. Writing code that orchestrates these tasks can be painful and hard to debug and maintain.

AWS Step Functions

AWS Step Functions is a web service that enables you to coordinate the components of distributed applications and microservices using visual workflows. Step Functions have been around since 2016, it was announced at re:Invent 2016.

Step Functions

1. Enables us to define visual workflows for processes by writing minimal or no code

2. Scales out automatically

3. Deals with failures and timeouts of tasks

4. Maintains an auditable log of all state changes

Step Functions are based on the concept of tasks, states, and state machines. The tasks do all the work. A task can be a Lambda Function, an activity which can be fulfilled on other AWS-hosted resources, or even an activity defined on our own machines.

The state machine

A state machine is the workflow and the biggest component of Step Functions. State machines are defined using JavaScript Object Notation (JSON) text written in the syntax of the AWS States Language.

States

The states are blocks that represent some action. Here's a list of states available in the States Language:

1. Pass: does nothing. Mainly used for debugging or as a placeholder state

2. Task: execute some code or run an activity

3. Choice: adds branching logic to the state machine (if/else?)

4. Wait: waits for a specified time before moving on to the next state

5. Succeed: terminates the execution with a success

6. Fail: terminates the execution with a failure

7. Parallel: adds parallel branching to the state function

You can use the AWS Step Functions console to start making simple flows (state machines) using Step Functions. There are pre-configured examples available. We’ll go through the example — which extracts Exchangeable Image File (EXIF) metadata from an image, crops the object from the image, and saves it to S3. We'll assume that Lambda functions are deployed. We'll use Step Functions to orchestrate the Lambda functions to process the images.

Image metadata extraction example with Step Functions

Goal: Extract metadata from images, resize images to medium size, and a thumbnail. Upload the details to the database.

We have two Lambda functions already set up

1. get-exif-lambda: downloads image from S3, extracts EXIF data

   Input: S3 image URI

   Output: image metadata

2. process-image-lambda: downloads image from S3, resizes it to the desired size

    Input: S3 image URI, desired size of the output image

    Output: S3 URIs of the processed images

Creating the state machine

A state machine can be created by the AWS Command Line Interface (CLI), or the AWS Step Functions console. It’s easier to create it from the console because you can see a visual representation of it. It's also easy to set up the required Identity and Access Management (IAM) role to give the state machine permission to the required AWS resources from the console.

Open the Step Functions console and create a new state machine with the following JSON.

{

 "StartAt":  "GetExif",

 "States":  {

   "GetExif":  {

     "Type":  "Task",

     "Resource":  "<get-exif-lambda arn>",

     "Next":  "ResizeImage"

   },

   "ResizeImage":  {

     "Type":  "Parallel",

     "Next":  "WriteToDb",

     "ResultPath":  "$.resizedLinks",

     "Branches":  [

       {

        "StartAt":  "MediumSize",

        "States":  {

          "MediumSize":  {

           "Type":  "Task",

           "Resource":  "<resize-image-lambda arn>",

           "Parameters":  {

             "thumbnail":  false,

             "source.$":  "$.source",

             "maxHeight":  600,

             "maxWidth":  600

           },

            "End":  true

          }

        }

      },

      {

        "StartAt":  "Thumbnail",

        "States":  {

          "Thumbnail":  {

            "Type":  "Task",

            "Resource":  "<resize-image-lambda arn>",

            "Parameters":  {

               "thumbnail":  true,

               "source.$":  "$.source",

               "maxHeight":  128,

               "maxWidth":  128

             },

            "End":  true

          }

        }

      }

     ]

   },

   "WriteToDb": {

     "Type":  "Task",

     "Resource":  "arn:aws:states:::dynamodb:putItem",

     "Parameters":  {

       "TableName":  "image-details",

       "Item":  {

         "key":  {

           "S.$":  "$.source.key"

         },

         "exif":  {

           "S.$":  "$.exif"

         },

         "mediumURL":  {

           "S.$":  "$.resizedLinks[0]"

         },

         "thumbnailURL":  {

           "S.$":  "$.resizedLinks[1]"

         }

       }

     },

     "End":  true

   }

 }

}

In the next step, provide an IAM role for the state machine. Select “Create an IAM role,” give a valid role name, and proceed. This will create a new IAM role, which provides it access only to the resources that are used by the state machine. The state machine created will look like this

Visual representation of the state machine
Visual representation of the state machine

The flow starts at the GetExif state, which executes a Lambda function to retrieve metadata from the image.

Then it executes two image conversion tasks:MediumSize, and Thumbnail, parallelly.

This data is passed to WriteToDb, which writes the output to DynamoDB.

This is a simple example. It's very easy to design more complex workflows with other states available in the States Language.

We can also add error handling and retry functionality to the states.

Manipulating input and output of a state

The States Language allows us to manipulate and control the JSON data that flows between states.

In Amazon State Language, these fields manipulate and control the flow of data between states:

1. InputPath

2. OutputPath

3. ResultPath

4. Parameters

This diagram shows the sequence in which these fields are applied to the JSON data.

Input and output state as applied to JSON

InputPath and parameters

Let's assume the state receives the following input.

{

   "message": {

          "title": "Msg Title",

          "content": "Hello World!"

   },

   "timestamp": 12312432

}

We can add InputPath and Parameters to the state.

"InputPath": "$.message",

"Parameters": {

        "messageType": "text",

        "messageTitle.$": "$.title",

        "messageContent.$": "$.content"

}

This will give us the following as input to the worker.

{

         "messageType": "text",

         "messageTitle": "Msg Title",

         "messageContent": "Hello World!"

}

ResultPath

Assume the worker returns the following output for the input in the previous example.

   "HELLO WORLD!"

We can add ResultPath to add the output to the input.

   "ResultPath": "$.taskOutput"

This will include the result of the worker to the input.

{

        "messageType": "text",

        "messageTitle": "Msg Title",

        "messageContent": "Hello World!"

        "taskOutput": "HELLO WORLD!"

}

OutputPath

OutputPath field filters data to be sent to the next state. In this case, "OutputPath":"$.messageContent" will send "Hello World!" as input to the next task.

Read the Input and Output Processing documentation for more details on this.

Logging, debugging, and monitoring

We can run tests, debug, and monitor Step Functions executions on the Step Functions console. If you are using Lambda functions to run tasks, logs will be delivered to the Lambda’s CloudWatch log group as usual.

Listing of Step Functions executions in the console
All StepFunction executions are listed in the console with status, and we can dive into an execution to check the details about the underlying states

Execution event history
Each step of a particular execution can be viewed for debugging. The CloudWatch logs link redirects to the log stream of that AWS service.

It also has a CloudWatch metrics integration to monitor failures in production, which are available under CloudWatch > Metrics > States.

Conclusion

Step Functions is an easy-to-use service for orchestrating backend workflows. Very complex flows can be designed easily with the States Language. It maintains the state of all the tasks and orchestrates them to run when needed, and scale automatically. Step Functions are easily pluggable with existing architecture. Since an SFN can stay alive for one year, it can also be used for long running workflows using activity worker.

Step Functions is probably the coolest console among all the AWS services.

A version of this blog was published earlier on Medium.

Recent Publications

ZEMOSO ENGINEERING STUDIO

How we built a big data platform for a futuristic AgriTech product

June 3, 2022
8 min read
ZEMOSO NEWS

Zemoso Labs starts operations in Waterloo, Canada

May 25, 2022
5 min read
ZEMOSO ENGINEERING STUDIO

Honorable mention at O’Reilly’s Architectural Katas event

May 17, 2021
5 min read
ZEMOSO ENGINEERING STUDIO

Product dev with testable spring boot applications, from day one

May 4, 2021
5 min read
ZEMOSO ENGINEERING STUDIO

When not to @Autowire in Spring or Spring Boot applications

May 1, 2021
5 min read
ZEMOSO ENGINEERING STUDIO

Efficiently handle data and integrations in Spring Boot

January 24, 2021
5 min read
ZEMOSO ENGINEERING STUDIO

Our favorite CI/CD DevOps Practice: Simplify with GitHub Actions

October 25, 2020
5 min read
ZEMOSO ENGINEERING STUDIO

GraphQL — Why is it essential for agile product development?

April 30, 2019
12 min read
ZEMOSO PRODUCT STUDIO

How to validate your Innovation: Mastering Experiment Design

November 22, 2018
8 min read
ZEMOSO PRODUCT STUDIO

Working Backwards: Amazon's Culture of Innovation: My notes

November 19, 2018
8 min read
ZEMOSO ENGINEERING STUDIO

Product developer POV: Caveats when building with Spark

November 5, 2018
2 min read

Want more best practices?

Access thought-leadership and best practice content across
the product development lifecycle

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

© 2021 Zemoso Technologies
Privacy Policy

Terms of Use
LinkedIn Page - Zemoso TechnologiesFacebook Page - Zemoso TechnologiesTwitter Account - Zemoso Technologies

© 2021 Zemoso Technologies
Privacy Policy

LinkedIn Page - Zemoso TechnologiesFacebook Page - Zemoso TechnologiesTwitter Account - Zemoso Technologies
June 7, 2022