By Ashwin Kumar Naik．Dec 4, 2022
This blog gives an high level overview of terraform-google-cloud-workflows module. In this blog we will showcase some use cases where this module can be used. Apart from this we will show some examples of using the module.
Introduction
Customers who want to implement IT process automation, Business process automation, App integration & micro-service orchestration , Data and ML pipelines on GCP can implement it using Cloud Workflows Terraform module.This module deploys a Cloud Workflow on GCP which can be triggered by a Cloud Scheduler or based on an Event Arc Trigger. This will benefit the users by combining Google Cloud services and APIs to build reliable applications, process automation, and data and machine learning pipelines , following the Infrastructure as Code principles.
- Google Cloud Workflows is a fully-managed orchestration platform that executes services in an order that you define: a workflow. These workflows can combine services including custom services hosted on Google Cloud services like Cloud Run,Cloud Functions, Cloud Vision AI, BigQuery etc. OR any HTTP-based API.
- Cloud Scheduler is a fully managed enterprise-grade cron job scheduler. It allows you to schedule virtually any job, including batch, big data jobs, cloud infrastructure operations, and more.
- Eventarc lets you asynchronously deliver events from Google services, SaaS, and your own apps using loosely coupled services that react to state changes. Eventarc requires no infrastructure management — you can optimize productivity and costs while building a modern, event-driven solution.
Use Cases
There are several use cases where you can use the Cloud Workflow terraform module integrated with Cloud Scheduler OR Event Arc
- Service orchestration : Create solutions by combining services — Perform a sequence of operations across multiple systems, waiting for all operations to complete. Can be event-driven.
- Batch jobs: Operate on multiple items — Perform operations on a set of items or batch data. Often scheduled. For example:
- Business processes: Automate line-of-business workflows — Encode the steps in a business process, including conditions, actions, and human-in-the-loop events.
- IT process automation: Managed execution of service operations — Easily script sequences of Google Cloud service operations.
Terraform Module Details
Below are the variables necessary to run the terraform workflow modules.
The workflow_trigger variable defines the trigger for the Workflow. This will let the user define either a Cloud Scheduler or Event Arc Trigger .Below is the terraform variable schema for the same.
Defining a Cloud Scheduler Trigger
To define Cloud Scheduler as the trigger user has to create a map with cloud_scheduler with following params
- name: Cloud Scheduler Name
- cron: Cronjob format Schedule
- time_zone: Time Zone to run the cron in
- deadline: The deadline for job attempts
- service_account_email: The Service Account to run the cloud scheduler.
Below is an example
Defining an Event Arc Trigger
To define Event Arc as the trigger user has to create a map with cloud_scheduler with following params
- name: Event Arc Trigger Name
- service_account_email: The Service Account to run the Event Arc Trigger.
- matching_criteria: The filtering criteria for the event arc trigger.This is an array.Hence Multiple matching criteria can be defined.
Below is an example
Module Usage Examples
1. Scheduling a Workflow to invoke a REST API
To implement refer the example which is using the terraform module to schedule a Workflow.In this example we have a workflow which is calling a REST API based on the cron */3 * * * * .
Cloud Workflow with Scheduler
On running the above example a scheduler & workflow will be created. Workflow will have the cloud Scheduler as its trigger.
2. GCS new objects Event Arc triggering a Workflow
To implement refer the example which is using the terraform module to trigger a Workflow based on new objects created in a certain GCS Bucket via Event Arc.
Cloud Workflow with Event Arc Trigger on new GCS Object
On running the above example you will have a Event Arc Trigger & workflow created. The Event Arc trigger will have event type as google.cloud.storage.object.v1.finalized .Workflow will have the Event Arc Trigger as its trigger.
3. Pubsub Event Arc triggering a Workflow
To implement refer the example which is using the terraform module to trigger a Workflow based on new messages published to Pubsub via Event Arc.
Cloud Workflow with Event Arc Trigger on new Pubsub Message
On running the above example you will have a Event Arc Trigger & workflow created. The Event Arc trigger will have event type as google.cloud.pubsub.topic.v1.messagePublished .Workflow will have the Event Arc Trigger as its trigger.
Conclusion
In conclusion users can use terraform-google-cloud-workflows to deploy workflows while trigger being either cloud scheduler or event arc trigger. Examples shown above should help users implement their own workflows.
If there are any concerns or issues with terraform-google-cloud-workflows module kindly raise an issue here.
The original article published on Medium.
