Event-driven architectures change the way information is exchanged between the services that produce them and those that consume them. Systems that sit idle waiting for data are becoming obsolete as software architectures continue to transition from API-driven to event-driven and real-time event streaming becomes more popular.
As businesses, projects, and infrastructure grow, so does the need for more real-time, multicloud data flows. Organizations are demanding easier workflows for DevOps engineers, real-time notifications after testing is complete, and tracking of security logs between different SaaS (Software as a Service) applications.
All are necessary in today’s growing cloud-native ecosystem, and without open-source tools or the constraint of vendor lock-in, they’re hard to find.
TriggerMesh is a serverless event router with a simple main goal: to process events in real time and route them to the right consumers. It’s a multi-purpose tool with an ever-growing list of use cases.
TriggerMesh’s open-source integration platform provides users with a configuration-driven solution that will feel familiar to users of Infrastructure as Code (IaC) offerings such as Ansible, Chef, and Terraform. It is built on top of Kubernetes and includes a declarative API to quickly and programmatically connect data and services from multiple sources to event-driven applications.
Message consumption sources include Amazon Web Services (AWS) SQS, Google Pub/Sub, Azure Event Hubs, and Kafka. TriggerMesh filters and transforms messages, combining processing functions with serverless functions and connecting them to messaging sinks that link Elasticsearch, AWS Simple Storage Service (S3), and Apache Kafka.
TriggerMesh also connects to legacy service buses to create workflows between applications or data sinks to other systems like Azure Data Lakes or Snowflake.
“Often people use it to do things like trigger a function or a service or … microservices,” Jonathan Michaux, product manager at TriggerMesh, told The New Stack.
TriggerMesh can also play a role in a larger streaming ecosystem, he added, such as providing “a function [for] Process events and forward them to another system like Kafka for further processing later.”
For a quick comparison, consider Amazon’s EventBridge. Event Router, introduced in 2019, allows developers to write rules that take action as soon as an event occurs in Amazon’s S3 bucket.
TriggerMesh is the open-source alternative, offering similar functionality with no vendor lock-in requirements. TriggerMesh recently announced its newest open source offering, Shaker, which continues the push toward wider adoption of TriggerMesh. Shaker eliminates the Kubernetes requirement and allows TriggerMesh to run on Docker.
Open source not only offers freedom from vendor lock-in. This means TriggerMesh is fully cloud-agnostic, allowing users to produce and consume between multiple clouds and on-premises data centers.
“Think about what that is and how powerful it could be if we could just take any application and connect them through events in such a simple way,” Michaux said. “You can deploy it anywhere on any Kubernetes cluster or on any machine with Docker on OpenShift, on Amazon, on Azure or on Google.”
Michael Edenzon, co-founder of Fianu Labs, a software governance firm, told The New Stack that TriggerMesh’s cloud-agnosticism “absolutely puts the ball forward when it comes to making it easier for developers to build event-driven applications “.
Case Studies and Use Cases
How do real organizations use TriggerMesh in their event-driven architectures? Here are two examples.
ManoMano: Save costs when running microservices
ManoMano, a Europe-based home improvement marketplace, is an e-commerce site that offers web and mobile experiences. The many business needs require hundreds of microservices. With such a large code base, it relies on various performance tests.
By adding TriggerMesh to its stack, ManoMano’s Site Reliability Engineering (SRE) team now had the right software tool to provide a serverless eventing experience that allows test developers to run code when specific AWS events occur by the test developers subscribe to specific events.
More recently, ManoMano dived into its longstanding microservices. Do they have to be durable and always available? Does their use justify the continuous consumption of computing resources and the associated costs?
The answer was no, sparking the company’s desire to offer services only on an as-needed basis. The research found that many of these services only handle a few requests per day and spend most of their time idle.
ManoMano replaced long-running idle tasks with containers scheduled on-demand to run EKS in response to AWS S3 events. This was made possible thanks to the new event-driven architecture, which collects events from AWS services, injects them into a central broker, and allows developers to subscribe to specific events for consumption. The platform abstracts security and infrastructure concerns.
Fianu Labs: Automated on-prem software governance
Fianu Labs’ fully automated, event-driven software governance tool provides instrumentation throughout the CI/CD process, capturing events throughout the software development lifecycle and comparing them to pre-defined policies and automated compliance documentation.
TriggerMesh, built on top of Google Kubernetes Engine with Knative, is a key part of Fianu’s architecture and permeates the fabric of its workflow. Although Fianu does not operate at scale, it requires redundancy and reliability in supporting 200,000 code repositories, which equates to approximately 500 to 1,000 events per minute.
“The TriggerMesh tooling and the way we’re using it allows us to maintain a robust, event-driven system and keep it pretty neat,” Fianu Labs’ Edenzon told The New Stack.
An example of the typical TriggerMesh workflow in Fianu starts with recording events. When events are ingested, Fianu uses TriggerMesh functions (Fianu uses embedded Python, but TriggerMesh also has Node.js and Ruby options) to perform transformations and computations on the payloads.
Edenzon discussed the example of changing strings to floats to ensure incoming data conforms to policy. While there may be an incoming data source, a number of different outputs could be required, and TriggerMesh functions abstract the need to create core tools.
“Without building a serverless function, we can embed the TriggerMesh functions to run small bits of business logic on the fly,” he said.
TriggerMesh Targets was another aspect of TriggerMesh functionality that Endezon pointed out. Targets allow Fianu customers to build customizations on top of the Fianu software as it helps with complicated event targets by abstracting complexity for events going to a Kafka stream or to a Slack.
TriggerMesh helps not only Fianu’s developers, but also the company’s customers. Fianu offers many out-of-the-box features to its end users, and while the company does not provide dedicated TriggerMesh tools, due to TriggerMesh’s high level of abstraction, customers can write custom plugins as Knative features in any language and snap them into the running instance, since TriggerMesh treats them as just another event processor.
“If we, as developers, only ever had to focus on the business functions, we would be extremely productive,” said Edenzon. “So that’s one of the things we like [TriggerMesh]. It makes it very easy for us to isolate and test business functions.”
Embedded in SaaS apps
TriggerMesh enables SaaS applications to ingest events from multiple cloud providers, transform the events to match specific schemas, and then send those events to any destination.
“TriggerMesh makes it very easy to add new event sources to an application,” Michaux said. He added that “B2B SaaS providers can expand their app’s ecosystem while development teams focus on the core product.”
In cybersecurity, this means transforming various cloud security events into standardized schemas, such as the recently announced Open Cybersecurity Schema Framework, before being sent to the customer’s preferred security solution for analysis and threat detection.
The value to the security community is incredibly high. Because security threats occur in real-time, the value lies not only in collecting data or integrating applications, but also in processing events as they occur.
“A typical example would be someone making a failed attempt to log into a system, say, a cloud provider like Oracle or AWS,” Michaux said. “This is an event that a threat detection system would be interested in analyzing. TriggerMesh can help capture the event and deliver it to the security system in real-time, in the right format, and with minimal effort.”
Diploma
TriggerMesh is an example of a use case that fulfills the demand and allows the demand to be accelerated to interact with the technology. It’s open source with the only requirement now being Docker, with the newly released Shaker making acceptance even more available.
“If you take full advantage of TriggerMesh with the right application design, you can isolate business functionality without worrying about the mechanics of how data goes where,” Edenzon said. “So you can just focus on the business functionality.”
