Running Datadog as a SideCar on AWS ECS Fargate

By Surajtikoo．Feb 21, 2023

The Datadog agent can be deployed in various ways, including as a standalone application, as a Docker container, or as part of an orchestration system like Kubernetes.

For our use case, we were running our microservice application on ECS Fargate, initially, we were thinking of deploying the Datadog agent altogether with a single package.

The below picture depicts how the Datadog container bundled with the Employee Application

With this approach, there are a couple of challenges

In order to overcome these challenges the best approach is to follow the sidecar pattern

SideCar Pattern:

The sidecar pattern is a software design pattern in which a container (known as a “sidecar”) is attached to a primary container in order to enhance or augment its functionality. The sidecar container runs in the same pod as the primary container and provides support for the primary container by adding features such as logging, monitoring, caching, or networking.

The sidecar pattern offers several advantages, including:

1. Modularity: The sidecar container can be designed to provide a specific set of functionalities that are independent of the main application, making it easier to add, remove, or modify functionality without affecting the main application.
2. Separation of concerns: The main application can focus on its core functionality, while the sidecar container can handle additional concerns such as monitoring, logging, or security.
3. Scalability: Because the sidecar container is a separate process, it can be scaled independently from the main application. This can help to improve the overall performance and reliability of the application.
4. Fault tolerance: If the sidecar container fails, the main application can continue to function without being affected, and the failed container can be replaced or restarted without impacting the main application.
5. Consistency: By using a standard sidecar container for multiple applications, you can ensure that all the applications have a consistent set of auxiliary services, reducing the risk of compatibility issues or configuration drift.
6. Extensibility: New functionality can be added to the sidecar container without requiring changes to the main application, making it easier to introduce new features or technologies.

To set up the Datadog agent as a sidecar in an ECS Fargate task definition, you can use the following Terraform code:

In the above example, the aws_ecs_task_definition resource creates an ECS task definition that includes two containers: the myapp-container container, which runs your application it can be any container like redis, Jenkins, nginx, etc and the datadog-agent container, which runs the Datadog agent as a sidecar.
The datadog-agent container is marked as non-essential, meaning that it will not cause the task to fail if it fails to start. It is also configured with environment variables for the Datadog API key and APM, and with port mappings for the Datadog agent to receive data from your application.
Note that you will also need to configure your ECS service to use this task definition, and to ensure that the containers can communicate with each other. You can do this using Terraform by creating an aws_ecs_service resource and an aws_ecs_service_network_configuration resource, and configuring the network mode and security groups appropriately.

Application log Forwarding

To forward application logs to Datadog using the sidecar pattern in Fargate, you can use the Datadog agent container to collect logs from your application container and forward them to Datadog for analysis and visualization.

Here are the general steps for setting up log forwarding using the Datadog agent as a sidecar in Fargate:

1. In your application container, make sure your logs are written to standard output and/or standard error streams. The format of the logs should be compatible with the log driver used by Fargate.
2. Configure the Datadog agent container to collect logs from your application container. This can be done by setting the log_processing_enabled and logs_config options in the agent container definition. Here is an example:

This configuration sets the log_driver option to “awslogs” and specifies the options for writing logs to Amazon CloudWatch Logs. It also sets the DD_LOGS_ENABLED environment variable to enable log collection in the Datadog agent and sets the DD_LOGS_CONFIG environment variable to configure which logs the agent should collect. In this example, the agent is configured to collect logs from the source named myapp and with the service name myapp It is also configured to use JSON format for messages and process multiline logs with a pattern that matches dates.

1. In your Datadog account, set up a log pipeline to process and analyze the logs collected by the Datadog agent. You can do this by going to the Logs section of the Datadog UI and creating a new pipeline. You can define rules for parsing and filtering your logs, as well as configure alerting visualization, and integration with other Datadog features.

By setting up log forwarding in this way, you can use the Datadog agent as a sidecar in Fargate to collect logs from your application container and forward them to Datadog for analysis and visualization.

---

Conclusion

Overall, the sidecar pattern can help to improve the flexibility, scalability, and fault tolerance of distributed applications, while also simplifying the development and deployment process

---

The original article published on Medium.