Infrastructure Composition

We plan for a complete infrastructure starting with a three-tier architecture comprising a Frontend, an API for the Backend, and a Database.

• Front
• API
• Database

We want the infrastructure to be scalable and duplicable, with the database backed up. The entire setup will represent our Production version in a Prod cluster and in a Staging cluster for testing versions.

We also want to be able to manage multiple environments, each represented by a cluster.
All of these clusters will be managed by Kubernetes.

• Production Cluster
• Staging Cluster
• Technical Cluster with observability and control tools

Infrastructure Stack

The whole setup is managed by Kubernetes, which is a Docker container orchestrator.
But first, let's describe the tools composing each cluster.

Three-tier architecture for Prod and Staging versions

Technical Cluster

Management of the development environment and infrastructure

New technologies enable managing infrastructure with code. This is called Infrastructure as Code (IaC).

The entire setup is deployed using a version control system that archives our code and communicates with all the tools installed on our clusters, including Kubernetes, which orchestrates all deployments and processes for cloning, scalability, and cluster resources.

Configurations, environment variables, and secure access persistence are managed respectively by the tools Terraform, Ansible, and Vault.

History of Infrastructure and Source Code Deployment

The infrastructure code will be deposited in GitLab.
In GitLab, we will find the infrastructure code and tools like Terraform and Ansible.

Summary of Operations

• Setting up a complete infrastructure
• On Kubernetes
• With a scaled three-tier application
• And an observability framework
• All deployed through a pipeline

Account Creation

• Creating an account with AWS hosting (S3 - Route 53 service)
• Renting multiple dedicated servers (Cluster)
• Creating a primary domain name with a domain name provider
  (imported into an AWS Route 53 service)

Definitions and Basic Principles

Infrastructure As Code

3 Main Steps:

• Infrastructure Provisioning
• Configuration Management
• Application Orchestration

Infrastructure Provisioning (Hardware)

This involves storage, networking, and computing.
We need servers, and with cloud providers, we can rent equipment, describe its configuration, and connect them to the network.

We provision servers, network, and storage with Terraform.

Configuration Management (Software)

This involves installing the operating system, securing, and configuring it to be ready to run applications.

We configure the servers to receive applications using Ansible.

Application Orchestration

This involves deploying applications, managing their configurations, lifecycles, and publishing them on hostnames.

We deploy and orchestrate the execution, configuration, and publication of applications using the Kubernetes ecosystem - Helm.

GitOps

The Git tool allows us to manage the evolution of the entire code (versioning) collaboratively with multiple developers.

Infrastructure as Code describes the state of an infrastructure at a specific moment. Since our code evolves, we also need to evolve our infrastructure. This technique, called GitOps, allows us to synchronize the plan of our infrastructure with its actual deployed state at a provider.

This synchronization is done from Git.

The GitLab platform, where we deposit all of our code (Infra + Apps), is precisely optimized to integrate all GitOps operations.

Docker and Kubernetes (container and orchestrator)

Definition and Basic Principles

Let's take the example of a Java application.

• The developer, from their .java code, generates the compilation of their .war code.
• To execute the .war file, you need to use the Java Runtime Environment (JRE), which is the application's runtime.

In general, the application needs a runtime environment (with libraries) called the Runtime.

Docker

The Docker tool, in a way, 'ships' the application. It provides containers in which both the runtime and the application are introduced. Each container is an executable that runs the application and its runtime environment with all the libraries needed by the system, ensuring that all components are compatible with each other.

A Docker image consists of both the source code and its runtime.
A container also has the characteristic of being idempotent regardless of the environment on which it runs. This image is also immutable.

So, in Docker Hub, we have patterns of images for both application code and infrastructure as code.

Another advantage, if we consider multiple containers on a machine, communicating with each other via different network ports, we begin to see the notion of Microservices.

Microservices

Microservices are small, isolated applications that can communicate with each other like APIs, facilitating maintenance and optimization of available resources.

If we want our microservices to be distributed according to the load, we can also consider the concept of replicas (cloning) on multiple servers (clusters), so that all requests can be load-balanced across these replicas.

For the progressive deployment of a new API, we could also perform a rollback if the changes in the new version of the API do not work correctly to restore the previous version.

The Conductor

In order to manage all these microservices, we need a conductor who will control all our containers across multiple clusters. For this task, the indispensable tool is Kubernetes.

Kubernetes

In a highly distributed environment, we talk about a fleet of containers on a fleet of servers, and the whole ensemble that manages all of this is the standard Kubernetes.

Gitflow

Revision management and teamwork for updates and different versions of the application follow a process that prioritizes testing and merges following deployment pipelines managed by GitLab.

Front

Backend

Monitoring