The application worked.

That was the exciting part.

The server was running. The application was accessible from the internet. Requests were reaching the backend, the database was responding, and everything looked fine.

For a while, that felt like success.

Then I asked myself a question that changed the way I started looking at cloud engineering:

What happens when 10,000 people try to use this application at the same time?

The answer is not simply, “Get a bigger server.”

That might work for a while, but it does not solve every problem.

What happens when the server crashes?

What happens when traffic suddenly increases?

What happens when the database becomes the bottleneck? 

What happens when users are accessing the application from different parts of the world?

What happens when we deploy a new version and something breaks?

And perhaps most importantly, how do we know that something is going wrong before our users start complaining?

These questions took me beyond the simple idea of putting an application on a cloud server.

They introduced me to something much more important.

Architecture.

Building an application that works is one thing.

Building an application that can handle growth, survive failures, maintain reasonable performance, and be monitored and updated reliably is a completely different challenge.

That is where Cloud Engineering and DevOps start becoming much more interesting.

Before We Scale, Let Us Start with Something Simple

Imagine our application has three basic parts.

The frontend is what the user interacts with.

The backend contains the application logic.

The database stores the information the application needs.

At the beginning, everything could technically run on one server.

It might look something like this:  User → Server → Database

For a small project, this may be perfectly reasonable.

There is nothing wrong with starting small.

In fact, starting with a simple architecture can be a good decision when the application is still small and the requirements are straightforward.

The problem begins when we assume that the same architecture will continue to work regardless of how much the application grows.

A server has limits.

It has a certain amount of CPU.

It has a certain amount of memory.

It has a certain network capacity.

It can only process so many requests at a time.

Eventually, we reach a point where the server starts struggling.

The application becomes slower.

Requests take longer to complete.

Users begin experiencing errors.

And if the server completely fails, the entire application could become unavailable.

That creates another problem.

The server has become a single point of failure.

So instead of asking how to make one server do everything, we can start asking a better question:

How can we design the system so that the workload can be distributed?

That question leads us to our next component.

What Happens When One Server Is Not Enough? 

There are two common ways to approach this problem.

The first is vertical scaling.

We make the existing server bigger.

We give it more CPU, more memory, more storage, or other resources.

This can be useful, especially when the application is still relatively small.

But there is a limit to how far we can go.

A bigger server can also become increasingly expensive.

Eventually, we may be paying a lot of money for one machine that is still a single point of failure.

The second approach is horizontal scaling.

Instead of making one server bigger, we add more servers.

For example, we could move from one application server to three.

Now we have:

User → Server 1
User → Server 2
User → Server 3

 But this creates another question.

How does the user know which server to connect to?

We do not want users manually choosing servers.

We need something in front of those servers that can receive incoming requests and distribute them appropriately.

This is where a load balancer becomes useful.

 The Load Balancer Becomes the Traffic Controller

Think about a busy restaurant.

If there is only one waiter and fifty customers arrive at almost the same time, that waiter is going to have a difficult day.

Now imagine there are five waiters.

The customers can be distributed among them.

The same basic idea applies to a load balancer.

Instead of users connecting directly to one application server, they connect to the load balancer.

The load balancer receives the incoming requests and distributes them across available servers.

Our architecture now looks more like this:

User → Load Balancer → Server 1
→ Server 2
→ Server 3

 This immediately gives us an advantage.

The workload can be distributed.

If Server 1 is busy, other servers can handle requests.

If Server 1 becomes unhealthy, the load balancer can stop sending traffic to it while the other servers continue serving users.

In AWS, Elastic Load Balancing provides this type of traffic distribution.

But we still have a problem.

 What happens when our three servers are no longer enough?

What If Traffic Keeps Increasing?

Suppose our application starts with three servers.

Everything is working well.

Then our application becomes popular.

Traffic increases.

Our servers are now operating at very high capacity.

We could manually create more servers.

But that does not sound particularly efficient.

Imagine having someone monitoring server usage throughout the night and manually launching new servers whenever traffic increases.

One of the reasons cloud computing is useful is that we can automate this kind of process.

 This is where Auto Scaling becomes important.

With an Auto Scaling setup, we can define rules that determine when additional compute resources should be added or removed.

For example, if the workload remains high for a certain period, additional servers can be launched.

When demand drops, unnecessary servers can be removed.

The exact configuration depends on the application and its requirements, but the underlying idea is simple.

Resources should be able to respond to demand.

During periods of high traffic, we may need more capacity.

During periods of low traffic, we may need less.

This allows infrastructure to become more flexible.

Instead of building everything around the highest amount of traffic we might ever receive, we can design an environment that can adjust as demand changes.

But there is still another major part of the system we need to think about.

 The database.

The Database Can Become the Bottleneck

Our application servers may be doing well.

We have three servers.

We have a load balancer.

We have Auto Scaling.

Everything looks great.

Then we discover that the application is still slow.

Why?

The database.

Every application request that needs data has to communicate with the database.

Users are logging in.

Orders are being created.

Products are being retrieved.

Messages are being stored.

Transactions are being processed.

If all those requests eventually reach one database that cannot handle the workload, the database becomes the bottleneck.

Adding more application servers will not automatically solve that problem.

This is one of the things I find interesting about cloud architecture.

You cannot simply look at one component in isolation.

You have to understand how the components depend on one another.

A fast application server does not help much if it spends most of its time waiting for a slow database.

Using a Managed Database

Instead of installing and managing a database ourselves on an ordinary server, we can use a managed database service.

Amazon RDS is one example.

A managed service does not mean that we no longer have responsibilities.

We still need to think about the database engine, instance size, storage, security, backups, access permissions, performance, and other configuration decisions.

However, many of the underlying administrative responsibilities can be handled by the cloud provider.

That can allow an engineering team to spend more time focusing on the application itself.

But there is an important lesson here.

A database is not simply somewhere we put data.

It is a critical part of the application architecture.

If the database becomes unavailable, the application may become unavailable too.

That means reliability and recovery have to be considered from the beginning.

 What Happens When Users Are Far Away?

Now imagine that our application becomes popular outside our original target location.

We have users in Nigeria.

We have users in the United States.

We have users in India.

We have users in Australia.

All of them are accessing infrastructure that may be physically far away from them.

The internet is incredibly capable, but physical distance still matters when it comes to latency.

 This is where a Content Delivery Network can help.

A CDN can cache and deliver certain types of content from locations closer to users.

AWS CloudFront is an example.

Imagine our application contains images, JavaScript files, CSS files, videos, and other static resources.

Instead of making every user retrieve those files directly from our main infrastructure, a CDN can deliver cached content from locations closer to the users.

This can improve the user experience while also reducing the amount of traffic reaching the origin infrastructure.

That gives us another lesson.

Cloud architecture is also about where content is delivered from, not just where the application is running.

 Where Should User Files Go?

There is another question we need to answer.

What happens when users upload files?

Maybe they upload profile pictures.

Maybe they upload documents.

Maybe they upload videos.

Maybe the application generates reports that need to be stored.

Should we keep all these files on our application server?

It might work at first.

But it creates a problem.

Application servers should ideally be replaceable.

If one server disappears, we should not lose important user data because that data happened to be stored on that particular machine.

 This is where object storage becomes useful.

Amazon S3 is an example of an object storage service.

Instead of treating uploaded files as something that belongs permanently to a particular application server, we can store those objects separately.

The application can then access them whenever they are needed.

This separation is important.

Our application servers can come and go.

The data should not have to disappear with them.

 Designing for Failure

One of the biggest changes in my thinking about cloud infrastructure has been understanding that failure is not necessarily an unusual event.

Servers fail.

Applications crash.

Networks have problems.

Databases can become unavailable.

Configurations can be wrong.

Deployments can introduce bugs.

The question is not simply whether something will fail.

The better question is:

What happens when it does?

Suppose we have three application servers and one suddenly crashes.

If our load balancer has health checks configured correctly, it can identify that the server is unhealthy and stop sending traffic to it.

The other servers can continue handling requests.

Meanwhile, another server can be launched to replace the failed one if the environment is configured for it.

The users may never know what happened.

That is the idea behind resilient architecture.

It does not mean failures stop happening.

It means one failure does not necessarily bring down the entire system.

 Monitoring Is Not Optional

There is another part of the architecture that is easy to overlook.

How do we know what is happening inside the environment?

Imagine waking up one morning and finding messages from users saying the application is slow.

You check the application.

It is running.

You check the servers.

They are running.

But something is clearly wrong.

Without proper monitoring and logs, you may spend hours guessing.

With monitoring, you may discover that CPU usage has suddenly increased.

Maybe memory usage is unusually high.

Maybe database connections have reached their limit.

Maybe response times have increased.

Maybe one API endpoint is returning thousands of errors.

Monitoring gives us visibility.

Instead of guessing what is wrong, we can investigate what the infrastructure is actually doing.

Services such as Amazon CloudWatch can help collect metrics, logs, and other information that can be used to understand the health and performance of our environment.

This becomes even more important as the infrastructure grows.

When you have one server, you might be able to check it manually.

When you have dozens or hundreds of resources, manual inspection is no longer practical.

 So Where Does DevOps Come In?

At this point, someone might ask:

Isn't everything we have discussed just cloud computing?

Yes.

But this is where DevOps connects the different pieces.

Imagine that a developer makes a change to the application.

The code is pushed to GitHub.

A CI pipeline can automatically build the application and run tests.

If the tests pass, the application can move to the next stage.

The application can be packaged into a container.

That container can be stored in a container registry.

Infrastructure can be provisioned using Infrastructure as Code.

The application can then be deployed to the required environment.

After deployment, monitoring can help determine whether the application is behaving as expected.

If something goes wrong, engineers investigate the problem, fix it, and improve the process.

This is where DevOps becomes more than knowing how to use individual tools.

Git is useful.

Docker is useful.

Kubernetes is useful.

Terraform is useful.

AWS is useful.

But knowing how these tools fit into the bigger picture is even more important.

The goal is not to collect tools.

The goal is to build a reliable process for developing, deploying, operating, monitoring, and improving software.

The Architecture Keeps Evolving

Look at where we started.

One server.

One application.

One database.

Then we encountered problems.

We needed more capacity, so we introduced additional servers.

We needed a way to distribute traffic, so we introduced a load balancer.

We needed the infrastructure to respond to changing demand, so we introduced Auto Scaling.

We needed to think about database reliability and performance, so we considered a managed database.

We needed to serve content efficiently to users in different locations, so we introduced a CDN.

We needed somewhere reliable to store files, so we considered object storage.

We needed visibility into the environment, so we introduced monitoring.

We needed a reliable way to build and deploy changes, so DevOps practices became important.

Notice something.

We did not introduce all these components simply because they were available.

Each one addressed a problem.

That distinction matters.

Good cloud architecture is not about using as many services as possible.

It is about understanding the problem and choosing an appropriate solution.

More Services Do Not Always Mean a Better Architecture

There is a temptation when learning cloud computing to think that a more complicated architecture must be a better architecture.

I do not think that is true.

If an application has a small number of users and simple requirements, it may not need an elaborate architecture.

There may be no reason to introduce Kubernetes simply because Kubernetes is popular.

There may be no reason to build a complicated multi region architecture if the application does not require it.

There may be no reason to introduce ten different services when three can solve the problem properly.

Every additional component introduces complexity.

It needs to be configured.

It needs to be secured.

It needs to be monitored.

It may need to be maintained.

And it may increase the cost of running the application. 

The goal is not complexity.

The goal is to build an environment that is reliable, scalable, secure, maintainable, and appropriate for the application's actual requirements.

So, What Happens When 10,000 Users Show Up?

Let's return to the question that started this whole discussion.

What happens when 10,000 users show up?

The answer depends on how we designed the system.

Users send requests to the application.

A load balancer receives the traffic and distributes requests across available application servers.

If demand increases, Auto Scaling can add more compute resources based on the rules we have configured.

The application servers communicate with the database.

Files can be stored separately using object storage.

A CDN can deliver cached content closer to users.

Monitoring helps us understand what is happening across the environment.

Automation helps us deploy changes consistently.

If one application server fails, the remaining servers can continue serving users.

If traffic decreases, unnecessary compute resources can be removed.

None of this happens by magic.

It happens because the architecture was designed with these possibilities in mind.

What I Took Away From This

The biggest lesson for me is that cloud engineering is not really about memorising cloud services.

It is about understanding problems.

A load balancer exists because distributing traffic can be better than sending everything to one server.

Auto Scaling exists because demand changes.

A managed database exists because databases require reliability, performance, backups, and proper management.

A CDN exists because users are not all sitting next to our servers.

Object storage exists because application servers should not necessarily be responsible for storing everything.

Monitoring exists because we cannot manage what we cannot see.

DevOps practices exist because deploying software manually and repeatedly is not a reliable way to operate modern applications at scale.

Once I started looking at cloud architecture this way, the services became easier to understand.

Instead of asking:

“What does this AWS service do?”

I started asking:

“What problem does this solve?”

That small change in perspective makes a huge difference.

Because the goal of cloud engineering is not to build the most complicated architecture possible.

The goal is to build an architecture that makes sense for the problem.

Maybe your application has ten users today.

Maybe it will have 10,000 tomorrow.

You may not need a complicated architecture from day one.

But you should understand what happens when the simple solution eventually stops being enough.

That is the part of Cloud and DevOps Engineering I am beginning to appreciate the most.

It is not just about getting an application to work.

It is about thinking ahead.

Because when 10,000 users eventually show up, you do not want to be asking for the first time:

“What are we going to do now?”

You want your architecture to already have an answer.

That is exactly what my team and I walked into during our DMI Cohort 2 final project. And by the time it was over, we had built something I am genuinely proud of.

What We Built and Why

We took the Spring PetClinic application, an open source veterinary management system, and turned it into a fully cloud native deployment running on AWS. The goal was never just to get the app online. The goal was to build a real world DevOps pipeline from the ground up. That meant infrastructure as code, containerization, Kubernetes orchestration, automated CI/CD, GitOps delivery, and full observability.

The Spring PetClinic platform manages pet owners, veterinarians, pets, and visit records through multiple independently running microservices. We deployed the entire thing on Amazon EKS in the AWS eu north 1 region. We provisioned the infrastructure with Terraform, containerized every service with Docker, stored images in Amazon ECR, connected a managed database through Amazon RDS, and built a CI/CD pipeline using GitHub Actions.

This was not a tutorial project. It was a full production grade setup handled by eleven engineers working across every layer of the stack.

My Role: GitOps and Continuous Delivery with ArgoCD

My specific assignment was the GitOps layer using ArgoCD. This is where deployment stops being a manual process and becomes an automated, self healing workflow.

Here is what I actually did.

I installed ArgoCD inside the Amazon EKS cluster in a dedicated namespace. After installation, I configured the ArgoCD server and set up access to the user interface. The next step was connecting our private infrastructure repository to ArgoCD using a GitHub Personal Access Token. This gave ArgoCD the ability to watch the repo continuously and detect whenever Kubernetes manifests changed.

I then wrote the ArgoCD Application resource manifest using an application.yaml file. This file defined the GitHub repository source, the path to our Kubernetes manifests, the deployment namespace, and the automatic synchronization behavior. Once this was applied to the cluster, ArgoCD took over.

The technical decision I am most satisfied with was choosing automated sync over manual sync. Some teams prefer to review changes before ArgoCD applies them. In this project, I configured ours to sync automatically because we already had GitHub Actions handling the CI side. The pipeline would test, build, and push the updated image. ArgoCD would detect the manifest change and reconcile the cluster to match Git. No human needed to run kubectl apply. No room for human error in the deployment step.

When we validated this in practice, watching the ArgoCD dashboard transition an application from OutOfSync to Synced after a repository push was genuinely satisfying. That visual confirmation meant our GitOps workflow was live.

The Full Architecture

The stack our team built together looks like this.

Cloud Platform: AWS EKS for Kubernetes orchestration, Amazon ECR for container image storage, Amazon RDS for the managed relational database.

Containerization: Docker and Docker Compose for local development and image packaging.

Infrastructure as Code: Terraform for provisioning every AWS resource in a repeatable and version controlled way.

CI/CD: GitHub Actions for automated build, test, and image push workflows.

GitOps: ArgoCD for continuous delivery and Kubernetes state synchronization.

Observability: Prometheus and Grafana for metrics and dashboards, Zipkin for distributed tracing.

Security: AWS Secrets Manager integrated through the External Secrets Operator and IAM Roles for Service Accounts to keep sensitive credentials out of Kubernetes manifests entirely.

What Went Well and What I Would Do Differently

What worked: The separation of concerns across the pipeline. GitHub Actions owned the CI side. ArgoCD owned the CD side. The two systems handed off cleanly without overlapping responsibilities.

What I would do differently: I would implement the ArgoCD App of Apps pattern from the start. We deployed individual ArgoCD applications for each microservice, which worked but created overhead when changes needed to be made across the board. App of Apps would have given us a single parent application to manage the entire platform hierarchy.

I would also invest more time in ArgoCD notifications early on. Having automated alerts when a sync fails or an application drifts from desired state would have saved us debugging time during the final sprint.

This Is Part of DMI

This project was built as part of the DevOps Micro-Internship by Pravin Mishra, a structured program that gives engineers hands-on experience with real production workflows. If you are looking to build practical DevOps skills in a collaborative environment, Join the discord community, do not miss it: https://discord.pravinmishra.com/

Eleven engineers. One cluster. Zero excuses.

What part of this stack would you want to deep dive into next? Do share with me.