How to Build Automated Cloud Infrastructure for Your Internal App Cloud

TL;DR

When business users start building apps with AI, the bottleneck is never the AI. It is the infrastructure behind it. This guide walks through the five layers every organization needs to provision resources automatically, safely, and without a ticket queue: a service catalog, a provisioning engine, a policy layer, secrets management, and observability. We use this architecture with clients at Forest Digital and it changes how IT relates to the rest of the business.

The real bottleneck

A VP wants to build a data pipeline. Claude writes the code in ten minutes. But the pipeline needs a database. It needs credentials. It needs to land in the right cloud region with the right encryption, tagged to the right cost center. Someone has to provision those resources.

In most organizations, that someone is IT, via a ticket, via a backlog, via a two-week wait.

This is the gap that kills the promise of AI-accelerated development. Not Claude's ability to build, but the distance between "Claude wrote it" and "it runs safely in production." Closing that gap is what building an automated provisioning layer is actually about.

The good news is that the pieces are available now. Terraform, Open Policy Agent, n8n, HashiCorp Vault, and Databricks Unity Catalog are all mature enough to wire together into a coherent system. The architecture is not exotic. What is rare is the decision to actually build it.

Layer 1: The service catalog

The service catalog defines what resources are available for self-serve provisioning. Not everything, not whatever the user can think of. A curated, pre-approved list.

Typical entries in an enterprise catalog might include a Databricks workspace for analytics workloads, a managed Postgres instance for lightweight application data, an n8n workflow environment for orchestration, an S3 or Azure Blob container for file storage, or a read-only Salesforce API connection scoped to specific objects.

Each entry comes with a configuration template that encodes your baseline requirements. Region. Encryption standard. Access tier. Network policy. Cost ceiling. The user requesting the resource does not choose these parameters. They are already decided, reviewed once by the platform team, and applied automatically on every provisioning request.

This is the first shift in how IT works: from reviewing individual requests to defining the rules that make individual review unnecessary.

Tools like Backstage or Port offer catalog interfaces out of the box. You can also build a lightweight version with a structured JSON/YAML definition file and an internal API that reads from it.

Layer 2: The provisioning engine

The provisioning engine takes a catalog entry and a set of parameters and creates the actual resource. This is where automation replaces the ticket.

At Forest Digital, we use n8n for this layer. An agent or user sends a POST request to a provisioning endpoint with a resource type and a small set of inputs: a name, a business unit identifier, an environment tag. n8n receives the request, validates the inputs, calls Terraform via a CLI step or a Terraform Cloud API call, waits for the resource to be ready, and returns a connection string or workspace URL.

The whole cycle runs in under two minutes. No human reviews the request. No email thread. No JIRA ticket.

Terraform handles the actual cloud API calls and manages state. n8n handles the orchestration: what happens before the Terraform run (input validation, cost pre-check), what happens during (status polling), and what happens after (credential injection, notification, audit log entry).

For teams that want to avoid managing Terraform state themselves, Terraform Cloud or Pulumi Cloud both offer an API-first approach that fits cleanly into this pattern.

Layer 3: The policy layer

Automation without policy is just faster chaos. The policy layer is what makes it safe to provision resources without human review on every request.

Before any provisioning run starts, a policy check validates three things: does the requesting identity have permission to create this resource type, does the requested configuration match the approved template, and will this request exceed cost or quota limits for this team?

If any check fails, the request is rejected with a clear reason. No resource is created.

We use Open Policy Agent (OPA) for this. Policies are written in Rego, OPA's declarative policy language, and stored in a Git repository. Every policy change goes through a pull request, giving you an audit trail of who changed what governance rule and when.

The policy layer also handles configuration drift. Once a resource is provisioned, periodic policy checks confirm it still matches its approved template. If someone manually changed a database's network settings, the next check flags it.

Layer 4: Secrets management

This is the layer most teams skip, and the one that causes the most damage when missing.

When the provisioning engine creates a database, it generates credentials. Handed back in plain text in an API response, they end up in logs, in Slack messages, in a developer's notes app. Hardcoded into an n8n workflow by a business user who did not know better. Still active six months after the person who created them has left the company.

The right pattern: credentials go directly from the provisioning engine into a secrets manager. The calling application or agent gets a reference, not the secret itself, and uses that reference to retrieve the credential at runtime. The secret is never exposed in transit, never stored in code, and can be rotated or revoked in one place.

HashiCorp Vault is the most capable option here. AWS Secrets Manager and Azure Key Vault are simpler alternatives that work well if you are already on those clouds.

The business user who requested the database never sees the password. Claude, writing code against that database, uses an environment variable that resolves at runtime. The credential lifecycle is managed by the platform, not by the person who built the app.

Layer 5: Observability and tagging

Every resource provisioned through the system gets a standard set of tags applied automatically at creation time: requesting user, business unit, environment, provisioning date, short description of purpose, and cost center code.

These tags flow through to your cloud billing console, your CMDB, and your security tooling. They are not optional fields on a form. They are injected by the provisioning engine from the request metadata, so they are always present and always consistent.

Beyond tagging, every API call from an agent that touches a provisioned resource should produce a log entry. Not a verbose trace of every query, but enough to answer the question "what is this resource doing and does it still need to exist?" You want to be able to see that a Databricks workspace provisioned three months ago has had zero activity in six weeks and should be decommissioned.

Datadog and Grafana both work well here. For Databricks specifically, Unity Catalog provides lineage tracking that gives you a clear view of what data assets are being accessed and by whom.

How AI agents plug into this

Claude and other AI agents interact with the provisioning system through a tool interface, typically via the Model Context Protocol (MCP) or a simple REST API wrapper.

The agent has access to provisioning tools: provision_database, provision_workflow_environment, provision_storage_bucket, and so on. When a business user describes what they want to build, Claude identifies what resources are needed and calls the appropriate tool with the required parameters.

The provisioning API handles the rest: policy check, Terraform run, credential injection. It returns a workspace URL or a secrets reference that Claude uses to write code connecting to the new resource.

From the user's perspective, they described what they needed and Claude built it, including the infrastructure. They never opened a ticket. They never waited. They never saw a credential.

What this looks like end to end

A business analyst at a manufacturing company wants to build a dashboard pulling weekly production data from SAP and comparing it against targets stored in Salesforce.

She opens Claude, describes what she needs. Claude identifies this requires a Databricks workspace for data transformation and a read-only Salesforce API connection.

Claude calls provision_databricks_workspace and provision_salesforce_connection via the MCP tool layer. Both requests hit the provisioning API. The policy layer checks permissions, template compliance, and cost quotas. Both resources provision in parallel. Credentials go to Vault. The provisioning API returns a Databricks workspace URL and a Vault secret reference for the Salesforce token.

Claude writes a Databricks notebook that reads from Salesforce via the managed connection, runs the comparison logic, and outputs a dashboard. It uses the Vault reference for authentication.

The analyst has a working dashboard on approved infrastructure, in the right region, with auditable data access, tagged to her cost center. Nobody from IT was involved after the platform team originally configured the service catalog templates.

Where to start

Start with the service catalog. Define two or three resource types your business users actually need most often. Write the configuration templates. Get sign-off from security on what the baseline requirements are. This is the hardest conversation, not because it is technically complex but because it requires agreement between IT, security, and the business on what "safe by default" looks like.

Build the provisioning engine next. Wire n8n to Terraform or your cloud provider's API. Test it against your catalog entries. Make sure it returns useful errors when requests are malformed.

Add the policy layer before you open this up to anyone outside the platform team. Even a basic OPA policy that checks resource type and requesting identity is better than no check at all.

Add secrets management and tagging in parallel. Observability can come slightly later, but get it in place before the first business user builds something that goes to production.

At Forest Digital, we help clients design and build this architecture from the ground up, with a stack that fits their existing cloud environment and compliance requirements. If you are mapping out where to start, reach out directly.

Forest Digital is a digital transformation consultancy based in Košice, Slovakia. We specialize in AI-driven development, system integration, and enterprise data platforms for mid-market and enterprise clients across Central and Eastern Europe.