Skip to content

Add Dsql AWS component#6496

Open
DamienPace15 wants to merge 22 commits intoanomalyco:devfrom
DamienPace15:damienpace/dsql
Open

Add Dsql AWS component#6496
DamienPace15 wants to merge 22 commits intoanomalyco:devfrom
DamienPace15:damienpace/dsql

Conversation

@DamienPace15
Copy link

@DamienPace15 DamienPace15 commented Feb 27, 2026

Built off the back of PR #5969, with testing and additional code suggested by @natac13. Also merged in latest dev with the latest 4.x version.

This adds the sst.aws.Dsql component for Amazon Aurora DSQL.

What's included

Single-region

Creates an Aurora DSQL cluster in the current region. Linked functions get endpoint and region via Resource.

Multi-region

Creates a primary cluster in the current region and a peer cluster in another region. An AWS provider for the peer region is created automatically so you don't need to configure one yourself. The two-way ClusterPeering handshake that DSQL requires is also handled automatically. Linked functions get endpoint and region for the primary cluster and peer.endpoint and peer.region for the peer. The peer cluster is accessed via its public endpoint which is secured by short-lived IAM auth tokens and TLS.

VPC endpoints

You can optionally create AWS PrivateLink interface endpoints inside a VPC. Two types are supported: management for control plane operations and connection for PostgreSQL client connections. Both default to false since they cost money and need to be explicitly enabled. A security group is created automatically that allows inbound TCP/5432 from within the VPC CIDR. When a connection endpoint is created, Resource.MyCluster.endpoint automatically resolves to the private hostname instead of the public one, so your function code doesn't need to change.

Note: Multi-region VPC endpoint support is not included yet. It has some nuances around cross-region VPC creation and routing that are better handled separately.

Dsql.get

Reference an existing cluster by its identifier, useful for sharing a cluster across stages. Supports both single-region and multi-region clusters via peerIdentifier and peerRegion.

Tested

  • Single region deployment
  • Multi region deployment
  • Single region connection within Lambda
  • Multi region connection within Lambda connecting to both clusters
  • Remove single region cluster
  • Remove multi region cluster

chrisfowles and others added 15 commits July 14, 2025 17:53
This commit introduces the new  component, providing a high-level construct for provisioning and managing Amazon Aurora DSQL clusters.

The component supports both single-region and multi-region active-active deployments. For multi-region setups, it automatically handles the creation and peering of clusters across regions. The  integration has been configured to provide functions with all necessary connection details for both primary and peer clusters, including ARNs, endpoints, and regions, accessible via the type-safe  object.

Two new examples have been added to demonstrate usage:
- : A basic single-region cluster setup.
- : A more advanced example showcasing how to connect to both the primary and peer clusters from a single Lambda function.

Additionally, this commit updates the pulumi/aws dependency to v6.83.0
This commit refactors the DSQL component to resolve a critical deletion deadlock in multi-region deployments. The architecture is separated into two components: `Dsql` for single-region cluster management and `DsqlPeering` for coordinating multi-region connections.

This new design leverages a Pulumi dynamic provider to orchestrate the simultaneous deletion of peered clusters, which is required by AWS and was the root cause of the deadlock.

Key changes:
- `Dsql` component is now simplified for single-region clusters.
- New `DsqlPeering` component manages the lifecycle of multi-region peering.
- A dynamic provider in TypeScript handles the complex creation and deletion logic, ensuring clusters are deleted in a coordinated manner.
- The `aws-dsql-multiregion` example is updated to reflect the new, more robust usage pattern.
@vimtor vimtor self-assigned this Feb 27, 2026
Copy link
Collaborator

@vimtor vimtor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for your contribution @DamienPace15

i've left a bunch of comments and suggestions

thanks again for taking a look at this

Comment on lines +207 to +222
private static publicEndpointFromArn(clusterArn: Output<string>) {
return clusterArn.apply((arn) => {
const parts = arn.split(":");
const region = parts[3];
const clusterId = parts[5].split("/")[1];
return `${clusterId}.dsql.${region}.on.aws`;
});
}

private static regionFromArn(clusterArn: Output<string>) {
return clusterArn.apply((arn) => {
const parts = arn.split(":");
const region = parts[3];
return region;
});
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are several ARN utilities in the codebase. we should check if any of the existing ones cover this use case

if this is sql specific, the existing code is fine

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find any generic region from arn like utility that works with Output from what I can see. The existing utilities are all service specific parsers for example parseQueueArn, parseLambdaEdgeArn and have some specific logic around that.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably it still makes sense to extract service-specific arn functions so we can test them. if you update the branch with origin-dev, you'll see new tests for those functions

you can keep them working with regular strings and just do the apply on the component side.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bumping this

```typescript
// sst.config.ts
const cluster = new sst.aws.Dsql('MyCluster', {
multiRegion: {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this property doesn't exist in the component

import type { Dsql } from "./dsql";
import { DsqlPeeringProvider } from "./providers/dsql-peering";

export interface DsqlPeeringArgs {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i believe we no longer need the custom provider since pulumi has a resource for this: https://www.pulumi.com/registry/packages/aws-v6/api-docs/dsql/clusterpeering/

Copy link
Author

@DamienPace15 DamienPace15 Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I be removing the dsql peering component completely?

Using the native pulumi peering component means that I would have to do something like this.

`
const witnessRegion = 'us-west-2';

const primary = new sst.aws.Dsql('Primary', {
  witnessRegion: witnessRegion,
});

const peerProvider = new aws.Provider('PeerRegion', {
  region: 'us-east-2',
});

const peer = new sst.aws.Dsql(
  'Peer',
  { witnessRegion: witnessRegion },
  { provider: peerProvider }
);

// Create peering between clusters
new aws.dsql.ClusterPeering('PrimaryPeering', {
  identifier: primary.identifier,
  clusters: [peer.arn],
  witnessRegion,
});

new aws.dsql.ClusterPeering(
  'PeerPeering',
  {
    identifier: peer.identifier,
    clusters: [primary.arn],
    witnessRegion,
  },
  { provider: peerProvider }
);`

Or should we be keeping the peering component so some of this is abstracted?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i believe we can remove the custom peering component and either provide this as an example or abstract this logic inside the existing DSQL component

an example of the latter would be:

const primary = new sst.aws.Dsql('Primary', {
  witness: { region: 'us-east-1' },
  peers: [
    { region: 'us-east-2' },
    { region: 'eu-west-1' },
  ],
});

i'm not super familiar with dsql but this latter approach feels very clean

@vimtor vimtor changed the title Damienpace/dsql Add Dsql AWS component Feb 27, 2026
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't worry i will fix these before merging the pr.

if you want to fix it deleting it and running bun install should work (in fact if you remove the unnecessary dep no bun changes should appear)

[
"aws:dynamodb/table:Table",
"aws:rds/instance:Instance",
"aws:dsql/cluster:Cluster",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wonder if we should also add the peering stuff here? can you confirm?

Copy link
Author

@DamienPace15 DamienPace15 Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to keep the peering resource then yes. It doesn't contain any data.

Currently what it will do in multi region is the following

Before Delete
[Cluster 1] -> [Peering relationship] -> [Cluster 2]

After Delete 
[Cluster 1] [Cluster 2]

The peering relationship is easy to recreate if needed.

If we add it then it will keep the multi region relationship.

Is this function strictly for resources that hold data? If so Cluster peering isn't the right spot for this imo.

Comment on lines +3 to +21
/* # AWS Aurora DSQL Example
## How to use

1. **Deploy the stack**

```bash
npm install
sst deploy
```

2. **Test the connection**
The Lambda function when invoked will connect to the DSQL cluster and run a test query.

3. **View the results**
Check the Lambda function logs to see the successful connection and query results.

## Clean up

```bash
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we restructure these docs to follow the pattern of the others?

if you don't want to bother i will do it before merging

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants