PostgreSQL connector

PostgreSQL is enabled for saved source and destination connections. Use postgres for both connector_id and connector_type; its connector category is databases. It connects to PostgreSQL servers you run, Amazon RDS and Aurora PostgreSQL (including RDS Proxy), and Azure Database for PostgreSQL.

Database settings

Authentication

Choose exactly one authentication branch. Store credentials in a Secret Vault and select references containing namespace and key.

Preview means the method is supported but has not yet been validated on every configuration of the cloud service. Its behavior may still change. Contact support if something does not work as described.

username is always required. For password sign-in it is the database user. For AWS IAM it is the database user that has been granted the rds_iam role. For Microsoft Entra ID it is the PostgreSQL role created for the Entra principal. For Kerberos it is both the user part of the Kerberos principal and the PostgreSQL role.

Connections saved with password_auth or ssh_tunnel_auth keep working unchanged.

Password authentication

Use password_auth.password_ref to reference the database password. This example is the connection configuration object:

{
  "host": "db.example.internal",
  "port": 5432,
  "database": "analytics",
  "username": "data_reader",
  "ssl_mode": "verify-full",
  "password_auth": {
    "password_ref": {
      "namespace": "production-data",
      "key": "postgres-password"
    }
  }
}

AWS IAM database authentication

VegaFlow signs a short-lived IAM authentication token each time it opens a database connection, so nothing expires between runs. The token is generated from access keys you store as secrets; VegaFlow never uses an AWS role or credentials of its own.

{
  "host": "analytics.cluster-abc123.eu-west-1.rds.amazonaws.com",
  "port": 5432,
  "database": "analytics",
  "username": "vegaflow_reader",
  "ssl_mode": "verify-full",
  "aws_iam_auth": {
    "access_key_id_ref": {
      "namespace": "production-data",
      "key": "aws-access-key-id"
    },
    "secret_access_key_ref": {
      "namespace": "production-data",
      "key": "aws-secret-access-key"
    }
  }
}

Set up the database before you save the connection:

1. Enable IAM database authentication on the RDS instance, Aurora cluster or RDS Proxy.
2. Create the database user and grant it the rds_iam role: CREATE USER vegaflow_reader; GRANT rds_iam TO vegaflow_reader;, then grant the user the privileges the flow needs.
3. Attach an IAM policy that allows rds-db:connect on the database user's resource ARN, arn:aws:rds-db:<region>:<account-id>:dbuser:<resource-id>/vegaflow_reader, to the IAM user or role whose access keys you store. For an RDS Proxy, use the proxy's resource ID.
4. Create the access key and store its parts in a Secret Vault.

Microsoft Entra ID

Azure Database for PostgreSQL accepts a Microsoft Entra ID access token as the password. Two branches are available. VegaFlow never uses an Azure identity of its own, and Entra user name and password sign-in is not offered.

Service principal. VegaFlow requests a fresh access token from Microsoft Entra ID each time it opens a database connection, using the application's client secret.

{
  "host": "analytics.postgres.database.azure.com",
  "port": 5432,
  "database": "analytics",
  "username": "vegaflow-app",
  "ssl_mode": "verify-full",
  "entra_service_principal_auth": {
    "tenant_id": "11111111-2222-3333-4444-555555555555",
    "client_id": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
    "client_secret_ref": {
      "namespace": "production-data",
      "key": "entra-client-secret"
    }
  }
}

Access token. entra_access_token_auth.access_token_ref references a token you issued for the https://ossrdbms-aad.database.windows.net resource. VegaFlow does not refresh it, so the connection stops working when the token expires. Use it for tests; use the service principal for scheduled flows.

Set up the database before you save the connection:

1. Enable Microsoft Entra authentication on the Azure Database for PostgreSQL server.
2. Register an application in Microsoft Entra ID and create a client secret. Store the secret in a Secret Vault.
3. As a Microsoft Entra administrator of the server, connect to the database and create a role for the application with pgaadauth_create_principal, for example SELECT * FROM pgaadauth_create_principal('vegaflow-app', false, false);. The role name is the connection's username. Then grant the role the privileges the flow needs.

Kerberos

Use Kerberos when your PostgreSQL server accepts gss authentication. VegaFlow signs in to your Kerberos key distribution centers (KDCs) as the user, asks for a ticket to the server's service principal, and presents it to PostgreSQL. No password is sent to the database.

{
  "host": "db.corp.example.com",
  "port": 5432,
  "database": "analytics",
  "username": "vegaflow_reader",
  "ssl_mode": "verify-full",
  "kerberos_auth": {
    "realm": "CORP.EXAMPLE.COM",
    "kdc_hosts": ["kdc1.corp.example.com", "kdc2.corp.example.com:88"],
    "keytab_ref": {
      "namespace": "production-data",
      "key": "postgres-keytab"
    }
  }
}

Kerberos does not require TLS, because the ticket is not a password, but TLS still protects your data; choose an ssl_mode as you would for any other connection. Kerberos works with the top-level ssh_tunnel object. Do not combine it with ssh_tunnel_auth.

To set it up:

1. Create the server's service principal. The server needs the principal postgres/<host>@REALM, where <host> is the name you put in host, spelled the way the principal is registered (VegaFlow lowercases it and does not look it up in DNS, so use the fully qualified name, not an alias). Export its keytab to the server and point PostgreSQL at it in postgresql.conf:

   krb_server_keyfile = '/etc/postgresql/postgres.keytab'

   The keytab must be readable by the operating system user that runs PostgreSQL. Change krb_srvname on the server only if you also set kerberos_auth.service_name.

2. Allow gss in pg_hba.conf. Use a host or hostssl line, not hostgssenc, because VegaFlow does not use GSS encryption (TLS is chosen with ssl_mode):

   host  analytics  all  all  gss  include_realm=0

   include_realm=0 removes the realm from the principal, so vegaflow_reader@CORP.EXAMPLE.COM becomes the role vegaflow_reader. Without it, the role must be mapped from the full principal with a pg_ident.conf map. Add krb_realm=CORP.EXAMPLE.COM to accept only your realm. PostgreSQL compares principals case-sensitively unless you set krb_caseins_users.

3. Create the role. CREATE ROLE vegaflow_reader LOGIN;, then grant it the privileges the flow needs. The role has no password.

4. Let VegaFlow reach the KDCs. kdc_hosts are network destinations like host: they follow the connection's network access, so private KDCs need a private connection on dedicated compute, and TCP port 88 (or the port you give) must be open to it. If the database is reached through an SSH tunnel, the KDCs are reached through the same bastion, so the bastion must be able to reach them. The KDC address must be resolvable from VegaFlow compute, as host is.

5. Provide the credential. Use a password secret, or a keytab for the user's principal. With MIT Kerberos, ktutil writes one:

   ktutil
   addent -password -p vegaflow_reader@CORP.EXAMPLE.COM -k 1 -e aes256-cts-hmac-sha1-96
   wkt vegaflow_reader.keytab

   Then store the base64 text of the file in your Secret Vault, for example base64 -w0 vegaflow_reader.keytab, and reference it with keytab_ref. Create a new keytab whenever the principal's key changes.

6. Keep clocks in step. Kerberos rejects requests when the clocks of VegaFlow, the KDC and the database server differ by more than a few minutes (five by default). Keep all three on network time.

If a connection fails, the error tells you which step to check:

SSH tunnel

Use an SSH tunnel when the database is reachable through a bastion. There are two ways to configure it.

With ssh_tunnel_auth, the tunnel settings and the PostgreSQL password_ref live in one branch.

{
  "host": "db.private.internal",
  "port": 5432,
  "database": "analytics",
  "username": "data_reader",
  "ssl_mode": "require",
  "ssh_tunnel_auth": {
    "password_ref": {
      "namespace": "production-data",
      "key": "postgres-password"
    },
    "ssh_host": "bastion.example.internal",
    "ssh_port": 22,
    "ssh_user": "connector",
    "ssh_host_fingerprint": "SHA256:replace-with-verified-fingerprint",
    "ssh_private_key_ref": {
      "namespace": "production-data",
      "key": "ssh-private-key"
    }
  }
}

To use a tunnel with password_auth, aws_iam_auth, entra_service_principal_auth or entra_access_token_auth, add the top-level ssh_tunnel object instead. Do not combine it with ssh_tunnel_auth.

{
  "host": "analytics.cluster-abc123.eu-west-1.rds.amazonaws.com",
  "database": "analytics",
  "username": "vegaflow_reader",
  "ssl_mode": "require",
  "aws_iam_auth": {
    "access_key_id_ref": { "namespace": "production-data", "key": "aws-access-key-id" },
    "secret_access_key_ref": { "namespace": "production-data", "key": "aws-secret-access-key" }
  },
  "ssh_tunnel": {
    "host": "bastion.example.internal",
    "user": "connector",
    "host_key_fingerprint": "SHA256:replace-with-verified-fingerprint",
    "private_key_ref": { "namespace": "production-data", "key": "ssh-private-key" }
  }
}

With AWS IAM, the token is signed for the host and port you configure, not for the tunnel's local end. With Kerberos, the server's principal is built from the host you configure. Through a tunnel the database certificate is checked against that same host, so verify-full works as it does without a tunnel.

SSL modes

Choose the strongest mode supported by the database deployment. The default is disable, so set the intended mode explicitly.

AWS IAM and Microsoft Entra ID sign-in send a token as the database password, so a connection that uses them is refused with disable when you save it. Set ssl_mode to require, verify-ca or, preferably, verify-full.

Validate and save

1. Create the database and any SSH secret references.
2. Open VegaFlow → Connections → New connection and choose PostgreSQL.
3. Enter database settings and select one authentication branch. For AWS IAM or Microsoft Entra ID, also set ssl_mode to verify-full (at least require).
4. Validate connectivity, authentication and the required source or destination privileges.
5. Save and grant the QuickFlow operator permission to use the connection.

Revalidate after changes to credentials, database settings or network access. Use catalog discovery to select streams and supported sync modes for a flow.

Troubleshooting