Connect to VegaDB

Arrow Flight SQL drivers, Spark Connect and the HTTP SQL API, with authentication, sessions and warehouse selection.

VegaDB has three client interfaces. They share one SQL language, one set of permissions and the same warehouses.

Interface	Use it for	Clients
Arrow Flight SQL	Applications, notebooks, BI and SQL tools	ADBC (Python, Go, Java and others), the Arrow Flight SQL JDBC driver, ODBC
Spark Connect	Existing PySpark and Spark DataFrame code	PySpark and other Spark Connect clients
HTTP SQL API	Scripts, services and languages without a driver	Any HTTPS client, such as `curl`

Copy the endpoint, organization ID and workspace ID from the warehouse's Connect panel. The endpoint is stable: it stays the same when a warehouse starts, stops or scales. Every interface requires TLS.

Endpoint      <warehouse endpoint>
Organization  <organization id>
Workspace     <workspace id>
Warehouse     the warehouse name, when the workspace has more than one

Authentication

Credential	Header	Use
User access token	`Authorization: Bearer <token>`	A person signed in to Vegalake.
API key	`X-API-Key: <api key>`	A service principal for automation and integrations.
Personal access token	`Authorization: Bearer <token>`	Drivers and tools that keep one long-lived credential.

Every request also names its scope with X-Vega-Organization and X-Vega-Workspace. The identity needs membership in the workspace and USAGE on the warehouse; object permissions then apply to every table and view a statement reads or writes. See users, roles and service principals.

On Flight SQL and Spark Connect, the first request opens a session: a driver that performs the Flight SQL handshake does this itself; otherwise send x-vega-create-session: true with the credential, as the examples below do. The response carries a session token, which the driver uses for the rest of the connection.

Do not put credentials in shell history or source code. Read them from your organization's credential manager or an environment variable.

Choose a warehouse

When the workspace has exactly one warehouse, every session uses it. When it has several, name one:

Interface	Setting
Flight SQL	Session option `warehouse`
Spark Connect	Configuration `vega.warehouse`
HTTP SQL API	Request field or parameter `warehouse`

A query on a stopped warehouse starts it and waits until it is ready; see warehouses and scaling. Metadata calls and catalog browsing do not need a running warehouse.

Arrow Flight SQL

Flight SQL returns results as Arrow record batches, so columnar data reaches the client without row-by-row conversion. Drivers connect with a grpc+tls:// URI.

Python (ADBC)

pip install adbc-driver-flightsql pyarrow

import os
import adbc_driver_flightsql.dbapi as flightsql

with flightsql.connect(
    "grpc+tls://<warehouse endpoint>",
    db_kwargs={
        "adbc.flight.sql.authorization_header": f"Bearer {os.environ['VEGADB_TOKEN']}",
        "adbc.flight.sql.rpc.call_header.x-vega-organization": "<organization id>",
        "adbc.flight.sql.rpc.call_header.x-vega-workspace": "<workspace id>",
        "adbc.flight.sql.rpc.call_header.x-vega-create-session": "true",
    },
    conn_kwargs={"adbc.flight.sql.session.option.warehouse": "bi"},
) as conn:
    with conn.cursor() as cur:
        cur.execute(
            "select region, sum(amount) from sales.orders where day >= ? group by region",
            ["2026-01-01"],
        )
        print(cur.fetch_arrow_table().to_pandas())

For a service principal, replace the authorization header with "adbc.flight.sql.rpc.call_header.x-api-key": os.environ["VEGADB_API_KEY"].

cursor.adbc_ingest("orders", arrow_table, mode="append") loads an Arrow table with Flight SQL bulk ingest.

Go (ADBC)

import (
	"context"
	"os"

	"github.com/apache/arrow-adbc/go/adbc"
	"github.com/apache/arrow-adbc/go/adbc/driver/flightsql"
	"github.com/apache/arrow-go/v18/arrow/memory"
)

func query(ctx context.Context) error {
	db, err := flightsql.NewDriver(memory.DefaultAllocator).NewDatabase(map[string]string{
		adbc.OptionKeyURI:                   "grpc+tls://<warehouse endpoint>",
		flightsql.OptionAuthorizationHeader: "Bearer " + os.Getenv("VEGADB_TOKEN"),
		flightsql.OptionRPCCallHeaderPrefix + "x-vega-organization":   "<organization id>",
		flightsql.OptionRPCCallHeaderPrefix + "x-vega-workspace":      "<workspace id>",
		flightsql.OptionRPCCallHeaderPrefix + "x-vega-create-session": "true",
	})
	if err != nil {
		return err
	}
	defer db.Close()

	conn, err := db.Open(ctx)
	if err != nil {
		return err
	}
	defer conn.Close()

	stmt, err := conn.NewStatement()
	if err != nil {
		return err
	}
	defer stmt.Close()

	stmt.SetSqlQuery("select region, sum(amount) from sales.orders group by region")
	reader, _, err := stmt.ExecuteQuery(ctx)
	if err != nil {
		return err
	}
	defer reader.Release()
	for reader.Next() {
		batch := reader.Record()
		_ = batch // process the Arrow record batch
	}
	return reader.Err()
}

Java (ADBC)

import java.util.Map;
import org.apache.arrow.adbc.driver.flightsql.FlightSqlDriver;
import org.apache.arrow.memory.RootAllocator;

try (var allocator = new RootAllocator();
     var database = new FlightSqlDriver(allocator).open(Map.of(
         "uri", "grpc+tls://<warehouse endpoint>",
         "adbc.flight.sql.rpc.call_header.authorization", "Bearer " + System.getenv("VEGADB_TOKEN"),
         "adbc.flight.sql.rpc.call_header.x-vega-organization", "<organization id>",
         "adbc.flight.sql.rpc.call_header.x-vega-workspace", "<workspace id>",
         "adbc.flight.sql.rpc.call_header.x-vega-create-session", "true"));
     var connection = database.connect();
     var statement = connection.createStatement()) {
  statement.setSqlQuery("select region, sum(amount) from sales.orders group by region");
  try (var result = statement.executeQuery()) {
    var reader = result.getReader();
    while (reader.loadNextBatch()) {
      System.out.println(reader.getVectorSchemaRoot().contentToTSVString());
    }
  }
}

JDBC: DBeaver and Java tools

Use the Apache Arrow Flight SQL JDBC driver (org.apache.arrow:flight-sql-jdbc-driver). In DBeaver, add it as a driver with the class org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver, then connect with:

jdbc:arrow-flight-sql://<warehouse endpoint>?useEncryption=true&x-vega-organization=<organization id>&x-vega-workspace=<workspace id>&x-vega-create-session=true

Put the token in the connection's token property, kept in the tool's credential store rather than in a shared connection definition. The driver sends URL parameters it does not recognize as request headers, which carries the organization, workspace and session request.

var url = "jdbc:arrow-flight-sql://<warehouse endpoint>?useEncryption=true"
    + "&x-vega-organization=<organization id>&x-vega-workspace=<workspace id>"
    + "&x-vega-create-session=true";
var properties = new java.util.Properties();
properties.put("token", System.getenv("VEGADB_TOKEN"));

try (var connection = DriverManager.getConnection(url, properties);
     var statement = connection.prepareStatement(
         "select region, sum(amount) from sales.orders where day >= ? group by region")) {
  statement.setDate(1, java.sql.Date.valueOf("2026-01-01"));
  try (var rows = statement.executeQuery()) {
    while (rows.next()) {
      System.out.println(rows.getString(1) + ": " + rows.getBigDecimal(2));
    }
  }
}

Tools that speak ODBC use a Flight SQL ODBC driver with the same endpoint, credential and headers.

What Flight SQL provides

Statements and prepared statements with bound parameters, updates and bulk ingest.
Transactions: BEGIN/COMMIT/ROLLBACK in SQL, or the driver's transaction API (ADBC autocommit off, JDBC setAutoCommit(false)).
Catalog metadata: catalogs, schemas, tables, table types, primary keys and type information.
Long-running queries report queued, resuming (while the warehouse starts) and running through polling, and can be cancelled.
Session options and session close.

Spark Connect

Point PySpark at the endpoint with remote(...). Connection-string parameters other than Spark's own are sent as request headers.

from pyspark.sql import SparkSession

spark = (
    SparkSession.builder
    .remote(
        "sc://<warehouse endpoint>/;use_ssl=true;token=<token>"
        ";x-vega-organization=<organization id>;x-vega-workspace=<workspace id>"
        ";x-vega-create-session=true"
    )
    .getOrCreate()
)
spark.conf.set("vega.warehouse", "bi")

orders = spark.table("sales.orders")
orders.groupBy("region").sum("amount").show()

spark.sql("select region, count(*) from sales.orders group by region").show()

DataFrame operations follow Spark semantics. Temporary views belong to the Spark session.

HTTP SQL API

The HTTP SQL API runs SQL over HTTPS. POST /v1/sql streams one statement's result; POST /v1/statements submits a statement asynchronously so you can poll and cancel it. The complete reference is HTTP SQL API.

curl --request POST "https://<warehouse endpoint>/v1/sql?format=JSONEachRow&warehouse=bi" \
  --header "Authorization: Bearer $VEGADB_TOKEN" \
  --header "X-Vega-Organization: <organization id>" \
  --header "X-Vega-Workspace: <workspace id>" \
  --header "Content-Type: text/plain" \
  --data-binary "select region, sum(amount) as amount from sales.orders group by region"

format is one of JSON (the default), JSONEachRow, CSV, TSV, Arrow (an Arrow IPC stream) and Parquet. Send Accept-Encoding: gzip or zstd for a compressed response.

# Submit; wait up to 10 seconds for the result.
curl --request POST "https://<warehouse endpoint>/v1/statements" \
  --header "X-API-Key: $VEGADB_API_KEY" \
  --header "X-Vega-Organization: <organization id>" \
  --header "X-Vega-Workspace: <workspace id>" \
  --header "Content-Type: application/json" \
  --data '{"sql": "select count(*) from sales.orders", "warehouse": "bi", "wait_timeout": 10}'

# Poll the returned statement id until its state is succeeded, failed or cancelled.
curl "https://<warehouse endpoint>/v1/statements/<statement id>" \
  --header "X-API-Key: $VEGADB_API_KEY" \
  --header "X-Vega-Organization: <organization id>" \
  --header "X-Vega-Workspace: <workspace id>"

# Cancel it.
curl --request DELETE "https://<warehouse endpoint>/v1/statements/<statement id>" \
  --header "X-API-Key: $VEGADB_API_KEY" \
  --header "X-Vega-Organization: <organization id>" \
  --header "X-Vega-Workspace: <workspace id>"

Sessions

A session carries the current catalog and schema (USE), settings, the selected warehouse, prepared statements, temporary objects and an open transaction.

Interface	Session
Flight SQL	The driver connection.
Spark Connect	The Spark session.
HTTP SQL API	Each request is independent unless it sets `create_session: true`; later requests pass the returned `session_id`.

A session does not keep its warehouse running: a warehouse that stopped while idle starts again on the session's next query. Idle sessions expire after a configured time.

One SQL dialect

Every interface runs the same VegaDB SQL, whose dialect follows DuckDB's. Use VegaDB data types and functions from the SQL reference whichever client you use.

Current catalog metadata through HTTP

The workspace catalog API accepts fresh=true on GET /catalogs/{catalog_id}/databases and GET /databases/{database_id}/objects, under /orgs/{org_id}/workspaces/{workspace_id}/vegadb. These reads require the caller's existing workspace/catalog use permission. A successful current refresh returns X-VegaDB-Catalog-Fresh: 1; refresh failures return an error instead of a cached empty list. Unavailable or uninitialized metadata and refresh timeouts return a retryable 503, with a stable code such as unavailable, COORDINATOR_UNAVAILABLE or WAREHOUSE_UNAVAILABLE and Retry-After guidance. Authorization failures retain 403, missing resources retain 404, and existing public errors keep their status and code. Omitting the option keeps the existing cached-metadata behavior. Clients that require confirmed absence must verify the freshness header before treating an empty list as proof that an object is missing.

On this page