Data Source Setup

How to connect a PostgreSQL database to CRAFT as a data source for querying and governance.

Connect a PostgreSQL Database

This guide walks you through registering a PostgreSQL database as a data connection in CRAFT. Once connected, the database is available to solutions like Data Insights (for NL-to-SQL queries) and Data Governance (for profiling and enrichment).

Prerequisites

Before you begin, ensure you have:

• A running CRAFT instance
• A valid JWT token with the developer or admin role
• Network connectivity between the platform and your PostgreSQL instance
• A PostgreSQL user with read access to the target schemas

How Data Connections Work

Data connections are managed by the Assets service (port 8002). When you register a connection:

1. The connection metadata (host, port, database name) is stored in the Assets database
2. Credentials (username, password) are stored securely via the platform Secrets API (Infisical or ESO + GCP Secret Manager) — never in the Assets database
3. Solutions retrieve the connection at runtime via the Assets SDK and use the credentials to establish a live database session
4. All connections are scoped to an organization and project for multi-tenant isolation

Data connection credentials are never exposed in API responses. They are injected at runtime only when a solution needs to establish a connection.

Step 1: Prepare Your Database

Ensure your PostgreSQL instance is configured to accept connections from the platform.

1. Create a read-only user

   -- Connect to your PostgreSQL instance as a superuser
   CREATE USER emergence_reader WITH PASSWORD 'your-secure-password';
   
   -- Grant read access to the target schema
   GRANT USAGE ON SCHEMA public TO emergence_reader;
   GRANT SELECT ON ALL TABLES IN SCHEMA public TO emergence_reader;
   
   -- Ensure future tables are also accessible
   ALTER DEFAULT PRIVILEGES IN SCHEMA public
     GRANT SELECT ON TABLES TO emergence_reader;
   
2. Configure network access

   Update pg_hba.conf to allow connections from the platform’s network:

   # Allow connections from the Kubernetes cluster CIDR
   host    your_database    emergence_reader    192.0.2.0/24    scram-sha-256
   

   Reload the configuration:

   sudo systemctl reload postgresql
   
3. Verify connectivity

   From a pod in the platform’s Kubernetes cluster, test the connection:

   psql -h <db-host> -U emergence_reader -d your_database -c "SELECT 1;"
   

Step 2: Register the Data Connection

Use the Assets API to register the connection in the platform.

curl -X POST "https://<platform-host>:8002/assets/data" \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-Project-ID: <your-project-id>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "production-analytics",
    "description": "Production analytics database (read-only)",
    "type": "postgresql",
    "config": {
      "host": "db.example.com",
      "port": 5432,
      "database": "analytics",
      "schema": "public",
      "ssl_mode": "require"
    },
    "credentials": {
      "username": "emergence_reader",
      "password": "your-secure-password"
    }
  }'

The credentials field is encrypted and stored via the platform Secrets API upon registration. It will not appear in subsequent GET responses.

Step 3: Test the Connection

Verify the platform can reach your database:

curl -X POST \
  "https://<platform-host>:8002/assets/data/<connection-id>:test" \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-Project-ID: <your-project-id>"

A successful test returns:

{
  "status": "connected",
  "latency_ms": 15,
  "server_version": "17.2",
  "schemas": ["public", "analytics"],
  "table_count": 42
}

Step 4: Use the Connection with Data Insights

Once registered, the Data Insights solution can use this connection for natural-language queries:

# Start a chat session with the connected data source
curl -X POST "https://<platform-host>:8080/talk2data/sessions" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "data_connection_id": "<connection-id>",
    "name": "Analytics exploration"
  }'

You can now ask questions like “What were the top 10 products by revenue last quarter?” and the Text2SQL agent will generate and execute the appropriate SQL against your connected database.

Connection Configuration Reference

PostgreSQL

Parameter	Type	Required	Description
host	string	Yes	Database hostname or IP
port	integer	No	Port number (default: 5432)
database	string	Yes	Database name
schema	string	No	Default schema (default: public)
ssl_mode	string	No	SSL mode: disable, require, verify-ca, verify-full

Credentials

Parameter	Type	Required	Description
username	string	Yes	Database username
password	string	Yes	Database password

Security Considerations

Credential Storage

Credentials are stored via the platform Secrets API (backed by Infisical or ESO + GCP Secret Manager). They are encrypted at rest and organization-scoped. Credentials are injected at runtime only when a solution needs to establish a connection.

Network Security

Configure your database firewall to accept connections only from the platform’s Kubernetes cluster CIDR. Use ssl_mode: verify-full for production deployments to prevent man-in-the-middle attacks.

Principle of Least Privilege

Always create a dedicated read-only user for the platform. Never use a superuser or an account with write privileges unless the solution explicitly requires write access.

Troubleshooting

Connection test fails with timeout

Verify network connectivity between the Kubernetes cluster and your database. Check firewall rules, security groups, and pg_hba.conf configuration. Ensure the database host is resolvable from within the cluster.

Authentication failure

Confirm the username and password are correct. Check that the user exists in the target database and has the CONNECT privilege. Verify the pg_hba.conf entry allows the authentication method being used.

SSL connection required

If your database requires SSL, set ssl_mode to require or higher. For verify-ca and verify-full, ensure the platform has access to the CA certificate.

Next Steps

Chat with Data

Learn how to ask natural-language questions against your connected database.

Data Profiling

Profile your connected database to understand data quality and structure.

Backup & Restore

Learn how to back up your platform data including connection configurations.

Network Security

Review network security policies for data connections.