# Managed Data

FullStacked Cloud runs PostgreSQL, Redis, MongoDB and S3-compatible object storage (RustFS) instances, each in its own container. A project reaches an instance linked to it through `fullstacked/tunnel`: register the tunnels hub with the user's `AUTH_TOKEN` and the instance's data token, then point the standard npm driver at the returned name. The runtime keeps a WebSocket session to the hub for that tunnel and carries every driver connection as a stream on it. Credentials come from `process.env` and match the current user's access (read-write or read-only), so the same code serves every user.

## Setup

1. Provision an instance and link it to the project: Console (https://console.fullstacked.cloud, Data, then Project, Linked data) or MCP (`data_provision`, then `data_link_project`).
2. The project environment then carries the data token and credentials (see [Injected process.env](https://docs.fullstacked.cloud/project-runtime)):

| Engine | Variables |
| :--- | :--- |
| PostgreSQL | `PG_DATA_TOKEN`, `DATA_USER`, `DATA_PASSWORD`, `DATA_DB` |
| Redis | `REDIS_DATA_TOKEN`, `DATA_USER`, `DATA_PASSWORD` |
| MongoDB | `MONGO_DATA_TOKEN`, `DATA_USER`, `DATA_PASSWORD`, `DATA_DB` |
| S3 | `S3_DATA_TOKEN`, `DATA_KEY_ID`, `DATA_KEY_SECRET`, `DATA_BUCKET` |

The oldest linked instance uses these names; other instances add a prefix from their name (`ANALYTICS_DB_DATA_USER`, `ANALYTICS_DB_PG_DATA_TOKEN`, ...).

3. Connect with the code below. Always `ssl: false` (the tunnel is already `wss://`); the port given to the driver is ignored (the token selects the instance), keep the engine's default for readability.

## PostgreSQL (`pg`)

```ts
// db.ts
import tunnel from "fullstacked/tunnel";
import pg from "pg";

let pool: pg.Pool | null = null;

export async function getPool(): Promise<pg.Pool> {
    if (pool) return pool;
    const host = await tunnel.register({
        host: process.env.TUNNELS_HOST!,
        authorization: `${process.env.AUTH_TOKEN}:${process.env.PG_DATA_TOKEN}`,
    });
    pool = new pg.Pool({
        host,
        port: 5432,
        user: process.env.DATA_USER,
        password: process.env.DATA_PASSWORD,
        database: process.env.DATA_DB,
        ssl: false,
        max: 10,
    });
    pool.on("error", (err) => console.error("pg pool error", err));
    return pool;
}

const { rows } = await (await getPool()).query("SELECT now()");
```

## Redis (`ioredis`)

```ts
import tunnel from "fullstacked/tunnel";
import Redis from "ioredis";

const host = await tunnel.register({
    host: process.env.TUNNELS_HOST!,
    authorization: `${process.env.AUTH_TOKEN}:${process.env.REDIS_DATA_TOKEN}`,
});
const redis = new Redis({
    host,
    port: 6379,
    username: process.env.DATA_USER,
    password: process.env.DATA_PASSWORD,
});
redis.on("error", (err) => console.error("redis error", err));
await redis.set("greeting", "hello");
```

## MongoDB (`mongodb`)

```ts
import tunnel from "fullstacked/tunnel";
import { MongoClient } from "mongodb";

const host = await tunnel.register({
    host: process.env.TUNNELS_HOST!,
    authorization: `${process.env.AUTH_TOKEN}:${process.env.MONGO_DATA_TOKEN}`,
});
const { DATA_USER, DATA_PASSWORD, DATA_DB } = process.env;
const client = new MongoClient(
    `mongodb://${DATA_USER}:${DATA_PASSWORD}@${host}:27017/${DATA_DB}?authSource=${DATA_DB}&directConnection=true`
);
await client.connect();
const db = client.db(DATA_DB);
```

Keep `directConnection=true` (a single server reached through the tunnel name) and `authSource` set to the database: the app users live there.

## S3 (`@aws-sdk/client-s3`)

```ts
import tunnel from "fullstacked/tunnel";
import { S3Client, PutObjectCommand, ListObjectsV2Command } from "@aws-sdk/client-s3";

const host = await tunnel.register({
    host: process.env.TUNNELS_HOST!,
    authorization: `${process.env.AUTH_TOKEN}:${process.env.S3_DATA_TOKEN}`,
});
const s3 = new S3Client({
    endpoint: `http://${host}`,
    region: "us-east-1",
    forcePathStyle: true,
    credentials: {
        accessKeyId: process.env.DATA_KEY_ID!,
        secretAccessKey: process.env.DATA_KEY_SECRET!,
    },
});
const Bucket = process.env.DATA_BUCKET!;
await s3.send(new PutObjectCommand({ Bucket, Key: "hello.txt", Body: "hi", ContentType: "text/plain" }));
const list = await s3.send(new ListObjectsV2Command({ Bucket }));
```

Use `http://` (not `https://`) and `forcePathStyle: true`. Each instance has one bucket, `DATA_BUCKET`.

## Read-only users

A user with read access only receives the `app_ro` credentials and the read-only token: PostgreSQL rejects writes (`SQLSTATE 25006` read-only transaction, or `42501` insufficient privilege), Redis allows only read and subscribe commands, MongoDB and S3 refuse writes. A frozen instance (over quota) also resolves to read-only credentials for everyone. Design the UI to handle a failed write, and gate write controls with `GET /api/v1/user/access`: `resources.data[<instanceId>].write && !resources.data[<instanceId>].frozen` (`write` stays true on a frozen instance).

## Container limits

| Engine | Image | Port | Memory | CPUs | PIDs |
| :--- | :--- | :--- | :--- | :--- | :--- |
| PostgreSQL | `postgres:18.6-alpine` | `5432` | 256 MB | 2 | 256 |
| Redis | `redis:8.10.2-alpine` | `6379` | 256 MB | 1 | 128 |
| MongoDB | `mongo:8.3.11-noble` | `27017` | 512 MB | 2 | 512 |
| S3 (RustFS) | `rustfs/rustfs:1.0.0` | `9000` | 768 MB | 2 | 1024 |

Engine settings:

- **PostgreSQL**: `shared_buffers=32MB`, `maintenance_work_mem=16MB`, `max_connections=110`, and each backend's private memory capped at 64 MB. A query that needs more fails alone with `out of memory` (SQLSTATE `53200`); if backends together exceed the container memory, one is killed and PostgreSQL resets every connection of the instance. A query's temporary files stop at 1 GB (`temp_file_limit`, SQLSTATE `53400` for that query alone).
- **Redis**: AOF persistence (`appendonly yes`) and `maxmemory 192mb` with `noeviction`: past 192 MB of data, writes fail with an `OOM` error while reads keep working; nothing is evicted. Keep keys bounded (TTLs).
- **MongoDB**: default settings inside the container memory limit (diagnostic data capped at 20 MB).
- **S3 (RustFS)**: at most 8 concurrent disk reads and 4 concurrent uploads per instance; further uploads queue for up to 60 s.

Containers have no swap and restart automatically after being killed.

## Fair use and driver practices

1. **Page large results**: `LIMIT` with keyset pagination or cursors in PostgreSQL so no single query needs more than 64 MB of backend memory; `SCAN` instead of `KEYS *` in Redis; batch MongoDB reads with `limit` and cursors.
2. **Bound concurrency**: keep `pg.Pool` small (`max` 10 or less per app) and avoid many heavy queries at once (`Promise.all` over dozens of large result sets) that together exceed the instance memory.
3. **Bound uploads**: run at most 4 S3 uploads at a time per instance and keep the AWS SDK's default retries.
4. **Handle errors**: attach `error` listeners to pools and clients so a dropped connection (revoked access, instance restart) does not surface as an unhandled error; reconnect on demand.
5. **Register once**: call `tunnel.register` once per instance and reuse the name and the client.
6. **Plan quotas**: on the Base plan an owner keeps 5 data instances, 1 GiB of storage across their projects and data instances together, and 5 GiB per UTC calendar month of data tunnel transfer (both directions). Instances beyond the 5 oldest are frozen (read-only) until others are deleted; over 1 GiB of storage every non-sponsored instance is frozen until storage falls below 95% of it; once the monthly transfer is spent, new connections are refused (`402 quota_exceeded`) until the next month. A Pro owner, or a Pro member accepted on a project the instance is linked to, lifts the limits.

## Troubleshooting

A refused tunnel session fails every connection with an error naming the tunnel and the WebSocket handshake status (for example `expected handshake response status code 101 but got 401`); the reason word below is in the hub's response body, not in the error.

| Symptom | Cause |
| :--- | :--- |
| Variable undefined | Instance not linked, or the user has no access to it (omitted from the environment). |
| Connection fails, status `401` | `AUTH_TOKEN` missing, expired or revoked, or the data token was rolled (re-read the environment). |
| Status `403` (`read_access_denied` / `write_access_denied`) | The user lost access to the instance. |
| Status `402` (`quota_exceeded`) | Frozen instance (read-write token) or monthly data transfer spent. |
| Status `503` (`auth_unavailable`) | The hub could not check the token: retry. |
| SSL negotiation error | Driver TLS enabled: set `ssl: false` / plain `http://`. |

## Endpoints

### GET /api/v1/data

Managed data instances visible to the caller (?engine=).

- **Service**: `data` (`https://tunnels.fullstacked.cloud`)
- **Auth**: Bearer fsc_*

curl:

```bash
curl -sS -H "Authorization: Bearer $AUTH_TOKEN" "https://tunnels.fullstacked.cloud/api/v1/data"
```

### POST /api/v1/data/provision

Provisions an isolated instance { engine: postgres | redis | mongodb | s3, label? } (201).

- **Service**: `data` (`https://tunnels.fullstacked.cloud`)
- **Auth**: Bearer fsc_*

Request:

```json
{
  "engine": "postgres"
}
```

### GET /api/v1/data/:id

One instance.

- **Service**: `data` (`https://tunnels.fullstacked.cloud`)
- **Auth**: Bearer fsc_*

### DELETE /api/v1/data/:id

Deprovisions an instance (owner only, 204).

- **Service**: `data` (`https://tunnels.fullstacked.cloud`)
- **Auth**: Bearer fsc_*

### GET /api/v1/data/:id/connection

Connection parameters, DATA_* variables and fullstacked/tunnel snippets for ?access_level=rw|ro within the caller's permission.

- **Service**: `data` (`https://tunnels.fullstacked.cloud`)
- **Auth**: Bearer fsc_*

### GET /api/v1/data/:id/schema

Introspects tables, keys, collections or buckets (?schema=).

- **Service**: `data` (`https://tunnels.fullstacked.cloud`)
- **Auth**: Bearer fsc_*

### POST /api/v1/data/:id/query

Runs one operation: { sql, params? } (postgres), { command } (redis), { operation, collection, ... } (mongodb) or { operation, key?, prefix?, ... } (s3).

- **Service**: `data` (`https://tunnels.fullstacked.cloud`)
- **Auth**: Bearer fsc_*

Request:

```json
{
  "sql": "select now()"
}
```