🌐 API Instance¶

The Meerschaum API instance (and Web Console) lets you manage pipes, jobs, and connectors over the Internet. Start the API server with the stack or via start api; see Deployment below for ways to run Meerschaum in production.

Usage¶

Starting the Meerschaum API server lets you treat your host as the following:

• API instance For managing pipes.
• Plugin repository For storing plugins.
• Remote executor For running jobs.

Adding Connectors¶

Add new api connectors with bootstrap connector or declaring an environment connector:

# URI is in the form 'protocol://user:pass@host:port'
export MRSM_API_FOO='https://foo:bar@mrsm.example.com'

Remote Actions¶

Add -e to client commands to execute them remotely on your API instance:

sync pipes -e api:main

When running the mrsm shell, the executor command allows you to temporarily run all the commands in your session remotely (like the instance command):

executor api:main

API Documentation¶

You can visit your API instance's Swagger documentation by navigating to /docs or /redoc. Every route is listed there with the scopes it needs. Saved Pipelines, their runs and the console's actions have routes of their own, which answer {"success": ..., "message": ..., ...} with real status codes: see Pipelines, Runs and Actions over HTTP.

Permissions and Rate Limits¶

Which routes a server serves, who may run actions and how often a client may call are settings under api:permissions: see Permissions and Rate Limits. To publish only plugin routes and plugin pages (a public dashboard), use the public preset:

mrsm start api --public

Deployment¶

It's common in production deployments to stand up a public-facing API instance. See below for deployment tips:

On a new instance, the first account becomes an administrator. Create that account before opening the instance to visitors. Self-registration then closes unless you set api:permissions:registration:users: true. New users are Editors by default: they may work with pipes, but actions and the web terminal need the Operator role (actions:execute and webterm:use). Administrators manage users, roles and teams in the console's Users & Teams page.

mrsm start api --secure restricts actions and terminals to administrators; --private requires authentication on every endpoint. Enabling registration with Operator as the default role lets any visitor register and open a shell on this server. See roles, teams and permissions and HTTPS.

Meerschaum Stack (easiest)¶

If you have Docker installed, the most straightforward way to deploy is via the Meerschaum stack:

mrsm stack up -d

This starts an internally-managed, pre-configured Docker Compose project with an api service that runs on http://localhost:8000.

The environment in the api service is updated such that connectors from your host are passed to the container.

Bare Metal¶

If you don't want to deploy with Docker, you can run an API instance on your host directly:

mrsm start api --production

For example, this is a common pattern for standing up a systemd service to start the API server:

mrsm start api --production --name api-instance -d

TLS / SSL / HTTPS¶

Give start api a private key and its certificate, and the API server speaks HTTPS itself (HTTP/2 and HTTP/1.1):

mrsm start api --production \
  --keyfile /etc/meerschaum/tls/privkey.pem \
  --certfile /etc/meerschaum/tls/fullchain.pem

Or set the files in the config (mrsm edit config api), so every start api (and the background job of start api -d) uses them:

api:
  server:
    ssl_keyfile: /etc/meerschaum/tls/privkey.pem
    ssl_certfile: /etc/meerschaum/tls/fullchain.pem

The server prints where it serves, for example Serving HTTPS on https://mrsm.example.com:8000 (certificate for mrsm.example.com, valid until 2027-01-25). Without a key and a certificate it speaks plain HTTP, as always.

• The files are PEM. The key may be unencrypted PKCS#8, PKCS#1 (RSA) or SEC1 (EC). The certificate file holds the server's certificate first, then its intermediates, as Let's Encrypt's fullchain.pem does. start api checks both before it starts and stops with the reason when something is off: a key without a certificate (or the reverse), a file it cannot read, swapped files, a key that does not belong to the certificate.
• Renewals need no restart. The server reads both files again every 30 seconds and on SIGHUP (kill -HUP <pid>), and new connections get the renewed certificate. When the new files do not load yet (a renewal caught between writing the key and the certificate), it keeps serving the previous certificate and says why if the problem persists. (So with HTTPS, SIGHUP reloads instead of stopping the server: stop it with Ctrl-C, SIGTERM or mrsm stop jobs.)
• Plain HTTP on the HTTPS port gets /healthcheck answered, so container healthchecks such as curl --fail http://localhost:8000/healthcheck keep working, and a redirect to https:// for everything else.
• Clients. Point api connectors at the server with https, e.g. MRSM_API_MAIN='https://user:pass@mrsm.example.com:8000' or protocol: https. For a self-signed certificate, add its CA to the client's trust store, or set verify: false on the connector (no certificate check: for testing only).
• The web console, the web terminal and MCP work over HTTPS unchanged. The server's own callers on the same machine (the console's Python host, jobs that install plugins from this server) reach it through a separate plain-HTTP listener on 127.0.0.1 with a random port, which is never exposed.

sudo mkdir -p /etc/meerschaum/tls
sudo certbot certonly --standalone -d mrsm.example.com --deploy-hook \
  'install -m 640 -g meerschaum "$RENEWED_LINEAGE/privkey.pem" "$RENEWED_LINEAGE/fullchain.pem" /etc/meerschaum/tls/'

In the stack, mount the certificate directory into the api service (mrsm edit config stack, under docker-compose.yaml) and set api:server:ssl_keyfile / ssl_certfile to the paths inside the container; its healthcheck needs no change.

The 4.1 keys api:uvicorn:ssl_keyfile and api:uvicorn:ssl_certfile still work (with a notice to move them to api:server). --keyfile '' --certfile '' serves plain HTTP even when the config sets a key and a certificate.

A reverse proxy that terminates TLS, such as Caddy or NGINX, remains a fine choice, for example to serve several sites on port 443: see the example NGINX configuration. Behind a proxy, set api:server:forwarded_allow_ips and api:server:proxy_headers so that the server sees each client's address (see Permissions).

Stopping and Restarting¶

When the API server gets SIGTERM or SIGINT (Ctrl-C, docker stop, mrsm stop job), it stops gracefully:

• It stops accepting new connections at once.
• Requests in progress finish. The server exits as soon as the last one is done.
• An idle server exits immediately.
• Event streams, log streams and websockets (the web terminal) close at once. A tmux session in the web terminal survives.
• A second SIGTERM or SIGINT stops the server immediately.

api:shutdown:grace_seconds caps how long the server waits for requests in progress (default 8). It is an upper bound, not a delay. 0 stops at once without waiting.

api:
  shutdown:
    grace_seconds: 8

The default stays under Docker's stop timeout (10 seconds, then SIGKILL). If you raise grace_seconds, also raise the time your supervisor waits: stop_grace_period in Docker Compose, docker stop --time, or terminationGracePeriodSeconds in Kubernetes. mrsm stop job waits for an API job's grace period plus 2 seconds before it signals again. With the local executor, an explicit --timeout wins.

Behind a load balancer, probe GET /healthcheck/ready. It answers 200 only once the server can serve everything it mounts, including the Python API host that serves Python plugins and the web console. Otherwise it answers 503 with a reason:

{"success": false, "message": "The server is shutting down.", "reason": "draining"}

A server whose plugins need no API host (only Rust plugins, or --no-dash without Python plugins) does not wait for one. GET /healthcheck is unchanged, so uptime monitors keep matching it. A proxy that retries the next upstream on a refused connection or a 503 (NGINX's proxy_next_upstream error http_503) moves clients to another server during a restart.

Docker¶

If you're building your own Docker images, consider basing off the bmeares/meerschaum Docker image, as is done in the mrsm compose template repository:

FROM bmeares/meerschaum

COPY --chown=meerschaum:meerschaum ./ /app
WORKDIR /app
RUN mrsm compose init

ENTRYPOINT ["/app/docker/bootstrap.sh"]

This image comes pre-installed with common database drivers and package dependencies. Note this image is configured to run as a non-privileged user meerschaum.

Of course, you can always pip install Meerschaum into your own custom image:

FROM python

# Install dependencies into the virtual environment:
RUN pip install meerschaum && mrsm upgrade packages api -y

# or to install dependencies globally:
# RUN pip install meerschaum[api]

ENTRYPOINT ["python", "-m", "meerschaum"]

AWS¶

Run the bmeares/meerschaum Docker image with ECS or on an EC2 like you would any other Docker container. See the Docker section above if you want to build your own images.

Helm Chart (k8s)¶

If you are running Kubernetes, consider the Meerschaum Helm chart on Artifact Hub. It's configured to run the bmeares/meerschaum Docker Image for k8s environments.