Deployment prefix in routes

The VH_DEFAULT_PREFIX environment variable contains the default prefix the application is served under; i.e. something like organization/project/deployment/version/endpoint. This will not take into account any deployment aliases.

The WSGI/HTTP application will also receive a X-VH-Prefix HTTP header containing the root path the application is served under regardless of whether it’s being served via the default version moniker or an alias; i.e. something like organization/project/deployment/versionalias/endpoint.

If you use path-based routing (as opposed to e.g. RPC style) in your deployment code, you may need to use this header or variable to properly route your requests. (Another option is to simply allow any prefix for all of your app’s routes, e.g. ^.*/foo/$).

Handling deployment prefixes in FastAPI

In a FastAPI application, you can use a middleware function that rewrites the internal path based on the prefix environment variable or headers.

Feel free to copy-paste the middleware function into your app.

import os
from fastapi import FastAPI
from starlette.requests import Request

app = FastAPI()

@app.middleware("http")
async def process_valohai_prefix(request: Request, call_next):
    path = request.scope["path"]
    for prefix in (
        request.headers.get("X-VH-Prefix"),
        os.environ.get("VH_DEFAULT_PREFIX"),
    ):
        if not prefix:  # Could have no header or no envvar, so skip
            continue
        if path.startswith(prefix):  # If the path starts with this prefix,
            # ... tell FastAPI (0.56.0+) that that is the prefix we're mounted under...
            request.scope["root_path"] = prefix
            # ... then strip the prefix out as far as FastAPI is concerned.
            request.scope["path"] = "/" + path[len(prefix) :].lstrip("/")
            break
    return await call_next(request)

@app.get("/predict")
def predict(name: str):
    return {"predicted_first_letter": name[:1].lower()}

Handling deployment prefixes in Flask

Below you'll find a simple Flask app and a generic WSGI middleware that rewrites the internal request path based on the prefix environment variable or headers.

import os
from functools import wraps
from flask import Flask, request, jsonify
from werkzeug import run_simple

def handle_valohai_prefix(environ):
    path = environ["PATH_INFO"]
    for prefix in (
        environ.get("HTTP_X_VH_PREFIX"),
        os.environ.get("VH_DEFAULT_PREFIX"),
    ):
        if not prefix:  # Could have no header or no envvar, so skip
            continue
        if path.startswith(prefix):  # If the path starts with this prefix,
            # ... then strip the prefix out as far as WSGI is concerned.
            environ["PATH_INFO"] = "/" + path[len(prefix) :].lstrip("/")
            break

def manage_prefixes(app):
    """
    Decorator to apply Valohai prefix management to a WSGI app callable.
    """
    @wraps(app)
    def prefix_managed_app(environ, start_response):
        handle_valohai_prefix(environ)
        return app(environ, start_response)

    return prefix_managed_app

app = Flask(__name__)

@app.route("/predict")
def predict():
    name = request.args.get("name", "")
    return jsonify({"predicted_first_letter": name[:1].lower()})

app = manage_prefixes(app)

if __name__ == "__main__":
    run_simple("0", 8000, use_reloader=True, application=app)