# Deploying to production

MpxjEx runs [MPXJ](https://www.mpxj.org), a Java library, in a JVM next to
your Elixir application. This guide covers what that means in production and
how to package it, using Docker as the example.

A complete, working setup lives in
[`examples/mpxj_service`](https://github.com/rocket4ce/mpxj_ex/tree/main/examples/mpxj_service):
a small HTTP API with a production `Dockerfile` and `docker-compose.yml`. Its
README explains how to run it.

## What mpxj_ex needs at runtime

| Requirement | Why |
| ----------- | --- |
| A Java runtime, version 8 or newer | MPXJ is a Java library. A headless JRE is enough; Java 21 LTS is a good default. |
| MPXJ's jars (about 30 MB) | Downloaded once, see [Installing MPXJ](#installing-mpxj-at-build-time). |
| A writable temporary directory | MPXJ writes each schedule as JSON to `System.tmp_dir!/0` before it's decoded. The JVM also uses it, and some readers extract a native library there, so it must allow executing files. |
| Memory for the JVM | MPXJ holds the whole schedule in memory while reading it. |

Nothing else: the Java bridge (`priv/mpxj_ex_bridge.jar`) ships with the
package and is included in your release automatically.

## Checklist

1. Install MPXJ at build time and turn off `auto_install`.
2. Put a Java runtime in the image that runs your release.
3. Start an `MpxjEx.Server` in your supervision tree.
4. Size the JVM heap, the pool and the container memory together.
5. Limit upload sizes and set timeouts.
6. Add a health check.

## Installing MPXJ at build time

By default MpxjEx downloads MPXJ the first time it's used. That's handy in
development, but in production it means a 70 MB download from GitHub when
the first request arrives, and a failure if GitHub isn't reachable.

Download it while building instead:

```sh
mix mpxj_ex.install --path /opt/mpxj
```

The task downloads the MPXJ distribution, verifies its SHA-256 checksum and
keeps only the jars. Then tell MpxjEx where they are and never to download:

```elixir
# config/prod.exs (or config/config.exs inside `if config_env() == :prod`)
config :mpxj_ex, auto_install: false
```

```sh
# runtime environment
MPXJ_EX_JAR_DIR=/opt/mpxj
```

You can also set `config :mpxj_ex, jar_dir: "/opt/mpxj"` instead of the
environment variable. With `auto_install: false`, a missing install fails
with a `:not_installed` error rather than downloading.

### Choosing the MPXJ version

MpxjEx pins a tested MPXJ version and knows its checksum. To use another
one, set it in `config/config.exs` so both the install task and the
application see it:

```elixir
config :mpxj_ex, version: "16.10.0"
```

For a version MpxjEx doesn't know, the download isn't verified and a warning
is logged. Run your test suite against it before deploying.

### Using an MPXJ you already have

Point `jar_dir` (or `MPXJ_EX_JAR_DIR`) at a directory that contains
`mpxj.jar` and all of its dependencies side by side (the contents of the
distribution's `lib` directory, flattened), and set `auto_install: false`.

## The Java runtime

Install a JRE in the image that runs the release (not only in the build
image):

| Base image | Package |
| ---------- | ------- |
| Debian trixie | `openjdk-21-jre-headless` |
| Debian bookworm | `openjdk-17-jre-headless` |
| Ubuntu 24.04 | `openjdk-21-jre-headless` |
| Alpine | `openjdk21-jre-headless` |

The example image is built and tested on Debian trixie.

Alternatively copy a JRE from the official image, for example
`COPY --from=eclipse-temurin:21-jre /opt/java/openjdk /opt/java/openjdk`, and
set `JAVA_HOME=/opt/java/openjdk`.

MpxjEx looks for java in this order: the `:java` option, `config :mpxj_ex,
java: ...`, `$JAVA_HOME/bin/java`, then `java` on the `PATH`.

## Running MPXJ: one JVM per call, or a server

Every `MpxjEx.read/2` or `MpxjEx.convert/3` call without a server starts a
JVM, which costs about half a second. That's fine for a nightly job, but for
anything serving users, run an `MpxjEx.Server`, which keeps one JVM up and
handles several files at once:

```elixir
# lib/my_app/application.ex
children = [
  {MpxjEx.Server, name: MpxjEx.Server, pool_size: 4, max_memory: "1g"},
  # ... your other children, such as your endpoint
]
```

```elixir
# config/config.exs: send every call to it
config :mpxj_ex, server: MpxjEx.Server
```

Measured on the test fixtures: about 565 ms per file with a JVM per call,
about 33 ms per file through a warm server, and less when files are read
concurrently.

If the JVM dies (killed by the out-of-memory killer, for instance), pending
callers get `{:error, %MpxjEx.Error{reason: :jvm_crashed}}`, the server
stops, and its supervisor starts a new JVM. If it keeps crashing faster than
the supervisor's restart intensity allows, your application stops, and the
container's restart policy takes over.

You can run several servers under different names, for example a separate
one with a larger heap for bulk imports.

## Sizing

* **`pool_size`**: how many files the JVM processes at the same time.
  Parsing is CPU bound, so the number of cores available to the container is
  a good start (`System.schedulers_online/0`, the default). Extra requests
  wait in the JVM's queue.
* **`max_memory`**: the JVM heap (`-Xmx`). Small and medium schedules are
  fine with `1g`. Large MPP or XER files (tens of MB) with timephased data
  may need `2g`-`4g`. A file that doesn't fit fails with `:out_of_memory`.
* **Container memory limit**: the heap, plus about 250 MB of JVM overhead
  (metaspace, threads, code cache), plus the BEAM (about 100 MB), plus
  anything you keep in a `tmpfs` (it counts as memory). With a 1 GB heap,
  1.5 GB to 2 GB is reasonable. If the limit is lower than the heap can
  grow, the kernel kills the JVM and you'll see `:jvm_crashed` with exit
  status 137.
* **`:timeout`** (per call, default 60 seconds): how long a caller waits for
  the server. MPXJ can't be interrupted, so a request that timed out keeps
  running in the JVM until it finishes (the server then removes its
  temporary output). Timeouts protect callers, not capacity: size
  `pool_size` for your slowest files.

An idle container running the example service uses about 225 MB.

## Docker

The example's
[`Dockerfile`](https://github.com/rocket4ce/mpxj_ex/blob/main/examples/mpxj_service/Dockerfile)
is a standard two stage Elixir release build. The lines specific to mpxj_ex
are:

```dockerfile
# Build stage: ca-certificates to download MPXJ over HTTPS ...
RUN apt-get update -y \
    && apt-get install -y --no-install-recommends ca-certificates \
    && rm -rf /var/lib/apt/lists/*

# ... and install it after fetching dependencies, so the layer is cached
RUN mix deps.get --only prod
RUN mix deps.compile
RUN mix mpxj_ex.install --path /opt/mpxj

# Runtime stage: a Java runtime ...
RUN apt-get update -y \
    && apt-get install -y --no-install-recommends \
       openjdk-21-jre-headless libstdc++6 openssl libncurses6 ca-certificates \
    && rm -rf /var/lib/apt/lists/*

# ... the jars, and where to find them
ENV MPXJ_EX_JAR_DIR=/opt/mpxj
COPY --from=build /opt/mpxj /opt/mpxj
```

The example builds mpxj_ex from this repository, so its build context is
the repository root and it copies the library's sources. In your own app,
depend on mpxj_ex from Hex, build from your app's root and skip that step.

The resulting image is about 550 MB, most of it the JRE.

### Docker Compose

The example's
[`docker-compose.yml`](https://github.com/rocket4ce/mpxj_ex/blob/main/examples/mpxj_service/docker-compose.yml)
runs the service hardened for production:

```yaml
services:
  mpxj_service:
    build:
      context: ../..
      dockerfile: examples/mpxj_service/Dockerfile
    ports:
      - "4000:4000"
    environment:
      MPXJ_POOL_SIZE: "4"
      MPXJ_MAX_MEMORY: "1g"
      MPXJ_TIMEOUT_MS: "60000"
      MAX_UPLOAD_MB: "50"
    read_only: true
    tmpfs:
      - /tmp:exec,mode=1777,size=512m
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    deploy:
      resources:
        limits:
          cpus: "2"
          memory: 1600m
    restart: unless-stopped
```

Notes:

* `read_only: true` works because every runtime write goes to `/tmp`: set
  `RELEASE_TMP=/tmp` (the release writes its runtime configuration there)
  and `ERL_CRASH_DUMP=/tmp/erl_crash.dump`.
* `/tmp` is mounted with `exec`: Docker mounts `tmpfs` with `noexec` by
  default, which stops MPXJ readers that load sqlite-jdbc's native library.
* The image runs as an unprivileged `app` user.

Build and run it from the example directory:

```sh
cd examples/mpxj_service
docker compose up --build -d
docker compose ps        # wait for "healthy"
curl -F file=@plan.mpp http://localhost:4000/projects
```

## Handling untrusted files

Schedule files uploaded by users are untrusted input parsed by complex
binary and XML readers. Contain the work:

* Limit upload sizes (`Plug.Parsers`' `:length` option).
* Set a `:timeout` and a heap size (`:max_memory`), and a memory limit on
  the container.
* Run as an unprivileged user with a read-only filesystem,
  `no-new-privileges` and no capabilities, as the example does.
* Don't send `MpxjEx.Error` messages to clients verbatim: they can contain
  server paths and Java internals. Log them, and answer with a description
  based on the error's `reason` (see the example's router).

Password protected files fail with `:password_protected`; MpxjEx doesn't
read them.

## Health checks and logs

The example's `GET /health` answers 200 while `MpxjEx.Server` is running,
and the image's `HEALTHCHECK` calls it. For a deeper check, read a small
schedule bundled with your app and check the result.

When the JVM exits, `MpxjEx.Server` logs the termination with the JVM's exit
status. Anything the JVM writes to stderr goes to your application's stderr,
so it ends up in the container logs.

## Configuration reference

Application environment (`config :mpxj_ex, ...`):

| Key | Default | Description |
| --- | ------- | ----------- |
| `:version` | `"16.10.0"` | MPXJ version to install and use |
| `:jar_dir` | `$MPXJ_EX_JAR_DIR`, else `<user cache>/mpxj_ex/<version>` | Directory containing `mpxj.jar` and its dependencies |
| `:auto_install` | `true` | Download MPXJ on first use when it's missing |
| `:server` | `nil` | `MpxjEx.Server` used when a call doesn't pass `:server` |
| `:java` | `$JAVA_HOME/bin/java`, else `java` on the `PATH` | Java executable |
| `:max_memory` | JVM default | Heap size for JVMs started per call, such as `"1g"` |
| `:jvm_args` | `[]` | Extra JVM arguments for JVMs started per call |

`MpxjEx.Server` options: `:name`, `:pool_size`, `:max_memory`, `:java`,
`:jvm_args`.

Per call options for `MpxjEx.read/2`, `MpxjEx.read_all/2` and
`MpxjEx.convert/3`: `:server`, `:timeout`, `:max_memory`, `:java`,
`:jvm_args`, plus `:time_units`, `:timephased` and `:raw` for reads.

## Upgrading MPXJ

1. Set `config :mpxj_ex, version: "x.y.z"` (or upgrade mpxj_ex, which may
   bump its pinned version).
2. Run your tests: they download the new version into the user cache.
3. Rebuild the image: the `mix mpxj_ex.install` layer is rebuilt because
   the configuration changed.

## Troubleshooting

| Symptom | Cause and fix |
| ------- | ------------- |
| `:java_not_found` | No Java in the runtime image. Install a JRE or set `JAVA_HOME`. |
| `:not_installed` | `auto_install` is off and `jar_dir` has no `mpxj.jar`. Check `MPXJ_EX_JAR_DIR` and that the image copies the jars. |
| `:install_failed` during the build | GitHub isn't reachable, a proxy blocks it, or `ca-certificates` is missing from the build image. |
| `:out_of_memory` | The schedule doesn't fit in the heap. Raise `max_memory`, and the container limit with it. |
| `:jvm_crashed` with exit status 137 | The kernel killed the JVM: the container limit is too low for the heap. |
| `:timeout` | The file takes longer than `:timeout`, or every pool slot is busy. Raise the timeout or `pool_size`. |
| `UnsatisfiedLinkError` mentioning sqlite in the logs | `/tmp` is mounted `noexec`. |
| `Read-only file system` when the release starts | Set `RELEASE_TMP` to a writable directory. |
| The first requests after a deploy are slower | The JVM is warming up (JIT compilation); they speed up after a few files. |