Self Hosting My Way - Immich
4th post in a series about self-hosting
A popular problem privacy-minded self-hosters are trying to solve is photo management and automatic backup from mobile devices. The big players, specifically Google and Apple dominate this field, with their main advantage being full control over the respective mobile operating systems. I haven’t used Apple devices after iPhone 5s, but on Android you just sign in to you Google account to be able to use the Play store, and Google Photos is already there, with all your old photos available.
It’s hard to beat the experience, and having root-level access to the
OS makes automatic uploads seamless and efficient. And according to
Google, “they don’t use your photos for <place your nefarious intent here>
”.
Immich
Image borrowed from Immich website
Immich is a really good alternative, when the promises of privacy from the big companies just don’t sound right. There are too many features to list here, but I will give a special accomodation for their face recognition feature. It was quite eerie to see my nowadays 5-year-old son’s pictures linked to those of him on his first day. This all happened on a 6th gen i7, without breaking a sweat, and fast. It also made me realize what a disservice it is to our kids to put their pictures online, even when they are supposedly private. Please don’t post pictures of your kids to social media.
Back on topic, Immich is really, really good at what it’s made for. But the silver lining has a cloud attached to it, it’s also one of, if not the most laboursome service I’ve self-hosted. This is not that it’s buggy or breaks often, but it’s unstable. To be fair, this is clearly stated by the devs, but their update pace is fast, and many of the updates are breaking ones. They also don’t put much attention in backwards compatibility between the server and the mobile app. The server config needing changes is something one can live with, but making sure that the family’s apps don’t update before you have the time to upgrade the server is not fun.
That said, I have opted to still rely on Nextcloud for the background photo upload, and the photos from Nextcloud are added as an external library in Immich. The experience this way is quite nice, and since no one is relying on the mobile apps, it won’t matter if I don’t update the server right after a new version is released. Most of the updates are not breaking ones!.
Summary
Here's the short version:
- Create directories:
export BASE_PATH="/path/to/immich" # The parent directory for YAML and host volumes
mkdir -p "$BASE_PATH"/{data,model-cache,redis,quadlet}
- Under
$BASE_PATH/quadlet/
- Create immich-configMap.yaml
- Change DB_PASSWORD, optionally other values according to the Immich documentation.
- Don’t edit DB_HOST, DB_PORT, REDIS_PORT or REDIS_PASSWORD
- Download immich-pod.yaml
- Replace
/path/to/immich
with your chosen $BASE_PATH in volumes section:sed -i "s|/path/to/immich|$BASE_PATH|g" immich-pod.yaml
- Create quadlet file (pod-immich.kube)
- Change
PublishPort
if 3001 doesn’t suit you, with e.g.PublishPort=8080:3001
- Symlink files for Quadlet:
mkdir -p ~/.config/containers/systemd/immich cd ~/.config/containers/systemd/immich cp -s "$BASE_PATH"/quadlet/* .
- Start the service
Requirements
Volumes:
The base server requires two persistent volumes, one for the actual photos and another for the ML model cache. It also uses a PostgreSQL database, so Postgres will need a volume for its data too. The web UI uses Redis, and the experience is nices if its data has a volume too, so logins survive server restarts and the like.
In total, 4 volumes are necessary:
- data
- model-cache
- Postgresql
- Redis.
I also mount volumes from my Nextcloud as read-only to the mix, to be able to add them as external libraries. I’ve opted to using a named volume for the PostgreSQL container, and bind mounts for the others.
Networking:
As Immich is a more traditional web app, it will work fine without any special network configuration. By default Immich exposes only TCP port 3001.
Containers:
Before version 1.106.1 (June 2024), Immich used a separate
microservices
container, which executed the background tasks while the mainserver
container handled the API (note: I haven’t investigated if this is the actual division, please excuse if this is incorrect). Since 1.106.1, the microservices were integrated into the mainserver
container, so the required containers are nowadays the following:immich-server
immich-machine-learning
- PostgreSQL:
pgvecto-rs
redis
I however liked the separation of the API and other microservices, i.e. for better log separation. The separate containers can be achieved by running two instances of the
immich-server
container with different environment variables.To run the single combined
server + microservices
container, remove themicroservices
container from thecontainers
section, and remove theIMMICH_WORKERS_INCLUDE
environment variable from the mainserver
container definition
Pod definition
As there’s a bit more to go through compared to HomeAssistant, I’ll not go over the spec with as much detail. The principle is exactly the same as with Home Assistant.
Configuration (ConfigMap, a.k.a. .env
in Docker compse)
|
|
The ConfigMap defines key-value variables which are set as environment variables in the containers, similar to what a .env
file
does when using Docker compose. All the variables in
the Immich documentation
can be set here. I’ve included some examples commented here, but those are not necessary for the (almost) standard deployment used
here.
Don’t change the values of
DB_HOSTNAME, DB_PORT, REDIS_HOST
orREDIS_PORT
, if you use the dedicated instances in the same pod according to this post
Pod
You can see the full definition here: immich.yaml
Volumes:
|
|
Like described in the
2nd post in the series, all the necessary files and volumes
live under a single directory (ZFS dataset), in these examples it’s /path/to/immich
. We define the 4 volumes required:
- immich-data-host ->
/path/to/immich/data
- immich-model-cache-host ->
/path/to/immich/model-cache
- immich-redis-host ->
/path/to/immich/redis
- immich-psql (named volume)
Additionally, I included an example on how to define the read-only Nextcloud user directory for use as external library in Immich.
/path/to/nextcloud/app/data/nc-user/files
is the path on the host, which is mounted to /var/www/nextcloud/data/nc-user/files
in
the Nextcloud container. This is all the files from a Nextcloud user named “nc-user”.
More on Nextcloud later on a separate post, probably.
Immich-server
container
|
|
The server
container has the environment variable IMMICH_WORKERS_INCLUDE=api
set, to only serve the Immich API. The configuration
from immich-config
is mapped as environment variables for the container. The volumes immich-data-host
and optional Nextcloud user
data directories are mounted inside the container. The path for the Immich data must be /usr/src/app/upload
, the Nextcloud
directories can be anything, as long as they don’t overwrite anything in the immich-server
image.
To use the default deployment model in the Immich upstream documentation, remove the highlighted lines
Immich-microservices
container (optional)
|
|
The microservices
container is defined the same way as the server
container, but we use the environment variable
IMMICH_WORKERS_EXCLUDE=api
, so this container will not serve everything except the API.
Remove this section entirely if you want to run using the default single-container deployment.
Immich-machine-learning
container
|
|
The machine-learning
container runs the ML tasks, like face recognition. immich-config
is available as environment
variables here too. In addition to the data volume and the possible external directories, the model-cache
volume
should be mounted under /cache
for this container.
PostgreSQL
Immich utlizes the
pgvecto.rs that is not available in the official PostgreSQL containers. The image used is
docker.io/tensorchord/pgvecto-rs
. The container needs some special arguments to start.
|
|
Some configuration values from the immich-config
configMap are set as environment values for this container (POSTGRES_USER
, POSTGRES_PASSWORD
, POSTGRES_DB
).
These are mapped from immich-config
to avoid the need to define them twice, as Immich itself uses different variable names for the same values.
Additionally, there’s POSTGRES_INITDB_ARGS=´--data-checksums'
. By default, that container starts with the arguments
postgres -c shared_preload_libraries=vectors.so -c "search_path=\"$user\", public, vectors," -c logging_collector=on
,
but Immich operates best with some additional start arguments ("-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"
), so the args
array
is redefined with the additional ones.
This is converted for Podman from the default docker-compose.yaml in the official Immich deployment docs.
Redis
|
|
The redis container has nothing special, we start with the arguments redis-server --save 60 1 --loglevel warning
. Since the TCP port is only accessible from inside
the pod, defining authentication is optional. For the official Redis container, this could be done similarly to the PostgreSQL container using environment variables.
SELinux
At the time of writing,
podman kube play
won’t change the SELinux context automatically for directories that are bind mounted inside the containers. For the container to be able to access the host directory, set the context manually:chcon -t container_file_t -R /path/to/immich/{data,model-cache,redis}
I’m using the recursive
-R
flag here, even though at this stage the directory is empty. This command, doesn’t need root privileges, as all the things described in this post should be done as an unprivileged user since we’re using rootless containers.The same applies to possible mounted external library directories.
Quadlet
Like with Home Assistant, Quadlet will handle starting and keeping the service running via systemd.
[Install]
WantedBy=default.target
[Kube]
Yaml=immich.yaml
PublishPort=3001:3001
ConfigMap=immich-configMap.yaml
The Quadlet file is very simple. We tell Quadlet to run a pod using Kube YAML file named immich.yaml
, and port 3001 is
exposed from the pod. In addition, Quadlet should use the ConfigMap immich-configMap.yaml
. If port 3001 is already reserved on the host,
or you just wish to use a different port, change the first number, similar to podman/docker run -p ...
.
Note here that this is the HTTP port, and if you plan to expose the service, a separate reverse proxy is required to provide TLS. I may be covering NGINX configurations for the services in this series later. Most, if not all projects have example configurations for the most popular reverse proxies in their documentation. Immich has one here.
The file should be named after the systemd service we want to create. In my case I want the service to be called pod-immich.service
,
so the filename will be pod-immich.kube
. Place the file, along with the pod YAML and configMap YAML under $HOME/.config/containers/systemd
,
or a subfolder under it.
I store the Kube YAMLs along with Quadlet files under the parent ZFS dataset, in a quadlet
folder, from which I symlink them
to the correct place:
mkdir -p ~/.config/containers/systemd/immich
cd ~/.config/containers/systemd/immich
cp -s /path/to/immich/quadlet/* .
Again, this is so backups will have everything necessary included, and there’s no duplicates that may not be in sync.
With these files, the folder layout looks like the following:
# /path/to/immich
.
├── data
├── model-cache
├── redis
└── quadlet
├── immich.yaml
├── immich-configMap.yaml
└── pod-immich.kube
Start the service
It is a good idea to pre-pull the images manually to avoid timeouts
Reload systemd and start the service, navigate to http://<your-server-ip>:3001
and you should see the Immich welcome page!
systemctl --user daemon-reload
systemctl --user start pod-immich.service
Immich welcome page (finnish locale)
Next up, I’m planning to cover Nextcloud