Bring your own model

Some AI features run only a model you supply: Depth and Subject have none built in, and show Needs a model until you add one. The edge matte and Denoise come with a built-in method and can also run a model of yours. You add models in File › Preferences… › AI Models, and choose one for its feature there. With Auto chosen, a feature that has no built-in model uses the first model you added for it. DarkRay never copies the model: it reads it where it is, so a large model on a network share costs no disk space. You are responsible for the license of any model you add, including the data it was trained on.

What each feature accepts

Feature	Takes your models	A model takes, and gives
Depth (Depth Range masks, Atmosphere)	yes; needed, none is built in	one RGB image; gives a depth map, either nearness (near is bright) or distance (far is bright)
Subject (Select Subject masks)	yes, as an ONNX file; needed, none is built in	one RGB image; gives one map of where the subject is, as probabilities or as scores
Edge matte (the soft edge of every AI mask)	yes, as an ONNX file or a matting model folder; optional	a tile of the photo and its trimap (sure background, unsure, sure subject); gives the tile’s alpha
Denoise (RAW Process › Denoise)	yes, as an ONNX file; optional	a tile of the RAW’s sensor data before demosaic, its four colors as channels (R, G, G, B), scaled 0 to 1; gives the same tile back, cleaned
Content search	no
Sky, Smart Select, Select by name	no

Depth and Subject models look the same from outside (one image in, one map out), so when a model could be either, DarkRay says so: check For before you add it. A Subject model’s edge is then refined at full resolution, as every AI mask’s is.

A depth model’s direction doesn’t need to be known: when you add one, DarkRay runs it once on a reference photo and sets Invert direction for a model where far comes out bright. You can change it later with the pen icon.

What you can add

• An ONNX file (.onnx). A model larger than 2 GB comes as the .onnx plus a data file beside it; keep both in the same folder, and point DarkRay at the .onnx.
• A model folder as downloaded from a model hub: its config.json, which names a standard model type, beside its weights as .safetensors files.

DarkRay refuses, with the reason:

• a weights file on its own (.safetensors, .pt, .bin): it holds the learned numbers but not the model, so nothing can run it, and a .pt or .bin file can run code when it opens;
• a folder whose model type needs code of its own (its config.json has an auto_map), since DarkRay never runs code that comes with a model;
• a folder holding only .bin weights.

If a folder has a preprocessor_config.json, DarkRay reads the input size and color scaling from it. Without one it runs the model at 518 pixels with the usual ImageNet scaling.

Turning a model into ONNX

If the model’s publisher offers an ONNX download, use that. Otherwise, a model folder with .safetensors weights works as it is, as long as its type is a standard one. For any other model, convert it once with the tools made for it, on a computer with Python:

• a PyTorch model: torch.onnx.export, with the image input shaped (1, 3, height, width) and the height and width left dynamic;
• a model from a model hub: Hugging Face Optimum’s optimum-cli export onnx.

A depth or Subject model must take one image and give one map. An edge matte takes either one input of four channels (the image’s three, then the trimap) or two inputs (the image, then a one-channel trimap), the trimap as 0 for background, 0.5 for unsure and 1 for subject, and gives one alpha map. DarkRay runs it on tiles along the mask’s edge, so any tile size works; if a matte you added can’t be found, DarkRay uses its own until it’s back.

A denoise model takes one input shaped (1, 4, height, width), a RAW denoiser’s usual packed layout, and must accept any height and width: DarkRay runs it on tiles of the sensor data. A denoise model and an edge matte can look alike (four channels in), so check For before adding one. The Denoise choice under AI Models is what a photo’s RAW Process › Denoise runs unless the photo was prepared with another; a photo already prepared keeps its result until it is prepared again. DarkRay checks both when you add it and says what doesn’t fit.

Model packs

To hand a model to someone else ready to use, put a darkray-model.json in its folder. Whoever adds the folder then confirms without changing anything:

{
  "darkray_model": 1,
  "feature": "depth",
  "name": "Studio depth",
  "model": "model.onnx",
  "options": {"invert": false},
  "settings": {"size": 518, "image_mean": [0.485, 0.456, 0.406], "image_std": [0.229, 0.224, 0.225]}
}

• feature is depth, subject, matte or denoise.
• model is the .onnx file inside the folder; leave it out for a model folder with config.json and .safetensors weights.
• options.invert is the depth direction: true for a model where far comes out bright.
• settings is optional, and overrides the folder’s preprocessor_config.json: size, image_mean, image_std, do_normalize, ensure_multiple_of.

Setting up many computers

Add the models once, on one computer, with their files on a network share. Then Export Model List… under Your models saves them, and each feature’s choice of them, to a file. On every other computer, Import Model List… adds them and takes the same choices. Importing the same list again changes nothing. A model whose share is offline at the time still comes in, and shows File not found until the share is back.