[Feature Request] Support for bundled LoRA/embedding files

[efhosci]

Onetrainer recently added an option to simultaneously train embeddings and LoRAs (I think this is called pivotal tuning?) and bundle them into a single safetensor file. I believe the format for this was actually created a while ago (see the first link). I experimented a bit with training using this method and got good results, and it seems like it can be an effective method for multi-concept LoRAs.

In A1111/web UI these work as expected, just include "lora:loraname:1 embeddingname" in the prompt. However I can't find a way to activate the embedding in a bundled file in Stable Swarm or the ComfyUI backend, only the LoRA. Separate embedding files work when loaded with embed:embeddingname but bundled files are a lot more convenient for large multi-concept LoRAs. Would support for this be something that Stable Swarm can implement, or would this need to be handled by ComfyUI?

Ideally, bundled embeddings should only be "accessible" when the associated LoRA is active, which is how web UI appears to work from my testing. Or there may be a better way to avoid conflicts between identically named embeddings in multiple bundle files.

Some links for reference:
Original web UI pull request for the bundle format
OneTrainer pull request
Relevant section of the OneTrainer code

[mcmonkey4eva]

I think it'd make more sense to request this as a Comfy feature rather than Swarm? https://github.com/comfyanonymous/ComfyUI

Comfy is the core backend/engine, swarm is the friendly frontend wrapper