Add support for integrating Hudini with self-hosted LLMs, allowing organizations to use their own on-premises or private cloud AI models without relying on third-party AI providers.
+1 from an MSP in Germany, and this is a real blocker for us.
Most of our Hudu KB is client documentation: network setups, system configurations and processes for businesses such as medical practices, care services and law firms. Under GDPR, every external AI provider that processes this data needs a data processing agreement, and sending it to providers outside the EU raises third-country transfer questions that many of our clients are not comfortable with.
For clients bound by professional secrecy, like doctors and lawyers, the bar is even higher. In practice this means we can't simply switch on a US cloud AI provider for the whole KB, even if the features are great.
A self-hosted option would solve this completely: the data never leaves our own infrastructure, and we can document that clearly for our clients and their auditors. It also removes the ongoing per-token costs for indexing and daily use.
We already run Semantic Search through our internal LLM gateway, and the results are very good, including for German content. So local models are clearly up to the task.
What's missing is a supported way to run Hudini chat on a self-hosted, OpenAI-compatible endpoint as well, ideally with separate provider settings for chat and Semantic Search.
Sapphire Pinwheel
•
Sep 2
Das wäre super, wir können einfach unsere Kundendaten nicht nach Übersee schicken :-(
Turquoise Lake
•
Jul 9
Yes please!!
Sapphire Geyser
•
Jul 8
This would be great for us as well as we have an internal Llama server OR at least allow the endpoint URL to be changed as Granite suggested as then we can redirect to our own router.
Bronze Narwhal
•
May 18
Would love this! Should be pretty trivial to allow Llama.
Coffee Asteroid
•
Dec 24, 2025
Allowing to override the endpoint URL for OpenAI provider will help greatly.
Log in to comment and vote
Comments6
Bronze Narwhal
Sep 24
+1 from an MSP in Germany, and this is a real blocker for us.
Most of our Hudu KB is client documentation: network setups, system configurations and processes for businesses such as medical practices, care services and law firms. Under GDPR, every external AI provider that processes this data needs a data processing agreement, and sending it to providers outside the EU raises third-country transfer questions that many of our clients are not comfortable with.
For clients bound by professional secrecy, like doctors and lawyers, the bar is even higher. In practice this means we can't simply switch on a US cloud AI provider for the whole KB, even if the features are great.
A self-hosted option would solve this completely: the data never leaves our own infrastructure, and we can document that clearly for our clients and their auditors. It also removes the ongoing per-token costs for indexing and daily use.
We already run Semantic Search through our internal LLM gateway, and the results are very good, including for German content. So local models are clearly up to the task.
What's missing is a supported way to run Hudini chat on a self-hosted, OpenAI-compatible endpoint as well, ideally with separate provider settings for chat and Semantic Search.
Sapphire Pinwheel
Sep 2
Das wäre super, wir können einfach unsere Kundendaten nicht nach Übersee schicken :-(
Turquoise Lake
Jul 9
Yes please!!
Sapphire Geyser
Jul 8
This would be great for us as well as we have an internal Llama server OR at least allow the endpoint URL to be changed as Granite suggested as then we can redirect to our own router.
Bronze Narwhal
May 18
Would love this! Should be pretty trivial to allow Llama.
Coffee Asteroid
Dec 24, 2025
Allowing to override the endpoint URL for OpenAI provider will help greatly.