Overview

Expand Hudu's AI integration capabilities to support other (inc) self-hosted AI models, addressing the current limitation where public-only model support restricts adoption due to privacy concerns. The system should provide a flexible framework that allows organisations to use their preferred AI solutions while maintaining security and compliance requirements.

Business Value

• Enables adoption by privacy-conscious organisations by supporting private and self-hosted AI models

• Provides flexibility for organisations to choose AI solutions that match their compliance requirements

• Allows organisations to optimise their AI usage based on cost, performance, and privacy needs

Functional Requirements

Model Integration Framework

1. Support for private model deployments:

• Self-hosted model support

• Custom API endpoint configuration

1. Configuration options:

• API endpoint management

• Authentication settings

• Model-specific parameters

1. Fallback configuration:

• Primary/secondary model selection

• Automatic failover settings

Admin Interface

1. Model configuration management:

• Add/remove model endpoints

• Configure authentication

• Set usage parameters

1. Access control:

• Role-based permissions for model usage

• Team/department access settings?

1. Usage monitoring:

• API call tracking

• Error monitoring

• Usage analytics

User Interface

1. Model selection dropdown (when multiple configured)

2. Clear indication of which model is being used

3. Consistent experience across different models

4. Real-time response streaming

5. Error handling and fallback notifications