Overview
Expand Hudu's AI integration capabilities to support other (inc) self-hosted AI models, addressing the current limitation where public-only model support restricts adoption due to privacy concerns. The system should provide a flexible framework that allows organisations to use their preferred AI solutions while maintaining security and compliance requirements.
Business Value
Enables adoption by privacy-conscious organisations by supporting private and self-hosted AI models
Provides flexibility for organisations to choose AI solutions that match their compliance requirements
Allows organisations to optimise their AI usage based on cost, performance, and privacy needs
Functional Requirements
Model Integration Framework
- Support for private model deployments:
Self-hosted model support
Custom API endpoint configuration
- Configuration options:
API endpoint management
Authentication settings
Model-specific parameters
- Fallback configuration:
Primary/secondary model selection
Automatic failover settings
Admin Interface
- Model configuration management:
Add/remove model endpoints
Configure authentication
Set usage parameters
- Access control:
Role-based permissions for model usage
Team/department access settings?
- Usage monitoring:
API call tracking
Error monitoring
Usage analytics
User Interface
Model selection dropdown (when multiple configured)
Clear indication of which model is being used
Consistent experience across different models
Real-time response streaming
Error handling and fallback notifications