Skip to content

Ollama Integration#

The LIT Platform seamlessly integrates with Ollama, a popular tool for running large language models locally. This integration allows you to leverage Ollama's efficient model management while enjoying all the features of the LIT Platform.

Benefits of Ollama Integration#

  • Model Management: Easily download, update, and manage models
  • Efficient Serving: Optimized inference performance
  • Simplified Setup: One-click configuration
  • Wide Model Support: Access to Ollama's growing library of models

Setup Instructions#

Prerequisites#

  • Ollama installed on your system or a remote server
  • Network connectivity between LIT Platform and Ollama

Configuration Steps#

  1. Navigate to Settings > Integrations > Ollama
  2. Enter the Ollama server URL (default: http://localhost:11434)
  3. Click "Test Connection" to verify connectivity
  4. Select "Enable Ollama Integration"
  5. Save settings

Using Ollama Models in LIT#

Once configured, Ollama-served models will appear alongside native LIT models in:

  • Model selection dropdowns
  • Workflow canvas components
  • Chat interfaces
  • Python scripts

Model Management#

The LIT Platform provides an interface to manage your Ollama models:

  1. Navigate to Models > Ollama Models
  2. View available models
  3. Download new models
  4. Update existing models
  5. View model details and parameters

Advanced Configuration#

Custom Model Tags#

You can create custom model tags in Ollama and use them in LIT:

ollama pull llama3:8b
ollama pull llama3:8b-instruct

These tagged models will be available in the LIT Platform with their respective tags.

Server Configuration#

For production environments, consider:

  • Configuring Ollama on a dedicated GPU server
  • Setting up proper authentication and network security
  • Adjusting Ollama's resource limits in the configuration file

Troubleshooting#

If you encounter issues with the Ollama integration:

  1. Verify Ollama is running (ollama ps)
  2. Check network connectivity between LIT and Ollama
  3. Ensure models are properly downloaded in Ollama
  4. Check Ollama logs for specific errors

For detailed information on Ollama itself, refer to the official Ollama documentation.