I’ve been exploring MariaDB 11.8’s new vector search capabilities for building AI-driven applications, particularly with local LLMs for retrieval-augmented generation (RAG) of fully private data that never leaves the computer. I’m curious about how others in the community are leveraging these features in their projects.
I’m especially interested in using it with local LLMs (like Llama or Mistral) to keep data on-premise and avoid cloud-based API costs or security concerns.
Does anyone have experiences to share, in particular what LLMs are you using when generating embeddings to store in MariaDB?
I used all-minilm which worked well for documents. But it doesn’t work for images (or didn’t when I was using it last time). Although I did in combination with N8N and Qdrant.
Have a look at Ollams embeddings. Easy to set up and the models are much smaller than a typical LLM.
