BrainFog
An AI powered second brain. Capture, summarieze, retrieve and process all your personal data on the go.
Problem Statement
We are using calendars, todo apps, contact apps, budget apps and so on because it is harder for us to track things. However, at its core, the service all these apps provide is the same. They remember something allowing us to recall later, either processed or in raw form. With the developments in AI, we are able to now eliminate intermediary UIs and can directly use natural language. Hence, AI might be able to unify all of those tracking apps. If such a product is possible, it will improve efficiency massively.
Project Description
BrainFog architecture is as follows. Frontend is developed using NextJS. Backend is developed using Java with Spring Boot framework. However since python has a rich ecosystem for AI applications, AI engine is developed using python, with LangChain. There's a RAG pipeline which uses a locally installed embedding model (IBM Granite embeddings) to generate embeddings for user queries. Then by using FAISS, answers to user queries will be chosen.
Project's core engine can be accessed as a CLI tool to test out. I developed this CLI tool to test my approach and thereby develop further. Hence CLI application acts like a test bed. This portion of the project is open source. Can be accessed here.
- Automatic notifications
- Automatic calendar entries
- Easy content retrieval
Those are some of the advantages of this application.
Challenges & Learning Experience
I thought a lot about data privacy and initially wanted to make this application E2E encrypted. However to do that, all the LLM features should work on device. Hence I tried out different small language models and experimented with them. This gave me an understanding of limitations of number of small language models out there. However, due to limitations, eventually had to gave up on E2E encrypted requirement. To reduce cost, I resolved to use a locally installed model for embeddings for the RAG pipeline. This gave me the experience of implementing efficient RAG pipelines. Overall, while this application gave me a solid learning experience in the following areas,
- Building LLM agents
- Building RAG pipelines
- Tracking conversation memory in different stages of the application
- Key limitations when building privacy first applications