deen.ai

Overview

deen.ai can be summarised as "Muslim ChatGPT". It's a one-shot AI assistant that answers your questions from the perspective of an Islamic scholar. The scope was deliberately small, as I wanted to do this in a short amount of time to try out a few different things. I bought the domain and build everything on the weekend of the 7th of January 2023. The code is on GitHub. The reason I picked this vertical / use case, is because:

Architecture

I set out wanting to hone the workflow for creating products like this, as well as experiment with a new stack. On the front-end, I wanted to try using htmx, but it turned out to not be very well suited to this, so I ended up going pretty vanilla for this. I wanted to try building a specific kind of journey that goes like this:

  1. Land on a page with one very clear action that creates value
  2. That action also creates an easily shareable artefact, that has a domain watermark on it, which becomes the main growth driver
  3. People can also sign up to a newsletter to keep up to date about future projects, such that when those launch, growth is not from scratch
  4. Or, they can repeat the journey once more

The main very useful bits of code here (that I plan to use in some other projects -- already have ideas!) are:

Besides that, I streamlined my process for different bits and pieces like UI, logo, social media preview cards, favicons, loading states with an SVG spinner, etc.

More important is the back-end however. I've been using Node-RED more and more to create little APIs, and it's super convenient and easy to build things in. Currently, this is what the back-end to deen.ai looks like:

I even have some nodes in there just for testing (e.g. sending a test prompt, or bypassing OpenAI and instead waiting a second and responding "Done") which sped up development. The heavy lifting is of course done by GPT-3, which I would like to fine-tune based on the training data I collect from this first release. Besides that, I mainly transform prompts and answers, and append these with a timestamp to a CSV file.

On the bottom, we have the flagging flow. If a user flags a result, it gets logged to a separate CSV, so I can use this later to improve the model.

Challenges

The answers you get back are sometimes quite off. The worst thing that can happen is that the answer is just completely made up, including the citations (I try to force it to always give primary source citations). I think I could introduce some steps later in the pipeline to verify these against quran.com and sunnah.com, but my friends who've been testing this have found cases where a hadith talks about photography, it insists there are only 3 fardh prayers, or it says something not quite right, then backs it up with a completely unrelated hadith. Like many other GPT-3-based products, it's just not quite there yet, and that last part makes all the difference. Maybe GPT-4?

The other questions is: who's actually asking questions? Is it the curious non-Muslim asking "easy" exploratory questions? Is it the Muslim asking very technical questions that depend on the madhhab? Is it the anti-Muslim looking to get an out-of-context quote? Human Islamic scholars tailor their answers to the person who's asking the question, but deen.ai can't do that currently. Similarly, there may be several correct answers to a question (cf madhhab) and indeed you might get different answers to the same question if you ask it again. I don't want to be responsible for misinformation.

So overall, I think these problems might be intractable for now, so I won't present this as anything more than an experiment, and try to improve it to reach that ideal accuracy. Maybe this could spawn other ideas in the same space (I already have a bunch) or help build useful datasets!

Amar Memoranda > deen.ai

Log

ChatGPT as an Islamic scholar

Last weekend I built a small AI product: https://deen.ai. Over the course of the week I've been gathering feedback from friends and family (Muslim and non-Muslim). In the process I learned a bunch and made things that will be quite useful for future projects too. More info here!

Jan 13, 2023 • #projects #ai