update README, add better docs

README.md

@@ -24,9 +24,9 @@ RAGTheDocs is an open-source library that allows you to deploy retrieval augment
 ![image](https://github.com/jerpint/buster/assets/18450628/0c89038c-c3af-4c1f-9d3b-9b4d83db4910)
 
 3) Set your environment variables:
-* OPENAI_API_KEY: Needed for the app to work, e.g. `sk-...`
-* READTHEDOCS_URL: The url of the website you are interested in scraping
-* READTHEDOCS_VERSION: This is important only if there exist multiple versions of the docs (e.g. "en/v0.2.7" or "en/latest"). If left empty, it will scrape all available versions.
+* `OPENAI_API_KEY`: Needed for the app to work, e.g. `sk-...`
+* `READTHEDOCS_URL`: The url of the website you are interested in scraping
+* `READTHEDOCS_VERSION`: This is important only if there exist multiple versions of the docs (e.g. "en/v0.2.7" or "en/latest"). If left empty, it will scrape all available versions.
 
 **WARNING** This library is experimental and automatically calls OpenAI APIs for you. Use at your own risk! ⚠️
 
@@ -35,6 +35,6 @@ RAGTheDocs is an open-source library that allows you to deploy retrieval augment
 
 - **Web Scraping and embeddings:** RAGtheDocs automatically scrapes and embeds documentation from any website generated by ReadTheDocs/Sphinx using OpenAI embeddings
 
-- **RAG Interface:** It comes built-in with a gradio UI for users to interact with [Buster 🤖](github.com/jerpint/buster) our RAG agent.
+- **RAG Interface:** It comes built-in with a gradio UI for users to interact with [Buster 🤖](https://github.com/jerpint/buste) our RAG agent.
 
 - **Customization Options:** Tailor RAGtheDocs to your needs with customizable settings and options.

app.py

@@ -22,6 +22,7 @@ homepage_url = os.getenv("READTHEDOCS_URL") # e.g. "https://orion.readthedocs.io
 target_version = os.getenv("READTHEDOCS_VERSION") # e.g. "en/stable"
 
 # scrape and embed content from readthedocs website
+# comment out if already embedded locally to avoid extra costs
 scrape_rtd(
     homepage_url=homepage_url, save_directory="outputs/", target_version=target_version
 )
@@ -101,7 +102,24 @@ def chat(chat_history: ChatHistory) -> Tuple[ChatHistory, Completion]:
 demo = gr.Blocks()
 with demo:
     with gr.Row():
-        gr.Markdown("<h3><center>RAGTheDocs</center></h3>")
+        gr.Markdown("<h1><center>RAGTheDocs</center></h1>")
+
+    gr.Markdown(
+        """
+        ## About
+        RAGTheDocs allows you to ask questions about any documentation hosted on readthedocs.
+        Simply clone this space and point it to the right URL!
+
+        Try it out by asking a question below about [orion](https://orion.readthedocs.io/), an open-source hyperparameter optimization library.
+
+        ## How it works
+        This app uses [Buster 🤖](https://github.com/jerpint/buster) and ChatGPT to search the docs for relevant info and
+        answer questions.
+        View the code on the [project homepage](https://github.com/jerpint/RAGTheDocs)
+        """
+
+    )
+
 
     chatbot = gr.Chatbot()
 
@@ -122,10 +140,6 @@ with demo:
         inputs=question,
     )
 
-    gr.Markdown(
-        "This app uses [Buster 🤖](github.com/jerpint/buster) and ChatGPT to search the docs for relevant info and answer questions."
-    )
-
     response = gr.State()
 
     # fmt: off