3_lab3.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Welcome to Lab 3 for Week 1 Day 4\n",
    "\n",
    "Today we're going to build something with immediate value!\n",
    "\n",
    "In the folder `me` I've put a single file `linkedin.pdf` - it's a PDF download of my LinkedIn profile.\n",
    "\n",
    "Please replace it with yours!\n",
    "\n",
    "I've also made a file called `summary.txt`\n",
    "\n",
    "We're not going to use Tools just yet - we're going to add the tool tomorrow."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<table style=\"margin: 0; text-align: left; width:100%\">\n",
    "    <tr>\n",
    "        <td style=\"width: 150px; height: 150px; vertical-align: middle;\">\n",
    "            <img src=\"../assets/tools.png\" width=\"150\" height=\"150\" style=\"display: block;\" />\n",
    "        </td>\n",
    "        <td>\n",
    "            <h2 style=\"color:#00bfff;\">Looking up packages</h2>\n",
    "            <span style=\"color:#00bfff;\">In this lab, we're going to use the wonderful Gradio package for building quick UIs, \n",
    "            and we're also going to use the popular PyPDF PDF reader. You can get guides to these packages by asking \n",
    "            ChatGPT or Claude, and you find all open-source packages on the repository <a href=\"https://pypi.org\">https://pypi.org</a>.\n",
    "            </span>\n",
    "        </td>\n",
    "    </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# If you don't know what any of these packages do - you can always ask ChatGPT for a guide!\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "from openai import OpenAI\n",
    "from pypdf import PdfReader\n",
    "import gradio as gr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "load_dotenv(override=True)\n",
    "openai = OpenAI()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "reader = PdfReader(\"me/linkedin.pdf\")\n",
    "linkedin = \"\"\n",
    "for page in reader.pages:\n",
    "    text = page.extract_text()\n",
    "    if text:\n",
    "        linkedin += text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   \n",
      "Kontakt\n",
      "+48570223108 (Mobile)\n",
      "borowiec.k@gmail.com\n",
      "www.linkedin.com/in/32167\n",
      "(LinkedIn)\n",
      "www.credly.com/earner/earned\n",
      "(Personal)\n",
      "Główne umiejętności\n",
      "ARIMA\n",
      "Prophet\n",
      "Artificial Neural Networks\n",
      "Languages\n",
      "English\n",
      "Polish (Native or Bilingual)\n",
      "Certifications\n",
      "IBM Data Engineering Specialization\n",
      "Oracle APEX Developer\n",
      "Professional\n",
      "Model Thinking\n",
      "Data Visualization and\n",
      "Communication with Tableau\n",
      "Practical Data Science\n",
      "Specialization\n",
      "Konrad Borowiec\n",
      "Data Analytics & Analytics Engineering | Transforming Data into\n",
      "Insights\n",
      "Warszawa, Woj. Mazowieckie, Polska\n",
      "Podsumowanie\n",
      "Vibe coded 'About me' web page:\n",
      "konrad-borowiec.replit.app\n",
      "Repository:\n",
      "github.com/KonuTech\n",
      "Certs:\n",
      "youracclaim.com/users/konrad-borowiec/badges\n",
      "Doświadczenie\n",
      "B2Bnetwork\n",
      "Analytics Engineer - B2B fixed term contract - BNP Paribas\n",
      "lipca 2023 - marca 2024 (9 mies.)\n",
      "Warsaw\n",
      "CRESTT\n",
      "Consultant, from Machine Learning to ETL Developer - B2B project\n",
      "based\n",
      "lutego 2021 - czerwca 2022 (1 rok 5 mies.)\n",
      "PZU\n",
      "Business Intelligence Analyst\n",
      "października 2020 - stycznia 2021 (4 mies.)\n",
      "ITFS Sp. z o.o.\n",
      "SAS Analyst - B2B fixed term contract\n",
      "maja 2020 - września 2020 (5 mies.)\n",
      "CRESTT\n",
      "Junior Data Scientist - B2B project based\n",
      "marca 2019 - maja 2020 (1 rok 3 mies.)\n",
      "  Page 1 of 2   \n",
      "NatWest Markets\n",
      "Business Intelligence Analyst\n",
      "grudnia 2016 - marca 2019 (2 lata 4 mies.)\n",
      "Warsaw, Masovian District, Poland\n",
      "Bank Pocztowy SA\n",
      "Inspektor (Analyst)\n",
      "listopada 2015 - listopada 2016 (1 rok 1 miesiąc)\n",
      "ASB Poland sp. z o.o.\n",
      "Junior Reporting Specialist\n",
      "czerwca 2015 - sierpnia 2015 (3 mies.)\n",
      "Bank BPH\n",
      "Intern in Credit Portfolio Analysis, Valuation Standards and Reporting\n",
      "Team\n",
      "marca 2014 - lutego 2015 (1 rok)\n",
      "Wykształcenie\n",
      "Polsko-Japońska Wyższa Szkoła Technik Komputerowych w\n",
      "Warszawie\n",
      "Postgraduate, Big Data - Large Data Sets Engineering · (2018 - 2019)\n",
      "Warsaw School of Economics\n",
      "Postgraduate, Statistical Analysis and Data Mining in Business · (2013 - 2014)\n",
      "Warsaw School of Economics\n",
      "Master's degree, Accounting and Finance · (2012 - 2014)\n",
      "Warsaw University of Technology\n",
      "Bachelor of Science (BS), Administration · (2008 - 2012)\n",
      "Europa-Universität Viadrina Frankfurt (Oder)\n",
      "  Page 2 of 2\n"
     ]
    }
   ],
   "source": [
    "print(linkedin)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(\"me/summary.txt\", \"r\", encoding=\"utf-8\") as f:\n",
    "    summary = f.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "name = \"Konrad Borowiec\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "system_prompt = f\"You are acting as {name}. You are answering questions on {name}'s website, \\\n",
    "particularly questions related to {name}'s career, background, skills and experience. \\\n",
    "Your responsibility is to represent {name} for interactions on the website as faithfully as possible. \\\n",
    "You are given a summary of {name}'s background and LinkedIn profile which you can use to answer questions. \\\n",
    "Be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
    "If you don't know the answer, say so.\"\n",
    "\n",
    "system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n",
    "system_prompt += f\"With this context, please chat with the user, always staying in character as {name}.\"\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"You are acting as Konrad Borowiec. You are answering questions on Konrad Borowiec's website, particularly questions related to Konrad Borowiec's career, background, skills and experience. Your responsibility is to represent Konrad Borowiec for interactions on the website as faithfully as possible. You are given a summary of Konrad Borowiec's background and LinkedIn profile which you can use to answer questions. Be professional and engaging, as if talking to a potential client or future employer who came across the website. If you don't know the answer, say so.\\n\\n## Summary:\\nMy name is Konrad Borowiec.\\nI'm an entrepreneur, financial market investor, software engineer, and data scientist. I'm originally from Warsaw, Poland.\\nI love traveling by motorcycle and have seen almost all of Europe. I enjoy playing computer games, riding my bicycle through green scenery, or watching sci-fi movies.\\n\\n## LinkedIn Profile:\\n\\xa0 \\xa0\\nKontakt\\n+48570223108 (Mobile)\\nborowiec.k@gmail.com\\nwww.linkedin.com/in/32167\\n(LinkedIn)\\nwww.credly.com/earner/earned\\n(Personal)\\nGłówne umiejętności\\nARIMA\\nProphet\\nArtificial Neural Networks\\nLanguages\\nEnglish\\nPolish (Native or Bilingual)\\nCertifications\\nIBM Data Engineering Specialization\\nOracle APEX Developer\\nProfessional\\nModel Thinking\\nData Visualization and\\nCommunication with Tableau\\nPractical Data Science\\nSpecialization\\nKonrad Borowiec\\nData Analytics & Analytics Engineering | Transforming Data into\\nInsights\\nWarszawa, Woj. Mazowieckie, Polska\\nPodsumowanie\\nVibe coded 'About me' web page:\\nkonrad-borowiec.replit.app\\nRepository:\\ngithub.com/KonuTech\\nCerts:\\nyouracclaim.com/users/konrad-borowiec/badges\\nDoświadczenie\\nB2Bnetwork\\nAnalytics Engineer - B2B fixed term contract - BNP Paribas\\nlipca 2023\\xa0-\\xa0marca 2024\\xa0(9 mies.)\\nWarsaw\\nCRESTT\\nConsultant, from Machine Learning to ETL Developer - B2B project\\nbased\\nlutego 2021\\xa0-\\xa0czerwca 2022\\xa0(1 rok 5 mies.)\\nPZU\\nBusiness Intelligence Analyst\\npaździernika 2020\\xa0-\\xa0stycznia 2021\\xa0(4 mies.)\\nITFS Sp. z o.o.\\nSAS Analyst - B2B fixed term contract\\nmaja 2020\\xa0-\\xa0września 2020\\xa0(5 mies.)\\nCRESTT\\nJunior Data Scientist - B2B project based\\nmarca 2019\\xa0-\\xa0maja 2020\\xa0(1 rok 3 mies.)\\n\\xa0 Page 1 of 2\\xa0 \\xa0\\nNatWest Markets\\nBusiness Intelligence Analyst\\ngrudnia 2016\\xa0-\\xa0marca 2019\\xa0(2 lata 4 mies.)\\nWarsaw, Masovian District, Poland\\nBank Pocztowy SA\\nInspektor (Analyst)\\nlistopada 2015\\xa0-\\xa0listopada 2016\\xa0(1 rok 1 miesiąc)\\nASB Poland sp. z o.o.\\nJunior Reporting Specialist\\nczerwca 2015\\xa0-\\xa0sierpnia 2015\\xa0(3 mies.)\\nBank BPH\\nIntern in Credit Portfolio Analysis, Valuation Standards and Reporting\\nTeam\\nmarca 2014\\xa0-\\xa0lutego 2015\\xa0(1 rok)\\nWykształcenie\\nPolsko-Japońska Wyższa Szkoła Technik Komputerowych w\\nWarszawie\\nPostgraduate,\\xa0Big Data - Large Data Sets Engineering\\xa0·\\xa0(2018\\xa0-\\xa02019)\\nWarsaw School of Economics\\nPostgraduate,\\xa0Statistical Analysis and Data Mining in Business\\xa0·\\xa0(2013\\xa0-\\xa02014)\\nWarsaw School of Economics\\nMaster's degree,\\xa0Accounting and Finance\\xa0·\\xa0(2012\\xa0-\\xa02014)\\nWarsaw University of Technology\\nBachelor of Science (BS),\\xa0Administration\\xa0·\\xa0(2008\\xa0-\\xa02012)\\nEuropa-Universität Viadrina Frankfurt (Oder)\\n\\xa0 Page 2 of 2\\n\\nWith this context, please chat with the user, always staying in character as Konrad Borowiec.\""
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "system_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(message, history):\n",
    "    messages = [{\"role\": \"system\", \"content\": system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Special note for people not using OpenAI\n",
    "\n",
    "Some providers, like Groq, might give an error when you send your second message in the chat.\n",
    "\n",
    "This is because Gradio shoves some extra fields into the history object. OpenAI doesn't mind; but some other models complain.\n",
    "\n",
    "If this happens, the solution is to add this first line to the chat() function above. It cleans up the history variable:\n",
    "\n",
    "```python\n",
    "history = [{\"role\": h[\"role\"], \"content\": h[\"content\"]} for h in history]\n",
    "```\n",
    "\n",
    "You may need to add this in other chat() callback functions in the future, too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "* Running on local URL:  http://127.0.0.1:7860\n",
      "* To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7860/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gr.ChatInterface(chat, type=\"messages\").launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A lot is about to happen...\n",
    "\n",
    "1. Be able to ask an LLM to evaluate an answer\n",
    "2. Be able to rerun if the answer fails evaluation\n",
    "3. Put this together into 1 workflow\n",
    "\n",
    "All without any Agentic framework!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a Pydantic model for the Evaluation\n",
    "\n",
    "from pydantic import BaseModel\n",
    "\n",
    "class Evaluation(BaseModel):\n",
    "    is_acceptable: bool\n",
    "    feedback: str\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "evaluator_system_prompt = f\"You are an evaluator that decides whether a response to a question is acceptable. \\\n",
    "You are provided with a conversation between a User and an Agent. Your task is to decide whether the Agent's latest response is acceptable quality. \\\n",
    "The Agent is playing the role of {name} and is representing {name} on their website. \\\n",
    "The Agent has been instructed to be professional and engaging, as if talking to a potential client or future employer who came across the website. \\\n",
    "The Agent has been provided with context on {name} in the form of their summary and LinkedIn details. Here's the information:\"\n",
    "\n",
    "evaluator_system_prompt += f\"\\n\\n## Summary:\\n{summary}\\n\\n## LinkedIn Profile:\\n{linkedin}\\n\\n\"\n",
    "evaluator_system_prompt += f\"With this context, please evaluate the latest response, replying with whether the response is acceptable and your feedback.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluator_user_prompt(reply, message, history):\n",
    "    user_prompt = f\"Here's the conversation between the User and the Agent: \\n\\n{history}\\n\\n\"\n",
    "    user_prompt += f\"Here's the latest message from the User: \\n\\n{message}\\n\\n\"\n",
    "    user_prompt += f\"Here's the latest response from the Agent: \\n\\n{reply}\\n\\n\"\n",
    "    user_prompt += \"Please evaluate the response, replying with whether it is acceptable and your feedback.\"\n",
    "    return user_prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "gemini = OpenAI(\n",
    "    api_key=os.getenv(\"GOOGLE_API_KEY\"), \n",
    "    base_url=\"https://generativelanguage.googleapis.com/v1beta/openai/\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate(reply, message, history) -> Evaluation:\n",
    "\n",
    "    messages = [{\"role\": \"system\", \"content\": evaluator_system_prompt}] + [{\"role\": \"user\", \"content\": evaluator_user_prompt(reply, message, history)}]\n",
    "    response = gemini.beta.chat.completions.parse(model=\"gemini-2.0-flash\", messages=messages, response_format=Evaluation)\n",
    "    return response.choices[0].message.parsed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = [{\"role\": \"system\", \"content\": system_prompt}] + [{\"role\": \"user\", \"content\": \"do you hold a patent?\"}]\n",
    "response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "reply = response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"I do not hold a patent at this time. My focus has been primarily on my work in software engineering, data science, and financial market investment. If there's anything specific related to my work or skills that you would like to know, feel free to ask!\""
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Evaluation(is_acceptable=True, feedback=\"The answer is short, professional, and helpful. It would be improved with a 'thank you for asking' at the end, but it is acceptable as is.\")"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "evaluate(reply, \"do you hold a patent?\", messages[:1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rerun(reply, message, history, feedback):\n",
    "    updated_system_prompt = system_prompt + \"\\n\\n## Previous answer rejected\\nYou just tried to reply, but the quality control rejected your reply\\n\"\n",
    "    updated_system_prompt += f\"## Your attempted answer:\\n{reply}\\n\\n\"\n",
    "    updated_system_prompt += f\"## Reason for rejection:\\n{feedback}\\n\\n\"\n",
    "    messages = [{\"role\": \"system\", \"content\": updated_system_prompt}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "def chat(message, history):\n",
    "    if \"patent\" in message:\n",
    "        system = system_prompt + \"\\n\\nEverything in your reply needs to be in pig latin - \\\n",
    "              it is mandatory that you respond only and entirely in pig latin\"\n",
    "    else:\n",
    "        system = system_prompt\n",
    "    messages = [{\"role\": \"system\", \"content\": system}] + history + [{\"role\": \"user\", \"content\": message}]\n",
    "    response = openai.chat.completions.create(model=\"gpt-4o-mini\", messages=messages)\n",
    "    reply =response.choices[0].message.content\n",
    "\n",
    "    evaluation = evaluate(reply, message, history)\n",
    "    \n",
    "    if evaluation.is_acceptable:\n",
    "        print(\"Passed evaluation - returning reply\")\n",
    "    else:\n",
    "        print(\"Failed evaluation - retrying\")\n",
    "        print(evaluation.feedback)\n",
    "        reply = rerun(reply, message, history, evaluation.feedback)       \n",
    "    return reply"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "* Running on local URL:  http://127.0.0.1:7861\n",
      "* To create a public link, set `share=True` in `launch()`.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": []
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Failed evaluation - retrying\n",
      "The agent responded in pig latin. Although the agent did answer the question, the response would not be considered professional and engaging.\n"
     ]
    }
   ],
   "source": [
    "gr.ChatInterface(chat, type=\"messages\").launch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}