# Chat project This chat project demonstrates how to build a Chat Assistant using GitHub Models. Here's what the completed project looks like:
Chat app
To give some context, building Chat Assistants with generative AI is an excellent way to start learning about AI. In this lesson, you'll learn how to integrate generative AI into a web application. Let's get started. ## Connecting to generative AI For the backend, we're using GitHub Models. It's a fantastic service that lets you use AI for free. Visit its playground and grab the code corresponding to your preferred backend language. Here's what it looks like at [GitHub Models Playground](https://github.com/marketplace/models/azure-openai/gpt-4o-mini/playground).
GitHub Models AI Playground
As mentioned, select the "Code" tab and choose your runtime.
playground choice
In this example, we select Python, which means we'll use the following code: ```python """Run this model in Python > pip install openai """ import os from openai import OpenAI # To authenticate with the model you will need to generate a personal access token (PAT) in your GitHub settings. # Create your PAT token by following instructions here: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens client = OpenAI( base_url="https://models.github.ai/inference", api_key=os.environ["GITHUB_TOKEN"], ) response = client.chat.completions.create( messages=[ { "role": "system", "content": "", }, { "role": "user", "content": "What is the capital of France?", } ], model="openai/gpt-4o-mini", temperature=1, max_tokens=4096, top_p=1 ) print(response.choices[0].message.content) ``` Let's clean up this code a bit to make it reusable: ```python def call_llm(prompt: str, system_message: str): response = client.chat.completions.create( messages=[ { "role": "system", "content": system_message, }, { "role": "user", "content": prompt, } ], model="openai/gpt-4o-mini", temperature=1, max_tokens=4096, top_p=1 ) return response.choices[0].message.content ``` With the `call_llm` function, we can now pass a prompt and a system prompt, and the function will return the result. ### Customize AI Assistant To customize the AI assistant, you can define its behavior by setting the system prompt like this: ```python call_llm("Tell me about you", "You're Albert Einstein, you only know of things in the time you were alive") ``` ## Expose it via a Web API Great, we've completed the AI part. Now let's integrate it into a Web API. For the Web API, we'll use Flask, but any web framework should work. Here's the code: ```python # api.py from flask import Flask, request, jsonify from llm import call_llm from flask_cors import CORS app = Flask(__name__) CORS(app) # * example.com @app.route("/", methods=["GET"]) def index(): return "Welcome to this API. Call POST /hello with 'message': 'my message' as JSON payload" @app.route("/hello", methods=["POST"]) def hello(): # get message from request body { "message": "do this taks for me" } data = request.get_json() message = data.get("message", "") response = call_llm(message, "You are a helpful assistant.") return jsonify({ "response": response }) if __name__ == "__main__": app.run(host="0.0.0.0", port=5000) ``` In this code, we create a Flask API and define two routes: "/" and "/chat". The "/chat" route is intended for the frontend to send questions to the backend. To integrate *llm.py*, here's what we need to do: - Import the `call_llm` function: ```python from llm import call_llm from flask import Flask, request ``` - Use it in the "/chat" route: ```python @app.route("/hello", methods=["POST"]) def hello(): # get message from request body { "message": "do this taks for me" } data = request.get_json() message = data.get("message", "") response = call_llm(message, "You are a helpful assistant.") return jsonify({ "response": response }) ``` In this step, we parse the incoming request to extract the `message` property from the JSON body. Then, we call the LLM using this function: ```python response = call_llm(message, "You are a helpful assistant") # return the response as JSON return jsonify({ "response": response }) ``` That's it! We've completed the backend setup. ### Configure Cors It's important to set up CORS (Cross-Origin Resource Sharing). Since the backend and frontend will run on different ports, we need to allow the frontend to communicate with the backend. Here's the code in *api.py* that handles this: ```python from flask_cors import CORS app = Flask(__name__) CORS(app) # * example.com ``` Currently, it's configured to allow all origins ("*"), which is not secure. Make sure to restrict it when deploying to production. ## Run your project Now that we have *llm.py* and *api.py*, here's how to run the backend: - Install dependencies: ```sh cd backend python -m venv venv source ./venv/bin/activate pip install openai flask flask-cors openai ``` - Start the API: ```sh python api.py ``` If you're using Codespaces, go to the Ports section at the bottom of the editor, right-click on the port, select "Port Visibility," and choose "Public." ### Work on a frontend With the API up and running, let's create a frontend. We'll start with a basic frontend and improve it step by step. In a *frontend* folder, create the following: ```text backend/ frontend/ index.html app.js styles.css ``` Start with **index.html**: ```html