Skip to content

Commit f9d370a

Browse files
author
pjsingh28
authored
Merge pull request #632 from MicrosoftDocs/test_terraform
Add new doc: "Deploy and run an Azure OpenAI/ChatGPT app on AKS"
2 parents ccabc95 + fb40f0e commit f9d370a

11 files changed

Lines changed: 531 additions & 0 deletions

File tree

‎scenarios/AksOpenAiTerraform/.gitignore‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
# Local .terraform directories
2+
**/.terraform/*
3+
4+
# .tfstate files
5+
*.tfstate
6+
*.tfstate.*
7+
8+
# Crash log files
9+
crash.log
10+
crash.*.log
11+
12+
# Exclude all .tfvars files, which are likely to contain sensitive data, such as
13+
# password, private keys, and other secrets. These should not be part of version
14+
# control as they are data points which are potentially sensitive and subject
15+
# to change depending on the environment.
16+
*.tfvars
17+
*.tfvars.json
18+
19+
# Ignore override files as they are usually used to override resources locally and so
20+
# are not checked in
21+
override.tf
22+
override.tf.json
23+
*_override.tf
24+
*_override.tf.json
25+
26+
# Ignore transient lock info files created by terraform apply
27+
.terraform.tfstate.lock.info
28+
29+
# Include override files you do wish to add to version control using negated pattern
30+
# !example_override.tf
31+
32+
# Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan
33+
# example: *tfplan*
34+
35+
# Ignore CLI configuration files
36+
.terraformrc
37+
terraform.rc
38+
39+
.venv
40+
.vscode

‎scenarios/AksOpenAiTerraform/README.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
title: Deploy and run an Azure OpenAI ChatGPT application on AKS via Terraform
3+
description: This article shows how to deploy an AKS cluster and Azure OpenAI Service via Terraform and how to deploy a ChatGPT-like application in Python.
4+
ms.topic: quickstart
5+
ms.date: 09/06/2024
6+
author: aamini7
7+
ms.author: ariaamini
8+
ms.custom: innovation-engine, linux-related-content
9+
---
10+
11+
## Provision Resources with Terraform (~5 minutes)
12+
Run terraform to provision all the Azure resources required to setup your new OpenAI website.
13+
```bash
14+
# Terraform parses TF_VAR_* as vars (Ex: TF_VAR_name -> name)
15+
export TF_VAR_location="westus3"
16+
export TF_VAR_kubernetes_version="1.30.9"
17+
export TF_VAR_model_name="gpt-4o-mini"
18+
export TF_VAR_model_version="2024-07-18"
19+
# Terraform consumes sub id as $ARM_SUBSCRIPTION_ID
20+
export ARM_SUBSCRIPTION_ID=$SUBSCRIPTION_ID
21+
# Run Terraform
22+
terraform -chdir=terraform init
23+
terraform -chdir=terraform apply -auto-approve
24+
```
25+
26+
## Login to Cluster
27+
In order to use the kubectl to run commands on the newly created cluster, you must first login.
28+
```bash
29+
RESOURCE_GROUP=$(terraform -chdir=terraform output -raw resource_group_name)
30+
az aks get-credentials --admin --name AksCluster --resource-group $RESOURCE_GROUP --subscription $SUBSCRIPTION_ID
31+
```
32+
33+
# Install Helm Charts
34+
Install nginx and cert-manager through Helm
35+
```bash
36+
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
37+
helm repo add jetstack https://charts.jetstack.io
38+
helm repo update
39+
40+
STATIC_IP=$(terraform -chdir=terraform output -raw static_ip)
41+
DNS_LABEL=$(terraform -chdir=terraform output -raw dns_label)
42+
helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \
43+
--set controller.replicaCount=2 \
44+
--set controller.nodeSelector."kubernetes\.io/os"=linux \
45+
--set defaultBackend.nodeSelector."kubernetes\.io/os"=linux \
46+
--set controller.service.annotations."service\.beta\.kubernetes\.io/azure-dns-label-name"=$DNS_LABEL \
47+
--set controller.service.loadBalancerIP=$STATIC_IP \
48+
--set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz
49+
helm upgrade --install cert-manager jetstack/cert-manager \
50+
--set crds.enabled=true \
51+
--set nodeSelector."kubernetes\.io/os"=linux
52+
```
53+
54+
## Deploy
55+
Apply/Deploy Manifest File
56+
```bash
57+
export IMAGE="aamini8/magic8ball:latest"
58+
# Uncomment below to manually build docker image yourself instead of using pre-built image.
59+
# docker build -t <YOUR IMAGE NAME> ./magic8ball --push
60+
export HOSTNAME=$(terraform -chdir=terraform output -raw hostname)
61+
export WORKLOAD_IDENTITY_CLIENT_ID=$(terraform -chdir=terraform output -raw workload_identity_client_id)
62+
export AZURE_OPENAI_DEPLOYMENT=$(terraform -chdir=terraform output -raw openai_deployment)
63+
export AZURE_OPENAI_ENDPOINT=$(terraform -chdir=terraform output -raw openai_endpoint)
64+
envsubst < quickstart-app.yml | kubectl apply -f -
65+
```
66+
67+
## Wait for host to be ready
68+
```bash
69+
kubectl wait --for=condition=Ready certificate/tls-secret
70+
echo "Visit: https://$HOSTNAME"
71+
```

‎scenarios/AksOpenAiTerraform/magic8ball/Dockerfile‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
FROM python:3.13-slim
2+
WORKDIR /app
3+
4+
ENV PYTHONDONTWRITEBYTECODE=1
5+
ENV PYTHONUNBUFFERED=1
6+
7+
COPY requirements.txt ./
8+
RUN pip install --no-cache-dir -r requirements.txt
9+
10+
COPY . .
11+
EXPOSE 8501
12+
ENTRYPOINT ["streamlit", "run", "app.py", "--server.port=8501", "--server.address=0.0.0.0"]

‎scenarios/AksOpenAiTerraform/magic8ball/app.py‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
import os
2+
from openai import AzureOpenAI
3+
import streamlit as st
4+
from azure.identity import WorkloadIdentityCredential, get_bearer_token_provider
5+
6+
azure_deployment = os.getenv("AZURE_OPENAI_DEPLOYMENT")
7+
azure_endpoint = os.getenv("AZURE_OPENAI_ENDPOINT")
8+
workload_identity_client_id = os.getenv("WORKLOAD_IDENTITY_CLIENT_ID")
9+
10+
client = AzureOpenAI(
11+
api_version="2024-10-21",
12+
azure_endpoint=azure_endpoint,
13+
azure_ad_token_provider=get_bearer_token_provider(
14+
WorkloadIdentityCredential(client_id=workload_identity_client_id),
15+
"https://cognitiveservices.azure.com/.default",
16+
),
17+
)
18+
19+
20+
def ask_openai_api(messages: list[str]):
21+
completion = client.chat.completions.create(
22+
messages=messages, model=azure_deployment, stream=True, max_tokens=20
23+
)
24+
return completion
25+
26+
27+
assistant_prompt = """
28+
Answer as a magic 8 ball and make random predictions.
29+
If the question is not clear, respond with "Ask the Magic 8 Ball a question about your future."
30+
"""
31+
32+
# Init state
33+
if "messages" not in st.session_state:
34+
st.session_state.messages = [{"role": "system", "content": assistant_prompt}]
35+
if "disabled" not in st.session_state:
36+
st.session_state.disabled = False
37+
38+
st.title(":robot_face: Magic 8 Ball")
39+
for message in st.session_state.messages[1:]: # Print previous messages
40+
with st.chat_message(message["role"]):
41+
st.markdown(message["content"])
42+
43+
44+
def disable_chat():
45+
st.session_state.disabled = True
46+
47+
48+
if prompt := st.chat_input(
49+
"Ask your question", on_submit=disable_chat, disabled=st.session_state.disabled
50+
):
51+
# Print Question
52+
st.session_state.messages.append({"role": "user", "content": prompt})
53+
with st.chat_message("user"):
54+
st.write(prompt)
55+
56+
# Print Response
57+
with st.chat_message("assistant"):
58+
messages = st.session_state.messages
59+
with st.spinner("Loading..."):
60+
response = st.write_stream(ask_openai_api(messages))
61+
st.session_state.messages.append({"role": "assistant", "content": response})
62+
63+
# Re-enable textbox
64+
st.session_state.disabled = False
65+
st.rerun()

‎scenarios/AksOpenAiTerraform/magic8ball/requirements.txt‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
streamlit~=1.40.1
2+
azure-identity~=1.20.0
3+
openai~=1.65.2

‎scenarios/AksOpenAiTerraform/quickstart-app.yml‎

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
apiVersion: v1
2+
kind: ConfigMap
3+
metadata:
4+
name: magic8ball-configmap
5+
data:
6+
AZURE_OPENAI_ENDPOINT: $AZURE_OPENAI_ENDPOINT
7+
AZURE_OPENAI_DEPLOYMENT: $AZURE_OPENAI_DEPLOYMENT
8+
WORKLOAD_IDENTITY_CLIENT_ID: $WORKLOAD_IDENTITY_CLIENT_ID
9+
---
10+
apiVersion: apps/v1
11+
kind: Deployment
12+
metadata:
13+
name: magic8ball
14+
labels:
15+
app.kubernetes.io/name: magic8ball
16+
spec:
17+
replicas: 2
18+
selector:
19+
matchLabels:
20+
app.kubernetes.io/name: magic8ball
21+
template:
22+
metadata:
23+
labels:
24+
app.kubernetes.io/name: magic8ball
25+
azure.workload.identity/use: "true"
26+
spec:
27+
serviceAccountName: magic8ball-sa
28+
containers:
29+
- name: magic8ball
30+
image: $IMAGE
31+
imagePullPolicy: Always
32+
ports:
33+
- containerPort: 8501
34+
envFrom:
35+
- configMapRef:
36+
name: magic8ball-configmap
37+
---
38+
apiVersion: v1
39+
kind: Service
40+
metadata:
41+
name: magic8ball
42+
spec:
43+
selector:
44+
app.kubernetes.io/name: magic8ball
45+
ports:
46+
- port: 80
47+
targetPort: 8501
48+
protocol: TCP
49+
---
50+
apiVersion: v1
51+
kind: ServiceAccount
52+
metadata:
53+
name: magic8ball-sa
54+
annotations:
55+
azure.workload.identity/client-id: $WORKLOAD_IDENTITY_CLIENT_ID
56+
azure.workload.identity/tenant-id: $TENANT_ID
57+
---
58+
apiVersion: networking.k8s.io/v1
59+
kind: Ingress
60+
metadata:
61+
name: magic8ball
62+
annotations:
63+
cert-manager.io/issuer: letsencrypt-dev
64+
spec:
65+
ingressClassName: nginx
66+
tls:
67+
- hosts:
68+
- $HOSTNAME
69+
secretName: tls-secret
70+
rules:
71+
- host: $HOSTNAME
72+
http:
73+
paths:
74+
- path: /
75+
pathType: Prefix
76+
backend:
77+
service:
78+
name: magic8ball
79+
port:
80+
number: 80
81+
---
82+
apiVersion: cert-manager.io/v1
83+
kind: Issuer
84+
metadata:
85+
name: letsencrypt-dev
86+
spec:
87+
acme:
88+
server: https://acme-v02.api.letsencrypt.org/directory
89+
email: $EMAIL
90+
privateKeySecretRef:
91+
name: tls-secret
92+
solvers:
93+
- http01:
94+
ingress:
95+
ingressClassName: nginx

‎scenarios/AksOpenAiTerraform/terraform/.terraform.lock.hcl‎

Lines changed: 41 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)