You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/languages/python.md
+115Lines changed: 115 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -894,6 +894,121 @@ data: [DONE]
894
894
895
895
Streaming responses can run for longer than the default function timeout. Make sure your OpenFaaS [timeout values](/tutorials/expanded-timeouts/) are configured appropriately for your streaming workloads.
896
896
897
+
### Streaming OpenAI responses
898
+
899
+
The following example uses the same SSE pattern to stream OpenAI chat completions token by token.
900
+
901
+
**1. Create the function**
902
+
903
+
Pull the `python3-flask` template and scaffold a new function:
904
+
905
+
```bash
906
+
faas-cli template store pull python3-flask
907
+
faas-cli new --lang python3-flask openai-stream \
908
+
--prefix ttl.sh/openfaas-examples
909
+
```
910
+
911
+
**2. Add the openai dependency**
912
+
913
+
Add `openai` to the function's `requirements.txt`:
914
+
915
+
```
916
+
openai
917
+
```
918
+
919
+
**3. Create a secret for the API key**
920
+
921
+
Store the OpenAI API key as an OpenFaaS secret. This keeps the key out of environment variables and the function's container image.
922
+
923
+
Save your API key to `openai-api-key.txt`, then run:
The handler creates a streaming chat completion with `stream=True` and yields each content delta in SSE format. The OpenAI client is initialised once and reused across invocations.
You should see tokens appear incrementally as OpenAI generates them:
1000
+
1001
+
```
1002
+
data: Server
1003
+
data: -Sent
1004
+
data: Events
1005
+
data: (
1006
+
data: SSE
1007
+
data: )
1008
+
...
1009
+
data: [DONE]
1010
+
```
1011
+
897
1012
## OpenTelemetry zero-code instrumentation
898
1013
899
1014
Using [OpenTelemetry zero-code instrumentation](https://opentelemetry.io/docs/zero-code/python/) for python functions requires some minor modifications to the existing Python templates.
0 commit comments