Skip to content
Merged
2 changes: 1 addition & 1 deletion docs/_common/agents/remember-to-disconnect.mdx
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
:::danger Remember to disconnect your agents!
:::danger[Remember to disconnect your agents!]
It's important to disconnect agents, because **every connected agent generates usage** just as a normal peer.
:::
51 changes: 50 additions & 1 deletion docs/integrations/gemini-live-integration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ Create a Fishjam agent configured to match the audio format that the Google clie

### Step 3: Connect the Streams

:::warning Encoding
:::warning[Encoding]
Fishjam handles raw bytes, while Google GenAI SDKs often expect Base64 strings. Ensure you convert between them correctly as shown below.
:::

Expand Down Expand Up @@ -278,3 +278,52 @@ Fishjam handles raw bytes, while Google GenAI SDKs often expect Base64 strings.

</TabItem>
</Tabs>

## Troubleshooting Gemini API keys

If your agent joins the room but silently does nothing — no audio, no transcription, and no error — the issue is often related to the Gemini API key.

:::warning[API key errors surface asynchronously]
`createClient` is a thin synchronous wrapper around Google's `GoogleGenAI` and makes no network call, so it does **not** validate your key — an invalid or unauthorized key is not rejected when you create the client. Unless you verify it explicitly (see below), the key is first exercised when `live.connect()` opens the Live websocket, and authentication or access failures then typically surface through the Live session callbacks (`onerror` / `onclose`) rather than as a thrown exception. (Other problems, such as a malformed config, can still reject the awaited `connect()` call — keep a `try/catch` around it too.)
:::

To catch a bad key up front, call `checkCredentials` before connecting: it makes a single network request and throws if the key is rejected, so misconfiguration fails fast instead of silently. Still implement the `onerror` and `onclose` callbacks and log the close code and reason — that is where model-specific rejections (see below) and other Live failures appear:

```ts
import GeminiIntegration from "@fishjam-cloud/js-server-sdk/gemini";
import { Modality, type LiveServerMessage } from "@google/genai";

const genAi = GeminiIntegration.createClient({
apiKey: process.env.GOOGLE_API_KEY!,
});
const GEMINI_MODEL = "gemini-2.5-flash-native-audio-preview-12-2025";
const handleMessage = (msg: LiveServerMessage) => {};

// ---cut---
await GeminiIntegration.checkCredentials(genAi);

const session = await genAi.live.connect({
model: GEMINI_MODEL,
config: { responseModalities: [Modality.AUDIO] },
callbacks: {
onopen: () => console.log("Gemini Live connected"),
onerror: (e) => console.error("Gemini Live error:", e),
onclose: (e) =>
console.error(`Gemini Live closed: code=${e.code} reason=${e.reason}`),
onmessage: handleMessage,
},
});
```

### Common reasons a key doesn't work

- Wrong or mistyped key, or a key from the wrong Google Cloud project.
- The Gemini API is not enabled for the key's project.
- Region or country restriction — the Gemini API (or its free tier) is not available in your region.
- Model-specific rejection. The native-audio Live models (e.g. `gemini-2.5-flash-native-audio-preview-*`) can reject an otherwise-valid key: the websocket closes with code `1008` and the message _"Your API key was reported as leaked. Please use another API key."_ (even when the key is not actually leaked), or with code `1011` internal errors. A key that works for other models can still fail here.

### How to fix

- Verify or regenerate your key at [Google AI Studio](https://aistudio.google.com/app/apikey).
- Confirm the Gemini API is enabled for the key's project and that your region is supported.
- If you see the `1008` "leaked" message, rotate to a freshly generated key.
2 changes: 1 addition & 1 deletion packages/js-server-sdk
Submodule js-server-sdk updated 43 files
+7 −3 .github/workflows/release.yaml
+3 −0 examples/multimodal/.env.example
+10 −0 examples/multimodal/README.md
+3 −2 examples/multimodal/package.json
+4 −1 examples/multimodal/src/index.ts
+2 −2 examples/multimodal/src/service/multimodal.ts
+3 −3 examples/room-manager/package.json
+3 −2 examples/selective-subscription/backend/package.json
+3 −2 examples/selective-subscription/frontend/package.json
+3 −2 examples/selective-subscription/package.json
+11 −1 examples/transcription/README.md
+3 −2 examples/transcription/package.json
+4 −1 examples/transcription/src/index.ts
+2 −2 examples/transcription/src/service/transcription.ts
+6 −5 package.json
+201 −0 packages/composition/LICENSE
+15 −0 packages/composition/README.md
+26 −0 packages/composition/eslint.config.mjs
+86 −0 packages/composition/package.json
+2 −0 packages/composition/src/core.ts
+10 −0 packages/composition/src/eventBus.ts
+47 −0 packages/composition/src/hooks.ts
+10 −0 packages/composition/src/index.ts
+406 −0 packages/composition/src/store.ts
+43 −0 packages/composition/src/types.ts
+12 −0 packages/composition/tests/eventBus.test.ts
+73 −0 packages/composition/tests/hooks.test.ts
+352 −0 packages/composition/tests/store.test.ts
+20 −0 packages/composition/tsconfig.json
+3 −2 packages/fishjam-openapi/package.json
+3 −2 packages/fishjam-proto/package.json
+1 −1 packages/fishjam-proto/protos
+3 −2 packages/js-server-sdk/package.json
+3 −0 packages/js-server-sdk/src/index.ts
+44 −0 packages/js-server-sdk/src/integrations/gemini.ts
+32 −7 packages/js-server-sdk/src/notifications.ts
+5 −0 packages/js-server-sdk/src/types.ts
+8 −0 packages/js-server-sdk/src/ws_notifier.ts
+23 −0 packages/js-server-sdk/tests/gemini.test.ts
+97 −3 packages/js-server-sdk/tests/notifications.test.ts
+1 −1 packages/js-server-sdk/tests/webhook.test.ts
+33 −5 packages/js-server-sdk/tests/ws_notifier.test.ts
+53 −10 yarn.lock
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
:::danger Remember to disconnect your agents!
:::danger[Remember to disconnect your agents!]
It's important to disconnect agents, because **every connected agent generates usage** just as a normal peer.
:::
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ Create a Fishjam agent configured to match the audio format that the Google clie

### Step 3: Connect the Streams

:::warning Encoding
:::warning[Encoding]
Fishjam handles raw bytes, while Google GenAI SDKs often expect Base64 strings. Ensure you convert between them correctly as shown below.
:::

Expand Down
2 changes: 1 addition & 1 deletion versioned_docs/version-0.26.0/tutorials/livestreaming.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ In this tutorial we explain how to publish and view streams with the client SDKs
If you prefer to use WHIP or WHEP directly, then you should read [WHIP/WHEP with Fishjam](../how-to/backend/whip-whep).
:::

:::info Cost-effective audio livestreams
:::info[Cost-effective audio livestreams]
Fishjam supports audio-only livestreams at a reduced cost of $0.20 per 1000 minutes of the streamer and each listener (compared to $0.80 for video livestreams).
This is ideal for podcasts, radio-style broadcasts, or any scenario where video isn't necessary.
See [Audio-only calls](../how-to/backend/audio-only-calls) for more information.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
:::danger Remember to disconnect your agents!
:::danger[Remember to disconnect your agents!]
It's important to disconnect agents, because **every connected agent generates usage** just as a normal peer.
:::
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ Create a Fishjam agent configured to match the audio format that the Google clie

### Step 3: Connect the Streams

:::warning Encoding
:::warning[Encoding]
Fishjam handles raw bytes, while Google GenAI SDKs often expect Base64 strings. Ensure you convert between them correctly as shown below.
:::

Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
:::danger Remember to disconnect your agents!
:::danger[Remember to disconnect your agents!]
It's important to disconnect agents, because **every connected agent generates usage** just as a normal peer.
:::
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ Create a Fishjam agent configured to match the audio format that the Google clie

### Step 3: Connect the Streams

:::warning Encoding
:::warning[Encoding]
Fishjam handles raw bytes, while Google GenAI SDKs often expect Base64 strings. Ensure you convert between them correctly as shown below.
:::

Expand Down
Loading