Skip to main content

Google PaLM

note

This integration does not support gemini-* models. Check Google AI or VertexAI.

The Google PaLM API can be integrated by first installing the required packages:

tip

See this section for general instructions on installing integration packages.

npm install google-auth-library @google-ai/generativelanguage @langchain/community

Create an API key from Google MakerSuite. You can then set the key as GOOGLE_PALM_API_KEY environment variable or pass it as apiKey parameter while instantiating the model.

import { GooglePaLM } from "@langchain/community/llms/googlepalm";

export const run = async () => {
  const model = new GooglePaLM({
    apiKey: "<YOUR API KEY>", // or set it in environment variable as `GOOGLE_PALM_API_KEY`
    // other params
    temperature: 1, // OPTIONAL
    model: "models/text-bison-001", // OPTIONAL
    maxOutputTokens: 1024, // OPTIONAL
    topK: 40, // OPTIONAL
    topP: 3, // OPTIONAL
    safetySettings: [
      // OPTIONAL
      {
        category: "HARM_CATEGORY_DANGEROUS",
        threshold: "BLOCK_MEDIUM_AND_ABOVE",
      },
    ],
    stopSequences: ["stop"], // OPTIONAL
  });
  const res = await model.invoke(
    "What would be a good company name for a company that makes colorful socks?"
  );
  console.log({ res });
};

API Reference:

  • GooglePaLM from @langchain/community/llms/googlepalm

GooglePaLM

Langchain.js supports two different authentication methods based on whether you're running in a Node.js environment or a web environment.

Setup​

Node.js​

To call Vertex AI models in Node, you'll need to install Google's official auth client as a peer dependency.

You should make sure the Vertex AI API is enabled for the relevant project and that you've authenticated to Google Cloud using one of these methods:

  • You are logged into an account (using gcloud auth application-default login) permitted to that project.
  • You are running on a machine using a service account that is permitted to the project.
  • You have downloaded the credentials for a service account that is permitted to the project and set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path of this file.
npm install google-auth-library

Web​

To call Vertex AI models in web environments (like Edge functions), you'll need to install the web-auth-library pacakge as a peer dependency:

npm install web-auth-library

Then, you'll need to add your service account credentials directly as a GOOGLE_VERTEX_AI_WEB_CREDENTIALS environment variable:

GOOGLE_VERTEX_AI_WEB_CREDENTIALS={"type":"service_account","project_id":"YOUR_PROJECT-12345",...}

You can also pass your credentials directly in code like this:

tip

See this section for general instructions on installing integration packages.

npm install @langchain/community
import { GoogleVertexAI } from "@langchain/community/llms/googlevertexai";

const model = new GoogleVertexAI({
  authOptions: {
    credentials: {"type":"service_account","project_id":"YOUR_PROJECT-12345",...},
  },
});

Usage​

Several models are available and can be specified by the model attribute in the constructor. These include:

  • text-bison (default)
  • text-bison-32k
  • code-gecko
  • code-bison
import { GoogleVertexAI } from "@langchain/community/llms/googlevertexai";
// Or, if using the web entrypoint:
// import { GoogleVertexAI } from "@langchain/community/llms/googlevertexai/web";

/*
 * Before running this, you should make sure you have created a
 * Google Cloud Project that is permitted to the Vertex AI API.
 *
 * You will also need permission to access this project / API.
 * Typically, this is done in one of three ways:
 * - You are logged into an account permitted to that project.
 * - You are running this on a machine using a service account permitted to
 *   the project.
 * - The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is set to the
 *   path of a credentials file for a service account permitted to the project.
 */
const model = new GoogleVertexAI({
  temperature: 0.7,
});
const res = await model.invoke(
  "What would be a good company name for a company that makes colorful socks?"
);
console.log({ res });

API Reference:

Google also has separate models for their "Codey" code generation models.

The "code-gecko" model is useful for code completion:

import { GoogleVertexAI } from "@langchain/community/llms/googlevertexai";

/*
 * Before running this, you should make sure you have created a
 * Google Cloud Project that is permitted to the Vertex AI API.
 *
 * You will also need permission to access this project / API.
 * Typically, this is done in one of three ways:
 * - You are logged into an account permitted to that project.
 * - You are running this on a machine using a service account permitted to
 *   the project.
 * - The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is set to the
 *   path of a credentials file for a service account permitted to the project.
 */

const model = new GoogleVertexAI({
  model: "code-gecko",
});
const res = await model.invoke("for (let co=0;");
console.log({ res });

API Reference:

While the "code-bison" model is better at larger code generation based on a text prompt:

import { GoogleVertexAI } from "@langchain/community/llms/googlevertexai";

/*
 * Before running this, you should make sure you have created a
 * Google Cloud Project that is permitted to the Vertex AI API.
 *
 * You will also need permission to access this project / API.
 * Typically, this is done in one of three ways:
 * - You are logged into an account permitted to that project.
 * - You are running this on a machine using a service account permitted to
 *   the project.
 * - The `GOOGLE_APPLICATION_CREDENTIALS` environment variable is set to the
 *   path of a credentials file for a service account permitted to the project.
 */

const model = new GoogleVertexAI({
  model: "code-bison",
  maxOutputTokens: 2048,
});
const res = await model.invoke(
  "A Javascript function that counts from 1 to 10."
);
console.log({ res });

API Reference:

Streaming​

Streaming in multiple chunks is supported for faster responses:

import { GoogleVertexAI } from "@langchain/community/llms/googlevertexai";

const model = new GoogleVertexAI({
  temperature: 0.7,
});
const stream = await model.stream(
  "What would be a good company name for a company that makes colorful socks?"
);

for await (const chunk of stream) {
  console.log("\n---------\nChunk:\n---------\n", chunk);
}

/*
  ---------
  Chunk:
  ---------
    1. Toe-tally Awesome Socks
  2. The Sock Drawer
  3. Happy Feet
  4. 

  ---------
  Chunk:
  ---------
  Sock It to Me
  5. Crazy Color Socks
  6. Wild and Wacky Socks
  7. Fu

  ---------
  Chunk:
  ---------
  nky Feet
  8. Mismatched Socks
  9. Rainbow Socks
  10. Sole Mates

  ---------
  Chunk:
  ---------
  

*/

API Reference:

Help us out by providing feedback on this documentation page: