Node.js, a powerful JavaScript runtime built on Chrome's V8 engine, is renowned for its scalability and performance. Today, we'll harness these capabilities to build a command-line interface (CLI) tool that streams OpenAI responses using Server-Sent Events (SSE) and Server-Side Rendering (SSR).

Prerequisites

Basic understanding of JavaScript and Node.js. - Ensure you have Node.js installed. You can download it from the official website.

Project Setup

First, set up a new Node.js project:

mkdir openai-streaming-cli && cd openai-streaming-cli
npm init -y

Install Required Packages

Next, install the necessary packages:

npm install axios express

We'll use axios for making HTTP requests and express to set up our server for SSE.

Create the CLI Tool

touch cli.js

Edit cli.js with the following code:

#!/usr/bin/env node
 
const axios = require("axios");
const express = require("express");
const open = require("open");
 
const app = express();
 
const PORT = 3000;
 
app.get("/stream", (req, res) => {
  res.setHeader("Content-Type", "text/event-stream");
  res.setHeader("Cache-Control", "no-cache");
  res.setHeader("Connection", "keep-alive");
 
  axios
    .post(
      "https://api.openai.com/v1/engines/davinci-codex/completions",
      {
        prompt: "Hello, OpenAI!",
        max_tokens: 50,
      },
      {
        headers: {
          Authorization: `Bearer YOUR_OPENAI_API_KEY`,
          "Content-Type": "application/json",
        },
        responseType: "stream",
      }
    )
    .then((response) => {
      response.data.on("data", (chunk) => {
        res.write(`data: ${chunk}\n\n`);
      });
      response.data.on("end", () => {
        res.write("event: end\n\n");
        res.end();
      });
    })
    .catch((error) => {
      res.write(`data: ${JSON.stringify(error)}\n\n`);
      res.end();
    });
});
 
app.listen(PORT, () => {
  console.log(`Server running at http://localhost:${PORT}/stream`);
  open(`http://localhost:${PORT}/stream`);
});

Make the script executable:

chmod +x cli.js

Run the CLI Tool

./cli.js

How It Works

Setting Up Server

We use express to create a simple server that listens on port 3000. When a GET request is made to the /stream endpoint, the server responds with a series of events streamed over a single HTTP connection. This is achieved using Server-Sent Events (SSE), which allows the server to push updates to the client in real-time.

Ensure that the server is running on a secure environment and properly configured to handle incoming requests, especially if deploying this tool in production environments.

Making the OpenAI API Request

We use axios to make a POST request to the OpenAI API, specifically to the davinci-codex engine's completions endpoint. This request includes a prompt and the maximum number of tokens for the response. The responseType: "stream" option ensures that the response data is streamed back to us.

Make sure to replace YOUR_OPENAI_API_KEY with your actual OpenAI API key before running the CLI tool.

Handling the Streamed Response

When the response is received, we listen for data events on the response stream. Each chunk of data received is sent to the client as a new SSE message. Once the response ends, we send an end event to the client and close the connection.

There you go!

In this tutorial, we've built a simple Node.js CLI tool that streams responses from the OpenAI API using Server-Sent Events (SSE). This setup allows for efficient real-time data streaming, providing a robust foundation for building interactive and responsive applications.

Feel free to explore and expand upon this project, integrating more advanced features and customizing it to fit your specific needs. Happy coding!

Feel free to explore the entire codebase on GitHub if you ever need a helping hand.

OpenAI Response Streaming with Node.js CLI