AI as GraphQL Gateway

Introduction

Imagine you're diving into the world of an incredible app called "What if Pokemons attend StarWars?" Here, users can let their imaginations run wild, doing all sorts of crazy things like packing as many Snorlaxes as possible onto a starship or searching for the perfect planet to train their Squirtle army.

To make this app truly awesome, we've decided to use GraphQL as our API layer. The beauty of GraphQL is that it lets us grab exactly the data we need with just one query, saving us from writing extra code. And Graphs are powerful tools for modeling many real-world phenomena because they resemble our natural mental models and verbal descriptions of the underlying process. Lucky for us, we found some amazing third-party GraphQL APIs for both Pokemon and StarWars. The only thing left to do is figure out how to combine these APIs and give our app all the data it craves.

But here's the thing: we want our app to grow fast without breaking the GraphQL's core principle. That's why we came up with a brilliant idea - to use AI as our GraphQL gateway. So, picture this: a user fires up our app and asks, "Hey, is Pikachu taller than R2-D2?" Let me walk you through the flow of how our app handles that query.

The client initiates the query by sending a message to the server, asking, "Is Pikachu taller than R2-D2?"
The server, upon receiving the query, forwards it to the AI component, requesting a query plan and cost estimate based on the schemas and question provided.
The AI generates a query plan and estimates the cost, sending the plan back to the server. In this case, the estimated cost is $0.0001.
The server evaluates the query plan and checks if the cost is affordable. If it is, the server proceeds to execute the query plan.
The server sends separate queries to both the Pokemon Service and the StarWars Service to retrieve the height of Pikachu and R2-D2, respectively.
The Pokemon Service responds to the server with the height of Pikachu, which is 40.6cm.
Similarly, the StarWars Service responds to the server with the height of R2-D2, which is 110cm.
The server then consults the AI, providing the question ("Is Pikachu taller than R2-D2?") and the height responses (40.6cm and 110cm).
The AI processes the information and sends its conclusion back to the server, stating that Pikachu (40.6cm) is, in fact, shorter than R2-D2 (110cm).
Finally, the server relays the answer back to the client, confirming that Pikachu (40.6cm) is indeed shorter than R2-D2 (110cm).

Our app tackles this question effortlessly thanks to our backend system's capabilities in schema composition, query planning (e.g., GetPikachu and GetR2D2), and query execution. Interestingly, our architecture share some similarities with the Supergraph introduced by Apollo. It enables us to seamlessly combine the schemas of the Pokemon Service and StarWars Service, optimizing data retrieval and delivering comprehensive responses to enhance the user (developer) experience.

In Reality

Supergraph has revolutionized the way we approach GraphQL, and numerous companies have already embraced this architecture. It introduces a unified composition layer built in modules that continually evolve over time. In our case, the Server component, along with the AI component, forms a unified composition layer built in the Pokemon and StarWars modules, adapting to changing needs.

There are three essential architectural components: subgraph, schema registry, and GraphQL gateway. Each subgraph represents an independent GraphQL service, just like our Pokemon and StarWars Services. The schema registry ensures schema validation and composition. It composes together the unified schema and provides it to the gateway. Lastly, the GraphQL gateway serves as the primary interface for handling GraphQL queries from consumers. It takes a client's query, deconstructs it into smaller sub-queries (query plan), and executes this plan by seamlessly redirecting requests to the relevant downstream subgraphs.

Entity

Once our graph is federated, we can define entities that resolve their fields across multiple subgraphs. Each subgraph contributes different fields to the entity and handles the resolution of the fields it provides.

For instance, let's consider the Berry entity, which has fields defined and resolved across the Pokemon and StarWars subgraphs:

Pokemon subgraph

type Berry @key(fields: "id") {
  id: ID!
  name: String!
  effect: String!
}

StarWars subgraph

type Berry @key(fields: "id") {
  id: ID!
  growBy: [Species]!
  growOn: [Planet]!
}

Entities are a fundamental building block of Apollo Federation that enable subgraphs to adhere to the separation of concerns principle.

Next Step - Supergraph Linking [Not in the Spec yet]

Supergraph let us link across types published by different companies. Give us the ability to refer to external type, in this case, the PokemonCenter type.

pokemon.graphql

@link(to: "pokemon_center", as: "pc")
type Berry @key(fields: "id") {
  id: ID!
  name: String!
  effect: String!
  canBuyAt: [pc_PokemonCenter]!
}

pokemon_center/schema.graphql

type PokemonCenter {
  id: ID!
  name: String!
}

Tips for Federated GraphQL in Enterprise

Use federation. Not too much. Mostly along team boundaries.
One Schema, One Graph. Make data availability and sources clear.
Focus on making the schema intuitive and usable by everyone.
Think of subgraphs as the subdomains in your supergraph.
Adopt an entity-centric mindset, as features often span multiple types across different subgraphs.
Follow a schema-first approach, facilitating tooling integration and consumption. Backward-incompatible changes are more obvious with schema diffs. Backward compatibility is even more critical when working in a Federated GraphQL architecture.
Utilize schema linting to ensure schema quality and consistency.
Propose and document schema updates for visibility, feedback, and validation by all stakeholders.
Establish a schema working group to collaborate and grow GraphQL expertise.
Use a subgraph-focused workflow. If we're part of the Pokemon team, we shouldn't need to spin up an instance of StarWars subgraph to develop features.
Isolate the client development. If we're part of the frontend team, we shouldn't need to spin up all the subgraphs locally to develop features.
Isolate the federation environments to preview subgraph changes while keeping other services in a default state.
Add a caching layer to avoid making duplicate network requests for the same or similar data. GraphQL reduces client side over fetching, not server side.
Chain the type systems to gain type safety from the database layer all the way to the client.
Utilize entity relationships in GraphQL for data aggregation and updating search indexes.
Don't version the schema; instead, use @deprecate. Versioned fields are fine and we can add a lot of fields.
Mark slow parts of the query with @defer, preventing a single slow field from delaying the entire page's loading and displaying loading indicators everywhere.
Employ observability tools to track the health and performance of our graph. These tools help surface patterns in how our graph gets used, which helps us identify ways to continue improving our graph.

In the Future

Envision a future where the Graph expands exponentially, empowering the AI as the GraphQL gateway to provide answers to nearly all our queries. This advanced AI system handles everything we need to do ourselves now, including generating query plans, managing AuthN and AuthZ, estimating query costs, and optimizing performance through techniques like batching and caching. Just imagine the possibilities!

As the Graph evolves and becomes more comprehensive, it raises the question: What should we call this extraordinary entity? Could it be referred to as the "One True Graph" or perhaps the "Great Graph" (GG 😳)? The potential is immense, and the future of this unified and intelligent Graph holds endless possibilities.