Unifying AI Control Planes: Workers AI and AI Gateway

Executive Summary

Cloudflare's unification of its Workers AI and AI Gateway products into a single AI control plane creates a more streamlined and efficient architecture for accessing AI models. This integration is designed to simplify model routing and improve the observability, billing, and security management of AI applications.

The Architecture / Core Concept

The core concept behind this unification is to provide users with a single interface to manage their interactions with various AI providers. AI Gateway acts as a proxy to external model providers, while Workers AI hosts models on Cloudflare's GPU infrastructure. By unifying these control planes, users can connect to any model provider through a centralized point, managing observability, billing, security, and logging with enhanced simplicity.

The architecture allows for requests to be directed through AI Gateway, providing automatic logging and tracking of token usage and request costs, all while maintaining a high level of security and observability. This setup abstracts the infrastructure concerns away from the user, enabling a focus on application logic and model performance.

Implementation Details

The API and binding approach is unified, eliminating the need for separate binding calls for AI Gateway and Workers AI. Users can utilize a single AI binding to call any model. Here’s an example in JavaScript for making a request through the default AI Gateway:

export default {
  async fetch(request, env) {
    const response = await env.AI.run(
      '@cf/zai-org/glm-5.2',
      {
        messages: [
          { role: 'user', content: 'What is the capital of France?' },
        ]
      },
      {
        gateway: {
          id: 'default' // Use 'default' for the built-in gateway
        }
      }
    );

    return new Response(JSON.stringify(response), {
      headers: { 'Content-Type': 'application/json' }
    });
  }
};

To interact via the REST API:

curl "https://api.cloudflare.com/client/v4/accounts/{account_id}/ai/run/@cf/zai-org/glm-5.2" \
  -H "Authorization: Bearer {api_token}" \
  -H "Content-Type: application/json" \
  -H "cf-aig-gateway-id: default" \
  -d '{
    "messages": [{"role": "user", "content": "What is the capital of France?"}]
  }'

Engineering Implications

The transition to a unified control plane is expected to enhance scalability and reduce latency, as it allows for smarter model routing and better resource utilization. With built-in observability, the complexity of managing AI infrastructure is significantly mitigated, providing a more straightforward way of handling AI requests. Cost management is streamlined, supporting unified billing across diverse AI model providers and improving resource allocation when using the shared infrastructure.

My Take

This architectural choice to unify AI Gateway and Workers AI is a practical move toward creating a more user-friendly and resilient model-serving infrastructure. By focusing on model-first routing and smart routing algorithms, the system not only promises improved resilience during provider outages but also aligns with future trends in AI development towards more adaptive and transparent AI systems. As the system matures, it could become an essential component for developers seeking flexibility and robustness in AI model deployment.