The beta release of the Cloudflare R2 Data Catalog opens the door for a fully featured, lowest cost (for the cloud/hybrid space), and relatively simple deployment of AI Software as a Service (SaaS) products. Combined with Prefect's bring your own compute and a RAPIDS enabled Dask server running on a consumer "gaming" PC at your home or office, it's possible to develop, operate, and maintain a modern AI SaaS offering for under $100 a month and one developer. This post is not sponsored by Cloudflare, Prefect, or any other company.

Building a Low-Cost Artificial Intelligence (AI) Software as a Service (SaaS): Part 1

Goals

There are four driving goals for this series of blog posts and the corresponding technical implementation:

1. Refine my own knowledge of the individual and integrated technology involved
2. Help others on their own self-improvement journey
3. Develop a (hopefully) value added SaaS supporting youth science, technology, engineering, and math (STEM) education and achievement
4. Empirically validate the maximum value balance of on premises and cloud computing offerings

The initial design

Hybrid-Cloud Architecture Diagram

High-Level System Architecture

Frontend

React Router 7.0+ framework mode and Google's OpenID Connect as the primary ICAM provider via the BetterAuth TypeScript framework. Styling driven by Tailwind Plus.

Key Points for Tech Selection:

• Pure TypeScript
• Deployment agnostic (self-hosted, cloudflare, AWS, Google, Azure, etc.)
• The lead maintainer of React Router (Shopify) makes money by the framework having a great developer experience. Next.js is the other popular React Framework that's primarily maintained by Vercel. Vercel makes money by locking in Next.js devs into their higher cost cloud hosting service.
• No fees from a third-party identity management provider like Clerk, Auth0, or Firebase.
• Cognitively easy to understand developer experience through file based nested routing and error boundaries.
• Tailwind is the leading CSS design language system. Tailwind Plus provides loads of production ready templates and examples for an affordable one-time purchase.
• 1st class support for Vite and Cloudflare Wrangler makes the transition from localhost dev to global production become effortless, fast, & reliable.

Cloud Hosted Backend Services

Almost pure Cloudflare with the addition of Prefect Cloud to manage complex data engineering workflows triggered by the frontend but executed on premises.

Key Points for Tech Selection:

• 1st class support for a Python-only backend
• Storage services (D1, R2, KV, R2 data catalog) provide extreme value & performance
• Very minimal computation occurring in the cloud greatly minimizes cost & likely stays in free tier limits for small/medium projects. Best-in-industry pricing model when scaling to higher user volume

Self-Hosted Backend Services

Pure Python API and data engineering services capable of supporting production workloads and efficient data engineering pipelines.

Key Points for Tech Selection:

• FastAPI's ground-up support of asynchronous tasks, Pydantic input validation, and Swagger make it the superior Python REST API platform
• Cloudflare Tunnel and the provided container image makes for effortless and secure communication between cloud and self-hosted services
• Apache Airflow, Apache Beam, Dagster, Pachyderm, and many other options exist for DAG based data engineering pipeline creation and monitoring. Prefect provides a completely free, self-hosted, pure Python, and modern option that has first-class support for Dask and enterprise logging.
• Dask is slightly less performant for some tasks compared to options built on Ray and Polars. However, Dask remains the leader for ease of use, deployment, and compatibility with the rest of the Python ecosystem like scikit-learn and SciPy. Since this will be self-hosted, development velocity is preferred over saving a few seconds of computation time here and there (which would otherwise add up to expensive cloud service provider charges).
• RAPIDS is an NVIDIA sponsored project that has first class support by Dask and Prefect. It allows for easy integration of GPU accelerated computation in our data workflows.
• PyIceberg will allow us to perform CRUD operations on the Cloudflare R2 Data Catalog which implements the Apache Iceberg data standard and protocol.

In-Depth Workshops

For anyone curious about the many Python ecosystem data engineering tools available and wanting to try them out, check out my 2024 Avengercon Workshop and follow up 2025 Avengercon Workshop.

Hybrid-cloud is the sweet spot

Cloud compute costs kill otherwise promising efforts to improve, modernize, and quickly deliver value. Hybrid-cloud deployment models allow use of the cloud where it provides outsized value like cheap, globally accessible, and failsafe blob storage like AWS S3 or Cloudflare R2. Fixed cost and cheap on-premises self-hosted hardware can be seamlessly integrated with high value cloud services for free using technology like Cloudflare Tunnels and wireguard.

For less than an economy car, even individuals can afford to build a rack mount or desktop server with cutting edge GPUs, terabytes of ultra-fast NVMe storage, over a hundred physical CPU cores, and a decent network security setup. This effectively drops the cost to run AI models, data engineering pipelines, and APIs for the price of a few months doing the same on cloud hosted hardware. Better yet, if the business or project fails, the hardware purchased has residual value and can be sold to recoup some of the upfront cost. Self-hosting computational tasks also mitigates the risk of waking up to find an astronomical bill from your cloud provider because of a silly, but all to easy to make, mistake.

Using popular and proven technology

Using popular technology options maintains velocity thanks to search engines, LLMs, and stackoverflow.com likely having accurate answers to any question a developer may have. Python, React, and SQL are the lingua franca of modern software development related to AI-enabled SaaS webapps. The 2024 Stack Overflow survey overwhelmingly backs this up. Pursuing solutions that don't or barely support these languages will cause pain organizationally (hiring, training, knowledge management) and technologically by constraining your organization's options to pivot.

Heavy use of Cloudflare

This post and series is in no way sponsored by Cloudflarre. Cloud and web veterans will notice the conspicuous lack of AWS, Azure, Google Cloud, Vercel, and other mainstream cloud hosting options from this architecture. This is because Cloudflare has steadily positioned itself to steal market share and make our lives as entrepreneurs easier in three key areas:

No nonsense developer experience

Cloudflare's services are routinely simple enough that clicking around for a few minutes often solves blockers. Fully working and routinely updated templates made by cloudflare get projects up and going fast. Services are also generally global, redundant, and optimized by default. No more inter-zone, inter-region, 100+ hour cost optimization setup cycles semi-intentionally designed to squeeze you for cash at any minor mistake.

Aggressive pricing rates and policies

Cloudflare is the only major cloud service provider with no ingress or egress fees for data. This categorically eliminates a double-digit percentage of total cloud spend involved with storing data at any other reputable service provider. As mentioned in the intro, their introduction of Cloudflare R2 Data Catalog makes them a rock-bottom price data lake service provider orders of magnitude less costly than industry leaders like Snowflake, Data Bricks, Delta Lake, and BigQuery.

A few other notable free or nearly free key offerings:

• Cloudflare Tunnels for securely directing traffic to self-hosted API and services
• DNS Registrar that quickly and for no cost (beyond ICANN fee) acquires domain names like brent-stone.com
• Cloudflare Pages and Workers for serverless hosting of interactive websites
• DDoS protection
• Zero-trust access controls to effortlessly apply a world-class layer of security to self-hosted services

Concise but impactful portfolio of services

I believe Cloudflare's limited suite of service offerings, compared to the seemingly endless options at AWS, Google Cloud Platform, and Microsoft Azure, is one of its strengths. Time and again I discover a super-tailored and compelling offering like the Google Cloud Agones Project to provide a high performance multiplayer backend for game development. After half an hour of researching how to use it, random reddit comments and blog posts confirm the cloud provider no longer supports it.

With Cloudflare, I don't believe they've ever shut down a service. This slow and focused business growth strategy gives small teams confidence that development effort integrating their services will pay off for years to come.

Note

In this particular instance, Agones thankfully survived being killed off by the cloud service provider and lives on as an independent project

Comments