Google has made daily token quotas generally available for BigQuery’s generative AI functions to give enterprises a way to limit AI consumption by project and individual user.

These quotas, which can be managed via the IAM and Admin tab in the Google Cloud Console, cover both input and output tokens at the project and individual-user levels per day and apply to the AI.GENERATE, AI.GENERATE_TABLE, AI.GENERATE_TEXT, AI.CLASSIFY, AI.IF, AI.AGG, and AI.SCORE functions.

Last year in November, the hyperscaler added AI-based SQL functions to its managed data warehouse and analytics platform to help enterprises reduce the complexity of running large-scale analytics, especially on unstructured data.

These functions, which use LLMs to prepare unstructured data for analysis, reduce the traditionally time-consuming process of preparing such data because SQL cannot understand its nuances and meaning.

However, embedding functions such as AI.GENERATE_EMBEDDING, also available as ML.GENERATE_EMBEDDING and AI.EMBED are not subject to these quotas because they run on different, substantially cheaper models, the company wrote in a blog post.

For the other functions, by default, input tokens are capped at 200 billion per project and 150 billion per user per day, while output tokens are capped at 20 billion per project and 15 billion per user per day.

The output token quotas also include the reasoning tokens used by the LLM, the company wrote, adding that enterprises also have the option to raise a request to increase these default quotas.

There is also the option to manage these quotas programmatically through the Cloud Quotas API, the gcloud CLI, and Terraform.

How BigQuery enforces token quotas

Beyond that, however, there are nuances to how the data warehouse platform enforces token quotas.

BigQuery rejects a query with a “QuotaExceeded” error if the token quota it needs has already been exhausted. If the quota runs out while the query is running, BigQuery stops sending new requests to the LLM, the company wrote in its documentation.

That means the results of a query can be affected.

For AI.IF, BigQuery can return partial results if the number of rows it cannot process stays within a specified error limit. If that limit is exceeded, the query fails, the company added.

For AI.GENERATE, though, the query can complete execution, but rows that could not be processed have no generated output and carry an error message in their status field, Google further explained.

To help enterprises avoid hitting these quotas while queries are running, Google has also added a new AI.COUNT_TOKENS function to help enterprise teams estimate token consumption before running an AI function.

However, that function too has limitations. It estimates input tokens only and does not count output or reasoning tokens, additional instructions added by the AI functions, or non-text inputs such as images.

This means the estimate may be lower than the workload’s actual token consumption.

To get a more accurate estimate though, the hyperscaler is recommending enterprise teams to run a small sample workload to measure actual token consumption before setting quotas for larger workloads.

Token quotas offer cost control, but FinOps remains essential

Despite these limitations, Pareekh Jain, principal analyst at Pareekh Consulting, sees the daily token quotas as an “important improvement” for enterprises and their CIOs scaling workloads in BigQuery.

“Earlier, unpredictable token usage could lead to unexpected costs. The new controls help CIOs manage spending, and cost predictability is an important factor in scaling any enterprise workload,” Jain said.

That cost predictability, according to Nhat Bui, deputy director of engineering at Scopic Software, could provide more data professionals with more freedom: “Analysts and data scientists can experiment with less risk of an expensive mistake.”

It also abstracts away the complexity of building mechanisms or dashboards to keep tabs on AI function-related spending in BigQuery, Bui added: “The basic guardrail is now built in: daily limits, usage history, and a function to estimate tokens before running a job. Teams no longer need custom tooling just to avoid a runaway bill.”

However, it does not abstract away the need for FinOps, said Manoj Chandra Jha, principal analyst at Nord-IQ Research.

“For CIOs, BigQuery’s token quotas cap the worst case, not the spending story.  They make it safer to say yes to AI in SQL, but they count tokens rather than dollars and offer no team or business-unit budgets, so finance still can’t see who is spending what,” Jha added.

“The smart play is to treat the quotas as a floor, set real limits, structure projects by team so each becomes a de facto budget line, and layer FinOps reporting on top, because predictability without attribution is only half a governance strategy,” Jha further said.

The bigger challenge for enterprises will be balancing the trade-off between cost control and production reliability, Bui noted.

The lower the token limits, the greater the control over AI spending, but the higher the risk of disrupting production workloads, as queries may fail or return incomplete results when quotas are exhausted before the daily reset, Bui said.

To balance these competing priorities, enterprises should separate production workloads from development and experimental workloads, giving the former more room to operate while keeping tighter limits on the latter, Jha said.

Enterprises should also build pipelines that detect skipped rows and process them again after quotas reset, Jha added.

Using vibe coding to scratch an itch has many of the same issues as using low-code and no-code development tools. That’s not surprising. Putting together something quickly to solve a specific problem lacks the level of discipline of software engineering, introducing inefficiencies that come from not following architectural guidelines and policies.

I recently reviewed the AI-generated code I used to quickly build a PDF archive of some HTML-formatted content. The program used common libraries in unusual ways, often combining three or more libraries when all the necessary calls were in one, with a program flow that was hard to understand in places. The result was at best unmaintainable. The only way it was acceptable was that I’d thrown it together in an hour to solve a specific problem, and once I’d built that local PDF archive I could simply throw the program away.

But what if your sloppy AI-generated code was needed by an entire department, or had become part of an essential business process? The risk is that something, somewhere, would inevitably change, and that essential business process would fail, with no practical way of fixing the code. The risk only compounds when the author has left their role and there is no one left who remembers or understands the initial prompts that produced the code.

Adding AI-powered coding tools to products and platforms is risky. But they are too useful to ignore. Like low-code tools, AI tools put coding in the hands of the people who understand the business problems and helps them automate processes on their own schedule.

What’s needed is a way to improve the harnesses used by the coding agents, taking concepts from specification-driven development and using them to build guardrails and standards into the code they produce, with the aim of building code that’s maintainable.

Introducing Rayfin Fabric Apps

Microsoft Fabric is intended to be where all your businesses operational data lives, offering a way to query and explore data at scale. It builds on a mix of big data and data warehousing concepts, using open formats for storage and data mirroring tools to bring data from existing applications into its data lakehouses, as well as supporting streamed data.

Microsoft Fabric allows you to build and deploy your own agents inside the platform to work with your data. However, in many cases, you will need a more focused approach to exploring and using defined subsets of data. The recent addition of Fabric Apps to the platform allows business analysts to deliver applications that use data in Microsoft Fabric to provide essential business insights.

Fabric Apps are built on top of Rayfin, a new TypeScript-based SDK that is designed to work with both humans, via CLI, and AI agents. A partnership with Replit allows Microsoft Fabric users to build apps using Rayfin and AI assistance in Replit and deploy them directly into Microsoft Fabric. Here Replit support is best thought of as a tool for business users, whereas developers can use tools like the GitHub Copilot CLI and Visual Studio Code, which let you mix traditional and AI-supported development processes.

The intent is to provide the tooling that business users need to create data-powered applications inside Fabric, making application development part of the process of exploring data. Rayfin apps can be connected to raw data or draw on semantic models built with tools like Power BI, using DAX for analytics rather than SQL or KQL queries.

Data app development tools for coders and non-coders

The key to this approach is the open-source Rayfin SDK, which offers a way to build out the necessary definitions for data models that can then be used by agent-based tooling. The SDK allows you to set up access to specific models, along with the necessary permissions, before users begin to build on them. Rayfin turns those models into GraphQL endpoints, with TypeScript clients sitting between them and the data stored in Microsoft Fabric — adding an extra layer of protection as the APIs are automatically type safe.

Building a Rayfin-powered Fabric App starts with using Rayfin to define a data structure with decorators that apply types to the data you want to use in your application and defining each table as a class. For example, a table of employee names and contact details would be exposed as an entity that contains an employee class, with decorators that create ID number, name, and address strings, all using structures held in the @microsoft/rayfin-core definitions. Other options allow you to begin to define access controls around the data, so that it can be used only by authenticated users with the correct group privileges.

This is the most important part of any Rayfin-based application development, as it’s here that you’re setting up the guardrails for ongoing user-led development. You’re not limited to one basic application scaffold, either, as there are a series of different templates available to help you get started, from a ready-to-use “to-do” application that illustrates the structure of a complete Rayfin application, to applications that use Microsoft Fabric’s authentication framework to control access to data.

Using Rayfin from the command line

Microsoft provides a process for getting Rayfin scaffolding in place, starting with setting up an application template, then logging in to Microsoft Fabric via Entra ID. Once you’ve completed these steps you can run Rayfin’s development loop to create a runtime and configure the Microsoft Fabric back end, before deploying the base application scaffolding. You don’t need to do this manually; Microsoft’s documentation includes a ready-to-run prompt for your coding agents.

You can now define your data entities. These are placed in a specific directory within the application scaffolding, with separate TypeScript definitions for each class. While you can put all your definitions in a single file, having separate files for each entity makes more sense, as it keeps applications maintainable. Once you’ve added the entities you plan to use, you can rebuild the scaffolding and redeploy, ready for testing.

Usefully this technique lets you separate the back end and front end, ensuring that back-end services and entity definitions run in Microsoft Fabric using its authentication and security framework, with application code running on development hardware (and eventually on separate servers or as part of a local application). This is where you will develop Rayfin functions, TypeScript applications that access and process the defined entities, with predefined typed connectors to work with data already stored in Microsoft Fabric. Rayfin functions can call any HTTPS API, allowing you to mix Microsoft Fabric data with information from any other line-of-business application or service. They’re also where you put the business logic for any Fabric application.

Structuring apps to be coded by machines and managed by humans

Building with the Rayfin SDK produces a predictable application structure with YAML configurations and TypeScript code. That code can be tested locally against a Microsoft Fabric back end. When ready to be deployed and shared, it can be uploaded to Microsoft Fabric, which hosts both the Rayfin function code and any front-end assets. Apps will require Microsoft Fabric capacity to work, and can be accessed from inside the Microsoft Fabric portal, though users can access apps via published endpoints through their browsers.

Rayfin is one of the first examples of a new generation of SDKs and development tools that support both humans and agents. The human-readable Rayfin documentation is paired with an identical set of markdown files, ready for use by any agent working on Rayfin code. Microsoft’s partnership with Replit provides a coding agent to build Rayfin applications and a rendering canvas with familiar Power BI features, providing a place for non-coding business users to experiment with Fabric Apps using AI.

The strict structure Rayfin imposes on Fabric Apps aims to bridge the gap between AI-powered coding and traditional enterprise application development. By putting all the components of an application in a rigidly-defined environment, Rayfin helps ensure that AI-generated code remains secure, compliant, human readable, and human maintainable. Business users can quickly put together Fabric Apps with Rayfin and Replit. Then, if a Fabric App proves to have utility outside a narrow group of users, it can be transferred to the enterprise’s development team for further refinement and long-term operations.