Marcus wants four bags of bread flour, two tubs of vanilla bean paste, and a dozen packs of parchment sheets. His email also says "before friday the 15th ideally" and "around 340 bucks." In the recorded extraction below, Gemini turns that message into an order object with the right quantities, high priority, a total_usd of 340, and a deliver_by string of "Friday the 15th".

That's useful. It's also a good reason to look past the neat JSON. Which month is Friday the 15th? Is $340 a quote or the customer's estimate? A database can happily store both fields while the person fulfilling the order still needs answers.

Two saved order extractions. Marcus's email becomes three line items with quantities 4, 2, and 12. In the chat example, Priya changes her mind from one dripper to two; the result captures two drippers, one pack of filters, and low priority, with no total or delivery date. The final green block shows each saved structured result. Replay pacing is for readability, not a latency benchmark.

Priya's example shows why optional fields matter. Her message supplies neither a price nor a delivery date, and the result leaves both out. Marcus's example shows why the schema itself matters: a numeric total has nowhere to preserve "around," and a free-text date can retain ambiguity. These two successful runs demonstrate extraction behavior, not a general accuracy rate.

Structured output gives software a predictable object to work with. The rest of this article follows that object into an application: how to get its shape right, check what its values mean, and decide whether it should trigger an action.

The naive approach: prompt engineering and regex

A prompt can ask for "valid JSON with no additional text." That helps communicate intent, but an ordinary text response can still include a preamble or the wrong field types. A common patch is to search for braces and parse whatever is between them:

import re import json def extract_json(text: str) -> dict: # Try to find JSON in the response match = re.search(r'\{.*\}', text, re.DOTALL) if match: return json.loads(match.group()) raise ValueError("No JSON found in response")

This greedy regex can capture one nested object, but it also swallows everything between the first opening brace and the last closing brace. Two separate objects or braces in surrounding prose can break the extraction. Missing commas still fail in json.loads, and an array has no outer braces to find. Even successful parsing says nothing about whether the expected fields are present.

The real issue is that LLM decoding is unconstrained by default. At every token the model assigns probability across its full vocabulary, and nothing stops it from picking one that violates your schema. The fix is to constrain the decoder itself, not the prompt.

Constrained decoding: enforcing structure at the token level

Constrained decoding changes the sampling step. Before each token, the system filters the vocabulary down to tokens that keep the output consistent with a schema. Some teams call it grammar-guided generation, or logit masking. Same idea, different name.

Think of it as completing a form whose field types are enforced while you type. The model still chooses the values, but a constrained decoder rules out token sequences that cannot complete the supported schema. If age must be an integer, a quoted string such as "twenty-five" is not a valid value.

Schema: required age (integer), required name (string)

Object begins:             {
One allowed field:         "age":
An allowed integer value:  25
Next required field:       , "name":
An allowed string value:   "Ana"
Complete object:           }

This is a character-level illustration; model tokens can contain several characters, and whitespace or different field orders may also be valid. An incomplete generation can still stop before the closing brace. The separate article on grammar-constrained generation explains the decoder in more detail. Here, the useful boundary is that a supported schema controls the form of a completed answer. It cannot establish whether 25 is someone's real age.

Modern solutions in practice

The ecosystem has settled into a handful of approaches, each with its own trade-offs.

API-level JSON mode and structured outputs

JSON mode and schema enforcement solve different problems. JSON mode requests parseable JSON; a schema adds requirements about fields and types. For example, OpenAI's Chat Completions API accepts response_format with type: "json_schema" and a strict schema, while its Responses API puts the format under text.format. Refusals and incomplete responses need separate handling before using the result. OpenAI's structured-output guide documents those paths.

Gemini's Generate Content API accepts a JSON response MIME type and a schema. Supported schema features vary by provider and model, so test the actual contract you intend to use. A schema accepted by one service is not automatically portable.

Function calling and tool use

Function calling attaches a proposed operation to structured arguments. It fits a search, database write, or API request. This declaration describes a contact-extraction tool; it does not execute anything or, without a provider's strict mode, promise that every returned argument satisfies the schema:

tools = [{ "type": "function", "function": { "name": "extract_contact", "parameters": { "type": "object", "properties": { "name": {"type": "string"}, "email": {"type": "string", "format": "email"}, "phone": {"type": "string"} }, "required": ["name", "email"] } } }]

A concrete example: Gemini structured output and a function call

A calendar request makes the distinction concrete. Extracting "lunch with Maria next Friday" produces data. Creating an event changes someone's calendar. The following illustrative examples use the google-genai SDK's Generate Content API, which Google now labels legacy. They are separate from the recorded order runs above.

Step 1: define the schema with Pydantic

from pydantic import BaseModel, Field class MeetingRequest(BaseModel): title: str = Field(description="Short meeting title") attendees: list[str] = Field(description="Names or emails of attendees") start_iso: str = Field(description="ISO 8601 start with timezone offset") duration_minutes: int = Field(ge=5, le=480)

Pydantic defines the local contract. Descriptions help the model interpret fields; ge and le become numeric bounds. Google's structured-output documentation lists minimum and maximum among supported constraints. Local validation remains useful, especially when schemas or providers change.

Notice that start_iso is still a string. Its description alone does not validate a date, resolve "next Friday," or supply a timezone. We need those checks before creating an event.

Step 2: call Gemini with response_schema

from google import genai client = genai.Client() # reads GEMINI_API_KEY response = client.models.generate_content( model="gemini-2.5-flash", contents=( "Extract this confirmed request: lunch with Maria on " "2026-05-22 at 12:00, UTC-06:00, for 1 hour." ), config={ "response_mime_type": "application/json", "response_schema": MeetingRequest, }, ) # Treat missing, blocked, or unfinished output as a separate result. if not response.candidates or response.candidates[0].finish_reason != "STOP": raise RuntimeError("No completed extraction; inspect finish/block metadata") if not isinstance(response.parsed, MeetingRequest): raise ValueError("No validated meeting returned") meeting = response.parsed print(meeting.title, meeting.start_iso, meeting.duration_minutes)

The MIME type requests JSON, response_schema supplies the contract, and response.parsed provides the SDK's parsed result when available. The guard prevents the application from treating a missing or unfinished result as a meeting. Handle request errors as well; a schema does not make the network reliable.

The example supplies a confirmed date and offset on purpose. If the user only says "next Friday," first resolve that phrase against the user's date and timezone, and clarify it when necessary. Then check the parsed timestamp, identify which Maria is intended, and confirm the user can book the target calendar. A valid duration of 60 minutes says nothing about whether that hour is available.

Step 3: define the tool

Same domain, exposed as something the model can call instead of something it fills in:

def create_calendar_event( title: str, attendees: list[str], start_iso: str, duration_minutes: int, ) -> dict: """Simulate creating a calendar event; perform no external write.""" # A real adapter would call the calendar only after authorization. event_id = "evt_" + start_iso.replace(":", "").replace("-", "") return {"event_id": event_id, "status": "dry_run"}

The SDK derives a tool declaration from the function's signature and docstring. This stub performs no calendar write. Its return value is useful for demonstrating dispatch, but it is not proof that a booking exists.

Step 4: inspect the proposed call before dispatch

The Python SDK automatically executes supplied Python functions by default. To inspect calls yourself, disable that behavior explicitly, as shown in the SDK's function-calling documentation.

from datetime import datetime from google.genai import types response = client.models.generate_content( model="gemini-2.5-flash", contents=( "Prepare a 30 minute design review with Ana and Luis on " "2026-05-21 at 10:00, UTC-06:00." ), config=types.GenerateContentConfig( tools=[create_calendar_event], automatic_function_calling=types.AutomaticFunctionCallingConfig( disable=True ), ), ) if not response.candidates or response.candidates[0].finish_reason != "STOP": raise RuntimeError("No completed response to inspect") for fc in response.function_calls or []: if fc.name != "create_calendar_event": raise ValueError("Unexpected tool") meeting = MeetingRequest.model_validate(fc.args, strict=True) start = datetime.fromisoformat(meeting.start_iso) if start.tzinfo is None or start.utcoffset() is None: raise ValueError("A timezone offset is required") # This handler is a dry run. A real write requires application-owned # calendar permission, attendee resolution, and availability checks. result = create_calendar_event(**meeting.model_dump()) print(fc.name, "->", result)

The allowlist checks the operation, Pydantic checks the arguments, and datetime checks that the string represents a timestamp with an offset. None of these grants calendar access. For a real booking, authorization must come from the authenticated user's permissions, and the application must resolve attendees, check availability, and prevent a retried request from creating duplicate events.

A model may return no function call. Treat that as its own outcome, rather than assuming candidates[0] contains an executable request. If you want a final response after execution, send the tool result back as a function_response in the conversation. The Gemini function-calling guide shows that request-and-response cycle.

Choose the contract for the job

Use structured output when the result is data for your application to process. Use function calling when the model should propose an operation. Both involve schemas, but provider guarantees and execution behavior differ. In either case, keep the application responsible for accepting the values and permitting side effects.

Instructor: Pydantic-native structured outputs

Instructor is a Python library that patches the OpenAI client (and friends) so calls return Pydantic models instead of strings. You define your output type and Instructor handles schema generation, validation, and retries.

import instructor from pydantic import BaseModel from openai import OpenAI client = instructor.from_openai(OpenAI()) class ContactInfo(BaseModel): name: str email: str phone: str | None = None company: str | None = None contact = client.chat.completions.create( model="gpt-4o", response_model=ContactInfo, messages=[{ "role": "user", "content": "Extract: John Smith, [email protected], works at Acme Corp" }] ) print(contact.name) # "John Smith" print(contact.email) # "[email protected]"

Instructor returns a validated Python object and can retry failed validation with feedback. Set a retry budget and handle exhaustion: repeated attempts cost time and tokens, and they cannot supply facts missing from the input. Its retry documentation covers those controls. Here, email is a plain string, so the example does not establish that it is a valid or deliverable address.

Outlines: grammar-level control for open models

Outlines does constrained decoding for open-weight models you host yourself. It constrains generation using supported output types and grammars. The following example keeps the sentiment vocabulary and confidence range explicit, using the current Transformers integration and JSON output type:

import json import outlines from outlines.types import JsonSchema from transformers import AutoModelForCausalLM, AutoTokenizer model_name = "mistralai/Mistral-7B-v0.1" model = outlines.from_transformers( AutoModelForCausalLM.from_pretrained(model_name), AutoTokenizer.from_pretrained(model_name), ) schema = '''{ "type": "object", "properties": { "sentiment": {"type": "string", "enum": ["positive", "negative", "neutral"]}, "confidence": {"type": "number", "minimum": 0, "maximum": 1} }, "required": ["sentiment", "confidence"] }''' text = model( "Classify this review: The food was excellent but the service was slow.", output_type=JsonSchema(schema), max_new_tokens=128, ) result = json.loads(text) # A completed constrained result follows the supported schema. # The confidence value is model-generated, not a calibrated probability.

Choosing an integration

Provider structured outputs are a convenient starting point when you're already using that API. Test schema support and failure handling against the model you deploy. Function calling fits proposed actions, with execution controls kept in the application.

Instructor adds Pydantic validation and retries across supported provider modes. Outlines lets you constrain a model you host, which adds serving work and schema-processing costs. Neither choice removes the need to validate meaning. In the sentiment example, a confidence value between zero and one meets the contract even if the model is confidently wrong.

I'd choose based on where the application already runs, the schema features it needs, and who should control execution. Measure latency with that setup rather than assuming schema enforcement or retries are free.

Before the object becomes an action

Marcus's email became usable fields without a hand-written parser. Priya's changed quantity survived extraction, and absent details stayed absent. Those are concrete improvements over trying to recover JSON from a conversational answer.

To use the result, the application still needs a completed response, a validated shape, and values that make sense for the operation. For Marcus, that means confirming the delivery date and treating $340 as an estimate until the order system calculates the price. For the calendar example, it means resolving the time and attendees before an authorized write.

That's where I would draw the production boundary: accept structured output as a candidate record or proposed action. Commit it only after the same checks you would require from any other input to the system. The schema saves you from repairing the envelope; your application still has to decide what to do with what's inside.