SKILL.md
Terraform Resource Generator Skill
Generate new Terraform resources and data sources by learning from existing patterns in the codebase.
Tooling Rule (read first)
Always drive generation, formatting, and linting through Task:
task generate— generates code and runstask fmtautomatically at the endtask fmt— formats Go, Terraform, and whitespacetask lint— runs all linters
Do NOT use go run ./generators/..., go generate, gofmt, goimports, golangci-lint, or make for these workflows. See AGENTS.md and Taskfile.dist.yml (run task --list) for the full command surface.
Overview
The generator creates Terraform resources by combining:
- OpenAPI Spec (
openapi.json) - Aiven's API schema - YAML Definitions (
definitions/aiven_*.yml) - Configuration for resource generation
Note: This skill is the canonical reference for YAML syntax, adapter API, and implementation patterns. The tf-resource-migration skill builds on this for migrating existing SDK resources.
Generated outputs per resource:
zzresource.go/zzdatasource.go- Terraform schema + internal schemazz_view.go- CRUD operation handlers,ResourceOptions,DataSourceOptionsexamples/resources/*/import.sh- Import command examples
Generated aggregated output:
internal/plugin/zz_provider.go- Provider resource/datasource registry
User Interaction Pattern
When the user requests a new resource, gather requirements BEFORE starting:
First Question (REQUIRED)
ALWAYS start by asking what type of Terraform component is needed:
Use the AskUserQuestion tool with these options:
- Both resource and data source - Full CRUD resource + read-only data source
- Resource only - Just the resource (create, read, update, delete)
- Data source only - Read-only data source
Additional Information to Gather
- Resource name - What API resource/endpoint? (e.g., "MySQL database", "Kafka topic")
- Scope - Project-level, service-level, organization-level?
Discovery Process
- Update OpenAPI spec first by running
task get-specto ensure you have the latest API schema - Search OpenAPI spec for operation IDs
- Show the user what you found and ask for confirmation
- Find similar resources in
definitions/and show them - Clarify specifics if needed (composite ID, fields to exclude/rename, custom modifiers)
When NOT to Ask
- If the user provided specific operation IDs -> proceed
- If the user provided a complete YAML definition -> proceed
- If it's a modification to existing definition -> read the file and proceed
Confirmation Before Generation
Always summarize what you're about to create before proceeding.
Learning from Existing Resources
CRITICAL: Before creating a new resource, ALWAYS search for similar existing resources to learn patterns:
# Find all YAML definitions
ls definitions/aiven_*.yml
# Find resources with similar client handlers
grep -l "clientHandler: service" definitions/aiven_*.yml
# Find resources with custom modifiers
grep -l "expandModifier: true" definitions/aiven_*.yml
# Find composite ID examples
grep -l "idAttributeComposed:" definitions/aiven_*.yml
Key files to reference:
definitions/.schema.yml- Complete YAML schema specificationdefinitions/aiven_*.yml- Real working examples- Existing generated code in
internal/plugin/service//zz_.go
Generation Workflow
1. Update OpenAPI Spec
Always start by downloading the latest API schema:
task get-spec
2. Find the API Operations
Search OpenAPI spec for operation IDs:
jq '.paths | to_entries[] | .value | to_entries[] | .value.operationId' openapi.json | grep -i "keyword"
Common patterns: ResourceCreate, ResourceGet, ResourceUpdate, ResourceDelete, ResourceList
3. Find a Similar Resource
Search definitions/ for a resource with similar characteristics:
- Same client handler (project, service, organization, etc.)
- Similar ID structure (single vs composite)
- Similar operations (CRUD, list-based reads, etc.)
Read that YAML file to understand the pattern.
4. Create Your YAML Definition
File: definitions/aivenmyresource.yml
IMPORTANT: Definition files MUST have the aiven prefix. The filename (without .yml) becomes the resource name directly: aivenmyresource.yml -> resource aivenmy_resource.
Start minimal:
# yaml-language-server: $schema=.schema.yml
location: internal/plugin/service/myresource
operations:
- id: OperationIDFromOpenAPI
type: create|read|update|delete
resource:
description: "..."
idAttributeComposed: [field1, field2]
clientHandler: project
5. Generate and Iterate
task generate # Generate code
task build # Build to check for errors
task lint # Check code quality
Review generated zz_*.go files. Refine YAML definition as needed.
YAML Definition Reference
Top-Level Fields
IMPORTANT: Always check definitions/.schema.yml for the complete, authoritative schema specification.
Common fields:
location- Package path (e.g.,internal/plugin/service/mysql)operations- Array of CRUD operations (id, type, resultKey, etc.)resource/datasource- Configuration metadataidAttributeComposed- Fields that compose the ID (e.g.,[project, service_name])clientHandler- API client type:project,service,organization,organizationbilling, etc.remove- Fields to exclude from schema (exact paths or glob patterns)rename- Field name mappings (API field -> Terraform field)schema- Schema customizations (types, validation, behavior)expandModifier/flattenModifier/planModifier- Enable custom Go modifiersversion- Schema version (for state upgrades)beta- Mark resource as beta (requiresPROVIDERAIVENENABLE_BETAenv var)limitedAvailability- Mark resource as limited availabilitylegacyTimeouts- Enable SDK v2-style timeout blocks
Resource Configuration
resource:
description: "..."
deprecationMessage: "..."
importIDFormats: # Optional docs-only override for resources with multiple import forms
- PROJECT/RESOURCE_ID
- PROJECT/RESOURCE_ID/REGION
terminationProtection: true # Check field before delete
stateAttribute: state # Terraform attribute polled after Create/Update/Delete
refreshStateDesired: [ACTIVE, PENDING_PEER] # Call Read after Create/Update and wait for state convergence
refreshStateFailed: [ERROR, DELETED]
refreshStateDelay: "15s" # Wait before Read
removeMissing: true # Remove from state on 404
deleteStateDesired: [DELETED] # Poll Read after Delete until the resource reaches its terminal state
importIDFormats generates every listed form in both the resource's Import documentation and examples/resources/<resource>/import.sh. It does not change IDFields or runtime import parsing. Omit it when the single import form can be derived from idAttributeComposed.
refreshStateExists: true enables a post-Create/Update Read that completes on the first successful Read, retrying transient 404/403 errors. To also wait for an attribute value, set stateAttribute plus refreshStateDesired and optional refreshStateFailed instead — desired already includes that exists retry. Values within each list use OR semantics. A desired value completes the refresh, and any failed value stops it immediately. Values in neither list are treated as pending and retried until the operation timeout.
stateAttribute names a top-level computed-only attribute; the generator rejects anything else. When the value to wait for lives elsewhere — nested in a collection, spread over several attributes, or derived — keep refreshStateExists: true and set ResourceOptions.RefreshStateCheck in init(). It runs after each Read of the post-Create/Update refresh only (never during plain Read, import, or a data source read), and its error decides what happens next: adapter.ErrRefreshStateFailed stops the refresh, and any other error means "not converged yet" and is retried with the same backoff as a transient Read error.
// The status lives inside a nested collection, out of reach of the declarative condition.
// Do not index with Get("file_info.0.file_status"): Get panics when the list is still empty.
func init() {
ResourceOptions.RefreshStateCheck = fileIsReady
}
func fileIsReady(d adapter.ResourceData) error {
list, _ := d.Get("file_info").([]any)
if len(list) == 0 {
return fmt.Errorf("jar file status is not ready: %q", "")
}
info, _ := list[0].(map[string]any)
status, _ := info["file_status"].(string)
switch status {
case "READY":
return nil
case "FAILED":
return fmt.Errorf("%w: jar file verification failed", adapter.ErrRefreshStateFailed)
}
return fmt.Errorf("jar file status is not ready: %q", status)
}
Prefer the declarative form: reach for RefreshStateCheck only when the condition cannot be expressed as desired/failed values of one top-level attribute. See internal/plugin/service/flink/jarversion/jarversion.go and internal/plugin/serviceuser for reference implementations.
deleteStateGone: true enables the delete poller and completes when Read returns 404. deleteStateDesired waits until stateAttribute matches any listed value; a 404 also completes (gone is implied). Delete is re-issued while it errors (for example 409 from dependents still detaching); once Delete succeeds, only Read is polled. Omit both to delete without polling.
Values are validated against the field's enum when one is declared. stateAttribute is required when refreshStateDesired or deleteStateDesired is set.
Examples:
# Wait until the API 404s; no observable terminal state (e.g. aiven_azure_privatelink).
deleteStateGone: true
# Soft-delete API reports state: DELETED, and dependents may still be detaching
# (e.g. aiven_project_vpc, aiven_organization_vpc).
stateAttribute: state
deleteStateDesired: [DELETED]
Prefer deleteStateGone or deleteStateDesired over a hand-written delete disableView + custom poller whenever the terminal condition is "gone (404)" and/or a fixed attribute value. Keep a custom delete view only for genuinely bespoke flows (e.g. a cancel-then-delete state machine).
Datasource Configuration
datasource:
description: "..."
deprecationMessage: "..."
For alternative lookup keys (e.g. look up by name instead of id), see [Data Source with Alternative Lookup Key](#data-source-with-alternative-lookup-key) below.
Operations Configuration
Each operation maps a Terraform action to an OpenAPI operation ID:
operations:
- id: OperationIDFromOpenAPI
type: create|read|update|delete
disableView: true # Don't generate view function (for custom override)
datasourceLookup: true # Inline this read op as readView's id-empty branch on
# the data source (requires resultListLookupKeys).
# See "Data Source with Alternative Lookup Key" below.
resultIDField: OrganizationId # Go field on the lookup result that holds the
# primary id. When set, the lookup only resolves
# the id, then control falls through to the canonical
# read body in the same readView. Requires
# datasourceLookup + resultListLookupKeys.
resultKey: nested_field # Extract from response.nested_field (schema/JSON path,
# used at generation time to scope the OpenAPI schema)
resultKeyField: GoField # Go field name on the client response to drill into at
# runtime, e.g. d.Flatten(rsp.GoField). Use when the Go
# client doesn't strip an extra wrapper exposed by the API
# (response is {accessors: {aws: {...}}} but the client
# returns *CMKAccessorsListOut, so set resultKeyField: Aws).
# When combined with resultListLookupKeys, points at the
# list to search (e.g. resultKeyField: ConnectionPools).
resultToKey: wrapper_key # Wrap response as {wrapper_key: ...}
resultListLookupKeys: # For list responses
APIField: terraform_field # Match items by field
Find examples: grep -A 5 "operations:" definitions/aiven_*.yml
Removing Fields
remove drops attributes the resource should not expose. Entries are matched against the attribute's OpenAPI path, where nested attributes are slash-separated:
remove:
- update_time # top-level attribute
- plans/node_count # attribute nested under "plans"
Entries are glob patterns. * and ? match within a single path segment, ** matches across segments, and a pattern with no wildcards is an exact match:
remove:
- "azure_*" # every top-level attribute with that prefix
- "*google*" # every top-level attribute containing "google"
- "**/errors" # "errors" at any depth
Quote glob patterns — an unquoted leading * is an alias reference in YAML and fails to parse.
Reach for a glob when the removed attributes share an obvious naming pattern, most often a cloud provider that the resource does not support. Some Aiven endpoints serve every provider from one schema, so a provider-specific resource has to drop the rest. A glob keeps doing that when the API grows new fields, whereas an explicit list silently lets them into the schema on the next task generate. See definitions/aivenbyocaws_entity.yml for this pattern.
Globs only match attribute names, not descriptions or enum values, so a cloud_provider attribute whose enum lists azure is unaffected by "azure".
Schema Customization
Override generated schema fields:
schema:
field_name:
type: string|integer|number|boolean|array|arrayOrdered|object
description: "..."
required: true|false
computed: true|false
sensitive: true|false
writeOnly: true|false
forceNew: true|false # Triggers replacement
useStateForUnknown: true|false # Preserve prior state during planning
enum: [val1, val2]
minimum: 1
maximum: 100
conflictsWith: [other_field]
exactlyOneOf: [field_a, field_b]
atLeastOneOf: [field_a, field_b]
alsoRequires: [other_field]
default: value
dropDefault: true # Migration workaround, see tf-resource-migration. Drops the OpenAPI default
deprecationMessage: "..."
Important: Use arrayOrdered (list) for complex objects, not array (set). Sets cause performance issues.
Blocks vs attributes: a nested collection of objects renders as a block, unless it is read-only all the way down — then it renders as a computed nested attribute (Item.RendersAsAttribute). Blocks cannot be computed, so a read-only block could never hold the values the API returns without diffing forever. Making a single nested field optional flips the whole collection back to a block, which changes the syntax users must write, so do that deliberately. Data sources always use blocks.
An object is a one-element list in either form (Item.TFType), which is the SDKv2 state shape. Due for removal in v5.0.0: an object becomes a single nested value, dropping the SizeAtMost(1) validator, the Max: 1 label and .0 indexing. The sites carrying it are marked todo: ... v5.0.0. Generated examples use attribute assignment (file_info = [{ ... }]) for computed nested attributes, not block syntax.
Find examples: grep -A 10 "schema:" definitions/aiven_*.yml
Generated View Structure
The generator produces zz_view.go with this structure:
const typeName = "aiven_my_resource"
func idFields() []string {
return []string{"project", "service_name", "resource_name"}
}
var ResourceOptions = adapter.ResourceOptions{
Create: createView,
Delete: deleteView,
IDFields: idFields(),
Read: readView,
RefreshState: &adapter.RefreshStateCondition{
Attribute: "state",
Desired: []string{"ACTIVE", "PENDING_PEER"},
Failed: []string{"ERROR", "DELETED"},
},
RefreshStateDelay: adapter.MustParseDuration("15s"),
RemoveMissing: true,
Schema: resourceSchema,
SchemaInternal: resourceSchemaInternal(),
TypeName: typeName,
}
var DataSourceOptions = adapter.DataSourceOptions{
IDFields: idFields(),
Read: readView,
Schema: datasourceSchema,
SchemaInternal: datasourceSchemaInternal(),
TypeName: typeName,
}
CRUD function signature (all views follow this pattern):
func createView(ctx context.Context, client avngen.Client, d adapter.ResourceData) error
func readView(ctx context.Context, client avngen.Client, d adapter.ResourceData) error
func updateView(ctx context.Context, client avngen.Client, d adapter.ResourceData) error
func deleteView(ctx context.Context, client avngen.Client, d adapter.ResourceData) error
ResourceOptions Additional Fields
Beyond basic CRUD, ResourceOptions supports:
ModifyPlan— implementsresource.ResourceWithModifyPlan(separate fromplanModifierin YAML)ValidateConfig— implementsresource.ResourceWithValidateConfigConfigValidators— implementsresource.ResourceWithConfigValidatorsRefreshStateCheck— post-Create/Update convergence check thatrefreshStatecan't express (see [Resource Configuration](#resource-configuration))
ResourceData Interface
All views and modifiers interact with Terraform state through adapter.ResourceData:
type ResourceData interface {
Get(key string) any // Get from plan, then config, then state
GetOk(key string) (any, bool) // Get with existence check
GetState(key string) any // Get specifically from state
HasChange(key string) bool // Plan value differs from state
Set(key string, value any) error // Set a value in current state
SetID(parts ...string) error // Set composite ID
ID() string // Get the "id" field
IsNewResource() bool // True if ID is empty
RequiresReplace(keys ...string) // Force replacement over attributes (ModifyPlan only)
Schema() *Schema // Access internal schema
Expand(out any, modifiers ...MapModifier) error // Plan -> API request
Flatten(in any, modifiers ...MapModifier) error // API response -> state
}
Behavior by Operation
CRITICAL: ResourceData behaves differently depending on the CRUD operation:
- Create:
d.Get()reads from plan, then config. No state available. - Update:
d.Get()reads from plan, then config, then state (for ID fields only).d.GetState()reads from prior state.d.HasChange()compares plan vs state. - Read/Delete:
d.Get()reads from state. No plan available.
This matters when writing modifiers:
- In
flattenModifier,d.Get("field")returns the current state value (from plan during Create/Update, from state during Read) - In
expandModifier,d.Get("field")returns the plan value (what the user configured) - Use
d.HasChange("field")to detect if a field was modified (Update only)
Custom Modifiers
For complex data transformations, enable modifiers in YAML:
expandModifier: true # TF state -> API request
flattenModifier: true # API response -> TF state
planModifier: true # Pre-process state before read API call
Then implement functions in a .go file in the resource package.
expandModifier / flattenModifier
Transform data between Terraform state and API request/response formats.
Signature:
func expandModifier(ctx context.Context, client avngen.Client) adapter.MapModifier
func flattenModifier(ctx context.Context, client avngen.Client) adapter.MapModifier
Where MapModifier is:
type MapModifier func(d ResourceData, dto map[string]any) error
d— theadapter.ResourceDatainterface for accessing plan/state/config valuesdto— the raw map being sent to or received from the API
Example — nested API field (pgallowreplication stored as accesscontrol.pgallow_replication in API but exposed top-level in TF):
func expandModifier(_ context.Context, _ avngen.Client) adapter.MapModifier {
return func(d adapter.ResourceData, dto map[string]any) error {
if v, ok := d.GetOk("pg_allow_replication"); ok {
dto["access_control"] = map[string]any{"pg_allow_replication": v}
delete(dto, "pg_allow_replication")
}
return nil
}
}
func flattenModifier(_ context.Context, _ avngen.Client) adapter.MapModifier {
return func(d adapter.ResourceData, dto map[string]any) error {
if v, ok := dto["access_control"]; ok {
dto["pg_allow_replication"] = v.(map[string]any)["pg_allow_replication"]
delete(dto, "access_control")
}
return nil
}
}
Composing multiple modifiers with adapter.ComposeMapModifiers():
func expandModifier(ctx context.Context, client avngen.Client) adapter.MapModifier {
return adapter.ComposeMapModifiers(
getFullCardID(ctx, client),
expandParentID(ctx, client),
ExpandEmails("billing_emails"),
)
}
Find examples: grep -l "flattenModifier\|expandModifier" internal/plugin/service/**/*.go
planModifier
Runs at the start of the generated readView, before the API call. Use it to pre-process or fix up state so the read has all the data it needs.
Signature:
func planModifier(ctx context.Context, client avngen.Client, d adapter.ResourceData) error
When to use:
- A renamed ID field (
rename: {id: deployment_id}) isn't in old SDK state — extract it from the compositeid - A field needs to be resolved before the read (e.g., name -> ID lookup via API)
- Any state pre-processing required before the generated read API call
How it works: Setting planModifier: true in YAML causes the generator to insert planModifier(ctx, client, d) at the top of readView:
// generated readView
func readView(ctx context.Context, client avngen.Client, d adapter.ResourceData) error {
err := planModifier(ctx, client, d) // <-- inserted by generator
if err != nil {
return err
}
// ... API call using d.Get() fields
}
Reference implementations:
internal/plugin/service/billinggroup/billinggroup.go— extractsbillinggroupidfrom compositeidfor SDK backward compatinternal/plugin/service/flink/deployment/deployment.go— extractsdeployment_idfrom compositeidfor SDK backward compat
For name -> ID resolution in data sources, use the generated datasourceLookup pattern instead of a hand-written planModifier. See [Data Source with Alternative Lookup Key](#data-source-with-alternative-lookup-key).
modifyPlan
Despite the similar name, resource.modifyPlan: true has nothing to do with planModifier: it wires the package's modifyPlan function as the resource's ResourceWithModifyPlan handler, which the framework calls on every plan (except a destroy plan).
Signature:
func modifyPlan(ctx context.Context, client avngen.Client, d adapter.ResourceData) error
When to use:
- Plan-time checks that need prior state, e.g. forbidding a decrease in a numeric attribute (
d.GetStatevsd.Get) - Computing a plan value the configuration cannot express, e.g. hashing a file the resource uploads
Whatever modifyPlan writes with d.Set becomes the planned value, which is the equivalent of SDKv2's CustomizeDiff + SetNew. Only computed attributes may be written: Terraform rejects a plan whose value for a configured attribute differs from the configuration.
d.RequiresReplace("attr") marks attributes whose planned value forces the resource to be replaced. Schema plan modifiers (forceNew: true -> RequiresReplace()) run before modifyPlan, so they compare a value it has not written yet: a computed value produced here needs d.RequiresReplace to have any effect. Marking an unknown attribute panics.
// A version is immutable, so edited file content can only go into a new one.
func modifyPlan(_ context.Context, _ avngen.Client, d adapter.ResourceData) error {
checksum, err := fileChecksum(d.Get("source").(string))
if err != nil {
return err
}
if !d.IsNewResource() && checksum != d.GetState(sourceChecksumField) {
d.RequiresReplace(sourceChecksumField)
}
return d.Set(sourceChecksumField, checksum)
}
Reference implementations:
internal/plugin/service/kafka/topic/topic.go— plan-time checks against prior stateinternal/plugin/service/flink/jarversion/jarversion.go— computed checksum that drives replacement
Custom View Overrides
The generator produces standard CRUD views, but you can override any of them via init() when the generated logic is insufficient. Common reasons:
- Create needs extra steps (e.g., reset password after user creation)
- Update touches multiple API operations for different fields
- Delete requires a multi-step state machine (e.g., cancel then delete)
- Read needs post-processing not expressible via modifiers
Use disableView: true on the operation in YAML to skip generating the view you plan to override.
Override pattern:
func init() {
ResourceOptions.Create = createView // Custom create
ResourceOptions.Update = updateView // Custom update
ResourceOptions.Delete = deleteView // Custom delete
}
Example — custom update with multiple API operations (pg_user):
func updateView(ctx context.Context, client avngen.Client, d adapter.ResourceData) error {
if d.HasChange("pg_allow_replication") {
req := &service.ServiceUserCredentialsModifyIn{
Operation: service.ServiceUserCredentialsModifyOperationTypeSetAccessControl,
AccessControl: &service.AccessControlIn{
PgAllowReplication: new(d.Get("pg_allow_replication").(bool)),
},
}
_, err := client.ServiceUserCredentialsModify(ctx,
d.Get("project").(string),
d.Get("service_name").(string),
d.Get("username").(string), req)
if err != nil {
return err
}
}
return resetPassword(ctx, client, d)
}
Example — custom delete with state machine (flinkapplicationdeployment):
func deleteView(ctx context.Context, client avngen.Client, d adapter.ResourceData) error {
project := d.Get("project").(string)
serviceName := d.Get("service_name").(string)
applicationID := d.Get("application_id").(string)
deploymentID := d.Get("deployment_id").(string)
for {
_, err := client.ServiceFlinkGetApplicationDeployment(ctx, project, serviceName, applicationID, deploymentID)
if avngen.IsNotFound(err) {
return nil
}
_, err = client.ServiceFlinkCancelApplicationDeployment(ctx, project, serviceName, applicationID, deploymentID)
if err != nil {
_, _ = client.ServiceFlinkDeleteApplicationDeployment(ctx, project, serviceName, applicationID, deploymentID)
}
select {
case <-ctx.Done():
return fmt.Errorf("can't delete: %w", ctx.Err())
case <-time.After(time.Second):
continue
}
}
}
Reference: See internal/plugin/service/flink/deployment/deployment.go and internal/plugin/service/pg/user/user.go.
Write-Only Fields
Write-only fields are values sent to the API but never stored in Terraform state. Use writeOnly: true in YAML for any field where persisting the value in state is undesirable — passwords, tokens, API keys, configuration payloads, etc.
How it works:
- Write-only fields are automatically excluded from data sources
ResourceData.Get()reads write-only fields from config (not plan or state, since they're never stored)- The generator marks them with
WriteOnly: truein the Plugin Framework schema
YAML:
schema:
my_secret:
type: string
optional: true
writeOnly: true
sensitive: true
Write-Only with Stored Variant
A common pattern offers both a regular field (stored in state) and a write-only variant (not stored), letting users choose. Applicable to passwords, tokens, or any sensitive credential.
YAML:
schema:
password:
type: string
optional: true
computed: true
sensitive: true
conflictsWith: [password_wo]
password_wo:
type: string
optional: true
writeOnly: true
sensitive: true
conflictsWith: [password]
alsoRequires: [password_wo_version]
password_wo_version:
type: integer
optional: true
alsoRequires: [password_wo]
description: Increment to rotate password_wo.
Key points:
conflictsWithensures the user picks one approach or the otherpasswordwoversionacts as a trigger — incrementing it forces a change even though the write-only value itself isn't tracked in state- In
flattenModifier, clear the regular field from state when the write-only variant is active - In create/update logic, read the write-only field via
d.Get()(falls through to config)
Reference: See internal/plugin/service/pg/user/user.go for a complete implementation.
Data Source with Alternative Lookup Key
When a data source should look up by a field that's NOT in idAttributeComposed (e.g., lookup by name but resource ID uses application_id), add a second read operation marked datasourceLookup: true with resultListLookupKeys. The generator inlines the lookup body into readView as an id-empty branch:
func readView(ctx context.Context, client avngen.Client, d adapter.ResourceData) error {
// planModifier (if enabled) runs first
if d.Get("<id-attr>").(string) == "" {
// ... datasourceLookup list call + FindOne by resultListLookupKeys ...
return d.Flatten(&match, ...) // default: flatten match into state and return
// OR (when resultIDField is set):
// d.Set("<id-attr>", match.<resultIDField>) // resolve id, fall through to canonical read
}
// ... canonical read API call by id ...
}
The data source uses the same readView directly — no separate dataReadView is generated. The generator also wires ExactlyOneOf(id, alt_field...) (and RequiredTogether when composedOf has more than one field) on the data source config validators, and marks the alternative fields as optional in the data source schema.
YAML:
clientHandler: flinkapplication
idAttributeComposed: [project, service_name, application_id]
rename:
id: application_id
operations:
- id: ServiceFlinkCreateApplication
type: create
- id: ServiceFlinkGetApplication
type: read
- id: ServiceFlinkListApplications
type: read
datasourceLookup: true
resultKey: applications # Unwrap the response: rsp.applications -> []Application
resultListLookupKeys:
Name: name # API field "Name" matches TF attribute "name"
- id: ServiceFlinkDeleteApplication
type: delete
- id: ServiceFlinkUpdateApplication
type: update
The rename map (e.g. id: application_id) is forwarded to the lookup branch's Flatten call when the API field exists in the lookup response, so the resource id attribute is populated correctly from the list item.
Multi-field composite lookup: if resultListLookupKeys declares more than one mapping, the data source requires either the id attribute OR all alternative fields together. Example (aivenkafkaschemaregistryacl):
resultListLookupKeys:
Permission: permission
Resource: resource
Username: username
resultIDField — id resolution only
Set resultIDField to the Go field name on the lookup result item that holds the primary id when the lookup endpoint and the canonical read endpoint return different shapes (the list response is just a directory of ids; the actual payload comes from the canonical read). With resultIDField set, readView's id-empty branch:
- Calls the
datasourceLookuplist op. - Finds the matching item by
resultListLookupKeys. - Sets the resolved id on state via
d.Set(<id-attr>, match.<resultIDField>). - Falls through to the canonical read body in the same
readView(no recursive call).
The lookup response is not merged into the data source schema, so leaked fields from the directory endpoint cannot pollute it.
resultIDField requires datasourceLookup: true and resultListLookupKeys (enforced by .schema.yml dependencies).
operations:
- id: OrganizationUserList # canonical read, fetches the actual payload
type: read
resultToKey: users
- id: UserOrganizationsList # directory used only to resolve the id
type: read
datasourceLookup: true
resultIDField: OrganizationId # Go field on the list item holding the id
resultKey: organizations
resultListLookupKeys:
OrganizationName: name
rename:
organization_id: id
No planModifier needed: both patterns fully replace the previous datasource.exactlyOneOf + custom planModifier approach.
Reference implementations:
definitions/aivenflinkapplication.yml— single-field name lookup (flatten match into state)definitions/aivenkafkaschemaregistryacl.yml— three-field composite lookupdefinitions/aivenorganizationalunit.yml— name lookup with field renamesdefinitions/aivenorganizationuser_list.yml—resultIDField, lookup resolves id then falls through to the canonical read body
Computed + Optional Pattern
Use computed: true + optional: true when the API always returns a value for a field, even when not configured. Without computed, you'll get:
Error: Provider produced inconsistent result after apply
.field_name: was null, but now cty.False
Solution:
schema:
field_name:
type: boolean
optional: true
computed: true
useStateForUnknown: true # Preserves prior state during planning
Trade-off: Users who didn't configure the field will see a one-time cosmetic diff on upgrade (null -> false), but no resource recreation.
Common Resource Patterns
Search the codebase for similar patterns:
| Pattern | Search Command | |
|---|---|---|
| Data source (list) | `grep -l "datasource:" definitions/aiven_*.yml \ | xargs grep -l "resultToKey"` |
| Composite ID | `grep -l "idAttributeComposed:" definitions/aiven_*.yml \ | head -3` |
| Custom modifiers | grep -l "expandModifier: true" definitions/aiven_*.yml |
|
| Field renaming | grep -l "rename:" definitions/aiven_*.yml |
|
| Alt data source lookup key | grep -l "datasourceLookup: true" definitions/aiven_*.yml |
Best approach: Find a similar resource, read its YAML definition, understand the pattern, adapt to your needs.
Troubleshooting
| Error | Solution | |||||
|---|---|---|---|---|---|---|
operationID X not found |
Search OpenAPI: `jq '.paths \ | to_entries[] \ | .value \ | to_entries[] \ | .value.operationId' openapi.json \ | grep -i X` |
no 'id' field found |
Add idAttributeComposed: [field1, field2] to YAML |
|||||
ID field "X" not found |
Verify field exists in API response, check rename mappings |
|||||
| Generation fails | Run task get-spec, validate YAML syntax, check operation IDs |
|||||
| Infinite plan changes | Use arrayOrdered not array, ensure idAttributeComposed fields are stable |
|||||
| "was null, but now cty.X" | Add computed: true + useStateForUnknown: true to field |
Reference Implementations
| Resource | Patterns Demonstrated |
|---|---|
pg_user |
Nested API fields, computed+optional, custom create/update, write-only fields, expand/flatten modifiers |
mysql_database |
Simple CRUD, list-based read with lookup, termination protection |
billing_group |
planModifier, ComposeMapModifiers, card ID resolution, organization ID conversion |
flink_application |
Alt data source lookup key via datasourceLookup read op, field rename |
flinkapplicationdeployment |
Renamed ID field, planModifier for backward compat, custom delete with state machine |
kafkaschemaregistry_acl |
Composite alt data source lookup (multi-field datasourceLookup + resultListLookupKeys) |
organization/unit |
Alt data source lookup via datasourceLookup read op, expand/flatten for parent ID |
flink/jarversion |
Attributes the API doesn't have, modifyPlan computing a checksum that drives replacement, RefreshStateCheck, custom create that uploads a file |
Testing Requirements
CRITICAL: Every generated resource MUST have acceptance tests covering:
For Resources
- Create - Basic resource creation
- Read - Verify attributes are read correctly
- Update - Modify attributes, verify changes
- Delete - Implicit in test cleanup
- Import - Test
terraform importwith correct ID format
Example test structure:
func TestAccAivenMyResource_basic(t *testing.T) {
resource.ParallelTest(t, resource.TestCase{
PreCheck: func() { acc.TestAccPreCheck(t) },
ProtoV6ProviderFactories: acc.TestProtoV6ProviderFactories,
Steps: []resource.TestStep{
{
Config: testAccMyResourceConfig("value1"),
Check: resource.TestCheckResourceAttr(resourceName, "field", "value1"),
},
{
Config: testAccMyResourceConfig("value2"),
Check: resource.TestCheckResourceAttr(resourceName, "field", "value2"),
},
{
ResourceName: resourceName,
ImportState: true,
ImportStateVerify: true,
},
},
})
}
Find test examples: ls internal/plugin/service//mysql/_test.go
Commands
task get-spec # Update OpenAPI spec
task generate # Generate code
task generate no_spec=true # Generate without spec update
task build # Build provider
task lint # Run linters
task test-unit # Unit tests (no API)
task test-acc -- -run TestName # Specific acceptance test
Key Principles
- Learn from existing code - Always find and read similar resources in
definitions/ - Start minimal - Basic YAML first, add complexity incrementally
- Use arrayOrdered - For complex/large nested objects, always use
arrayOrdered(list) notarray(set) for performance - Mark sensitive fields - Use
sensitive: truefor passwords, tokens, API keys, credentials - Follow conventions - Match patterns from existing resources (naming, structure, etc.)
- **Don't edit
zz_*files** - These are generated. Custom logic goes in separate.gofiles
Important Notes
- New resources go in
internal/plugin/(Plugin Framework), NOTinternal/sdkprovider/(legacy) - Generated files have
zz_prefix - never edit them manually - Custom logic goes in separate
.gofiles in the same package (e.g.,billinggroup.go) - Run
task lintbefore committing - Definition files MUST have
aivenprefix:definitions/aivenmyresource.yml-> resourceaivenmy_resource - Files without
aiven_prefix indefinitions/are ignored by the generator
Migrating Existing Resources
If you need to migrate an existing SDK resource from internal/sdkprovider/ to Plugin Framework, use the tf-resource-migration skill instead. That skill covers:
- Analyzing existing SDK code
- SDK type -> YAML type mapping
- Preserving state compatibility
- Backward compatibility testing
- Deprecation strategy