Podcast about DevSecOps

#70 - System Initiative Goes Ga

A shared infrastructure canvas challenges pull request reviews and plans that go stale before apply.

16 min read

Andrey, Mattias, and Paulina are joined by Paul Stack, an IaC tools developer and a frequent guest on the show. He’s back to discuss the general availability of System Initiative and share what has changed since his last visit when they talked about the early beta of the tool. Will this be a revolution or evolution in Infrastructure as Code tooling? Do we really need collaborative infrastructure management tools? Tune in to find out!

Discuss the episode or ask us anything on LinkedIn


Who gets to change your infrastructure

Two engineers need to change the same VPC this week. One writes the HCL, the other reviews the diff, and what comes back is a note about naming and a question about whether the module should take a list or a map. Nobody says whether the network the change describes is the right network. Reading the code and picturing the network it produces are different skills, and only the first fits in a review window. The pull request merges. You find out what it actually did when the apply runs.

That’s the ordinary case, and the structural version is worse. One or two people on your team read infrastructure code fluently, so one or two people do the infrastructure and everybody else opens a ticket. Put the question to your own team: how many people could have rejected your last infrastructure change on what it does, rather than on how it reads? System Initiative reached general availability on 25 September 2024, five years into a project that treats that concentration as the bug.

Paul Stack has spent his career on the same problem from the tooling side. He worked on Terraform at HashiCorp early enough to remember writing providers before helper schema existed, and left around the time providers were split out of Terraform’s central repository. Then came years at Pulumi, and now director of product at System Initiative. This is the third infrastructure as code tool he has helped carry to a 1.0 or a GA, and he makes the strongest claim for it: a revolution, rather than one more step along the line that produced the other two.

GA is where that claim stops being free. The product now has a price, a trial, and a list of things it cannot do yet.

An automation platform that happens to do infrastructure

Start with the category, because the claim being made is bigger than a product launch.

Is System Initiative an infrastructure as code tool?

No, and the team is careful to say so: it’s an automation platform, and infrastructure is the first job it’s pointed at.

The lineage matters here. CFEngine, then Puppet, then Chef, then Terraform and Ansible: every one of them is desired state and reconciliation, each standing on the last. The pitch says none of that line solves complex workflows, and that claim overshoots. Ansible’s own documentation has described multi-tier rollouts with rolling updates, delegated actions and coordination with monitoring servers and load balancers since its 2.9 release in 2019. The narrower complaint survives: nothing on that line gives two people a shared, interactive model they can stand inside at the same time. A workflow layer on top of the declarations is fair to call an evolution. Paul’s claim is that System Initiative isn’t on that line at all.

Security is the next surface they want. A CISO writes compliance rules as qualifications, the graph marks every node that fails them, and the failing set lands in a change set that pulls in the people who can fix it. Same machinery for the nodes exposed to a CVE that dropped this morning. FinOps follows because cost tools sit off to one side.

None of that was built when we spoke. Treat it as the shape of the bet, not a feature list.

Multiplayer editing is the whole bet

If the platform claim is the long game, the thing you can touch today is narrower, and it’s the only feature the team calls a killer feature.

What’s wrong with agreeing on infrastructure in a pull request?

People review code the way they’d have written it, so the review lands on structure and naming and leaves the outcome unexamined.

The failure mode is easy to picture once you’ve seen it at scale. There’s a Terraform pull request that collected 1,800 automated bot comments, one per nit, with whatever a human reviewer added somewhere in the middle. The other form we all use is a Zoom call where one person types and the other talks, which is communication rather than collaboration. Neither puts two people in the same change at the same time.

The premise is the part we’ve argued with, though that argument happened on his last visit rather than this one. For the open beta a year earlier, Andrey put the objection to him: in most companies he sees, one to three people do nearly all the infrastructure work, their paths rarely cross, and real-time editing reads like a fix for coding rather than for infrastructure. Paul’s answer then was that the small group is the symptom. This time he put the weight on the enterprise shape: several teams on one canvas with designated approvers, in place of several teams in several repositories with a platform team in between. Cognitive load is the same case in developer experience terms: what costs you isn’t the typing, it’s the communication around it.

A canvas everyone shares also gets crowded, and a large one has been described as staring at the sun. Views are the planned fix: slices of the workspace by team, application or environment, so your team sees the twenty components it runs and an architect sees a wider slice. That work had a five-week budget ending on 15 November 2024.

Change sets replay the real world

Multiplayer only works if two people editing at once can’t wreck each other’s work, which is where the model underneath matters.

Why does a plan go stale between review and apply?

Because a plan is a reading of the world at one moment, and reading the world again gives you a new plan rather than an updated version of the one your colleague approved.

A change set is a fork of the whole graph, close enough to a Git branch that it’s the analogy everyone reaches for. You work in it, it keeps talking to the cloud underneath, validations run against the real API while you edit, and only applying it to HEAD touches anything real. What changed since the beta is the merge semantics. Merging graphs produced conflicts, so the engine moved to operational transform: small events, replayed. Anything anyone applies to HEAD replays through every open change set, so the thing you’re looking at is always the current infrastructure. The GA write-up puts the consequence plainly: you never wind up with a situation where the plan no longer reflects the apply.

Compare that with the afternoon your terraform plan passes review, the world moves, and terraform apply fails on something nobody changed on purpose. You can plan again, and Terraform Cloud has watched for drift since June 2022, alerting when state stops matching the real resources. Both tell you something moved. Neither carries the approval forward, because the plan your colleague signed off on is still the old one. Or you take turns on one state file because the lock says so.

This isn’t a claim that Terraform was designed badly. It is stateful and unidirectional by design, with the code as the source of truth, and everything above follows from inverting that.

One-to-one with the cloud

Inverting it only pays off if the model is close enough to the cloud to be trusted, which is the design decision everything else rests on.

What do you do when AWS ships something your tool doesn’t model?

You write the asset yourself, in your own workspace, the day the feature lands.

Assets are designed to mirror cloud resources one for one: an EC2 instance in System Initiative is meant to behave the way one behaves in AWS, with no provider schema in between and nobody to wait for. Coverage is a separate question from that design goal, and Adam Jacob answered it on the day of the GA: “Things aren’t always perfectly modeled today, but we’re working on it”. Check the resource types you actually run before you decide what the mapping buys you.

Where the mapping holds, it pays off in release weeks. December is the annual test: re:Invent ships a pile of new surface area and the normal experience is waiting for Terraform, Pulumi or CloudFormation to catch up. Here the code driving the resources lives in your workspace, so you can change it the same week. That code runs in a pool of Firecracker micro VMs, one VM per function and destroyed afterwards, which is what you’d design if every customer could run arbitrary TypeScript in your platform.

The same mapping is what the AI plan rests on. The team had started asking OpenAI models for an asset and its actions, reasoning that a model reproducing a public API gets further than one asked for a Terraform provider, which has an abstraction layer to get right too. The further idea is a model as another player on the canvas, told in natural language to connect the security group to the load balancer. Both were ideas rather than results when we spoke.

Coverage follows from who’s writing. System Initiative runs on AWS, so AWS is what’s modelled, and the rest came from users: Cloudflare assets from a customer with workloads there, Google assets starting to appear, DigitalOcean as an in-house experiment.

Importing what you already run

All of which assumes you’re modelling something new, and almost nobody is.

Can you start from the infrastructure you already have?

Yes for AWS VPCs and subnets, which shipped with import functions on 7 November 2024, and for anything else once you write the import function yourself.

When we recorded, import existed behind a feature flag in production and greenfield was the only supported way in. It landed publicly a few weeks later. Every schema picked up a universal resourceId property: you drop the asset on the canvas, set the resource ID, and run the import function, which fetches the resource from the cloud API, fills the component in and queues a refresh for when the change set applies. Only AWS VPC and AWS Subnet shipped with that function written, with the rest of the library promised over the following weeks. Any asset System Initiative can manage can carry one, and the announcement’s worked example is a GitHub repository, so the gap is work rather than a wall.

The difference from terraform import is worth being precise about. Terraform 1.5 made importing plannable in June 2023: an import block puts the import in a plan you review before state changes, and terraform plan -generate-config-out writes the matching resource blocks for you. What differs here is where that review happens. The import lands in a change set beside whatever else you’re modelling, and abandoning that change set throws the lot away with no risk to the upstream resource.

Everything starts from a credential, the same thing your Terraform provider block wants: API keys, or an assumed role. After that an AWS region is a frame on the canvas and your infrastructure goes inside it, with how-to guides for an S3 static site behind CloudFront, a VPC and macOS instances.

Approvals shipped, self-service didn’t

Import puts your estate in front of everyone in the workspace, which raises the question the pull request used to answer: who is allowed to press the button.

Who approves a change when there’s no pull request?

A named approver on the workspace, since 14 November 2024: a collaborator builds the change set and requests approval, and only an approver can let it reach HEAD.

It’s the first slice of a relationship-based access control system built on SpiceDB. Workspace owners are approvers by default and can hand the role to collaborators. That’s the piece enterprise teams asked for first, and it landed four weeks after we recorded.

Here’s where we’d temper expectations, from having tried the same move with Terraform. We set an account up, told the developers they could create their own buckets and access keys, and imagined the end of the queue.

That experience doesn’t tell us why. One theory worth testing on your own team is that an approval gate plus a worked template gives people something to start from, where a gate on its own leaves them where they were. The template half wasn’t there at GA. There’s no equivalent of a Terraform module, and System Initiative ran its own multi-account production estate by copying a chunk of the canvas, pasting it and changing the variables, which the team says plainly isn’t the answer. The planned answer is a management node: a node that manages other nodes, so one place enforces your tag standard and rolls a change across regions, accounts and environments.

One more constraint to plan around: there was no API. The product is the canvas, so there’s nothing for your pipeline to call, and several users had already asked for one.

What it costs

Pricing is where a tool tells you what it thinks it is, and this one prices the real world rather than your code.

What counts as a billable resource?

Something that exists in your cloud account, not every block in your configuration.

The published pricing is $0.007 per resource hour after a 30-day trial, with up to 100 concurrent resources under management free. Read that threshold carefully, because crossing it bills the whole month: exceed 100 concurrent resources in any hour and you’re charged for all resource hours that month, including the hours under the limit. Concurrency is the peak at a single point in the hour, so a burst that touches 101 resources at 02:00 on the 3rd puts every hour of that month on the invoice.

A resource costs roughly five dollars a month at that rate, so the free tier waives around $500 a month in management fees. There’s no charge for extra users and none for function executions, and modelling infrastructure you never deploy costs nothing, because nothing is under management.

The gap between a resource and an IaC resource is worth checking against your own estate. Adam Jacob’s estimate on the Lobsters thread the day of the GA is about ten times fewer billable resources than the equivalent Terraform or Pulumi code, because things like IAM role attachments aren’t counted separately. Security groups were the live question: someone with 100 rules asked whether that blows the free tier, and the team was exploring a charge for the group rather than the rules inside it.

Self-hosting is the other lever. The software is Apache 2.0 and the whole repository is public, though at GA running it yourself meant the local development setup, nix with flakes plus docker, rather than a supported deployment. Bring your own cloud and on-premises sat under the enterprise plan.

Where the rest of the market went

The contrast is easy to read that autumn, because the incumbent shipped its own answer to complex workflows three weeks after this GA.

HashiCorp announced the public beta of Terraform Stacks at HashiConf on 15 October 2024, with components wrapping existing modules and deployments rolling a stack across environments, free for up to 500 resources during the beta. It’s the evolution answer: keep the code, keep the state, add a layer that coordinates configurations that depend on each other. Both are aimed at the same complaint. One concludes the problem is orchestration above the code, the other concludes the code is the problem.

What this means for teams

As of 28 November 2024, System Initiative is real software you can put a credit card behind, with a free tier big enough for a small production estate and gaps big enough to matter: no API, no templates, no views yet, import functions for two AWS asset types, and AWS as the only cloud modelled properly. The argument underneath it is usable whatever you run.

Three things worth doing this week:

  • Time how long your plan stays true. Take a recent terraform apply that failed after a clean plan and find out what changed underneath it and when. That interval is what change sets are designed to remove, so your own number tells you what removing it is worth.
  • Count your reviewers. For your most sensitive infrastructure repository, list the people who could reject a change on intent rather than on style. If it’s one or two, you have the concentration problem whatever tool you use.
  • Model one existing system somewhere free. Import a VPC you already run into a scratch workspace and see whether the diagram matches the one in your head. You’re testing the mental model, and the trial covers the bill.

Common questions, answered

How do I try System Initiative without paying anything?

Use the 30-day trial, which has no resource limit, then stay under the free tier of 100 concurrent resources under management. Watch that ceiling: the pricing page says exceeding 100 concurrent resources in any hour makes every resource hour that month billable at $0.007, including the hours you were under it. Extra users and function runs cost nothing, and so does modelling infrastructure you never deploy, because billing counts resources that exist in your cloud account. The source is Apache 2.0 if you’d rather not sign up, though at GA that meant the local development setup.

Can I use System Initiative with Google Cloud or Cloudflare instead of AWS?

You can, as long as you’re willing to write the assets, because AWS is the only cloud with real coverage out of the box. Assets mirror the provider’s own API shapes, so there’s no provider abstraction to learn before you add one, though the team said at GA that the models aren’t perfect yet. One customer had started building Cloudflare assets, Google assets were beginning to appear, and DigitalOcean was an internal experiment. Budget for writing and testing what you need.

Can I trigger System Initiative from my CI/CD pipeline?

Not at general availability: there was no API, and the canvas was the only interface. The design argument is that the visual, multiplayer workspace is the product, and several users had already asked for an API. You can go the other way and model a workflow that kicks off a GitHub workflow, but nothing in your pipeline can drive a change set. If that’s a hard requirement, this isn’t a tool you can slot in yet.

Resources

Since we recorded

Views shipped on 13 December 2024, the last of those three gaps to close after import and approvals. The product itself didn’t survive the AI turn: the systeminit/si repository was archived in February 2026, and the same team now builds Swamp, an AI-native automation CLI, which is a very different bet about where the collaboration happens. Paul came back to explain the switch in Episode 92 and to walk through how the team builds with agents in Episode 94.