Top
Best
New

Posted by mfornasa 7 hours ago

The future of Terraform CDK(github.com)
87 points | 90 comments
vbernat 7 hours ago|
It's odd to always say "Hashicorp, an IBM company". Looks like they want to assign blame.

I did try Pulumi a while back, but the compatibility with Terraform modules was not great, so I've switched to CDKTF, which can handle unmodified modules. Dunno if I'll switch back to Pulumi or just use OpenTofu directly.

jjice 7 hours ago||
> It's odd to always say "Hashicorp, an IBM company". Looks like they want to assign blame.

All their branding does this now, including the HashiCorp logo on their website [0]. There's gotta be a name for this specific branding pattern, but I don't know it.

[0] https://www.hashicorp.com/en/blog/products/terraform

huddo121 3 hours ago|||
Metastatized branding
pretext-1 3 hours ago|||
I was recently working for a company which got acquired by IBM and we had to do it too. It’s an IBM thing. I bet most people at HashiCorp hate it, at least that was the case for us.
dandellion 1 hour ago||
Makes IBM look really bad. Do they also force people to bow when the CEO of IBM enters the room, and address them as sir or your highness?
miki123211 1 hour ago||
They used to have their employees sign songs praising the company...

Granted, that was in the 1930s or something, but still.

smithcoin 7 hours ago|||
We use OpenTofu it’s pretty seamless
benatkin 6 hours ago|||
Now more will be using a combination of OpenTofu and Terraform, and there will probably be some tacit endorsement of OpenTofu by Hashicorp folks in their communication with those who are using both. Good to see!
Hamuko 6 hours ago|||
Does it do ephemeral values yet?
cube2222 6 hours ago||
Yep, as of yesterday’s 1.11 release it’s supported!

That also includes a new “enabled” meta argument, so you don’t have to hack around conditional resources with count = 0.

[0]: https://opentofu.org/blog/opentofu-1-11-0/

Disclaimer: affiliated with the project

lijok 6 hours ago|||
How do you migrate from count/for_each to `enabled` ?
cube2222 6 hours ago||
You can just switch from `count = 1` to `enabled = true` (or vice-versa, works back-and-forth) for a resource and tofu will automatically move it next time you apply.

It's pretty seamless.

joombaga 5 hours ago|||
That's cool! We'll still need to change all of the references to `resource[0]`, right? Or does tofu obviate that need as well?
cube2222 3 hours ago||
I’m not sure I understand. You refer to the conditional resource fields normally - without list indices. You just have to make sure the object isn’t null.

There’s some samples in the docs[0] on safe access patterns!

[0]: https://opentofu.org/docs/language/meta-arguments/enabled/

lijok 5 hours ago|||
Amazing. Good work !
Hamuko 5 hours ago|||
Damn, might finally be able to use it. The lack of ephemeral values was a major blocker.
packetlost 6 hours ago|||
I have absolutely nothing good to say about Pulumi. Stay far, far away.
willio58 2 hours ago|||
My experience with Pulumi is you can write bad pulumi code and good pulumi code and just like everything else, it's easy to end up in a codebase where one poor soul was tasked with writing it all and they didn't do the best job with it.
weakfish 4 hours ago||||
Why? I’ve had nothing but good experiences, but I don’t run it and the team that does is extremely competent
jen20 6 hours ago||||
Strange, I have a lot of good things to say about both it and Terraform.

Probably some specifics might be more useful there...

mfornasa 6 hours ago|||
please expand on this, I am interested (for real!)
atonse 7 hours ago|||
I was thinking the same thing about the "an IBM company". My guess is that it's a lazy find/replace.
Pet_Ant 5 hours ago|||
I assume it's a matter of branding and making IBM look more modern by associating with the Hashicorp brand.
cr125rider 4 hours ago|||
It’s one thing to say it once but 3 times in the same paragraph seems weird for sure!
selkin 6 hours ago|||
> It's odd to always say "Hashicorp, an IBM company". Looks like they want to assign blame.

Or it's legal trying to preempt a risk.

If it was the author just wanting to point at IBM, they'd mention it just once or twice, but using that awkward phrase throughout the text makes me think it was an edit mandated by a careful lawyer.

firesteelrain 6 hours ago||
It’s how Red Hat identifies themselves too
richardfontana 46 minutes ago|||
Do you mean Red Hat identifies itself using the phrase "Red Hat, an IBM Company"? Because I don't see any use of this on redhat.com (including that website's corporate "about" content) and if any Red Hatters are using this phrasing (I'm a current Red Hat employee) I haven't been aware of it.
viraptor 1 hour ago|||
It's common when corps buy large enough companies that they don't want to kill the original brand. That's why you get hotels like "(something) by Hilton".
crimsonnoodle58 6 hours ago||
This is particularly frustrating as I've spent the last year writing many thousands of lines of CDKTF Python.

HCL just does not have the modularity and expressiveness that Python, or other languages CDKTF supports.

I guess I'll spend another year migrating to Pulumi now..

lijok 6 hours ago|
The lack of expressiveness of HCL is the point and what makes it so good
crimsonnoodle58 5 hours ago|||
Being able to inherit from Ingress and add a parameter of say public=True/False and then it change annotations, middleware, etc and then being able to re-use that across 100s of stacks is very powerful. DRY is not something HCL is good at.
lijok 5 hours ago|||
Getting too clever with an imperative language in what is inherently a declarative domain, is an idea bad enough that they invented a whole new language to avoid you doing it. But some lessons have to be learned the hard way I guess
everforward 2 hours ago|||
The problem is they did an exceptionally poor job at designing their language. A reasonably large Terraform codebase is almost universally hard to read for one of two reasons: it's either unexpressive (read: verbose to the point it's hard to read) or modularized but hard to read because it's fragmented into a bajillion reusable modules.

SQL is also declarative, but incredibly expressive. A thousand character query contains enough complexity that it's hard to reason about. A thousand characters of Terraform will barely stand up a CRUD app on AWS.

Designing a language from first principles for this was a mistake. HCL is awful; they should have gone the Starlark route and made a stripped-down version of an existing language instead of making their own language from scratch. This feels like the worst of both worlds. The language is practically imperative, but it has its own syntax that isn't useful outside of this one single domain.

pxc 1 hour ago||||
Declarative vs. imperative doesn't have anything to do with power or expressiveness. Some general purpose programming languages are declarative, and some declarative DSLs are Turing-complete.

I worry that comments like this lead the average newbie to overlook (or worse, avoid) declarative languages (both among DSLs and among general-purpose languages) because they will associate the term with hacky, confining, gotcha-ridden messes like Terraform's HCL, Azure DevOps' standards-breaking "YAML" DSL, etc.

Incidentally I agree that a language like Python is a terrible fit for this domain, but it's also plain to see that HCL is a shitty tarpit. It's not hard to understand why people want to get away from HCL.

And concretely, you can use Pulumi in a pure functional style with F# or Scala.

crimsonnoodle58 4 hours ago||||
Yet said language continues to add imperative-inspired constructs to make up for its limitations..

The end result is still declarative, your just using an imperative language to keep your IaC DRY.

lijok 3 hours ago||
If you have the expertise and restraint to not go off the rails, I agree, imperative is more powerful. That plan does not survive teams of sizes over 2 in the majority of cases.
Spivak 2 hours ago||
But it's not even imperative. Your code runs, declares all its resources up front and then normal terraform runs on it. With cdktf you can even have it output the HCL.

At the point where we are templating Terraform files we've already lost the plot. You might as well get to use a real programming language.

theevilsharpie 2 hours ago||||
I have used Terraform, Puppet, Helm, and Ansible (although that's not strictly declarative), and all of them ran into problems in real-world use cases that needed common imperative language features to solve.

Not only does grafting this functionality onto a language after-the-fact inevitably result in a usability nightmare, it also gets in the way of enabling developer self-service for these tools.

When a developer used to the features and functionality of full-featured language sees something ridiculous like Terraform's `count` parameter being overloaded as a conditional (because Terraform's HCL wasn't designed with conditional logic support, even though every tool in this class has always needed it), they go JoePesciWhatTheFuckIsThisPieceOfShit.mp4 at it, and just kick it over to Ops (or whoever gets saddled with grunt work) to deal with.

I'm seeing the team I'm working with going down that same road with Helm right now. It's just layers of templating YAML, and in addition to looking completely ugly and having no real support for introspection (so in order to see what the Helm chart actually does, you essentially have to compile it first), it has such a steep learning curve that no one other than the person that come up with this approach wants to even touch it, even though enabling developer self-service was an explicit goal of our Kubernetes efforts. It's absolutely maddening.

dastbe 2 hours ago|||
They invented a language to avoid you imperatively updating infrastructure, but that's not what CDKTF does; it just makes it easier to materialize that declarative output.

It also makes it easier to reason about that output as you can avoid awkward iteration in your declarative spec.

bigstrat2003 2 hours ago||||
That is... not a good idea at all imo. It's very, very easy to over-DRY infrastructure config and it sounds like you're well past that point.
JojoFatsani 2 hours ago|||
Make a module
pizza234 3 hours ago|||
That's very subjective. Concepts like iterations are inevitable, and they don't look great in a declarative language like HCL.

I also find refactorings considerably harder in a declarative language, since configurations have a rigid structure.

vanschelven 6 hours ago||
"Will be sunset on Dec 10"... commit date: Dec 10.

That seems like rather short notice.

HashiCorps 2 hours ago|
As I said here [0] there's more of this coming.

[0] - https://news.ycombinator.com/item?id=46192130#46198058

mfornasa 6 hours ago||
Rug pulls on infrastructure components seem even worse than other rug pulls as they can hit your entire infra codebase at once
lillecarl 6 hours ago|
This is why infrastructure people are conservative by nature, it's so damn much gruntwork to migrate without downtime
mfornasa 6 hours ago||
And it happens while we are all very enthusiastically dedicated to migrating off Kubernetes ingress-nginx. Just as planned.
preisschild 3 hours ago||
As an Infrastructure Engineer who used it: I blame people who didnt help fund/maintain it (including ourselves)
GardenLetter27 7 hours ago||
Damn, what are the best alternatives here? For pure AWS I guess CDK directly is okay, but locks you in.
tapoxi 6 hours ago||
I went with CDK, I'm locked into AWS already and it means my major dependency for IaC is my cloud vendor and not a third party.

If I really need to migrate off of AWS at some point I'll throw an LLM at it.

manquer 1 hour ago|||
IaaC code is one of those use cases just throwing LLM is painful for a refactor.

In my experience claude/codex to wrangle CDK constructs be complicated, it frequently hallucinates constructs that simply do not exist, options that are not supported etc.

While they can generate IaaC component mostly okay and these problems can be managed, Iterations can take a lot of time, each checkpoint, goes the deploy/ rollback cycles in CF. CloudFormation is also not particularly fast, other IaaC frameworks are not that different.

Running an agent to iterate until it gets it right is just more difficult with IaaC refactor projects. Hallucinations, stuck loops and other issues, can quickly run the infra bill up not to mention security.

ryandvm 6 hours ago|||
Exactly. It's just so much cleaner to do it in the Cloud provider's native tooling. The impedance mismatch from Cloud-agnostic abstractions always just makes thing shitty enough that in the long run you spend more time dealing with weird edge cases.

Besides, actual full-scale Cloud migrations are exceedingly rare.

emoII 5 hours ago|||
Terraform is not an abstraction on top of multiple cloud providers, you work with aws, azure etc explicitly. It is , however, agnostic in the sense that you can provision aws, azure, gcp, etc resources within the same iac project
raw_anon_1111 3 hours ago|||
I always hated this meme. Using Terraform no more makes you “cloud agnostic” than using Python to script AWS services and calling boto3 than using bash and calling the AWS CLI.
tetha 6 hours ago|||
Hm, we have a few very repetitive terraform projects to setup structured infrastructure clusters. For those, we just use ansible with a bunch of templating to generate a configurable, HCL-based terraform module and version that.

It's a bit of an "Caveman solve problem with rock" approach, but for very regular projects it's great. A new cluster is some group vars, larger changes to the structures can be easily reviewed - and if you really really have to, you can also just modify the generated code by hand to fix something your generation code can't deal with right now.

scruff3y 7 hours ago|||
Just use Terraform?
cholantesh 6 hours ago|||
Yeah I'm struggling to see the value here.
stackskipton 6 hours ago||
The value for TFCDK was Developers don't have to learn another language, they can just continue to use existing language they already know.

Downsides are doing infrastructure in a programming language was always problematic unless developer was skilled at Ops which most who used TFCDK were not.

cholantesh 3 hours ago|||
I ought to have phrased it I guess as "I don't agree with the value proposition", mainly because of the downside you point out. This seems superior to Pulumi, though, in that the abstraction is (was) at least owned by Hashicorp so there was less likelihood of it falling out of date and giving you footguns.
coenhyde 3 hours ago|||
That might have been the promise but never the real value. As you say in practice the engineer needs to know ops & terraform along side their language of choice.

The real value of cdktf was more dynamic infrastructure provisioning while still having the plan / apply pattern.

mfornasa 7 hours ago|||
Probably Pulumi
resonious 3 hours ago||
I'll be honest Pulumi is pretty cool but I'm a little worried by how high on the stack it is. I wonder if the same thing won't happen to them that's happening to CDKTF here.

Terraform is ugly but it works well enough for me and seems ingrained enough to be durable to this kind of thing (i.e. I bet for sure the community would pick it up (I wish I could say that I'm part of that community but I can't say I use it quite that often))

re-thc 46 minutes ago||
> I wonder if the same thing won't happen to them that's happening to CDKTF here.

This is clearly a business decision rather than technical.

Pulumi is meant to be semi-automated (in generating the bridges) so perhaps is slightly better off in maintenance.

srmatto 7 hours ago|||
If you want maximal complexity use Crossplane. :P
sshine 7 hours ago||
Terranix? ;-)
madjam002 6 hours ago|||
Not gonna lie Terranix has been working great for us, all our configuration is in Nix files anyway so it's so easy to just pass stuff in rather than using Tf variables etc
lillecarl 6 hours ago|||
Yes, the NixOS module system is so much more composable than the TF one
Havoc 1 hour ago||
As far as corporate mercy killings goes archived under mozilla license is better than a pivot to "you now pay per core" or whatever
deadfece 6 hours ago||
At least they gave us some notice, that’s much appreciated.
NeckBeardPrince 6 hours ago||
Hashicorp, an IBM company
tonnydourado 6 hours ago|
Hashicorp,an IBM company
chickensong 1 hour ago||
Hashicorp, an IBM company
zer0-c00l 7 hours ago||
This is a bummer. I don't particularly like Pulumi but use it anyways because for my use cases being able to write actual code is really impactful. Sucks to see fewer options in that space
joeduffy 3 hours ago||
[Pulumi founder here] Sorry to hear you don't particularly like Pulumi---any/all feedback welcome. If nothing else, we do listen and we do try to get better. -Joe
leetrout 6 hours ago||
The often excluded option is dynamically generating JSON and feeding that to TF instead of HCL.

You can combine it with tools like Dhall or my personal preference Jsonnet instead of imperative languages for an interesting experience for reusable pieces outside of module concepts.

Duologic 3 hours ago||
Any particular libraries you use to generate TF-JSON from jsonnet?

I wrote a generator a little while ago that can create jsonnet libraries from the TF schemas: https://github.com/Duologic/soysonnet

Example lib here: https://github.com/Duologic/soysonnet-aws

I only needed it for AWS so I didn't spend more time on it.

moltar 5 hours ago|
This is so sad. It’s a great project. Needs to be forked and maintained. If anyone forks please email me I’ll contribute.
More comments...