-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce reference implementation for standard EKS cluster using Terraform #6
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… variables; allows for easier configuration of AWS region and targeting deployments (staging, production, etc)
… when creating a new cluster from scratch; aws-auth ConfigMap doesn't exist by the time the module attempts to customize it
…n RDS instance and places a Franklin deployment and LoadBalancer service onto the cluster; Franklin is reachable via the resulting ELB; still need to supply a route53 resource to create a DNS alias to this ELB
I've deferred the outstanding tasks in this PR to separate issues so that I can just merge the feature. This code, after all, does work as desired, even if it isn't perfect. We can improve things at a later date. |
This was referenced Oct 20, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The core function of this entire repository is to describe an Azavea-standard Kubernetes deployment. What are the hardware, system, and application requirements that meet our minimum needs? What kind of interface for deploying this (potentially complex) infrastructure should we provide? How should users of this infrastructure interact with it as they deploy their own custom applications? These are some of the questions that we will need to answer in due time as we learn more about this space.
This PR is an initial answer to how we should proceed, suggesting one potential method for deploying a Kubernetes cluster to AWS. The approach taken here is an iteration on a Terraform-based deployment that has been used for other projects at Azavea, which is hopefully a refinement of those previous efforts, eventually offering a standard cluster architecture that can be targeted by application-specific repositories that wish to place resources onto company-wide clusters. The shape of these standard clusters is emerging, but not yet fixed. See the documentation in this repo for more details.
The Terraform code in this PR has been segmented into several stages: (1) configuring the cluster hardware and API access, including OIDC for IRSA and some RBAC setup; (2) setting up basic cluster services, in this case, Karpenter, and possibly an ingress controller; and (3) provisioning system-wide applications, beginning with Franklin.
This basic setup is delivered through the use of an updated STRTA infrastructure. The standard approach to deployment will consist of executing
cibuild
followed bycipublish
. The latter script will be aware of both the AWS region that we are targeting and the target environment. (The reference deployment of this system currently lives onus-west-2
, with only astaging
environment.) I've also improved the script infrastructure for iterating on these deployments, offering aconsole
script that facilitates interaction with theinfra
script during development.The basic structures suggested by this PR's contributions are to be considered as a starting point for future discussions as we develop best practices in the future.
This is still a bit WIPpy. Some amount of work is still required to
Figure out how to grant RBAC roles to users (the Terraform EKS module has a sequencing issue with the aws-auth ConfigMap not yet existing when custom user maps are supposed to be applied)Deferred to Improve RBAC management for aws-auth #9Create a route53 alias to the Franklin ELBRolled into Complete install of Franklin #7Roll infrastructure module intoDeferred to Don't use a module for infrastructure setup #100-hardware
stage