DevOps Topeaks

Send us a text

This week we discussed Chaos Engineering, where did it come from, what it means, how can you incorporate it into daily work and plans.

Links

• https://github.com/vmware-tanzu/velero
• https://principlesofchaos.org/
• https://aws.amazon.com/apprunner/
• https://aws.amazon.com/blogs/machine-learning/llama-2-foundation-models-from-meta-are-now-available-in-amazon-sagemaker-jumpstart/

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

This week we talked about the services you don't normally hear when it comes to AWS: LightSail, Rekognition, Kafka, Open search, Prometheus and many more!

Links:

• https://github.com/AlDanial/cloc
• https://cupogo.dev
• https://www.reddit.com/r/golang/comments/14xyido/the_gorilla_web_toolkit_project_is_being_revived/

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

This week we talked about yet another undiscussed (kidding) buzz word - "DevSecOps". What is it? Where did it come from? What falls under the category?

Things mentioned:

• https://helix-editor.com/
• https://httpd.apache.org/docs/2.4/programs/ab.html
• https://github.com/kubernetes/autoscaler

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

This week we went on a small adventure discovering the K8s control plane components as well as the node resource. We shared background, experience, and as usual - went too far with stories ;)

Links:

• https://chirper.ai/
• https://medium.com/prodopsio/an-8-minute-introduction-to-k8s-94fda1fa5184
• https://kubernetes.io/docs/concepts/overview/components/

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

This week we discussed whether "ops" engineers should know how to code, and if so, to what degree. We went into languages, frameworks, when and if should you start as a junior. We shared our views on everything around "Dev" as it relates to "Ops" when it comes to actual positions within a software company.

Things mentioned:

• The Grand Budapest Hotel
• https://github.com/spf13/cobra
• https://github.com/spf13/viper
• https://click.palletsprojects.com/en/8.1.x/ 

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

Is Amazon dropping serverless? What was the trigger to the huge backlash on serverless and microservices of the past few weeks?
We discussed AWS's blog post, DHH comments, Kelsey Hightower response and more!

• Amazon's Prime Video moving away from serverless: https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90
• DHH blog post: https://world.hey.com/dhh/even-amazon-can-t-make-sense-of-serverless-or-microservices-59625580
• Pprof (Go profiling): https://github.com/google/pprof
• Dex: https://dexidp.io/docs/kubernetes/

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

This week we had the pleasure of discussing EC2!
The basic building block of AWS has so much going on around it so we made an effort keeping it short (did not do all that well if you compare to previous episodes 😉)

Links:

• EC2 https://aws.amazon.com/ec2
• The SSM plugin for AWS CLI (Using SSM connect from a teminal)
• Golang Telegram bot API - https://github.com/go-telegram-bot-api/telegram-bot-api
• GORM: The Golang ORM - https://gorm.io/

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In this episode we discussed S3, which is not all that "simple"!
Policies, web hosting, tiering, smart tiering, Glacier, Cloudfront, indexing and MORE!

Links and things mentioned: 

• S3 search and access throughput with prefixes: https://docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-performance.html
• URL shortener with S3 redirect function
• https://github.com/ducaale/xh
• https://thebrowser.company/
• https://brave.com/
• https://minbrowser.org/

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

This week we discussed AWS IAM, on the infrastructure level, application level, what are users, roles, profiles, permission sets, temporary credentials and MORE!

Links promised:

• SSO Sync for Google to AWS SSO: https://github.com/awslabs/ssosync

Tools / Experience of the week:

• Omer mentioned "nocode": https://github.com/kelseyhightower/nocode

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In this episode we discussed all-things VPC!
From the very basics of structure and best practices, deep down to wiring the internals, hacks tips and lots of other stuff.

Things we mentioned:

• Omer's list of K8s ingress controllers: https://docs.google.com/spreadsheets/d/191WWNpjJ2za6-nbG4ZoUMXMpUK8KlCIosvQB0f-oq3k/edit#gid=907731238
• Meir's stack for this week: https://nestjs.com & https://vitejs.dev

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

We talked about container from the ground up!
Is it only Docker in the space (no!).
What are they for, why they're also incredible for local work, how we like to work with them.
Why they're amazing for production purposes and can be found everywhere.

Meir talked about his experience with Nest.js (Omer: I'm definitely adding a +1 here, Nest is amazing!): https://nestjs.com

Omer mentioned Neovim's collaboration plugin: https://github.com/jbyuki/instant.nvim

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

This week we talked about backups, what do they mean to us, how do we apply it as a principle, what it boils down to and more.
Enjoy listening!

In the tools section Omer mentioned
Testcontainers: https://github.com/testcontainers/testcontainers-go
Neovim remote container: https://github.com/jamestthompson3/nvim-remote-containers

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In this episode we discussed "scale". What does it mean in the context of operations / devlopment, but also in regards to management and internal systems.
We talked about empowering engineers to make decisions, prepare systems for you joiners but even more importantly preparing systems to scale to handle additional load and company growth!

Tools mentioned:

• Omer mentioned https://score.dev
• Meir mentioned https://github.com/GAM-team/GAM

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

This week we talked about Git! The version control all (most) of us are using daily.
How did it start? What's with the name? 
What should developers know?
Have you ever wondered how to delete stuff form the root of the tree? Maybe how to pin point a commit that introduced a bug?
We had a fun chat around functionality, productivity, do's and don'ts.

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

We discussed AI and ChatGPT, how (and if) we're using in our daily work. What we think it's good for and what not so much, and we see this world progressing and evolving!

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In this episode we covered Observability! From log collection through metric time-series, through the systems they're stored and to analyzing the data, coming up with insights, and understanding system and application behavior!

Experience / Tool of the week:

• Meir mentioned working with Make - the evergreen scripting tool that supports application builds (https://www.gnu.org/software/make/)
• Omer suggested Just (https://github.com/casey/just) as the modern Rust alternative 
• Omer's findings:
  • Since Jan 2023 AWS offer CloudTail Lake for querying logs rather than having to export them with a data model to Athena: https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-lake.html
  • Bashly - a Bash CLI framework! - https://github.com/DannyBen/bashly

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In this episode we talked about FinOps. What is it, but beyond the buzz, what does it mean to be financially aware. What are good systems to put in place, metrics to track, tools to use and concepts to adopt.

Trivy: https://www.aquasec.com/products/trivy/

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

Serverless is one of those topics we can have an entire season just surrounding this one architecture option. We tried packing as much as possible to one short talk but we can (should?) probably do another follow up talk about advanced Serverless options we didn't touch this time.

We started off talking about how it is perceived by many, then went into what it really is, and even some of its inner workings intricacies or what's "under the hood".

Some of what's mentioned:
AWS Lambda, Firecracker, Fargate vs Lambda, containers, KVMs, frameworks, local helpers, VPC connectivity, concurrency, throttling, application design and considerations (and MORE)...

Useful links:

• Firecracker
• AWS Lambda
• Serverless Framework
• Serverless Land
• AWS SAM
• Omer's blog post about many of these including the new networking scheme
• LocalStack
• Moto (Mock AWS Services)

Cool projects: 

1. SKHD 
2. Yabai

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

This week we brought up the recently promoted and latest-buzz: Platform Engineering. What is it? How does this differ from DevOps / SRE? What does it actually mean?

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In this episode we discussed all-things application security; from scanning, to designing with security in mind, through OWASP and sources of information we feel engineers in the world of dev / ops should be aware of and familiar with!

We talked about:

• OWASP Top 10 - https://owasp.org/www-project-top-ten
• Git leaks - https://github.com/zricethezav/gitleaks
• 12 Factor - https://12factor.net
• Scanners: [Python Bandit: https://bandit.readthedocs.io/en/latest, Go: https://pkg.go.dev/golang.org/x/vuln/cmd/govulncheck]
• Clair static analysis for containers: https://github.com/quay/clair
• Bug Bounty platforms: HackerOne, Bugcrowd, Intigrity
• BGP repo cleaner - remove secrets from git history: https://rtyley.github.io/bfg-repo-cleaner
• Harden EKS - https://github.com/aws-samples/hardeneks

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In our 7th episode we discussed all-things cloud-security: NACLs, security groups, VPCs, usage of public and private subnets, secure tunneling, whats important to keep an eye on, and which layer is used and when!

Things mentioned: AWS, K8s, Network security, VPC, Subnets, routing, secure access, running in private networks

Links as promised: 
K8s limits: https://home.robusta.dev/blog/stop-using-cpu-limits
AWS security group rules real time application: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/security-group-rules.html

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In this episode of DevOps Topeaks, we discussed what DevOps actually is (spoiler alert: not a job title) and which qualities we're looking for in DevOps candidates. We also talked about our approach to picking up new technologies and consume information in a world of constant stream of data.
We hope you enjoy!

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In this episode to DevOps Topeaks we discussed how we think CI runners should be handled; where they should run, how, and ways to secure, scale and manage them efficiently.

As usual we drifted into further topics like cleaning container images, saving disk space, efficient use of cache, secure connection to instances with SSM rather than a VPN and lots more!

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In this session of #topeaks, we talked about how to choose a container orchestrator. We compared the local development process across different container orchestrators, when to choose a proprietary solution vs a standalone K8s, and dabbled with getting into Hashicorp's Nomad and the why's

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In this session of #topeaks, we'll talk about container orchestrators, when to use which service or tool, scaling in-out-up-down, node management when scaling, and which metrics we should use for scaling.

In the next session, we'll cover more about this awesome topeak!

Important additional information on autoscaling ECS and EKS nodes, starting from ECS:
AWS offers now an automated way to scale and ECS cluster nodes in and out.
The way it works, is by creating a Capacity Provider based on a new or existing Auto Scaling Group, and then enabling "managed scaling" for the same provider.
This is all you need to know if you're looking for a quick "FARGATE" experience, without the overhead costs.
However, if you're a geek like us, and interested what's going on under the hood, and the way the autoscaler actually creates metrics and calculates every minutes the need to scale in or out, here's the documentation: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/cluster-auto-scaling.html, specifically look for "How cluster Auto Scaling works".

As for EKS, things are rather simpler using the OSS Cluster Autoscaler (https://github.com/kubernetes/autoscaler/tree/master/cluster-autoscaler).
The documentation for how to deploy it on EKS, with all the required IAM permissions and different configurations, can be found here: https://docs.aws.amazon.com/eks/latest/userguide/autoscaling.html

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

Our first #topeaks session discusses approaches to aligning CI code across a multi-repo structure. We touch on CI templates, scripting, central parameter store, GitLab CI, GitHub, Jenkins, Dagger, and more.

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops

Send us a text

In this session of #topeaks, we'll talk about how to provision internal services and protect them from external users (non-employees).

Meir's blog: https://meirg.co.il
Omer's blog: https://omerxx.com
Telegram channel: https://t.me/espressops