The GeekNarrator

In this episode I talk to Andras Gerlits, who founded omniledger.io. Andras has a very interesting view on how Distributed Consistency should work that can get rid of several bottlenecks when it comes to maintaining Distributed consistency. He argues how getting rid of a global wall clock and using causality to approach Distributed consistency helps you build resilient, simple and performant systems. We have gone deeper into how that can be achieved and how the product works. Chapters: 00:00 Introduction 00:52 Andras's Journey into Distributed Consistency 03:04 The Evolution of Data Consistency in Banking and Beyond 08:04 Introducing Client-Centric Consistency 10:36 Exploring the Standard Model of Distributed Consistency 16:01 Redefining Strong Consistency with a Relativistic Approach 34:25 Practical Implications of Client-Centric Consistency in Banking 36:20 Mitigating Latencies and Partitions in Distributed Systems 41:08 Exploring System Reliability and Availability 41:52 Tuning System Properties for Specific Use Cases 43:07 Comparing Standard and New Models for Data Management 45:08 Understanding Local Progress and Mutex-Free Updates 47:23 Deep Dive into Token-Based Ordering and Global Calibration 58:30 Introducing OmniLedger: A New Approach to Distributed Consistency 01:02:41 Performance Optimizations and Tunable Consistency 01:08:20 Ideal Use Cases and Potential Limitations of OmniLedger 01:14:30 Future Directions and Closing Thoughts Links: Our website: https://omniledger.io A long-form essay on the thinking behind our model: https://medium.com/p/5e397cb12e63 A demo of transactionality https://www.youtube.com/watch?v=XJSSjY4szZE I think my blog in general might be interesting to some https://medium.com/@andrasgerlits The science-paper with all its mathematical rigour: https://www.researchgate.net/publication/359578461_Continuous_Integration_of_Data_Histories_into_Consistent_Namespaces =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #databases #sql #consistency #distributedsystems

In this video, I talk to Stephan Ewen from Restate, who is popularly known from the world of Apache Flink. We have talked about the problems in the world of Distributed systems and the complex solutions developers have to deal with. This complexity makes the architecture so complex that it eventually creates reliability, Observability and delivery velocity problems. Restate aims to solve it by making resilience and durability for your services, functions and RPC a lot simpler. Chapters: 00:00 Introduction 00:45 Introducing Restate: A Solution for Distributed System Challenges 01:22 Deep Dive into Restate with Stefan: From Apache Flink to Building Resilient Systems 06:04 The Complexities of Distributed Systems and How Restate Addresses Them 15:49 The Vision of Restate: Simplifying Developer Experience in Distributed Systems 24:42 Integrating Restate into Your Architecture: A User's Perspective 33:16 Exploring Restate: The Durable Service Mesh 33:32 The Power of Restate in Handling Transactions 34:26 Restate's Role in Service Communication and Durability 35:40 Deep Dive into Restate's Mechanisms and Benefits 38:04 Practical Example: Email Pipeline with Restate 39:40 Understanding Restate's Log and Event Handling 58:43 Restate's Unique Features and Programming Model 01:04:22 Final Thoughts on Restate's Impact and Deployment Restate: https://restate.dev/ =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #distributedsystems #faulttolerance #reliability #resilience

In this episode of The GeekNarrator podcast, our host Kaivalya talks to Seeta Somagani from Volt Active Data, a low latency stream processing platform. They discuss fascinating topics about what low latency stream processing means, the different guarantees that Volt Active Data provides, and the various problems it can solve. They delve into the evolution of VoltDB to Volt Active Data, real-time data processing use cases, the high-level architecture, and how the platform effectively addresses high-concurrency challenges. This is a must-listen for anyone interested in understanding latency critical applications, data processing, and high performance computing. Chapters: 00:00 Welcome to The GeekNarrator Podcast with Special Guest from Volt Active Data 00:41 Introduction 01:45 The Evolution of VoltDB to Volt Active Data 06:13 Exploring Real-Time Data Processing and Use Cases 08:25 Addressing High-Concurrency Challenges in Various Industries 12:57 High-Level Architecture of Volt Active Data 19:26 Understanding Stored Procedures and Data Processing in Volt 22:48 Practical Application: Tracking Data Usage with Volt Active Data 25:16 Diving into Replicated and Partitioned Tables 25:44 Exploring Event Processing and Exporting 26:57 Understanding Stored Procedures and Performance 29:03 Partitioning Strategies and Recommendations 31:39 Ensuring Determinism in Stored Procedures 35:02 Handling Complex Requirements with Compound Procedures 37:25 Fault Tolerance and Data Replication Strategies 40:44 Exploring Use Cases for VoltActiveData 43:30 The Future of Streaming and VoltActiveData's Role 47:05 Closing Remarks and How to Learn More Volt Active Data: https://www.voltactivedata.com/use-cases/activesd-streaming-data/ =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #sql #streamprocessing #java #acid

In an enlightening episode of the GeekNarrator Podcast, host Kaivalya Apte and TigerBeetle's CEO, Joran, delve deep into the world of online transaction processing (OLTP). They discuss the origin, unique architecture, and innovative methodologies behind TigerBeetle, a database tailored to efficiently handle high-volume transaction systems. The podcast explores the system's key features such as efficient scalability, performance-oriented design, and optimized memory usage, demonstrating its robustness in handling business transactions and accounting. It also elucidates TigerBeetle’s adaptability to various domains beyond finance, like energy management and gaming, while highlighting the rigorous testing it undergoes for impeccable quality assurance. Chapters: 00:00 Introduction 01:19 Joran's Journey into Databases 03:59 Understanding Financial Transaction Databases 07:41 The Evolution of OLTP and OLAP 16:13 The Need for a New Database: TigerBeetle 16:53 Performance and Safety Features of TigerBeetle 28:49 The Importance of Safety in Financial Transactions 36:49 Changing Developer Experience with TigerBeetle 41:43 Understanding the CPU and Memory Bandwidth 42:12 The Importance of Data Format Language 43:27 The Concept of Serialization and its Impact 46:23 The Architecture of TigerBeetle 46:29 The Role of Replicated State Machine 48:18 The Importance of Consensus in Replication 50:20 The Structure of TigerBeetle 50:37 The Importance of Log in Systems 50:51 Understanding the State in Replicated State Machine 52:55 The Role of LSM in TigerBeetle 53:55 The Impact of Compaction Process on Performance 57:06 The Importance of Predictability in Software 01:06:15 The Read and Write Path in TigerBeetle 01:14:46 Potential Use Cases for TigerBeetle 01:17:09 Understanding the Limitations of TigerBeetle =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! #tigerbeetledb #databases #acid #olap #oltp #postgres #mysql

In this episode, we dive deep into the world of clean coding with none other than the master and pioneer of the field, Uncle Bob. We explore the nuances and the art behind writing effective and efficient scripts. This conversation covers the nitty-gritty of writing and editing scripts, from understanding how to break down large functions, to discussing principles like 'Single Responsibility Principle', 'Dependency Inversion Principle' and how to balance the 'DRY' (Don't Repeat Yourself) principles. Uncle Bob also shares valuable insights on testing, handling errors, naming conventions and how to work with different types of duplication in coding. He shares recommended resources and books that every coder should read. Chapters: 00:00 Introduction and Welcome 00:06 The Importance of Code Quality 00:29 Introducing Robert Martin (Uncle Bob) 01:39 Uncle Bob's Journey in Programming 02:34 Discussion on Functional Design and New Book 03:52 The Evolution of Software Development 04:28 Revisiting the Clean Code Book 04:49 The Impact of Hardware Changes on Software 06:13 The Evolution of Programming Languages 07:33 The Importance of Code Structure and Organization 09:07 The Impact of Microservices and Open Source 11:14 The Role of Modular Programming 22:07 The Importance of Naming in Code 26:31 The Role of Functions in Code 34:12 The Role of Switch Statements in Code 42:36 The Importance of Immutability 51:00 Dealing with Complex Steps in Programming 51:21 Implementing State Machines in Programming 51:46 The Pragmatic Approach to Programming 53:01 Understanding Error Handling in Programming 54:08 The Challenge of Exception Handling 57:27 The Importance of Log Messages in Debugging 01:03:05 The Dilemma of Code Duplication 01:05:51 The Intricacies of Error Handling 01:07:40 The Role of Abstraction in Programming 01:13:55 The Importance of Testing in Programming 01:19:43 The Challenges of Mocking in Testing 01:25:11 The Essence of Programming: Discipline, Ethics, and Standards Book Recommendations: Tidy First: https://www.oreilly.com/library/view/tidy-first/9781098151232/ Design Patterns: https://www.amazon.de/-/en/Erich-Gamma/dp/0201633612 Analysis Pattern: https://martinfowler.com/books/ap.html Structured Analysis and System Specification: https://www.amazon.de/-/en/Tom-Demarco/dp/0138543801 Fundamental Algorithms: https://www.amazon.com/Art-Computer-Programming-Vol-Fundamental/dp/0201896834 Sorting and Searching: https://www.amazon.de/-/en/Donald-Knuth/dp/0201896850 Structure and Interpretation of Computer Programs: https://web.mit.edu/6.001/6.037/sicp.pdf =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning!

In this episode of the Geek Narrator Podcast, Kaivalya Apte engages in an invigorating discussion with Dominik Tornow, the founder and CEO of Resonate. They explore Durable Async-Await, an interesting concept in distributed systems world, along with other nuances of distributed programming. Dominik also talks about the development and execution of Resonate to simplify distributed systems with a focus on observability, usability, and it's future direction. The conversation concludes with a discussion of different concurrency models and the future of distributed systems Chapters: 00:00 Introduction and Guest Background 02:44 Understanding Async Await 10:25 Challenges with Current Async Await Model 12:53 Introducing Resonate: A Solution for Distributed Async Await 13:34 Practical Application: E-commerce Example 24:57 Understanding the Role of the Platform in Distributed Systems 30:12 Dealing with Partial Failures in Distributed Systems 39:44 Getting Started with Resonate 40:40 Introduction to Resonate and its Simplicity 41:09 Getting Started with Resonate: Installation and Setup 42:22 Understanding the Durability Aspect of Resonate 42:49 Exploring the Resonate Durable Promise Server 44:10 Scaling Up: Introducing Workers into the System 48:35 The Importance of Open Standards in Resonate 50:17 Exploring the Integration Capabilities of Durable Promises 01:04:31 Understanding the Role of Timeouts in Durable Promises 01:07:29 The Future of Resonate: Challenges and Upcoming Features 01:13:04 Understanding the Limitations of Durable Promises 01:14:51 Wrapping Up: Final Thoughts on Resonate and Durable Promises References: A note on Distributed Systems: https://scholar.harvard.edu/files/waldo/files/waldo-94.pdf Thinking in Distributed Systems: https://dtornow.gumroad.com/l/distributed-systems McCarthy's paper: https://www-formal.stanford.edu/jmc/recursive/recursive.html =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning!

Join host Kaivalya Apte in this episode of The Geek Narrator Podcast as he discusses observability engineering with field CTO at Honeycomb, Liz Fong-Jones. They delve into the importance of observability for software engineers, the role of Honeycomb in popularizing this concept, and how observability has evolved over the years. Liz shares her experiences transitioning from being an SRE at Google to advocating for observability at Honeycomb and walking the journey from developer advocate to Field CTO. They discuss the definitions and misconceptions surrounding observability and elucidate on Service-Level Objectives (SLOs) & indicators (SLIs) and challenges they solve. Tune in for an informative and in-depth conversation on observability engineering. Chapters: 00:00 Introduction 00:08 Understanding Observability Engineering 00:37 Guest Introduction: Liz Fong Jones 00:53 Liz's Journey to Field CTO at Honeycomb 27:38 Understanding Site Reliability Workbook Materials 27:57 Identifying Critical User Journeys 29:49 Different Types of Services and Their SLOs 33:05 Setting Up SLOs: Granularity and Number 42:42 Understanding Service Level Indicators (SLIs) 50:26 Common Mistakes in Setting Up SLOs 52:09 Cultivating an Observability-Driven Development Culture References: Observability Engineering: https://www.oreilly.com/library/view/observability-engineering/9781492076438/  @Google  SRE book: https://sre.google/books/ =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning!

In this video I talk about Apache Pulsar with Matteo Merli, CTO at StreamNative. This episode will provide you good insight about how Apache Pulsar works and more importantly differs with the most popular Pub/Sub and streaming platform Apache Kafka. Things like, what enables possibility of 1 million topics? Why is rebalancing not required? How does decoupled storage and compute architecture works? How it uses the concept of Subscriptions to avoid retaining data unnecessarily? And much more... Chapters: 00:00 Introduction and Guest Introduction 00:08 Understanding Apache Pulsar and its Origin 01:22 The Problem Apache Pulsar was Designed to Solve 02:35 The Evolution of Apache Pulsar 05:15 Understanding Basic Concepts of Apache Pulsar 09:27 Deep Dive into Apache Pulsar's Architecture 21:16 Understanding the Flow of Data in Apache Pulsar 28:54 Understanding Subscriptions in Apache Pulsar 31:57 Understanding End-to-End Latency and Subscription Creation 32:32 Broker's Role and Handling Metadata 33:05 Memory Management and Consumer Handling 34:07 Message Processing and Flow Control 34:32 Message Storage and Retrieval 36:00 Comparing Pulsar with Kafka 43:52 Understanding Multi-Tenancy in Pulsar 49:17 Exploring Tiered Storage and Future Developments Important links: StreamNative: https://streamnative.io/ Apache Pulsar: https://pulsar.apache.org/ Matteo Merli: https://twitter.com/merlimat =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning!

Deep Dive into Victoria Metrics with Alex and Roman Join the insightful discussion with Vitoriametrics creators, Alex and Roman, in the Geekneritor podcast hosted by Kaivalya Apte. This episode explores the internals of Victoria Metrics - a highly scalable monitoring solution and time series database. Discover the origins of Victoria Metrics, understand how it evolved, and learn about its unique architecture and functionality. From the concept of time series, the usage of consistent hashing in data distribution to real-world applications, it's all packed into this engaging conversation. 00:00 Introduction 01:52 The Genesis of VictoriaMetrics 02:18 The Journey from Postgres to Clickhouse 03:19 The Transition from Prometheus to Victoria Metrics 05:08 The Birth and Evolution of Victoria Metrics 13:01 The Architecture of Victoria Metrics 20:10 Data Ingestion and Integration in Victoria Metrics 29:15 Understanding the Vector Metric Architecture 30:30 Comparing Shared Storage and Object Store 31:00 Designing the VictoriaMetrics Architecture 32:01 The Role of Object Storage 36:15 The Importance of Indexing 43:19 Understanding the Ingestion Process 45:46 Exploring the Select Process 55:55 Future Plans for Victoria Metrics Important Links: 1. Architecture Overview: https://docs.victoriametrics.com/Cluster-VictoriaMetrics.html#architecture-overview 2. How ClickHouse Inspired Us to Build a High Performance Time Series Database https://altinity.com/wp-content/uploads/2021/11/How-ClickHouse-Inspired-Us-to-Build-a-High-Performance-Time-Series-Database.pdf 3. Frequently asked questions. https://docs.victoriametrics.com/FAQ.html =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning!

Join us on a deep dive into the intricacies of TiDB with Li Shen from PingCap. In this episode, Li Shen provides a comprehensive exploration of TiDB, its unique features, and how it tackles scalability and reliability issues commonly associated with MySQL. If you're dealing with struggles in your MySQL cluster and seeking a more dependable and scalable system, TiDB might be the solution for you. This conversation touches on various aspects of this cutting-edge database, its operational mechanism, use case scenarios, and how it's optimized for different workloads. Key topics include: the architecture of TiDB, the journey of data from API to storage node, embracing analytical use cases, the importance of database reliability, and the process of migrating to TiDB. Dive in now! 00:00 Introduction and Welcome 02:47 Defining TIDB: A Disputed SQL Database 04:55 The Role of MySQL Compatibility in TIDB 05:54 Primary Use Cases for TIDB 09:38 Understanding the Data Ingestion Process in TIDB 16:52 Understanding Indexing in TIDB 23:01 Pushing Down Table Scans and Partial Aggregation 24:39 Introduction to Columnary Extension: Flash 24:54 Understanding Data Replication and Learner Nodes 26:23 Ensuring Strong Consistency in Data 27:12 Balancing Transactional and Analytical Use Cases 27:57 Understanding Data Replication and Consistency Model 28:42 Exploring Ty Flash Storage Layer 28:54 Understanding High Concurrency Insert and Update 32:09 Exploring the Read Path and Caching Mechanism 37:50 Understanding the Importance of High Reliability 43:01 Exploring Migration from Other Databases 48:01 Comparing TiDB with Other Distributed SQL Databases 52:21 Identifying Use Cases Where TiDB Might Not Be the Best Choice Stay Curios! Keep Learning!

In this video I discuss Database tuning and Optimisation with Andy Pavlo, OtterTune. Andy is an Associate Professor with Indefinite Tenure of Databaseology in the Computer Science Department at Carnegie Mellon University. My research interest is in database management systems, specifically main memory systems, self-driving / autonomous architectures, transaction processing systems, and large-scale data analytics. 00:00 Introduction and Welcome 01:31 Understanding Database Optimization 05:48 Understanding When Database Tuning is Needed 08:45 Understanding Database Optimization Difficulties 16:16 Understanding Default Settings in Databases 22:35 Role of Machine Learning in Database Tuning 22:38 Introduction to Ottertune 28:36 Data Collection for Machine Learning Model 35:25 Deployment and Data Collection Process 38:03 Admitting the Limitations of Current Model 38:53 Challenges in Predicting Performance Improvements 39:28 The Importance of Data Collection Over Time 39:52 Avoiding Weekend and Holiday Tuning 40:05 Introducing New Features for Database Comparison 42:09 Provisioning Recommendations and Performance Predictions 43:03 The Importance of Telemetry in Understanding Database Performance 44:01 Handling Dramatic Changes in Database Workloads 44:48 Preparing for Predictable Traffic Spikes 48:13 The Importance of Testing in Database Optimization 53:33 The Future of Database Optimization 55:50 Common Mistakes in Database Management 01:09:15 The Future of Holistic Database Tuning Links: Ottertune: https://ottertune.com/ Andy Pavlo: https://www.cs.cmu.edu/~pavlo/ CMU youtube: https://www.youtube.com/@UCHnBsf2rH-K7pn09rb3qvkA Resources: CMU: https://15799.courses.cs.cmu.edu/spring2022/schedule.html Ottertune blog: https://ottertune.com/blog =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Stay Curios! Keep Learning!

Deep Dive into DuckDB with CTO Mark Raasveldt Decode the insights of databases with Geek Narrator podcast. In this episode, host Kaivalya Apte converses with Mark Raasveldt, the CTO of DuckDB labs, discussing his journey from being a database enthusiast to creating DuckDB. They delve into how DuckDB, an analytical database, differs from other databases, the design decisions, its internal mechanisms, and much more. The episode also highlights the advantages of DuckDB in analytics, the motivation behind its ACID compliance, and how DuckDB handles ingestion, transaction isolation, mutations, and queries. Join in to learn how your data workloads can benefit from DuckDB. 00:00 Introduction and Guest Introduction 00:44 Guest's Journey into Databases 03:40 The Birth of DuckDB 04:30 Challenges with Existing Databases 05:15 Technical Difficulties 05:16 Why Existing Databases Fall Short for Data Scientists 09:16 The Role of SQLite and Its Limitations 13:59 Defining DuckDB 16:48 Comparing DuckDB with Other Analytical Databases 19:50 Deployment Models for DuckDB 22:47 Data ingestion into DuckDB 22:51 Data Ingestion in DuckDB 30:24 How DuckDB Handles Updates and Mutations 35:35 Understanding Column Granularity and Rewrites 35:58 Implications of Compression on Data Updates 36:38 Trade-offs in Row Group Size 37:32 Benefits of Column Storage Model 38:15 Row Groups and Parallelism 39:02 Choosing Row Group Size: An Experimental Approach 40:00 Handling Data Type Changes in Columns 41:00 Internal Data Structures in DuckDB 42:21 Reading Data: Point Lookups, Aggregations, and Joins 47:22 Optimization for Full Table Scans 53:49 Understanding ACID Compliance in DuckDB 55:49 Multi-Version Concurrency Control (MVCC) in DuckDB 59:50 Use Cases and Applications of DuckDB 01:01:42 The Story Behind DuckDB's Name 01:02:34 Future Vision for DuckDB References: DuckDB: https://duckdb.org/ Mark's blog: https://mytherin.github.io/ =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! Cheers, The GeekNarrator

In this episode we talk about ScyllaDB internals with Felipe Mendes. Chapters: 0:00 ScyllaDB internals with Felipe Mendes 07:51 Write Path - API to Storage 11:40 What makes it faster than Cassandra? 13:39 Optimisations: Sea Star, shard per core architecture 15:49 Optimisations: No Garbage collection and Custom Cache Implementation 18:15 Optimisations: Scheduling groups and IO priority classes 20:07 Optimisations: IO scheduler 22:55 Benefits of shard per core architecture 30:16 Write path - Hows is a coordinator chosen? 38:20 Read path 39:27 Read path optimisations - Index Caching 41:48 Shard vs Partition 43:10 Shard per core architecture tradeoff 44:03 Observability of Database References: ScyllaDB architecture: https://opensource.docs.scylladb.com/stable/architecture/ Sea star: https://seastar.io/ ScyllaDB Caching: https://www.scylladb.com/2018/07/26/how-scylla-data-cache-works/ Shard per core architecture: https://www.scylladb.com/product/technology/shard-per-core-architecture/ Database performance at Scale: https://www.scylladb.com/2023/10/02/introducing-database-performance-at-scale-a-free-open-source-book/ =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! Cheers, The GeekNarrator

In this episode I talk to Michael Hunger from Neo4j about Graph Database Internals (Neo4J) Chapters: 0:00 Introduction and historical context 20:51 Data Modelling 25:16 Problem with SQL for Graph Model 26:21 Cypher - Query Language 28:23 Write Path 31:36 Neo4J Storage Layer 33:51 Graph API on top of Relational Model vs Native Graph Databases 37:05 Create Node Relationships 40:42 What makes Graph Database's performance better? 46:00 Partitioning Strategy 53:20 Read path 59:27 Schema Migration 01:04:41 Graph database use cases =============================================================================== For discount on the below courses: Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. =============================================================================== Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! Cheers, The GeekNarrator

In this episode I talk to Micah Wylde about why #Rust could be the best choice for writing distributed systems and how does it compare to #C++, #Java and #Go. Chapters: 00:00 Introduction 03:48 History of Systems Programming 09:42 Is C++ coming back? 13:31 Problems with C++ 16:24 Problems with Java 25:18 Problems with Go 31:21 Why did you choose Rust? 35:19 What makes Rust better? 41:49 Rust cannot save you from logical bugs 44:02 Problems in the context of Stream Processing 48:10 Challenges with Rust 51:28 Learning Rust 54:10 Future of Rust 56:41 A Summary Blog mentioned in the discussion: https://www.arroyo.dev/blog/rust-for-data-infra For the courses mentioned use the following links: Coupon code: "geeknarrator" Appsync: https://appsyncmasterclass.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Testing serverless: https://testserverlessapps.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Production-Ready Serverless: https://productionreadyserverless.com/?affiliateId=41c07a65-24c8-4499-af3c-b853a3495003 Use the button, Add Discount and enter "geeknarrator" discount code to get 20% discount. Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network. Also please subscribe if you haven't yet. Database internals series: https://youtu.be/yV_Zp0Mi3xs Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Stay Curios! Keep Learning! Cheers, The GeekNarrator

Hello Everyone, In this podcast I have invited John Crickett, who has been a Software Engineer since 27 years, having vast experience in variety of tech stacks. He is known for his newsletter "Coding Challenges" that helps developers build real world applications and becomming a better engineer. 00:00 Introduction 01:17 What made you start Coding Challenges? 03:21 What made you start learning Rust? 04:08 How should Software Engineers Prioritise learning? What should they learn? How would they know? 12:20 How to become a better engineer? 14:05 Knowing your passion? but how? 17:43 Should LeetCode be part of interviews? When does (and not) it make sense ? 25:39 System Design interviews 29:38 Building as a community. More about Coding Challenges : https://codingchallenges.fyi Join the discord server: https://discord.com/invite/zv4RKDcEKV Connect with John : https://www.linkedin.com/in/johncrickett/ Follow me on Linkedin and Twitter: https://www.linkedin.com/in/kaivalyaapte/ and https://twitter.com/thegeeknarrator If you like this episode, please hit the like button and share it with your network.

Cheers,

The GeekNarrator

Hey Everyone, In this video I talk to Franck Pachot about internals of YugabyteDB. Franck has joined the show previously to talk about general database internals and its again a pleasure to host him and talk about DistributedSQL, YugabyteDB, ACID properties, PostgreSQL compatibility etc. Chapters: 00:00 Introduction 01:26 What does Cloud Native means? 02:57 What is Distributed SQL? 03:47 Is DistributedSQL also based on Sharding? 05:44 What problem does DistributedSQL solves? 07:32 Writes - Behind the scenes. 10:59 Reads: Behind the scenes. 17:01 BTrees vs LSM: How is the data written do disc? 25:02 Why RocksDB? 29:52 How is data stored? Key Value? 33:56 Transactions: Complexity, SQL vs NoSQL 42:51 MVCC in YugabyteDB: How does it work? 45:08 Default Transaction Isolation level in YugabyteDB 51:57 Fault Tolerance & High Availability in Yugabyte 56:48 Thoughts on Postgres Compatibility and Future of Distributed SQL 01:03:53 Usecases not suitable for YugabyteDB Previous videos: Database Internals: Part1: https://youtu.be/DiLA0Ri6RfY?si=ToGv9NwjdyDE4LHO Part2: https://youtu.be/IW4cpnpVg7E?si=ep2Yb-j_eaWxvRwc Geo Distributed Applications: https://youtu.be/JQfnMp0OeTA?si=Rf2Y36-gnpQl18yj Postgres Compatibility: https://youtu.be/2dtu_Ki9TQY?si=rcUk4tiBmlsFPYzY I hope you liked this episode, please hit the like button and subscribe to the channel for more. Popular playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Franck's Twitter and Linkedin: https://twitter.com/FranckPachot and https://www.linkedin.com/in/franckpachot/ Connect and follow here: https://twitter.com/thegeeknarrator and https://www.linkedin.com/in/kaivalyaapte/ Keep learning and growing. Cheers, The GeekNarrator

Hey Everyone, In this video I talk to Gilad Kleinmann, CEO and Co-Founder of epsio.io, about Epsio and how it helps companies to run queries faster and cheaper. Chapters: 00:00 Introduction 02:09 Defining the problem statement 07:17 What is Epsio ? 09:58 How does Epsio change my architecture? 12:59 Use of CDC 14:05 Where is the query result stored ? (Foreign data wrappers) 15:40 What permissions does Epsio needs? 16:43 How does Epsio parses a query and creates a virtual table? 24:15 Consistency model of Epsio 27:48 How do I know if Epsio is suitable for me? 31:41 How does it compare with Caching? 35:59 What metrics are available with Epsio? 38:32 What other databases does Epsio support? (will support) 40:47 How to know more about Epsio? 41:37 Pricing model of Epsio Read more about epsio: https://www.epsio.io/ Docs: https://docs.epsio.io/ Foreign data wrappers: https://wiki.postgresql.org/wiki/Foreign_data_wrappers Other playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN I hope you like this episode, please hit the like button if you did and subscribe to the channel if you haven't. Cheers, The GeekNarrator

Hey Everyone, In this episode I talk to Anjul Sahu, Founder of CloudRaft, about all things Kubernetes. There are many common questions that we try to answer. If you ever wondered if you should adopt Kubernetes as a company? or learn Kubernetes as a Developer? or why people fail adopting Kubernetes? Or even if Kubernetes is worth the hype? Then this episode will bring a ton of clarity. Chapters 00:00 Introduction 02:37 History and Background 06:06 k8s use cases 08:24 k8s features 09:34 Some basics of k8s 11:54 Common problems people face with k8s 15:48 k8s - challenges 18:59 Should you use k8s? 24:02 Alternatives of k8s 26:54 Top reasons why people fail with k8s 31:38 As a dev - should I learn k8s? Is it good for my career? Do give this episode a like and share it with your network. Also please subscribe to the channel for content like this. Other playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Anjul Sahu: https://anjul.dev/ Cloudraft: https://www.cloudraft.io/ Our website: www.geeknarrator.com References: Kubernetes course: https://kodekloud.com/courses/kubernetes-for-the-absolute-beginners-hands-on/ Kubernetes Hard Way: https://github.com/kelseyhightower/kubernetes-the-hard-way Kind: https://kind.sigs.k8s.io/ k3s: https://k3s.io/ k0s: https://k0sproject.io/ Cheers, The GeekNarrator

In this video I talk to David Jacot who works as a Staff Software Engineer at  @Confluent  and has been a long time Kafka user, committer and PMC member. We covered how Kafka works internally in great depth. We use Kafka for various use cases and it works great, but going one level below the abstraction and truly understanding the protocols, techniques and algorithms used is a fun ride. Chapters: 00:00 Kafka Internals with David Jacot 03:33 Defining Kafka 05:16 Kafka Architecture(s) 11:39 Write Path - Producer sending data 18:35 How does replication work? 25:47 How do we track replication progress? 30:42 Failure Modes: Leader fails 38:18 Consumers: Push vs Pull 40:54 Consumers: How does fetch works? 49:03 Consuming number of bytes vs records 50:50 Optimising consumption 01:00:21 Offset management and choosing partitions 01:09:10 Ending notes I hope you like this episode and more importantly you learnt some amazing techniques Kafka uses to ensure durability, low latency, simplicity and scalability in its architecture. Do give this episode a like and share it with your network. Also please subscribe to the channel for content like this. Other playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Other episodes: KsqlDB: https://youtu.be/2yE86P6uD_0 Exactly once semantics: https://youtu.be/twgbAL_EaQw David's Linkedin: https://www.linkedin.com/in/davidjacot/ our website: www.geeknarrator.com Cheers, The GeekNarrator

In this episode I talk to @gkcs on the Practical system design for Senior Engineers. Gaurav brings in a ton of experience in Competitive Programming, Teaching System Design and working on real world problems at companies like @Uber and Directi. Currently he is running his startup called @interviewready_ that helps folks perform better at Interviews. Do check him out if you haven't. I truly believe System Design in the real world requires ton of other skills along with great technical skills. It becomes even more critical at Senior levels. So in this episode we discuss our view and experience on the same while providing some concrete examples from our real world experience. I hope this episode brings in some clarity and adds some value to folks at all levels. At junior level, you would know what to expect and how to approach, At senior level, you would know what you might be missing or can do more to get to the next level. Chapters: 00:00 System Design as Senior Engineers - Things that no one talks about about. 04:11 Fundamentals 08:32 Do you get to choose the Tech stack? 16:47 Thinking about the future while being fast 21:18 Real world vs Interviews 23:45 Importance of Communication Skills 30:50 IC does not mean Isolated Contributor 35:20 Senior Engineering is a journey not a Certificate. 39:17 Volatility, Uncertainity, Complexity, Ambiguity 42:18 Importance of Decision Making 47:49 Reducing the blast radius 53:10 Leading by example Give it a watch and share with your network. Hit the like button and don't forget to subscribe to the channel. Cheers, The GeekNarrator

Hey Everyone, In the 43rd episode I speak with Tim Berglund on Realtime Analytics with Apache Pinot. Chapters: 00:00 Introduction 01:22 What do we mean by analytics and realtime analytics? 05:35 Can we define realtime in millis, seconds or minutes? 08:54 What is the fundamental difference between traditional analytics systems and Apache Pinot? 12:19 Was Kafka one of the reasons Apache Pinot could reach its full potential? 16:50 E-commerce Application example - How do I get my data in? 20:07 How is data stored (structured) on the disk? 23:31 Are joins available in Apache Pinot? 26:07 Joins vs pre-computing at ingestion 27:15 How is historical data ingested into Apache Pinot? 28:14 Types of indexes available in Apache Pinot 35:42 Do indexes cause write amplification? Is that a problem in Apache Pinot? 40:02 Point lookups in Apache Pinot 42:54 Anamoly Detection 45:51 Coming up in Apache Pinot Links: StarTree https://startree.ai/ Apache Pinot: https://pinot.apache.org/ Joins in Pinot: https://startree.ai/blog/apache-pinot... Apache Pinot Indexes: https://docs.pinot.apache.org/basics/... Other playlists: Distributed systems:    • Distributed Syste...   Modern Databases:    • Modern Databases   Serverless Architecture:    • Serverless Archit...   Software Engineering:    • Software Engineering   I hope you like the episode. Like, share and subscribe to the channel. Cheers, The GeekNarrator

In this video I talk to Philip Fried from Estuary about Batch vs Realtime Stream Processing. Philip brings a ton of experience in the world of data processing and has shared some of the best practices in implementing these systems. We dive deep into the world of data processing, covering batch and streaming systems, their challenges, tradeoffs and use cases. Chapters: 00:00 Batch vs Realtime Stream Processing 03:25 What is Batch and Reatlime processing? 18:29 How does Batch and Realtime compare in terms of Latency and Throughput? 27:24 Where is the cost saving coming from? Compute?Storage? or Network? 31:38 Moving from Batch to Stream processing 37:50 How is Idempotency implemented in Streaming systems? 48:50 How do we approach Schema evolution in Batch and Streaming systems? 57:16 Summary - key points to keep in mind Do checkout Estuary if you deal with a ton of data, and don't want to deal with the painful operations, infrastructure management, schema migrations etc and only want to focus on building highly scalable and resilient applications. References: Estuary: https://estuary.dev/ Flow documentation: https://docs.estuary.dev If you like this video please hit the like button, share it with your network (whoever works with a ton of data) and subscribe to the channel. Feel free to watch related episodes in the playlist:    • Distributed Syste...   Modern Databases:    • Modern Databases   Software Engineering:    • Software Engineering   Distributed Systems:    • Distributed Systems   Cheers, The GeekNarrator

Hey Everyone, In this video I talk to Patrick McFadin from DataStax. We uncovered the new features in Cassandra 5.0 and discussed how ACID transactions are achieved in the new version. This is a deep dive into the features of Cassandra, consensus protocols and how Accord is different as compared to Paxos, RAFT, Spanner and Calvin. Chapters: Cassandra 5.0 - ACID transactions and Vector Search 00:00 Introduction 01:45 List of features in the new Cassandra 04:51 Who needs ACID properties? 07:20 Why didn't Cassandra have ACID properties so far? 10:35 Why is Accord consensus protocol well suited for Cassandra? 16:40 Lets take a gaming example to see how Transactions work 21:55 Whats happening behind the scenes in a Transactions? 27:44 What happens when there are failures? 33:41 How is upgrade to the new version going to look like? 35:48 How is the latency impacted because of transactions? 40:23 What was missing in lightweight transactions? 42:24 Vector Search - What is it? How does it work? Previous episode on Cassandra:    • Apache Cassandra ...   Other playlists to watch: Distributed systems and Databases:    • Distributed Syste...   Software Engineering:    • Software Engineering   Distributed systems:    • Distributed Systems   Modern Databases:    • Modern Databases   Patrick's Linkedin: https://www.linkedin.com/in/patrick-m... Astra: astra.datastax.com Cassandra: https://cassandra.apache.org/_/index.... I hope you liked this episode. If you did, please hit the like button, share it with your network and subscribe to the channel. Cheers, The GeekNarrator

Hey Everyone, In this episode I discuss Twisp's journey with Mike Parsons, CTO and Co-Founder of Twisp. We have discussed how they ended up building a cloud native distributed ledger database on top of DynamoDB, why they chose DynamoDB? what were the challenges they faced and many highly technical and very interesting topics. Chapters: 00:00 Introduction - Twisp - CTO and Co-Founder Mike Parsons 03:01 What do we mean by a Distributed Ledger Database? 07:13 Is it similar to Blockchain? 08:32 Challenges faced with Relational Database setup 19:39 Why DynamoDB was the right choice? 24:40 What was the scale you were building for? 26:05 Who uses a Ledger database? 29:30 What are the basic primtives in a ledger database? 33:55 Is every primitive Immutable? How is it stored? 38:07 Impact of Immutability on Data storage 41:33 How does idempotency work in Twisp? 44:53 How did you provide ACID guarantees on top of DynamoDB? 52:30 How do you ensure correctness? 57:13 Interesting real world usecases where Twisp is used. 01:02:33 Are you planning to build even higher level primitives? References: Clocks: https://cse.buffalo.edu/tech-reports/... and http://rystsov.info/2018/10/01/tso.html TicToc: https://people.csail.mit.edu/sanchez/... Hekaton Paper: https://www.microsoft.com/en-us/resea... Hermitage Test suite: https://github.com/ept/hermitage Online Async Schema Change in F1 : https://static.googleusercontent.com/... Designing Data intensive Applications: https://www.oreilly.com/library/view/... P Programming language: https://github.com/p-org/P Twisp: https://www.twisp.com/ AWS Dynamo DB: https://aws.amazon.com/pm/dynamodb/?t... Connect with Mike: https://www.linkedin.com/in/parsnips/ Follow me: Twitter: https://twitter.com/thegeeknarrator Linkedin: https://www.linkedin.com/in/kaivalyaa... Watch other playlists: Databases:    • Modern Databases   Distributed systems:    • Distributed Systems   Software Engineering:    • Software Engineering   Sofk skills:    • Soft Skills   Like, share and subscribe to the channel folks. See you in the other episode. Cheers, The GeekNarrator

In this episode I talk to Yan Cui, who is an AWS Serverless Hero, all about Serverless technologies. Chapters: 00:00 Serverless Architecture with Yan Cui 01:58 What do we mean by Serverless Architecture? 05:42 What is the core problem Serverless solves? 11:06 Do we need to think differently to be able to use Serverless? 15:27 What is the difference between serverless and managed services? 19:17 Is Vendor Lock-in really a problem? 27:42 Multicloud - Is it really worth it? 33:46 Is ColdStart a real problem? What kind of apps get impacted? 43:25 Monitoring serverless applications 48:22 Usecases when serverless may not be the best solution 54:27 Future of serverless 57:31 How should a developer learn about serverless? I hope you enjoy the discussion and learn from it. Please hit the like button, share it with your network and also subscribe to the channel. References: Yan Cui - https://theburningmonk.com Courses - https://productionreadyserverless.com/ Corey Quinn on MultiCloud -    • Corey Quinn: The ...   Linkedin Yan - https://www.linkedin.com/in/theburnin... Twitter Yan - https://twitter.com/theburningmonk Other playlists to watch: Distributed Systems and Databases -    • Distributed Syste...   Software Engineering -    • Software Engineering   Distributed systems practices -    • Distributed Systems   Cheers, The GeekNarrator

It was an honour to host Dr. Venkat Subramaniam on the podcast. Dr. Venkat Subramaniam is an award-winning author, founder of Agile Developer, Inc., and an instructional professor at the University of Houston. He has mentored tens of thousands of software developers in the US, Canada, Europe, and Asia, and is a regularly-invited speaker at several international conferences. Venkat helps his clients effectively apply and succeed with agile practices on their software projects. In this episode, Dr. Venkat has shared his experience on becoming productive, managing time effectively, learning and Software Engineering. There is so much to learn from the way Venkat approaches learning and keeping up with the pace. No wonder, how he is able to do so many different things. Chapters: 00:00:00 Introduction into Venkat's Journey 00:04:23 How is Venkat able to do so many things? 00:15:23 Saying NO and prioritisation 00:44:42 Keeping up with the pace and learn 00:53:55 Engineering practices every engineer must follow 01:14:40 Ending notes, resources etc.. This episode is full of stories, experiences and learnings from Dr. Venkat's vast experience. The analogies he has shared are amazing. I hope you like this discussion and also learn a ton of things that matter. Venkat Subramaniam: https://twitter.com/venkat_s Agile Developer: https://agiledeveloper.com/ If you like this discussion, please like, share, subscribe and press the bell icon. Keep supporting and more amazing discussions coming your way. Cheers, The GeekNarrator

Hey Everyone, In this video I was joined by Daniel Ciocîrlan from https://rockthejvm.com and we discussed Functional Programming concepts in great detail. Chapters: 00:00 Introduction 03:11 What is Functional Programming? How can I start learning? 11:40 Object Oriented Programming and Functional Programming 16:08 How should a beginner approach learning Functional Programming? 21:58 Handling Side Effects (Pure functions) 28:04 Immutability and Recursion 35:59 Concurrency Models (Futures, Co-routines, Actors) 42:47 Project Loom 46:42 Scala vs Kotlin 49:01 Current Limitations and Future RockTheJVM : https://rockthejvm.com I hope you like the episode, please like, share and subscribe for more videos. Cheers, The GeekNarrator

Hello Everyone, Time Management is a critical skill for Software Engineers and most of us struggle with it everyday. So I and Arpit decided to discuss and share our learnings and experience on time management for Software Engineers. We have discussed about: - How to balance Context Switching? - How to say NO? - How and when to ask for help? - Prioritisation of tasks - Visibility of your work - Documentation and helping others - Getting better at Time Management I hope this casual discussion will help you figure out gaps in your approach and help you get better at managing your time well :) If you like the episode, please like, share and subscribe to the channel. Also don't forget to press the bell icon to get notified on any new video added to the series. Arpit Bhayani : https://www.linkedin.com/in/arpitbhay... and https://arpitbhayani.me/

#softwareengineering #timemanagement Cheers, The GeekNarrator

Hey Everyone,

In this episode I was joined by Francesco Tisiot who is a Developer Advocate @Aiven. He has shared some interesting insights into how you can build robust data platforms. The approach we have discussed is also described here: https://aiven.io/blog/a-soft-methodology-to-define-robust-data-platforms

References:   

SOFT : https://aiven.io/blog/a-soft-methodology-to-define-robust-data-platforms

Aiven: https://aiven.io/Francesco 

Tisiot https://twitter.com/FTisiot and https://www.linkedin.com/in/francescotisiot/

I hope you like the episode. Please subscribe to the channel and share with your network.   

Cheers, The GeekNarrator

Hey Everyone,  

In this episode I and Matthias talk about KsqlDb. We have covered the topic in great depth talking about its history, architecture, different concepts, use cases, limitations, comparison to Kafka Streams and so on.

References: 

ksqlDB - https://ksqldb.io/

exactly once semantics podcast: https://youtu.be/twgbAL_EaQw

Matthias Sax: https://twitter.com/MatthiasJSax and https://www.linkedin.com/in/mjsax/ 

Cheers, The GeekNarrator

Hey Everyone, 

In this episode I have Denis Magda from YugabyteDb with me.

We have explained how to build Geo-Distributed Applications in depth.  

I am sure this will help you make your applications more scalable, highly resilient, compliant and highly performant. If you liked the video please subscribe to the channel and give me a like.  

Further reading: 

https://www.yugabyte.com/blog/what-is... 

https://cloud.google.com/load-balancing

AWS outage: https://aws.amazon.com/message/41926/

 Connect with Denis: https://www.linkedin.com/in/dmagda/ and https://twitter.com/denismagda 

Connect with me: https://www.linkedin.com/in/kaivalya-... 

Cheers, The GeekNarrator

Hey Everyone,

In this episode Gunnar Morling, who is the former project lead for Debezium and has recently started working for Decodable, has joined us to talk everything about Change Data Capture with Debezium.

We discussed basic and advance topics and features of Debezium. If you are doing dual writes, Debezium could be a great tool for you. 

I hope you like the episode.

Cheers,

The GeekNarrator

Hey Everyone,

In this episode I am joined by Richard Startin who works with DataDog on Java profiling. Richard has an amazing experience with profiling and benchmarking Java applications and he has explained us what profiling means, how one can get started, different concepts in the world of profiling, tools available, limitations and the future of profiling.

I hope you like the episode.

Cheers,

The GeekNarrator

Hey Everyone,

In this episode I am joined by Matthias Sax, who works with Confluent to build the amazing world of Kafka.

We have discussed in real depths of Kafka Streams and how Exactly once semantics is implemented.

This episode will give you all the details you need to understand how beautifully Kafka imeplements EOS.

I hope you like the episode.

Cheers,

The GeekNarrator

Hey Everyone,

In this episode Alex Debrie joined us to dive deep into the world of one of the most popular database, i.e. AWS DynamoDB.

This is a true masterclass where we go deep into different concepts, architecture and design choices of DynamoDB. We talked about CAP and PACELC, dynamo streams and many other advanced topics as well.

I hope you like this masterclass.

Cheers,

The GeekNarrator

Hey Everyone,

In this episode Paul Christmann has joined us to demystify the world of Non Functional Requirements.

We started with very basics and discussed on top of it, about how one can approach defining Non Functional requirements for a system.

I hope you like the discussion.

Cheers,

The GeekNarrator

Hey Everyone,

In this episode I invited Arpit Bhayani (#asliengineering) to talk about the famous Bitcoin whitepaper.

We have discussed in depth the architecture, design choices, challenges and suggested solutions.

I hope you like the episode.

Cheers,

The GeekNarrator

Hey Everyone,

This is the part2 of our episode with Franck Pachot.

In this episode I talk to Franck Pachot from YugabyteDB who has vast experience in the world of Databases.

We have talked about how modern databases work internally. BTrees, LSM trees and various design choices.

Cheers,

The GeekNarrator

Hey Everyone,

This is the part1 of our episode with Franck Pachot.

In this episode I talk to Franck Pachot from YugabyteDB who has vast experience in the world of Databases.

We have talked about how modern databases work internally. BTrees, LSM trees and various design choices. 

Cheers,

The GeekNarrator

Hey everyone,

This is the part-2 of our episode with Tim Berglund. We have covered some advanced topics on Kafka, Kafka Streams and Apache Pinot.

I hope you like the discussion.

Cheers,

The GeekNarrator

Hey Everyone,

In this episode I talked to Tim Berglund about his vast experience with Kafka, realtime analytics and Apache Pinot.

I hope you like the episode. Do watch the part-2.

Cheers,

The GeekNarrator

In this podcast I have invited Neha Pawar, who is one of the Founding Engineers are StarTree (the company powering Apache Pinot). We talked about how StarTree has implemented Tiered storage and how it differs from other available implementations.  Note: Currently tiered storage is available only in StarTree’s Pinot and not available in the open source version. But its only about time.

Chapters: 00:00 Introduction 03:28 What does Tiered Storage mean? 05:51 How many tiers are typically supported? 07:30 Is it mainly about Cost Optimisation? How do I compare the cost savings vs performance hit? 15:41 What is mmap and how does it help? 16:45 How do I implement/approach Tiered Storage? What are the challenges? 23:00 What is Apache Pinot? When we say low latency, how low it is? 25:00 How is it implemented in StarTree (Apache Pinot)? 36:45 What happens when I query for more number of (or all) columns? How is that optimised? 47:10 What are the failure modes? 50:15 How can we test and validate Tiered Storage as a feature? 54:30 How would bloom filter false positives affect performance and correctness? 56:15 Can I move back my data from Cold storage to Hot Storage? 57:45 What other cloud storage services are supported other than S3? 58:35 What is the future of Tiered Storage?

In this episode I invited Curtis Einsmann to join us and share his knowledge and experience on how to do code reviews effectively. 

Curtis has created a course called "Master the Code Review" which has some great testimonials and amazing feedback. 

We have discussed various suggestions on making code reviews more effective for teams and individuals.  

Chapters:

00:00 Introduction 

10:36 How should teams appoach Code Reviews?

 18:19 How should individuals approach Code Reviews? 

25:10 Does Code Reviews slow down the development process? 

28:15 Common mistakes done by Authors and Reviewers 

35:20 Are nitpicks useful? 

37:45 Where does Pair Programming fit into this process? 

40:50 Practical examples  

Links: Curtis Einsmann Twitter:  https://twitter.com/curtiseinsmann

Curtis Einsmann Linkedin: https://www.linkedin.com/in/curtisein...

Master the code review course: https://curtiseinsmann.gumroad.com/l/...

Here is the team license: https://curtiseinsmann.gumroad.com/l/... 

Hope you get amazing insights from the episode and make your code review process better. 

If you like the video please don't forget to hit the like button and share your feedback in the comments section.   

Also please press the bell icon and subscribe to The GeekNarrator for more content on Software Engineering.   

Cheers, 

The GeekNarrator

Hey Everyone, 

In this episode I had a fun conversation with Pratik Pandey who is currently working as a Senior Software Engineer at Booking.com and has vast experience in the Software Industry specialising in designing systems for large scale workloads.    

We have covered System Design in Depth and destroyed the myths around System Design in general.  

Agenda: 

00:00 Introduction 

04:30 System Design Framework 

19:33 Requirements Gathering 

28:00 Capacity Planning and Estimations 

31:50 Driving System design with Capacity planning 

42:21 Choosing the right tech stack 

58:45 Real world System Design vs System Design Interviews 

01:13:00 Must have components in a System Design Interviews 

01:17:32 How do I prepare for System Design? 

01:22:45 How do I gain experience in System Design? 

01:26:00 Resources to prepare for System Design  

I hope you like our discussion and learn more about System Design interviews and how it differs with the real world system design. Do hit the like button and subscribe to the channel if you like such content.  

Cheers, The GeekNarrator

Hey Everyone, 

In this episode I invited Patrick McFadin who is an expert in the world of Cassandra and Data Modelling. 

Patrick currently works for DataStax as a VP Of Dev Rel. Patrick has given several techtalks on Cassandra and the ecosystem around it. We have covered the architecture of Cassandra in depth.   

Heres what we have covered:  

00:00 Introduction 

04:00 History of Cassandra 

07:18 Patrick  Apache Cassandra? 

14:30 How writes work in Cassandra? 

21:30 How many copies are written on a single write? 

25:44 How does replication work? 

32:00 How do reads work? (Read consistency levels) 

39:00 Why is Allow Filtering not recommended? 

43:00 Data Modelling in Cassandra 

50:45 Modeling a Chat Application 

01:05:00 How does CAP theorem fits Cassandra? 

01:07:06 New features in Cassandra?  

References: 

Patrick McFadin: https://www.linkedin.com/in/patrick-m...

Kaivalya Apte: https://www.linkedin.com/in/kaivalya-...

Astra:  astra.datastax.com 

Cassandra: https://cassandra.apache.org/_/index....

Webinar on Data Modeling: https://www.youtube.com/watch?v=4D39w... 

Playlist on Distributed Systems and Databases: https://www.youtube.com/playlist?list... 

I hope you enjoyed our discussion and learned from it. Please like, share and subscribe to the channel and keep supporting.  

Cheers, The GeekNarrator

In this episode I talk to Luis Mineiro, Senior Director at Delivery Hero about Site Reliability Engineering. 

We have busted a lot of myths around Site Reliability Engineering as a concept. 

We have talked about the following: 

 00:00 Introduction 

02:00 What is SRE? What is NOT SRE? 

10:00 DevOps vs SRE vs Developers 

13:10 When do we need a dedicated SRE team? 

20:00 Traits of a Strong SRE culture 

29:00 Observability vs Monitoring vs Alerting 

41:40 Adding Observability to a Sandiwch Shop 

53:00 How to add distributed tracing to the Sandwich shop? 

57:10 How do we define SLO's for the Sandwich shop? 

01:01:00 How to I define timeouts between services? 

01:05:00 How do I determine cost of adding 9's to my SLOs? 

01:15:00 How do I transition from a Developer to SRE?  

References:  SRE Books from Google:  https://sre.google/books

Luis Linkedin: https://www.linkedin.com/in/lmineiro/

The GeekNarrator Page: https://www.linkedin.com/company/the-...

Kaivalya Apte Linkedin: https://www.linkedin.com/in/kaivalya-... 

I hope you like the video and learn a lot about SRE.  

Cheers,  The GeekNarrator

In this episode I speak with Arasu Shankher, who is working as a CTO at KBX Digital. Arasu has a vast experience as an Engineer, Manager and Technology Leader. He has worked for companies like Microsoft, Google and Amazon and has built many high performance teams.

We cover extremely important topics for Software Engineers and Managers at all levels.

Chapters:

00:00 Introduction

02:30 Traits of a Successful IC and Manager

07:11 Depth for ICs?

09:05 Do you miss being an IC? what is it that you miss?

10:10 How do you stay up to date? How do you keep learning?

16:30 What are some resources that you use to learn?

21:20 Productivity and Time Management

28:20 Dealing with toxic people?

37:10 How has the Tech space changed over the years?

41:40 What is the next big thing in the Tech Space?

48:00 Buy vs Build? How do we make a good decision?

54:10 How to manage Tech Debt effectively?

57:00 Attaching risk level to Tech Debt. Is it a good model?

59:20 Strategies to mitigate technical risks

01:01:20 One tip for Junior, Mid and Senior Engineers and Managers.

I hope you like this episode. Please hit the like button if you did and subscribe to the channel for more content like this.

Cheers, The GeekNarrator

Hey Everyone, 

In this episode I am talking to Mayank Shrivastava who has vast experience into building and maintaining high scale distributed systems. He was in the team that originally built Apache Pinot at Linkedin and is now working at StarTree as the Head of Core Data Engineering.  

He has shared some amazing insights from his experience and there is a lot to learn from our discussion.   

We discuss about the following: 

00:00 Introduction 

04:20 Practices to follow while designing and developing Distributed Systems 

05:47 What do we mean by Solid Scalable Design? How do we approach that? 

09:00 Safety Nets for developing Distributed systems 

10:21 When is the right time to do performance benchmarking? 

17:00 What is release certification? 

21:00 Deploying to Production 

24:45 Example when Canary Deployment might not be a good strategy? 

26:00 Example when Canary Deployment a good strategy? 

27:30 Post Deployment - how do we observe our system? 

33:30 How do we avoid on-call(alerting) noise? 

42:00 Maintaining a Large scale Distributed system 

47:15 Scaling up/down for stateful systems 

51:30 Handling Failures in Production (Disaster Recovery) 

01:00:30 Runbooks - How do we keep them updated?  

References: 

The GeekNarrator Linkedin page: https://www.linkedin.com/company/86276626

Kaivalya Apte: https://www.linkedin.com/in/kaivalya-apte-2217221a/

Geeknarrator website: www.geeknarrator.com 

Mayank Shrivastava: https://www.linkedin.com/in/mayankshriv/

StarTree: https://www.startree.ai/

Apache Pinot: https://pinot.apache.org/ 

Hope you enjoy the discussion and learn from it. Please hit the like button if you liked my discussion with Mayank and please subscribe to the channel for more content like this.  

Cheers, The GeekNarrator

Hey Everyone,

Welcome to the API design masterclass with our guest Deepak Patil, who works as a Principal Architect at iCertis (https://www.linkedin.com/in/01patilde...) and has decades of experience building and architecting mission critical experience.  

We talk about: 

Basic API Discussion 

00:00 Introduction 

03:50 What is an API? 

07:10 Why do we want to implement APIs? 

10:22 How do we plan to execute the API program?

14:45 Practical API Design 

24:40 Dominant Styles of API Entity vs Behaviour oriented 

31:50 Fine Grained vs Coarse grained APIs 

43:45 Pagination  

I hope you enjoyed the discussion in the part-1, please do listen to part-2 for more advanced topics and more practical examples.

Cheers,

The GeekNarrator

Hey All,   

In this episode, we have Maheep Bhalla with us, who has been a Senior Leader in the Product Management space for many years and has worked at amazing Tech companies who have built successful products over the years.   

We talked about Product Management in general and everything around it.  

We start with defining what is Product Management and start diving deep into answering all the confusing things in the Product Management world.   Are you aspiring to be a successful product manager? Then this Podcast is for you.  

Are you a developer who thinks how product management relates to you? Then this Podcast is for you.

If you are a product manager and want to reach that next level, then this podcast is for you. 

And last but not the least, if you love Product Management like our guest Maheep, then this podcast is for you.  

Some key topics we discussed: 

00:00 A Joke on Dev and PM relationship 

03:01 What is Product Management? 

07:00 How to transition into a Product Management Role? 

12:40 Are we solving the problem for majority of the customers? 

18:45 Why it might be sometimes important to solve for the 1% customers? 

19:10 How Technical should Product Managers be?

30:07 How can developers work effectively with PMs? 

43:00 Asking "Why" style questions is also very important. 

45:10 How to focus on the right problem to solve? 

53:10 How to measure product's success? and Examples of Great Software Products?

There is so much to talk about in this space, and for that we will be collaborating more in the future to bring you more insights into Product Management.  I hope you enjoy the podcast, please like, share and subscribe and let us know how do you like this podcast.  Also please feel free to share more questions that you need answers for.  

 Cheers,  The GeekNarrator

Hey 👋 ,  

In this podcast we are talking about Distributed Stream Processing at High scale with Maximilian Michels (https://www.linkedin.com/in/maximilianmichels/) , who is an expert in Data intensive applications and an open source maintainer for Apache Flink, Beam and other cool technologies.  

 We have covered an introduction on Stream Processing at high scale:

 - What is stream processing? 

- How does it compare with Batch Processing and also event processing? 

- What tools are available to perform stream processing? 

- Use cases where we need Stream processing. 

- Flink vs Kafka and some factors to decide between the two.  https://docs.confluent.io/platform/current/streams/index.html and https://flink.apache.org/

- Why Exactly once processing is a Hard problem and how Kafka and Flink approach that problem? Is it similar to exactly once delivery?

 - Do we really need exactly once?

 - Why data locality matters?

 - Should we make network calls from the stream processor OR have a join instead? How do decide?  And so many other important considerations based on real production experience in operating stream processing pipelines at a very high scale. 

 I hope you like the podcast. Please like, share and subscribe 🙏   The GeekNarrator channel needs your love and support 😃  

Regards, The GeekNarrator

In this video I (Kaivalya Apte) talk to Max Linden, CEO and Co-Founder of Lemon.markets, where he shares his insights into entrepreneurship, product mindset, setting goals and making HUGE impact.  - 

- What is lemon.markets? 

- Why do we need a trading API like lemon.markets? 

- What it takes to build a trading API like lemon.markets? 

- Why is it named lemon.markets? 

- How to be more productive?

 - Setting stretch goals

 - Getting Customer Feedback and solving problems for customers. 

- Biggest lesson from Max's entrepreneurship journey 

To try lemon markets  - https://www.lemon.markets

 How to use Gmail more effectively https://klinger.io/posts/dont-drown-i... 

Follow The GeekNarrator below:  Anchor: https://anchor.fm/kaivalya-apte 

Spotify: https://open.spotify.com/show/2peu9Op... 

Apple: https://podcasts.apple.com/us/podcast... 

Google: https://podcasts.google.com/feed/aHR0... 

Please Like, Share and Subscribe to the GeekNarrator.  

Cheers, The GeekNarrator

Hey 👋 People,

In this conversation we talk to Ananth about his experience with Microservices and Monolithic architectures. We also look at various examples of a good/bad monolith, good/bad Microservices.

We also talk about our personal experience with different architectural styles and share pros/cons.

Org structure must align with the way we develop software and we briefly touch about how business leadership and technical leadership should align on Software Architecture.

Having the right tooling in place is critical for Microservices architecture. We talk about the challenges of Microservices architecture when such tooling is missing.

We end the episode with talking about SLOs/SLIs and SLAs, and how a customer focused approach can help us.

Please like, share and follow the podcast. 

Cheers,

The GeekNarrator