Hacking Postgres

Today we take a deep dive into the world of PostgreSQL extensions, featuring our conversation with Heikki Linnakangas, co-founder of Neon, an expert Postgres hacker, and contributor to pgvector. Heikki shares the fascinating story of how Neon was born in 2021, driven by the vision of creating an open-source platform akin to Aurora Serverless. He delves into the early challenges and triumphs of building a fully remote startup during pandemic times, including the excitement of growing from a handful of employees to a bustling company with 600,000 databases under management.

In this episode, we explore:

• From a startup idea to managing over 600K databases
• Building a remote-first culture and the power of annual off-site meetings
• Time travel… how Neon's unique storage system enables quick point-in-time recovery
• Upcoming Postgres features like asynchronous IO and multi-threading
• How understanding new algorithms can help database management

Links mentioned:

Neon

Heikki Linnakangas on GitHub

Heikki Linnakangas on LinkedIn

Joining Aiven in 2021, Francesco Tisiot works as a Field CTO and AI Lead. He serves as a technical advisor to clients and prospects, helping to meet their needs without the focus on sales, acting as a 'rented' CTO to provide expert guidance. Join us as we dive into Francesco's insightful perspectives on the Postgres community, where he challenges the notion of valuing years of experience over actual contributions and expertise.

In this episode, we explore:

• How Postgres is a core focus for Aiven
• Strategies for incorporating diverse viewpoints within the community
• The perception of Oracle within the Postgres community
• Benefits and challenges of using AI tools for database optimisation
• Acquisition of EverSQL by Aiven for integrating AI tools in SQL performance tuning

Links mentioned:

Aiven for AlloyDB Omni

Francesco Tisiot on X

Francesco Tisiot on LinkedIn

Today we’re joined by Ray Lane and DJ Patil from GreatPoint Ventures. Ray, known for his pivotal role at Oracle during its turnaround, shares his journey from the US Army to top tech positions and eventually co-founding GreatPoint Ventures, where he supports entrepreneurs. DJ Patil – investor, entrepreneur, and former U.S. Chief Data Scientist – joins to dive into the evolution of databases.

Tune in as they share their experiences with early databases such as Db2, the rise of Postgres, and current challenges in the tech landscape. With insights on leadership, community-driven development, and more, this episode is a must-listen for developers, entrepreneurs, and tech enthusiasts.

In this episode, we explore:

• The critical need for evolving roles within a company to foster growth
• Challenges of data distribution vs. keeping data centralized and speedy
• “Siding with the community” and enhancing user capabilities
• The risks of rushing towards generative AI, and the importance data hygiene and data culture

Links mentioned:

GreatPoint Ventures

DJ Patil on LinkedIn

Ray Lane on LinkedIn

Sai Srirampur is the co-founder of PeerDB and a veteran Postgres Solutions Engineer with experience at Citus Data and Microsoft. He has been at the forefront of optimizing and scaling Postgres for large data workloads and is now spearheading innovation in data movement and replication with PeerDB. In this episode, we'll discuss the challenges of tuning massive Postgres systems, real-time data streaming solutions, and PeerDB’s vision for the future.

In this episode we explore:

• Leaving Microsoft and starting Sai’s own company, PeerDB
• Developing PeerDB's data movement and replication tool
• Why Postgres might not be suitable for everything at a large scale
• Having team members be database experts first before being software developers
• The growing customer demand for Clickhouse and Snowflake

Links mentioned:

PeerDB

Sai Srirampur on X (@saisrirampur)

Sai Srirampur on LinkedIn

In this episode, we talk Postgres committer Melanie Plagemean about her involvement with @PGConfdev, improvements to vacuum scheduling and auto vacuum configuration, the logical replication of DDL, and platforms such as CNCF and Kubernetes integrating with Postgres.

Links mentioned:

PGConf.dev

YouTube – CMU Database Group

Path To Citus Con Podcast

@[email protected]

Melanie on X (@melanieplageman)

Bharath Rupireddy has carved a niche for himself in the Postgres community since he began using the database system back in 2020. From his start at EnterpriseDB to making strides at Microsoft, and now contributing to the AWS open-source project, Bharath is entrenched in the inner workings of Postgres development. He has worked in many areas of Postgres such as WAL, Replication, pg_walinspect extension, bug fixes, performance improvements, new SQL functions etc.

In this episode we explore:

• Starting with Postgres in 2020 and learning the basics while contributing to Parallel COPY feature
• Postgres recent features such as pg_walinspect extension, WAL source switch from archive to streaming, WAL insertion lock improvements, WAL read from buffers, more replication slot invalidation mechanisms like XID age based and inactive_timeout based ones, new table access methods for multi inserts.
• The future of native active-active replication
• The complexities of conflict resolution
• Using PostgreSQL 16 in production scenarios for active-active replication
• The role of databases evolving within different industries

Links mentioned:

¡Databases! – A Database Seminar Series

Hacking Postgres, Ep. 9: Bertrand Drouvot

PGConf India

Hyderabad PostgreSQL User Group

PGConf.dev

Bharath on LinkedIn

Bharath on X (@BRupireddy)

Bharath on Github

As the podcast host of The Builders, member of Postgres Women, and Staff Engineer at EDB, Gülçin Yıldırım Jelínek is passionate to see where this space will take us. We kick off Season 2 with Gülçin as she shares her journey in the tech industry, CloudNativePG, the impact of AI on Postgres, and the representation of women in the Postgres community.

In this episode we explore:

• CloudNativePG in Gülçin’s products and the new SPGVector release
• Welcoming new developers to the PostgreSQL community
• Tembo Cloud, PG vector, and the challenges and rewards of contributing to such innovative projects
• Improving inclusivity and support for women in the tech industry
• The use of Postgres in AI workloads

Links mentioned:

The Builders: A Postgres Podcast

Gülçin Yıldırım Jelínek on X 

EDB

Postgres Women on X

Alongside his peers studying Systems Network Engineering, Jelte Fennema-Nio unearthed a security vulnerability within the framework of Postgres. Since then, Jelte joined the world of cybersecurity and network engineering. He is currently Principal Software Engineer at Microsoft working on Citus/ Postgres/ PgBouncer. 

In this episode we explore:

• Forking and the impact of commercial entities on the core project 
• The challenges of connection pooling
• The suitability of Citus for scalability
• The intricacies of configuring PgBouncer
• Paying attention to community needs and modernizing contribution methods

Links mentioned:

Jelte on X

Jelte on LinkedIn

Amit Kapila is a PostgreSQL Committer and a Senior Director at Fujitsu India. Amit joins the show to share his contributions to logical replication enhancements (achieving highly available systems), his involvement in the PostgreSQL community, and his perspective on contributions from tech giants in advancing PostgreSQL features.

In this episode we explore:

• Amit’s early work with PostgreSQL and Oracle
• The future of active-active topology
• The dependence on managed services for database management
• The importance of conflict resolution in replication
• Overcoming the lack of an open-source development environment in India

Links mentioned:

pgsql-hackers

Amit Kapila on LinkedIn

Amit Kapila on X (@kapila_amit)

Amit’s Blog

As Principal Architect at Tembo and the creator of PGXN, David Wheeler is a seminal figure in the Postgres world. Join us as we discuss David's tenure at The New York Times, his influential projects like Bricolage and pgTAP, and the game-changing database change management system, Sqitch. 

In this episode we explore:

• David Wheeler's transition from archaeology to databases
• Database management and learning with Sqitch
• Working with PGXN and Tembo
• Creating a central registry for Postgres extensions
• The state of Postgres and its versatility

Links mentioned:

pgTAP

PGXN

CITEXT 2

Sqitch

Better Call Saul Insider Podcast

Scriptnotes

[email protected]

Just a Theory

Denis Magda is the mastermind behind the innovative extension PgCompute and a pivotal figure in the world of Postgres development. With over a decade and a half of experience, Denis cut his teeth on Postgres during its use in high-traffic social networking projects in Eastern Europe and has continued to push the envelope at Yugabyte as the head of Developer Relations.

In this episode we explore:

• The role of Yugabyte in the Postgres ecosystem
• The need for local development experiences
• A more 'magical' and testing-friendly approach to handling stored procedures
• The intersection of DevRel with sales engineering and marketing
• The future of Postgres in educational institutions

Links mentioned:

Business Wars podcast

Postgres FM

Founders podcast

Denis on X

Denis on LinkedIn

Yugabyte

Andrew Atkinson is a Software Engineer who specializes in building high-performance web applications using PostgreSQL and Ruby on Rails. He wrote the book ‘High-Performance PostgreSQL for Rails’, published by Pragmatic Programmers in 2024.

Our discussion with Andrew spans the technical challenges of sharding and the concurrent evolution of Rails and Postgres. We'll pay homage to influential resources like Railscast, debate Rails' database tooling limitations, and discover tips from Andrew's new book.

In this episode we explore:

• Why newer developers favor Postgres over MySQL
• How Postgres might become a multi-primary database in the future
• The complexities of database decisions in a Rails environment
• Postgres innovations, such as composite primary keys and common table expressions, being supported from Active Record – the ORM for Ruby on Rails
• Key insights from writing ‘High Performance PostgreSQL for Rails’

Links mentioned:

Andrew Atkinson on LinkedIn

Andrew’s Blog

Newsletter

PGCasts

RailsCasts

‘High Performance PostgreSQL for Rails’ by Andrew Atkinson

Github rideshare

Postgres FM

Andrew Aktkinson’s interview on Postgres FM

Andrew Atkinson’s interview on Remote Ruby

Remote Ruby Podcast

GitHub doc: clarify logical decoding's deadlock of system tables

GitHub doc: Doc: fix grammatical errors for enable_partitionwise_aggregate 

GitHub Convert README to Markdown

Just describing something truly novel is challenging, let alone building it. That’s the story of our final episode from Season 1, where Ry sits down with Tim Sehn from Dolthub to talk about the journey of building Dolt, the new release of Doltgres, and the challenges of adapting to the changing ways users engage with your product. 

Build the thing you wish you had right now. That’s the story behind Hacking Postgres, Ep 9, where Ry sits down with Bertrand Drouvot of AWS Cloud. Today they talked about the struggles of trying to build from scratch vs having a template, what should be core vs an extension, and being able to see what’s really happening inside your database. 

Search is simple in theory. In practice? Anything but. Join Ry for Episode 8 of Hacking Postgres, where he sits down with Philippe Noël of ParadeDB to talk about how search is evolving, the influence of AI, and the lessons you'd tell your younger self.

Postgres for everything? How about building a new cloud provider? Join us for Ep 7 of Hacking Postgres where Ry talks with Burak Yucesoy of Ubicloud about new clouds, getting attached to extensions, and the future potential of Postgres.

Ever wonder what it's like to build a geospatial extension for PostgreSQL? Join us for Episode 6 of Hacking Postgres, where Ry sits down with Regina Obe and Paul Ramsey, two of the key creators of PostGIS.

In this episode, Ry and Alexander talk about OrioleDB (and the challenge of fighting bloat), fuzzy and vector search, and the challenges behind database management.

Pavlo Golub is the Co-Founder of PostgreSQL Ukraine and a PostgreSQL expert at Cybertec. Pavlo created pg_timetable, an advanced standalone job scheduler for Postgres. 

Eric Ridge is the primary developer of ZomboDB, “Making Postgres and Elasticsearch work together like it's 2023.” He’s also an active part of the pgrx and plrust teams.