Introducing the pg_analytica extension for Postgres.Tables can be exported to columnar format via this extension to massively speed up your analytics queries. Exported data is periodically refreshed to keep your query results fresh.

Hi HN! We're Evgeny and Arjun, and we’re building a better way to do analytics with Postgres.We love Postgres for its simplicity, power, and rich ecosystem. But engineers have to still get bogged down with heavyweight and expensive OLAP systems when connecting an analytics data stack.Postgres is amazing at OLTP queries, but not for OLAP queries (large data scans and aggregations). Even in this case, we’ve still heard from countless scaling startups that they still try to use only a read replica to run analytics workloads since they don’t want to deal with the data engineering complexity of the alternative. This actually works surprising well initially, but starts to break for them as they scale or when integrating multiple data sources. Adding lots of indexes to support analytics also slows down their transactional write performance.When growing out of “just use Postgres”, companies have to understand and wrangle complex ETL pipelines, CDC processes, and data warehouses — adding layers of complexity that defeat the simplicity that undermines their initial choice for Postgres as their data storage in the first place.We thought there had to be a better way, so we’re building BemiDB. It’s designed to handle complex analytical queries at scale without the usual overhead. It’s a single binary that automatically syncs with Postgres data and is Postgres-compatible, so it’s like querying standard Postgres and works with all existing tools.Under the hood, we use Apache Iceberg (with Parquet data files) stored in S3. This allows for bottomless inexpensive storage, compressed data in columnar files, and an open format that guarantees compatibility with other data tools.We embed DuckDB as the query engine for in-memory analytics that work for complex queries. With efficient columnar storage and vectorized execution, we’re aiming for faster results without heavy infra. BemiDB communicates over the Postgres wire protocol to make all querying Postgres-compatible.We want to simplify data stacks for companies that use Postgres by reducing complexity (single binary and S3), using non-proprietary data formats (Iceberg open tables), and removing vendor lock-in (open source). We'd love to hear your feedback! What do you think?

Hello HN!We're excited to announce MyDuck Server, an open-source project that seamlessly integrates the analytical power of DuckDB with your existing MySQL & Postgres databases.*Backstory*Currently, there are no fully satisfactory open-source OLAP solutions for MySQL & Postgres. In the MySQL ecosystem, HeatWave offers close integration, but it's a proprietary, commercial product from Oracle. The Postgres community has seen promising DuckDB-based extensions emerge, including the official pg_duckdb. However, extensions can introduce isolation concerns in mission-critical environments.Consequently, many organizations resort to setting up complex and costly data movement pipelines using tools like Debezium, Flink, or other commercial solutions to replicate data from MySQL & Postgres to OLAP systems (e.g., Snowflake, BigQuery, ClickHouse) or Lakehouses (e.g., Delta Lake + Spark). This approach introduces significant operational overhead and expense.Another emerging strategy is the zero-ETL approach, increasingly advocated by cloud providers. This model simplifies data integration by allowing the cloud provider to manage ETL pipelines, while necessitating reliance on specific cloud ecosystems and services.*Key features*MyDuck Server offers a real-time analytical replica that leverages DuckDB's native columnar storage and processing capabilities. It operates as a separate server, ensuring isolation and minimizing impact on your primary database. Key features include:- Easy Zero-ETL: Built-in real-time replication from MySQL & Postgres with no complex pipelines to manage. It feels like a standard MySQL replica or Postgres standby. With the Docker image, passing a connection string is enough.- MySQL & Postgres Protocol Compatibility: We take this seriously and are working to make this project integrate well with the existing ecosystem around MySQL & Postgres. Currently, it is already possible to connect to MyDuck with standard MySQL & PostgreSQL clients in many programming languages.- HTAP Support: A standard database proxy can be deployed in front of a MySQL/Postgres primary and its MyDuck replica to route write operations to the primary and read operations to the replica. It just works.- DuckDB SQL & Columnar I/O over Postgres Protocol: It's unnecessary to restrict ourselves to MySQL/Postgres's SQL expressiveness and row-oriented data transfer. The Postgres port accepts all DuckDB-valid SQL queries, and you can retrieve query results in columnar format via `COPY (SELECT ...) TO STDOUT (FORMAT parquet/arrow)`.- Standalone Mode: It does not need to be run as a replica. It can also act as a primary server that brings DuckDB into server mode and accepts updates from multiple connections, breaking DuckDB's single-process limitation.*Relevant Previous HN Threads*- pg_duckdb [1] (https://news.ycombinator.com/item?id=41275751) is the official Postgres extension for DuckDB. It uses DuckDB as an execution engine to accelerate analytical queries by scanning Postgres tables directly.- pg_mooncake [2] (https://news.ycombinator.com/item?id=41998247) is a Postgres extension that adds columnstore tables for PG. It uses pg_duckdb under the hood but stores data in Lakehouse formats (Iceberg & Delta Lake).- BemiDB [3] (https://news.ycombinator.com/item?id=42078067) is also a DuckDB-based Postgres replica. Unlike us, they focus on storing data in Lakehouse format.We believe MyDuck Server offers a compelling solution for those seeking high-performance analytics on their MySQL & Postgres data without the complexities and costs of traditional approaches. We're eager to hear your feedback and answer any questions you might have. Let me know what you think![0] https://github.com/apecloud/myduckserver[1] https://github.com/duckdb/pg_duckdb[2] https://github.com/Mooncake-Labs/pg_mooncake[3] https://github.com/BemiHQ/BemiDB

Hello HN, one of the founders of Mooncake Labs here.Today, we are launching pg_mooncake, an extension that brings columnstore tables with DuckDB execution to Postgres. Expect analytical performance akin to DuckDB on Parquet (clickbench results to come soon). You can run transactional updates, deletes, inserts on these tables, and they're written as Delta Lake tables (and soon Iceberg) to your object store (S3, etc.). pg_mooncake is live on Neon today. Let us know what you think. It'll be coming to Supabase shortly, and other Postgres providers in the future.Why another postgres analytics extension?We actually leverage pg_duckdb and DuckDB as our execution engine — this is how we were able to ship the extension in just 60 days. We wrote about this with the DuckDB team: https://motherduck.com/blog/pg-mooncake-columnstore/pg_mooncake shines in two scenarios: 1. Up-to-date analytics in Postgres - This is where having a table semantics, and not just exporting files is key. 2. Exporting Postgres Data to Iceberg/Delta Lake tables, and querying them outside of Postgres - Run ad-hoc analytics with Pandas, DuckDB, Polars. Or data transforms and processing with Polars and Spark directly on these tables. You don’t need to manage ad-hoc parquet files and complex pipelines. Mooncake Labs.We just came out of stealth today. Mooncake is a managed Lakehouse with clean Postgres and Python experiences. Our core belief is that open table formats (Iceberg and Delta) provide a lot of flexibility, and we can ensure great DevEx on top. We will bring it to more workloads, applications, developers and agents.Read about our beliefs: https://mooncake.dev/blog/3.Cheers!

Hi HN,I wanted to share an exciting new open-source project: "VulcanSQL"! If you're interested in seamlessly transitioning your operational and analytical use cases from data warehouses and databases to the edge API server, this open-source data API framework might be just what you're looking for.VulcanSQL (https://vulcansql.com/) is suitable for following use cases:* Customer-facing analytics - expose analytics in your SaaS product for customers to understand how the product is performing for them via customer dashboards, insights, and reports.* Data Sharing - sharing data with partners, vendors, or customers, which requires a secure and scalable way to expose data.* Internal tools - Integration with internal tools like Retools.it leverages the impressive capabilities of DuckDB as a caching layer. This combination brings about cost reduction and a significant boost in performance, making it an excellent choice for high-concurrency, low-latency scenarios that OLAP not suitable for.By utilizing VulcanSQL, you can move remote data computing in cloud data warehouses, such as Snowflake and BigQuery to the edge. This embedded approach ensures that your analytics and automation processes can be executed efficiently and seamlessly, even in resource-constrained environments.GitHub: https://github.com/Canner/vulcan-sql

I built Supr Analyst, a natural language to SQL analytics interface. It combines LLM-powered query generation (OpenAI + Claude) with database schema understanding and user-defined metrics for accurate data exploration.Technical stack: - FastAPI backend for API endpoints and query processing - Next.js + Shadcn/UI for a responsive frontend - PostgreSQL for storing metadata and user configurations - Custom prompt engineering to handle complex SQL generation - Table relationship management through foreign key detection and custom mappingsKey features I implemented: - AI-generated table and column metadata to improve query context - User-defined example queries that act as few-shot learning examples - Custom metric definitions that help generate accurate aggregations - Query validation layer to prevent unsafe operations - Support for MySQL, Postgres, and Snowflake connections - Interactive "Try Chat" with sample dataset to test natural language queries - SQL Playground for trying complex queries and understanding the query generationYou can try it live here with sample dataset: Chat: https://supranalyst.com/try-supr-analyst/chat Playground: https://supranalyst.com/try-supr-analyst/playgroundCurrently in private beta with people. Would appreciate feedback on the approach and UX.

My name is Elliott. For the last three years, I’ve been building a DevOps platform on the best-in-class open source platforms (Kubernetes, Elixir, PostgreSQL, Grafana, etc.). The goal is to give engineering teams access to a modern DevOps infrastructure without needing to have a full SRE/DevOps team dedicated.It’s also open source /fair source - all the source code is here → https://github.com/batteries-included/batteries-includedI shipped a public beta today and would love to hear initial reactions, thoughts, and feedback.Here are details of the platform: * The platform features a user-friendly suggestion-based interface that guides users on topics like PostgreSQL cluster memory/CPU ratios, serverless web hosting, and secure secret sharing. Advanced users can quickly access complete control over their data.* It’s an Elixir-based UI on a database-driven, self-hosted Kubernetes platform. It can automatically deploy a scalable cloud installation (currently on AWS, with more options to follow) without inflicting YAML or Terraform configurations. Alternatively, it can set up a development instance using Kind and Docker or Podman, facilitating a smooth transition from local to production environments.* The platform supports easy AI project hosting for various workloads. Use Ollama embedding models for text embedding, eliminating OpenAI costs and data leakage risks. With PGVector and Cloud Native PG for vector databases, you can achieve near-state-of-the-art performance without exposing your data to third-party APIs. Experiment with Jupyter Notebooks, featuring optional Nvidia Plugin batteries for no DevOps-required experimentation.* Single Sign-On is streamlined via Keycloak, Istio Ingress, and OAuth Proxy, and it is securely hosted on your machine or cloud account. We've simplified security with full mTLS, Istio, SSL generation, and automated routing with Let's Encrypt and Acme for HTTP2. Istio Ingress services are seamlessly configured down to the contents of config maps. Grafana and Victoria Metrics can be auto-configured with just a few clicks for easy installation.Here’s also a look at the demo of the database deployhttps://www.youtube.com/watch?v=YbvkWja3VIQThe platform follows all the best practices learned for configuring and running a maintainable system without Kubenete's GitOps pain.If you want to check it out, here are links to docs, site, repo, and signup to try it out: * https://www.batteriesincl.com/* https://home.batteriesincl.com/signup* https://github.com/batteries-included* https://www.batteriesincl.com/docs

We announced our startup pgEdge this morning. We have two options available: pgEdge Platform (self-hosted, fully open) and pgEdge Cloud (managed DBaaS), with the latter in private beta. Take a look and let us know what you think!

PeerDB is a Postgres-first data-movement platform that makes moving data in and out of Postgres fast and simple. We do multiple postgres-native and infrastructural optimization to provide 10x faster data-movement for Postgres users.