Should I Build It Should I Build It
02 Jul 2025
GitHub Developer Tools

Building ETLFunnel — a fresh take on solving ETL challenges. Unlike ...

...existing tools like HevoData or Fivetran, ETLFunnel is built for maximum flexibility and control. It's an on-premises ETL solution that empowers teams to design complex data pipelines using custom code in Golang — making it ideal for advanced and general-purpose use cases.

Confidence
Engagement
Net use signal
Net buy signal

Idea type: Freemium

People love using similar products but resist paying. You’ll need to either find who will pay or create additional value that’s worth paying for.

Should You Build It?

Build but think about differentiation and monetization.

Your are here

ETLFunnel enters a space where users appreciate accessible solutions but are hesitant to pay. Given the low number of directly comparable products found (n_matches=1), and thus low confidence in the market analysis, it's crucial to carefully consider differentiation and monetization strategies from the outset. The single similar product, Pathway, demonstrates high engagement (avg n_comments=17). Your proposed focus on on-premises solutions, custom code via Golang, and advanced use cases could be key differentiators to start with. While the provided metrics lack specific 'use' or 'buy' signals, the Pathway launch discussions highlight user interest in self-hosting, resource usage, and specific applications like RAG. However, concerns about limitations (like RAM caps) and build times also surfaced, suggesting areas for improvement in your approach. Therefore, it is imperative that you leverage the unique offering of ETLFunnel (Golang, on-prem) to try to capture a niche in the market.

Recommendations

  1. Begin by clearly identifying the specific user segments that derive the most value from open-source ETL tools, focusing on those with complex data pipeline needs that resonate with ETLFunnel's Golang-based customizability. Given Pathway's feedback, focus on users with a need for maximum flexibility and control, and on-prem solutions.
  2. Develop premium features that significantly enhance the capabilities of the free version for the target segment, such as advanced monitoring tools, dedicated support, or pre-built integrations with popular data sources relevant to complex data pipelines. This will encourage users to upgrade.
  3. Explore a team-based pricing model, as opposed to individual pricing, to appeal to larger organizations with multiple users who require ETLFunnel's advanced features, and who are more likely to have the budget for on-prem solutions. This aligns with the need to discover 'who will pay' within a freemium model.
  4. Offer personalized onboarding, consulting, or custom pipeline development services to paying customers, providing hands-on assistance that justifies the investment in a premium ETL solution, and directly addressing the need for solutions for the advanced user.
  5. Conduct targeted pricing experiments with small groups of users to determine the optimal pricing strategy, considering factors such as feature access, usage limits, and support levels. Closely monitor user behavior and feedback to fine-tune pricing and packaging, while taking into account Pathway’s criticism regarding licensing and hidden costs.
  6. Actively engage with the open-source community by contributing code, addressing user feedback, and building a strong reputation for ETLFunnel, fostering a loyal user base that can drive adoption and advocacy. This will help build 'use' and 'buy' signals.
  7. Create comprehensive documentation and tutorials that demonstrate the power and flexibility of ETLFunnel, helping users overcome initial learning curves and maximize their return on investment. Addressing concerns regarding build times like Pathway is also essential.
  8. Highlight security and compliance features of ETLFunnel to appeal to organizations that require on-premises solutions due to data sensitivity or regulatory requirements. This is essential to target advanced and general-purpose use cases.

Questions

  1. Considering ETLFunnel's focus on Golang and on-premises deployment, which specific industries or use cases have the greatest need for custom ETL solutions that balance flexibility with security and control?
  2. Given the concerns around hidden drawbacks and limitations with similar products, what proactive steps can be taken to build trust with potential users and clearly communicate the limitations and benefits of ETLFunnel?
  3. How can ETLFunnel differentiate itself in terms of ease of use and deployment, to make it an attractive and easier on-ramp alternative to current more complex, self-hosted ETL solutions?

Your are here

ETLFunnel enters a space where users appreciate accessible solutions but are hesitant to pay. Given the low number of directly comparable products found (n_matches=1), and thus low confidence in the market analysis, it's crucial to carefully consider differentiation and monetization strategies from the outset. The single similar product, Pathway, demonstrates high engagement (avg n_comments=17). Your proposed focus on on-premises solutions, custom code via Golang, and advanced use cases could be key differentiators to start with. While the provided metrics lack specific 'use' or 'buy' signals, the Pathway launch discussions highlight user interest in self-hosting, resource usage, and specific applications like RAG. However, concerns about limitations (like RAM caps) and build times also surfaced, suggesting areas for improvement in your approach. Therefore, it is imperative that you leverage the unique offering of ETLFunnel (Golang, on-prem) to try to capture a niche in the market.

Recommendations

  1. Begin by clearly identifying the specific user segments that derive the most value from open-source ETL tools, focusing on those with complex data pipeline needs that resonate with ETLFunnel's Golang-based customizability. Given Pathway's feedback, focus on users with a need for maximum flexibility and control, and on-prem solutions.
  2. Develop premium features that significantly enhance the capabilities of the free version for the target segment, such as advanced monitoring tools, dedicated support, or pre-built integrations with popular data sources relevant to complex data pipelines. This will encourage users to upgrade.
  3. Explore a team-based pricing model, as opposed to individual pricing, to appeal to larger organizations with multiple users who require ETLFunnel's advanced features, and who are more likely to have the budget for on-prem solutions. This aligns with the need to discover 'who will pay' within a freemium model.
  4. Offer personalized onboarding, consulting, or custom pipeline development services to paying customers, providing hands-on assistance that justifies the investment in a premium ETL solution, and directly addressing the need for solutions for the advanced user.
  5. Conduct targeted pricing experiments with small groups of users to determine the optimal pricing strategy, considering factors such as feature access, usage limits, and support levels. Closely monitor user behavior and feedback to fine-tune pricing and packaging, while taking into account Pathway’s criticism regarding licensing and hidden costs.
  6. Actively engage with the open-source community by contributing code, addressing user feedback, and building a strong reputation for ETLFunnel, fostering a loyal user base that can drive adoption and advocacy. This will help build 'use' and 'buy' signals.
  7. Create comprehensive documentation and tutorials that demonstrate the power and flexibility of ETLFunnel, helping users overcome initial learning curves and maximize their return on investment. Addressing concerns regarding build times like Pathway is also essential.
  8. Highlight security and compliance features of ETLFunnel to appeal to organizations that require on-premises solutions due to data sensitivity or regulatory requirements. This is essential to target advanced and general-purpose use cases.

Questions

  1. Considering ETLFunnel's focus on Golang and on-premises deployment, which specific industries or use cases have the greatest need for custom ETL solutions that balance flexibility with security and control?
  2. Given the concerns around hidden drawbacks and limitations with similar products, what proactive steps can be taken to build trust with potential users and clearly communicate the limitations and benefits of ETLFunnel?
  3. How can ETLFunnel differentiate itself in terms of ease of use and deployment, to make it an attractive and easier on-ramp alternative to current more complex, self-hosted ETL solutions?

  • Confidence: Low
    • Number of similar products: 1
  • Engagement: High
    • Average number of comments: 17
  • Net use signal: 5.3%
    • Positive use signal: 5.3%
    • Negative use signal: 0.0%
  • Net buy signal: 0.0%
    • Positive buy signal: 0.0%
    • Negative buy signal: 0.0%

This chart summarizes all the similar products we found for your idea in a single plot.

The x-axis represents the overall feedback each product received. This is calculated from the net use and buy signals that were expressed in the comments. The maximum is +1, which means all comments (across all similar products) were positive, expressed a willingness to use & buy said product. The minimum is -1 and it means the exact opposite.

The y-axis captures the strength of the signal, i.e. how many people commented and how does this rank against other products in this category. The maximum is +1, which means these products were the most liked, upvoted and talked about launches recently. The minimum is 0, meaning zero engagement or feedback was received.

The sizes of the product dots are determined by the relevance to your idea, where 10 is the maximum.

Your idea is the big blueish dot, which should lie somewhere in the polygon defined by these products. It can be off-center because we use custom weighting to summarize these metrics.

Similar products

Relevance

Pathway – Build Mission Critical ETL and RAG in Python (NATO, F1 Used)

13 Jun 2024 Open Source Developer Tools

Hi HN data folks,I am excited to share Pathway, a Python data processing framework we built for ETL and RAG pipelines.https://github.com/pathwaycom/pathwayWe started Pathway to solve event processing for IoT and geospatial indexing. Think freight train operations in unmapped depots bringing key merchandise from China to Europe. This was not something we could use Flink or Elastic for.Then we added more connectors for streaming ETL (Kafka, Postgres CDC…), data indexing (yay vectors!), and LLM wrappers for RAG. Today Pathway provides a data indexing layer for live data updates, stateless and stateful data transformations over streams, and retrieval of structured and unstructured data.Pathway ships with a Python API and a Rust runtime based on Differential Dataflow to perform incremental computation. All the pipeline is kept in memory and can be easily deployed with Docker and Kubernetes (pipelines-as-code).We built Pathway to support enterprises like F1 teams and NATO to build mission-critical data pipelines. We do this by putting security and performance first. For example, you can build and deploy self-hosted RAG pipelines with local LLM models and Pathway’s in-memory vector index, so no data ever leaves your infrastructure. Pathway connectors and transformations work with live data by default, so you can avoid expensive reprocessing and rely on fresh data.You can install Pathway with pip and Docker, and get started with templates and notebooks: https://pathway.com/developers/showcasesWe also host demo RAG pipelines implemented 100% in Pathway, feel free to interact with their API endpoints: https://pathway.com/solutions/rag-pipelines#try-it-outWe'd love to hear what you think of Pathway!

Users are intrigued by Pathway's capabilities, particularly its self-hosting and resource usage, including RAM requirements. There's interest in the Rust engine, licensing, and the build process for Python and Rust. Users have successfully built discrete event and AI solutions with it. Questions arose about vector indexes, RAG use cases, and state persistence, with some clarifications provided on RAM usage and file backends. The community is engaged, with mentions of NATO and NAFO, and there's appreciation for the clear documentation and impressive Python tools for ETL and RAG tasks.

Users have expressed concerns about potential hidden drawbacks and limitations such as an 8GB RAM cap on machines. Criticisms include the presence of license checking within the Rust source code, lengthy build times of one to two hours, and the requirement for audio transcription to be performed upstream. Additionally, there are suggestions for codebase improvements.

Avatar
73
17
5.9%
17
73
5.9%
Top