AgentQL

AI-powered alternative to XPath selectors

1.0•1 review•

904 followers

AI-powered alternative to XPath selectors

1.0•1 review•

904 followers

Visit website

Automation tools

•

AI Coding Agents

No more diving into complex DOM structures or writing fragile XPath expressions. Just specify what data you are scraping from the web with natural language-like queries, and AgentQL handles the rest.

Free Options

Launch tags:Developer Tools•Artificial Intelligence•Data & Analytics

Launch Team / Built With

AgentQL

Maker

📌

Hey Product Hunters! 👋 I am Shuhao, cofounder of the AgentQL team. Allow me to introduce AgentQL, an AI-powered semantic framework designed to enable AI agents to seamlessly interact with the web, using natural language. That's a lot of big words. Let's break down what AgentQL can do: 🕹️ Give it the command to find "price", and it will retrieve all the price data on an e-commerce page you spend less time building rigid selectors 🖥️ Take the same code to another site, and it can work just as well >> you spend less time maintaining fragile scripts 📝. Need to interact with sites as part of your work flow? Simply use natural language, tell AgentQL to click, scroll_page or fill. So why did we decide to build AgentQL? Traditional web automation struggles with identifying elements consistently through UI changes. When LLMs and agents first emerged, we envisioned a web infrastructure tailored for agentic interaction, which excels in handling these small but breaking changes, so we decided to tackle this challenge head-on. How did we go about this? At the heart of our framework is the AgentQL Query, a language crafted to describe locate web elements within a structured schema. We’ve integrated robust DOM processing with advanced prompt engineering, creating a powerful combination that dynamically generates context-aware prompts. By doing so, we overcome the fragility of static XPath or attribute selector-based scripts, enabling a more resilient and flexible web interaction infrastructure. Would you like to join our journey to an agentic future? Our early users have reported real productivity boosts with AgentQL, unlocking use cases that were previously too costly or complex to develop. Your feedback and questions will be invaluable to us. 👀 Please check out our technology at AgentQL.com 💬 Connect with our community on Discord and X.com Thank you for being a part of our story!

Report

2yr ago

@shuhao_zhang Congratulations to the AgentQL team on their release! Great job - up and running in seconds🦾

Report

2yr ago

@shuhao_zhang Congrats on your launch day! Wishing you great success and new opportunities. What challenges did you overcome to get here?

Report

2yr ago

@shuhao_zhang AgentQL's focus on natural language for web interaction with AI agents is a fascinating concept! The ability to handle dynamic website changes and overcome the limitations of traditional selectors is valuable. Here are some questions for further exploration: - How does AgentQL ensure the accuracy of its natural language parsing and element identification? How easy is it to maintain AgentQL queries when websites undergo significant changes? - Can AgentQL integrate with existing web scraping or automation frameworks? Can it learn and improve its understanding of user intent over time? - What security measures are in place to prevent unintended automation behavior or data breaches? Potential for Collaboration: As a developer with experience in AI and web automation, I'm interested in learning more about AgentQL's technology. Here are some potential areas for collaboration: - Contributing to further development of AgentQL's NLP capabilities for more robust understanding of user intent. - Exploring ways to leverage ML models for improved element identification and dynamic adaptation to website changes. - Collaborating on building a robust security framework for secure and controlled web interaction with AgentQL. I'd love to learn more about AgentQL's technical architecture and explore opportunities to contribute to its development.

Report

2yr ago

AgentQL

Maker

@shuhao_zhang @max_savonin1 Hi, Max! I may be able to help with some of these questions-- they're really great questions, btw! - How does AgentQL ensure the accuracy of its natural language parsing and element identification? How easy is it to maintain AgentQL queries when websites undergo significant changes? We do regular benchmarking to ensure our results are the best for accuracy and reproducibility, trading off against other dimensions. We also iterate against a set of real-world use cases to further improve AgentQL. We've found that providing context for certain terms is critical for accuracy and stabilization of results. As for maintaining AgentQL queries as websites undergo changes, we've found that we're generally able to handle most cases (even testing against some historic / archived versions of websites where available) so long as the elements exist, but obviously, if the structure changes too significantly, e.g. the elements or data no longer exists or the steps of a workflow are completely shuffled, the query wouldn't be able to adapt to this. - Can AgentQL integrate with existing web scraping or automation frameworks? Can it learn and improve its understanding of user intent over time? We currently provide an extension to the Playwright Python SDK, and are able to extend Page objects directly to run queries, interact, or provide some other utility functions. Most other frameworks that are leveraging Playwright where you still have access to the underlying Page object should be compatible. As for learning and understanding user intent, AgentQL itself is not personalized and doesn't adapt to usage, but we've provided tools like the aforementioned context to help disambiguate results. - What security measures are in place to prevent unintended automation behavior or data breaches? We currently view ourselves as a tool to enable developers, as it's simply a product which helps interpret data and provide elements for programmable interaction on web. We're curious to see the ingenuity of the community and believe our technology can be used to unlock a lot of good use-cases, but we will definitely remain vigilant and remain open to changing our position on this as we grow! Hope that helps, and definitely open to connecting to explore possible collaborations!

Report

2yr ago

AgentQL

Maker

@antonikozelski Thanks so much! We're excited to hear that the onboarding is smooth for you. Appreciate the support!

Report

2yr ago

only log in option is google ? I will wait then. I have email // apple // github// google // but cannot use something that only is for google users ! Ping me when we can use other ids

Report

2yr ago

AgentQL

Maker

Hey, Francoo! We'll have github SSO out by EOD-- stay tuned!

Report

2yr ago

AgentQL

Maker

@francoolaami just rolled out GitHub sign-in option! Thanks for the feedback!

Report

2yr ago

AgentQL

Maker

@francoolaami Thank you for the feedback and we enjoy building AgentQL together with the community. Looking forward to hearing more from you!

Report

2yr ago

Agents Base

Congrats! It appears to be very similar to Semantic Targets invented by Open Agent Studio(open source) last year https://www.openagent.studio/ would you be able to improve semantic targets or is it essentially the same?

Report

2yr ago

AgentQL

Maker

Hi, Rohan! The high level concept with Semantic Targets sounds similar, but I believe that our underlying approach to resolving query elements is quite different. Though I myself have not tested Semantic Targets extensively, we've seen really good results on accuracy and reliable consistency across a broad range of real world use-cases using AgentQL. We also choose to return the control flow back to the developer rather than trying to have an LLM reason about and generate the full end to end task, so that developers can be confident in actions taken, any validations, and graceful error handling across the workflow.

Report

2yr ago

Agents Base

@murkypan Thanks for the explanation yes open agent studio also has an Agent API that allows explicitly triggering events directly using json actions in English. I'd encourage you to try it out and see if you can improve it.

Report

2yr ago

AgentQL

Maker

@rohan_arun1 Thanks! Will look it up!

Report

2yr ago

Hunter

AgentQL is probably the easiest web data extraction tool that I have ever used. While it doesn't require all the back and forth editing to choose the right web elements, it still offers the ability to use code to process the data extracted ! Give it a try and you won't regret!

Report

2yr ago

AgentQL

Maker

Thanks for giving us a shot and the kind words! We're really excited to see positive engagement from the community!

Report

2yr ago

AgentQL

Maker

Thank you for your support, Shawn!

Report

2yr ago

AgentQL

Maker

@shunxinpang Thanks for your shoutout and love to explore more towards the future of Agent with you!

Report

2yr ago

AgentQL

Maker

My fellow data detectives, number crunchers, and info wranglers - remember those days of endless clicking through websites, hunting for elusive info? I do. Whether chasing sources as a journalist, crunching numbers in finance, or piecing together market intelligence in consulting, data gathering was always a challenge. That's why I am excited as both a user and a team member of AgentQL. We are building a tool that automates repetitive web interactions, e.g., point, click, scroll and get data from web or app UI. The nice thing is it uses an accessible query language that even I can learn :) It simplifies gathering and structuring web data for various projects. 👀 Please check out our technology at AgentQL.com

Report

2yr ago

AgentQL

Maker

@fiona_li5 🚀🚀🚀

Report

2yr ago

Oh this is sick! I can see lots of use cases to run E2E testing - my QA team will love this. How accurate are the results? I know on the website it says there is no black box, I wonder how does prompt engineering work together with DOM selection in this case? Congrats on launching this awesome product @shuhao_zhang and team!

Report

2yr ago

AgentQL

Maker

@shuhao_zhang @tonyhanded Hi Tony! Yes E2E testing is definitely a strong use case for this. The accuracy is dependent on the page complexity and the query quality, but has been consistently very high from our internal benchmarks. One of our key recommendations is to use semantic context https://docs.agentql.com/agentql... to optimize your queries. The more specific your queries are, the more accurate and consistent your responses will be. We have several tools listed on the website to help you get started if you want to test the accuracy out yourself!

Report

2yr ago

@andrew_chen22 thanks for sharing the guide very cool!

Report

2yr ago

AgentQL

Maker

@tonyhanded My pleasure! Let me know if you have any other questions 😊

Report

2yr ago

AgentQL sounds like a game changer for anyone who’s tired of the headache that comes with web scraping! 🙌 The idea of just using natural language to interact with elements is genius! No more wrestling with fragile selectors, which always seem to break when you least expect it. I'm really curious about how well it adapts to different site structures—does it handle them pretty seamlessly? The thought of writing less rigid code is definitely appealing. Plus, the integration of advanced prompt engineering is intriguing. I can see this being a real productivity booster, especially for teams who rely on frequent web data extraction. Kudos to you, @shuhao_zhang, for taking on the challenge of traditional web automation limitations! Can’t wait to see how AgentQL evolves and the new use cases it will unlock!

Report

2yr ago

AgentQL

Maker

@shuhao_zhang @blankwebdev Short answer: yes. Supporting reusable queries for varying site structures was definitely a focus of ours. You are right in that many websites do have significantly different layouts while providing similarly structured content - think e-commerce, travel, etc. With the right query, we very are capable of handling these seamlessly. If you have any questions or need help building such a query for your use case, feel free to reach out to us on Discord! I'd be happy to help

Report

2yr ago

AgentQL

Maker

Hi @blankwebdev ! To support @andrew_chen22 's point we have an example of handling 3 different e-commerce websites with exactly the same set of queries. You can check it out on our Github: https://github.com/tinyfish-io/f...

Report

2yr ago

1 2 3

•••

Review AgentQL?

AgentQL is highly praised for its intuitive and user-friendly approach to web scraping, eliminating the need for complex DOM navigation or fragile XPath expressions. Users appreciate its ability to simplify traditional web scraping methods, making it an invaluable tool for their work. The product is described as innovative, convenient, and frequently useful, with many users expressing admiration for the team's efforts. Overall, AgentQL receives strong recommendations for its impressive performance and ease of use.

Summarized with AI

Reviews

Most Informative