TinyFish has launched BigSet, an open-source multi-agent system designed to build structured datasets from plain-English descriptions, according to Marktechpost. Released on June 2, 2026, BigSet aims to streamline the process of creating datasets by automating data collection and organization.
What Is BigSet?
BigSet is a tool that converts user descriptions into structured datasets without the need for manual data scraping. It processes natural language inputs, identifies relevant web data, and generates exportable CSV or XLSX files. This system addresses the need for efficient data collection automation, eliminating the traditional steps of data sourcing and configuration.
How Does the Multi-Agent Architecture Work?
BigSet employs a structured two-tier agent system to perform its tasks. Initially, the system uses a large language model (LLM) called Claude Sonnet to infer the dataset schema from user input. This is followed by the orchestrator agent, which identifies and gathers relevant data entities from the web using TinyFish Search and Fetch tools. The system then removes duplicates and attributes data sources before exporting the final dataset.
What Are the Key Features of BigSet?
BigSet offers several key features, including schema inference, orchestrated data collection, and automatic dataset updates. Users can schedule these updates at various intervals, such as every 30 minutes or daily, ensuring that datasets remain current without manual intervention. Additionally, the system provides source attribution for each data entry, enhancing transparency and reliability.
Frequently Asked Questions
What is BigSet? BigSet is an open-source multi-agent system from TinyFish that automates the creation of structured datasets from natural language descriptions, streamlining data collection processes.
How does BigSet update datasets? BigSet allows users to set automatic update schedules, ensuring datasets are refreshed according to user-defined intervals without manual re-running of tasks.
What is the role of the orchestrator agent in BigSet? The orchestrator agent in BigSet identifies relevant data entities across the web and dispatches sub-agents to collect specific data, forming the rows of the final dataset.
Sources
