> For the complete documentation index, see [llms.txt](https://docs.harmony.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.harmony.io/data-sets/data-sets.md). # Data Sets {% hint style="info" %} **Path:** Settings → Data Sets {% endhint %} ### What are Data Sets? Data Sets are structured tables that let you store and manage rows of data directly in Harmony. Think of them as spreadsheets built into the platform. Each Data Set has typed columns you define and rows you populate, either manually or through workflows. Once created, a Data Set can be used across Harmony: * **Workflows** can query, insert, update, delete, and bulk upsert rows as part of automation steps. * **Service catalog forms** can pull a dataset's rows to populate dropdown options dynamically. * **Slack modals** can surface dataset rows as searchable select options for employees. * **Custom fields** on assets, employees, and applications can reference dataset values through data set selector fields. * **Custom Agents** can reference datasets to bring structured business data into agent workflows. *** ### Managing Data Sets Data Sets are managed from **Settings → Data Sets**. From this page you can create new datasets, view existing ones, and open any dataset to manage its columns and rows. #### Creating a Data Set 1. Go to **Settings → Data Sets** and click **New Data Set**. 2. Enter a **name** (must be unique) and an optional **description**. 3. Add columns to define the structure (see [Managing Columns](#managing-columns) below). 4. Click **Save** to create the dataset. #### Managing Columns Each column has a **name**, a **type**, and optional settings for whether it is required and whether it has a default value. | Column type | What it stores | | ----------- | --------------------------------- | | Text | Any string value | | Number | Numeric values | | Boolean | True / False | | Date | Date values | | Email | Email addresses | | URL | Web links | | Employee | A reference to a Harmony employee | You can add, rename, and reorder columns at any time. Deleting a column hides it from view but does not erase the data already stored in existing rows. This keeps historical records intact. #### Managing Rows Open a Data Set to view and edit its rows in a table view. You can: * **Add rows** manually by clicking **Add Row** and filling in the fields. * **Edit rows** inline by clicking a cell. * **Delete rows** individually using the row menu. * **Import rows** in bulk by uploading a CSV file (up to 1,000 rows at a time). #### Display Column Each Data Set has a **display column**: the column whose value is shown as the human-readable label when the dataset is used in a dropdown (e.g. in a Slack modal, service catalog form, or data set selector custom field). You can set this in the dataset settings. If no display column is set, Harmony defaults to the first column in the table. When a data set selector custom field is configured, you can also override the display column at the field level, so different uses of the same dataset can surface the most meaningful label for their context. *** ### Using Data Sets in Workflows The workflow block library includes blocks for interacting with Data Sets. Find them under the **Data Sets** category when adding a step to your workflow. #### Query Data Set Looks up rows from a dataset based on filters you define. Use this to retrieve data during a workflow, for example, finding the approver for a given department. * Configure filters using any column (e.g. "department equals Engineering") * Combine multiple filters with **All** (AND) or **Any** (OR) logic * Set a limit on how many rows to return (up to 100; defaults to 1) * The block outputs the matching rows and a `first_row` shortcut for single-lookup cases #### Insert Data Set Adds a new row to a dataset. Use this to write data from a workflow, for example, logging a completed action or recording a form submission. * Map workflow values to dataset columns * The block outputs the newly created row #### Update Data Set Updates all rows that match a set of filters. Use this to keep dataset records in sync, for example, marking an employee's status as inactive after offboarding. * Define filters to target the right rows * Provide the new values to write * The block reports how many rows were updated and surfaces any errors #### Delete Data Set Deletes all rows that match a set of filters. * Define filters to target the rows to remove * Optionally cap the number of rows deleted * The block reports how many rows were deleted #### Bulk Upsert Data Set Inserts or updates multiple dataset records in a single workflow step. Use this when you need to keep a dataset in sync as part of an automated workflow that processes batches of records - for example, syncing a list of tickets, contacts, or any other structured data your agents rely on. * Define the column to match on when deciding whether to insert or update each record * Map workflow values to dataset columns for each record in the batch * Removes the need to chain individual insert or update steps together, making bulk operations significantly faster to build and maintain *** ### Using Data Sets in Service Catalog Forms When building a service catalog form, you can configure a dropdown field to pull its options directly from a Data Set. The form will always show the current rows in the dataset, so you can update the available choices at any time without editing the form itself. The value shown in each dropdown option comes from the dataset's **display column**. #### Cascading Filters on Data Set Selector Fields You can configure cascading filters on data set selector fields in the Service Catalog form builder. This means the options shown in a data set dropdown can be automatically narrowed down based on the value a user selects in another field on the same form - keeping choices relevant and reducing noise. You can add multiple cascading filter rules to a single field. All active filters are applied simultaneously using AND logic, so only options that satisfy every rule are shown. Each filter rule includes the following settings: * **Filter column**: choose which column in the dataset to filter on * **Source field**: select another form field whose runtime value drives the filter * **Match operator**: choose between *equals* or *contains* to control how the comparison is made You can add and remove filter rules freely. Filtering is applied automatically when the form is rendered, with no additional configuration needed beyond defining the rules. *** ### Using Data Sets in Slack When a workflow presents a select menu in a Slack modal, you can point it at a Data Set to populate the options. Employees can type to search through the options live. Up to 100 options are shown at a time. The label for each option in the list comes from the dataset's **display column**. You can also use a **data set selector** field in Service Catalog form modals presented directly in Slack. When filling out a catalog form in Slack, users can search and select from dataset rows using a typeahead-backed dropdown without leaving the conversation. This gives catalog form builders a consistent, searchable experience for both data set selector and asset selector fields in Slack. *** ### Using Data Sets as Custom Fields You can add **data set selector** custom fields to assets, employees, and applications from **Field Management** settings. This gives you a consistent, dynamic field type across all your entity management workflows - not just service catalog forms. #### Configuring a Data Set Selector Custom Field When setting up a data set selector custom field, you can define the following options: * **Dataset**: choose which dataset the field draws its values from * **Allow multiple**: permit users to select more than one value * **Cascading filters**: filter available options based on other field values on the same record * **Display column**: specify which column from the dataset is shown to users Values entered into these fields are validated against the referenced dataset, ensuring data integrity across entities. #### Editing Data Set Selector Values on Entity Pages Once a data set selector custom field is added to an entity type, users can select and update values directly on asset, employee, and application detail pages using the full dropdown UI. Inline editing is supported on all entity detail pages. *** ### Using Data Sets in Custom Agents You can reference Data Sets inside your Custom Agents to bring structured business data - such as product catalogs, account lists, or any custom records - into your agent workflows without relying on external systems. Custom Agents support full CRUD operations on datasets, columns, and rows, including bulk updates. The **Employee** column type lets you link dataset records directly to members of your team, making it easy to build agent workflows that incorporate people data alongside other structured records. *** ### Common Use Cases #### Approver routing table Create a dataset with columns for **Department** and **Approver Email**. In your approval workflow, add a **Query Data Set** step that filters by the ticket's department and returns the matching approver. Use that email to route the approval request without hardcoding any values. #### Audit log Add an **Insert Data Set** step at the end of a workflow to record what happened, including who triggered it, what action was taken, and when. The dataset builds up a running log you can review in the Data Sets UI or export. #### Dynamic Slack dropdowns Create a dataset of office locations with columns for **City** and **Country**. Point a Slack select field at it. When a new office opens, just add a row to the dataset and the Slack dropdown updates automatically without any workflow changes. #### Cascading location selector Create a dataset with columns for **Country** and **City**. Add two fields to a service catalog form - one for Country and one for City. Configure a cascading filter on the City field so it only shows cities that match the selected Country. This keeps the options focused and reduces the chance of mismatched selections. #### Bulk syncing records via workflow Use the **Bulk Upsert Data Set** block to keep a dataset in sync as part of a scheduled or triggered workflow. For example, when a batch of onboarding tickets is processed, upsert all the relevant employee records into a tracking dataset in a single step rather than looping through individual inserts. *** ### Things to Know **Deleting a column does not delete existing data.** The column is hidden from the table view and no longer appears in workflows or forms, but the values already stored in rows are preserved behind the scenes. **Dataset names must be unique.** You cannot have two datasets with the same name. If you need to organize datasets, consider including a prefix in the name (e.g. "HR - Approvers", "IT - Office Locations"). **Bulk imports are capped at 1,000 rows.** If you need to import more, split the data into multiple CSV files. **Workflow operations on large datasets are capped at 10,000 rows.** Update and Delete workflow blocks process up to 10,000 matching rows per run. If your dataset is larger than this, use the **limit** setting on the block to control scope. **The display column affects all dropdowns.** Whichever column is set as the display column will be used as the label in every Slack modal, service catalog form, and data set selector custom field that references this dataset - unless overridden at the field level. Make sure it contains a value that is meaningful to end users. **Cascading filters use AND logic.** When multiple cascading filter rules are configured on a single field, all rules must be satisfied for an option to appear. There is no OR logic between rules. **Data set selector custom fields are validated against the dataset.** Values stored in data set selector custom fields on assets, employees, and applications must exist in the referenced dataset, keeping your entity data consistent. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.harmony.io/data-sets/data-sets.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.