Build a Custom Provider Dataset
This guide walks you through creating Datasets from custom data sources in NinjaCat’s Data Cloud. From establishing Data Source connections to utilizing the AI-powered auto-matching functionality to map your data efficiently. Then ultimately setting up account matching, sync schedule and requesting data.
Establish Custom Data Source
- Navigate to Settings > Data Sources.
- Select your desired provider.
- Follow the instructions in the linked articles
After following the steps to create the data sources, come directly back to this article to see instructions on creating mappings. Don't follow the mapping instructions in the other articles.
Google Sheets
- Follow the instructions from the article Create a Google Sheets Data Source until you get to the mapping section
Big Query
- Follow the instructions in the article Add a Native Data Source, selecting "Big Query" as the desired provider
Snowflake Share
- Follow the instructions in the article Setup Snowflake Share until you get to the mapping section
Configure the Data Connector
- Navigate to Data Cloud > Connectors
- Click "Add Connector" in the top right
- Choose the desired custom Provider
- Give your new Connector an original, distinct name
- Choose the desired authenticated Provider account
- Click on "Add Credential" under "Credential"
- Click on the desired data source
- Click "Save Connector"
Create a New Dataset
- Click on "Add Dataset" under Datasets
Google Sheets
- Provide an original, distinct Dataset Name
- Select the Google Sheet you want to import data from
Big Query
- Provide an original, distinct Dataset Name
- Enter the Big Query Project ID for the desired data source
- Enter a query that sources your desired data
- Click "Next"
Snowflake Share
- Provide an original, distinct Dataset Name
- Enter a query that sources your desired data
Date & Account Matching
After creating your dataset, you'll configure your date and account matching settings first. This step ensures your date and account settings are established before you begin configuring columns, helping reduce configuration errors.
-
Assign a Date column to identify the date field in your data.
-
Assign an Account Matching column if applicable.
Configure Columns (Auto-Mapping and Data Formatting)
Once your date and account matching is configured, you can proceed to set up your column mappings.
-
NinjaCat will automatically map column headers to known dimensions and metrics using AI-powered auto-matching.
-
Review the mappings:
- Ensure the correct data types (e.g., currency, integer, percentage) are assigned.
- Modify any mappings as needed.
-
Use the Refresh Columns option if you need to update the data structure without reloading the data source.
-
Exclude unnecessary columns if needed.
Set Sync Schedules
- Set the refresh time (once a day or multiple times per day) for the data sync.
More scheduling options are now availableYou can now run your dataset on the connector's existing schedule, add an additional schedule, or override the connector schedule entirely. For details, see Configuring Ingestion Controls for Connectors & Datasets.
Key Features of the New Custom Datasets
- Dataset-First Approach: All mapped data from providers is automatically created as a dataset.
- Dataset includes sync history with detailed information on each data sync
- AI-Powered Auto-Matching: Suggests optimal column mappings to reduce manual work.
- Editable and Refreshable Mappings: Modify datasets without reloading the entire data source.
- Legacy & New System Coexistence: Users can still use legacy mappings while transitioning to the new system.
Troubleshooting & Common Issues
Issue: Headers not loading correctly
Solution: Refresh the columns or verify access permissions
Issue: Auto-matching errors
Solution: Manually adjust column mappings and re-save
Issue: Missing data after update
Solution: Ensure that new columns are included in the dataset settings
Issue: Unable to select multiple credentials
Solution: The feature is planned for future updates; currently, use one credential per connection
Updated 23 days ago