Skip to main content

Snowflake Integration for Nodus

1. Introduction to the Snowflake Integration​

What Is This Integration?​

The Snowflake integration connects your Snowflake data warehousing platform with Nodus, allowing you to extract, analyze, and transform data from Snowflake tables and views. This integration functions as a source connector, bringing your Snowflake data into the Nodus ecosystem for advanced analytics and business intelligence.

Prerequisites:​

  • An active Snowflake account
  • Access to a Snowflake warehouse, database, and schema
  • Snowflake user credentials with appropriate permissions to read from the source data
  • Recommended: A dedicated user role for Nodus connections with appropriate privileges

Connection Overview:​

The integration uses Snowflake's SQL API to extract data and perform queries. Authentication is handled through standard Snowflake credentials, and data is extracted based on user-defined configurations that specify which schemas and tables to retrieve.

2. Platform Setup Documentation (Setup Form for Snowflake)​

Purpose & Scope​

This section covers how to set up the initial connection between Nodus and Snowflake by providing the necessary authentication credentials and account details.

Field-by-Field Breakdown:​

Integration Name​

  • Field Name & Label: Integration Name
  • Description & Purpose: A descriptive name to identify this Snowflake integration within your Nodus account.
  • Validation Rules & Format: Text string, required field.
  • Examples: "Production Snowflake", "Analytics Data Warehouse"
  • Troubleshooting Tips: Use a descriptive name that clearly identifies the specific Snowflake instance or purpose.

Host​

  • Field Name & Label: Host
  • Description & Purpose: The host domain of your Snowflake instance.
  • Validation Rules & Format: Text string, required field.
  • Examples: "accountname.us-east-2.aws.snowflakecomputing.com"
  • Troubleshooting Tips: This must include the account, region, cloud environment, and end with snowflakecomputing.com. You can find this in your Snowflake URL when you log in.

Database​

  • Field Name & Label: Database
  • Description & Purpose: The name of the Snowflake database you want to connect to.
  • Validation Rules & Format: Text string, required field.
  • Examples: "ANALYTICS_DB", "PRODUCTION_DB"
  • Troubleshooting Tips: Database names are case-sensitive in Snowflake. Ensure you use the exact same capitalization as in your Snowflake account.

Warehouse​

  • Field Name & Label: Warehouse
  • Description & Purpose: The name of the Snowflake compute warehouse to use for queries.
  • Validation Rules & Format: Text string, required field.
  • Examples: "ANALYTICS_WH", "REPORTING_WH"
  • Troubleshooting Tips: The specified warehouse must be running or auto-resumable. The Nodus user must have USAGE privileges on this warehouse.

Schema​

  • Field Name & Label: Schema
  • Description & Purpose: The default schema to use within the selected database.
  • Validation Rules & Format: Text string, required field.
  • Examples: "PUBLIC", "REPORTING"
  • Troubleshooting Tips: Schema names are case-sensitive in Snowflake. The schema must exist in the specified database, and the user must have appropriate access.

User​

  • Field Name & Label: User
  • Description & Purpose: The Snowflake username used to authenticate the connection.
  • Validation Rules & Format: Text string, required field.
  • Examples: "NODUS_USER", "ANALYTICS_SERVICE"
  • Troubleshooting Tips: For security best practices, consider creating a dedicated user for Nodus with appropriate permissions.

Password​

  • Field Name & Label: Password
  • Description & Purpose: The password associated with the Snowflake user.
  • Validation Rules & Format: Text string, required field, masked for security.
  • Troubleshooting Tips: Passwords are case-sensitive. Ensure the password has not expired in your Snowflake account.

Role​

  • Field Name & Label: Role
  • Description & Purpose: The Snowflake role to assume after connecting.
  • Validation Rules & Format: Text string, optional field.
  • Examples: "NODUS_ROLE", "ANALYST"
  • Troubleshooting Tips: If specified, the role must be granted to the connecting user. If not specified, the user's default role will be used.
  • Link Label: "Snowflake Documentation"
  • URL: https://www.snowflake.com/en/
  • Purpose: Provides access to official Snowflake documentation for additional help.

Step-by-Step Guide:​

  1. Prepare your Snowflake environment:
    • Ensure you have a Snowflake database and schema that you want to extract data from
    • Verify you have a warehouse with adequate compute resources
    • Set up a dedicated user with appropriate permissions (recommended)
  2. Gather your connection details:
    • Snowflake host URL (account name, region, and cloud provider)
    • Database and schema names
    • Warehouse name
    • User credentials (username and password)
    • Role name (if needed)
  3. Enter your integration details in the Nodus setup form:
    • Provide a meaningful Integration Name
    • Enter your Snowflake Host URL
    • Enter your Database name
    • Enter your Warehouse name
    • Enter your Schema name
    • Enter your User credentials
    • Optionally enter a specific Role
  4. Save the configuration and test the connection

3. Extraction/Query Configuration (Extraction Form for Snowflake)​

Purpose & Overview​

This section explains how to configure data extraction from Snowflake. The form allows you to select specific schemas and tables, define extraction strategies, and specify column filters for optimized data extraction.

Template & Field Documentation:​

Source Schema and Table Selection​

  • Field Name & Label: Source Schema / Source Table
  • Description & Purpose: Defines which schema and table to extract data from
  • Validation Rules & Format: Dropdown selection, required field
  • Troubleshooting Tips: The schema list is populated with available schemas in your Snowflake database, and the table list is populated based on the selected schema.

Destination Configuration​

  • Field Name & Label: Destination Schema / Destination Table
  • Description & Purpose: Defines where extracted data will be stored in the Nodus data warehouse
  • Validation Rules & Format: Schema dropdown and table name input, required fields
  • Troubleshooting Tips: Choose an existing schema and provide a name for the destination table where your data will be loaded.

Write Strategy​

  • Field Name & Label: Write Strategy
  • Description & Purpose: Defines how data should be written to the destination
  • Validation Rules & Format: Radio button selection, required field
  • Available Options:
    • Upsert - Update existing records and insert new ones
    • Append - Add new records to the existing table
    • Replace - Drop and recreate the table with new data
  • Troubleshooting Tips: Choose the appropriate strategy based on your use case. Upsert requires a primary key.

Workflow & Examples:​

  1. Select the source schema and table from the cascading dropdowns
  2. Select a destination schema and enter a table name
  3. Choose a write strategy
  4. Execute extraction

Example Use Cases:​

Sales Data Analysis:

  • Source Schema: SALES
  • Source Table: TRANSACTIONS
  • Write Strategy: Append

Customer Data Synchronization:

  • Source Schema: CRM
  • Source Table: CUSTOMERS
  • Write Strategy: Upsert (based on CUSTOMER_ID)

4. Troubleshooting & FAQs for Snowflake​

Common Issues & Error Messages​

Authentication Failures​

  • Error: "Failed to connect to Snowflake: authentication error"
  • Solution: Verify your username, password, account URL, and role. Ensure the user account is active and not locked.

Permission Issues​

  • Error: "Insufficient privileges" or "Object does not exist or not authorized"
  • Solution: Verify that the user has the necessary permissions on the schema, table, and warehouse. Check if the role has the required grants.

Warehouse Issues​

  • Error: "Warehouse cannot be found" or "Warehouse is suspended"
  • Solution: Verify the warehouse name and ensure it exists. Check if the warehouse is running or set to auto-resume.

Contact & Support Information​

Best Practices for Snowflake Connections​

  • Use a dedicated service account for Nodus integration
  • Grant the minimum required privileges to the service account
  • Consider using a dedicated virtual warehouse for Nodus queries
  • Use time-travel capabilities in Snowflake for data reconciliation if needed
  • Set appropriate resource monitors on warehouses to control costs