Showing posts with label Snowflake. Show all posts
Showing posts with label Snowflake. Show all posts

Monday, April 25, 2022

JDBC - Data type Mapping for Snowflake

 It is important to understand how we map various JDBC data types for the Snowflake data types.

BITBOOLEAN
TINYINTSMALLINT
SMALLINTSMALLINT
INTEGERINTEGER
BIGINTBIGINT
FLOATFLOAT
REALFLOAT
DECIMAL(m,n)NUMBER(m,n)
NUMERIC(m,n)NUMBER(m,n)
DOUBLEFLOAT
CHAR(m)CHAR(m)
LONGVARCHARSTRING
LONGNVARCHARSTRING
NCHAR(m)STRING
NVARCHARVARCHAR(m)
VARCHAR(m)VARCHAR(m)
DATEDATE
TIMETIMESTAMP
TIMESTAMPTIMESTAMP
ARRAYSTRING
BINARYBINARY
VARBINARYBINARY
LONGVARBINARYBINARY
JAVA_OBJECTSTRING
BLOBBINARY
CLOBSTRING
NCLOBSTRING
STRUCTSTRING
REFSTRING
BOOLEANBOOLEAN
ROWIDSTRING
SQLXMLSTRING
OTHERSTRING
NULLSTRING
DISTRICTSTRING
DATALINKSTRING

How-to-generate-log-file-on-Snowflake-connector- SnowSQL

 

SnowSQL

  • What is the Snowflake SnowSQL version?

          > snowsql --version

          > snowsql --bootstrap-version

          > snowsql --versions

  • Is it reproducible with the latest SnowSQL version?
  • Is SnowSQL used interactively or in a batch script?
  • Generate log files:
    • Add -o log_level=DEBUG to the usual command line arguments, or if you prefer to do it in the SnowSQL config file, update log_level=DEBUG:
      • Linux or Mac: ~/.snowsql/config
      • Windows: %USERPROFILE%\.snowsql\config
    • If you're generating DEBUG level logs to troubleshoot a specific issue, you might consider creating a separate logfile while reproducing this issue. This can be done with adding -o log_file, such as:
      • Linux or Mac: snowsql -o log_level=DEBUG -o log_file=/path/to/snowsql_debug.log <rest of the usual arguments>
      • Windows: snowsql.exe -o log_level=DEBUG -o log_file=C:\Temp\snowsql_debug.log <rest of the usual arguments>
  • Collect log files:
    • Linux or Mac: ~/.snowsql/log*
    • Windows: %USERPROFILE%\.snowsql\log*
    • or if using log_file, collect the logfile from the path specified in log_file. E.g. C:\Temp\snowsql_debug.log in the above example.
  • Documentation reference:

Sunday, March 27, 2022

Using Key Pair Authentication & Key Pair Rotation

 

Using Key Pair Authentication & Key Pair Rotation

SnowSQL supports key pair authentication and key rotation, but does not support unencrypted private keys.

  1. To start, follow the instructions to configure Key Pair Authentication & Key Pair Rotation.

  2. Specify the path to the private key file either in the configuration file or on the command line:

  • In the configuration file:

    • Add the private_key_path connection parameter to your connection settings and specify the local path to the private key file you created. The syntax is not OS-specific:

      Supported OS
      private_key_path = <path>/rsa_key.p8
      
    • Use the SNOWSQL_PRIVATE_KEY_PASSPHRASE environment variable to set the passphrase for decrypting the private key file. The syntax is OS-specific:

      Linux/macOS
      export SNOWSQL_PRIVATE_KEY_PASSPHRASE='<passphrase>'
      
    Windows
    set SNOWSQL_PRIVATE_KEY_PASSPHRASE='<passphrase>'
    
  • On the command line:

    Include the private-key-path connection parameter and specify the path to your encrypted private key file:

    $ snowsql -a <account_identifier> -u <user> --private-key-path <path>/rsa_key.p8
    

    SnowSQL prompts you for the passphrase. Alternatively, use the SNOWSQL_PRIVATE_KEY_PASSPHRASE environment variable to set the passphrase for decrypting the private key file (as described above).

Key Pair Authentication & Key Pair Rotation.

 

Key Pair Authentication & Key Pair Rotation

This topic describes using key pair authentication and key pair rotation in Snowflake.

In this Topic:

Overview

Snowflake supports using key pair authentication for enhanced authentication security as an alternative to basic authentication (i.e. username and password).

This authentication method requires, as a minimum, a 2048-bit RSA key pair. You can generate the Privacy Enhanced Mail (i.e. PEM) private-public key pair using OpenSSL. Some of the Supported Snowflake Clients allow using encrypted private keys to connect to Snowflake. The public key is assigned to the Snowflake user who uses the Snowflake client to connect and authenticate to Snowflake.

Snowflake also supports rotating public keys in an effort to allow compliance with more robust security and governance postures.

Supported Snowflake Clients

The following table summarizes support for key pair authentication among Snowflake Clients. A checkmark (i.e. ✔) indicates full support. A missing checkmark indicates key pair authentication is not supported.

Client

Key Pair Authentication

Key Pair Rotation

Unencrypted Private Keys

SnowSQL (CLI Client)

Snowflake Connector for Python

Snowflake Connector for Spark

Snowflake Connector for Kafka

Go driver

JDBC Driver

ODBC Driver

Node.js Driver

.NET Driver

Configuring Key Pair Authentication

Complete the following steps to configure key pair authentication for all supported Snowflake clients.

Step 1: Generate the Private Key

Depending on which one of the Supported Snowflake Clients you use to connect to Snowflake, you have the option to generate encrypted or unencrypted private keys. Generally, it is safer to generate encrypted keys. Snowflake recommends communicating with your internal security and governance officers to determine which key type to generate prior to completing this step.

Tip

The command to generate an encrypted key prompts for a passphrase to regulate access to the key. Snowflake recommends using a passphrase that complies with PCI DSS standards to protect the locally generated private key. Additionally, Snowflake recommends storing the passphrase in a secure location. If using an encrypted key to connect to Snowflake, you will input the passphrase during the initial connection. The passphrase is only used for protecting the private key and will never be sent to Snowflake.

To generate a long and complex passphrase based on PCI DSS standards:

  1. Access the PCI Security Standards Document Library.

  2. For PCI DSS, select the most recent version and your desired language.

  3. Complete the form to access the document.

  4. Search for Passwords/passphrases must meet the following: and follow the recommendations for password/passphrase requirements, testing, and guidance. Depending on the document version, the phrase is likely located in a section called Requirement 8: Identify and authenticate access to system components (or similar name).

To start, open a terminal window and generate a private key.

You can generate either an encrypted version of the private key or an unencrypted version of the private key.

To generate an unencrypted version, use the following command:

$ openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8 -nocrypt

To generate an encrypted version, use the following command (which omits “-nocrypt”):

$ openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out rsa_key.p8

The commands generate a private key in PEM format.

-----BEGIN ENCRYPTED PRIVATE KEY-----
MIIE6TAbBgkqhkiG9w0BBQMwDgQILYPyCppzOwECAggABIIEyLiGSpeeGSe3xHP1
wHLjfCYycUPennlX2bd8yX8xOxGSGfvB+99+PmSlex0FmY9ov1J8H1H9Y3lMWXbL
...
-----END ENCRYPTED PRIVATE KEY-----

Step 2: Generate a Public Key

From the command line, generate the public key by referencing the private key. The following command assumes the private key is encrypted and contained in the file named rsa_key.p8.

$ openssl rsa -in rsa_key.p8 -pubout -out rsa_key.pub

The command generates the public key in PEM format.

-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAy+Fw2qv4Roud3l6tjPH4
zxybHjmZ5rhtCz9jppCV8UTWvEXxa88IGRIHbJ/PwKW/mR8LXdfI7l/9vCMXX4mk
...
-----END PUBLIC KEY-----

Step 3: Store the Private and Public Keys Securely

Copy the public and private key files to a local directory for storage. Record the path to the files. Note that the private key is stored using the PKCS#8 (Public Key Cryptography Standards) format and is encrypted using the passphrase you specified in the previous step.

However, the file should still be protected from unauthorized access using the file permission mechanism provided by your operating system. It is your responsibility to secure the file when it is not being used.

Step 4: Assign the Public Key to a Snowflake User

Execute an ALTER USER command to assign the public key to a Snowflake user.

alter user jsmith set rsa_public_key='MIIBIjANBgkqh...';

Note

  • Only security administrators (i.e. users with the SECURITYADMIN role) or higher can alter a user.

  • Exclude the public key delimiters in the SQL statement.

Step 5: Verify the User’s Public Key Fingerprint

Execute a DESCRIBE USER command to verify the user’s public key.

desc user jsmith;
+---------------------+-----------------------------------------------------+---------+----------------------------------------------+
| property            | value                                               | default | description                                  |
+---------------------+-----------------------------------------------------+---------+----------------------------------------------+
| NAME                | JSMITH                                              | null    | Name                                         |
...
...
| RSA_PUBLIC_KEY      | MIIE6TAbBgkqhkiG9w0BBQMwDgQILYPyCppzOwECAggABIIE... | null    | RSA public key of the user                   |
| RSA_PUBLIC_KEY_FP   | SHA256:nvnONUsfiuycCLMXIEWG4eTp4FjhVUZQUQbNpbSHXiA= | null    | Fingerprint of user's RSA public key.        |
| RSA_PUBLIC_KEY_2    | null                                                | null    | Second RSA public key of the user            |
| RSA_PUBLIC_KEY_2_FP | null                                                | null    | Fingerprint of user's second RSA public key. |
...
+---------------------+-----------------------------------------------------+---------+----------------------------------------------+

Snowfalek connection Using Named Connections

 

Using Named Connections

To make multiple simultaneous connections to Snowflake, or to simply store different sets of connection configurations, you can define one or more named connections.

Defining Named Connections in the Configuration File

  1. Open the config configuration file in a text editor. By default, the file is located in:

    Linux/macOS

    ~/.snowsql/

    Windows

    %USERPROFILE%\.snowsql\

  2. Add a separate [connections] section with a unique name for each named connection.

    For example, the following illustrates a connection named my_example_connection for a Snowflake account with the account identifier myorganization-myaccount:

    [connections.my_example_connection]
    accountname = myorganization-myaccount
    username = jsmith
    password = xxxxxxxxxxxxxxxxxxxx
    dbname = mydb
    schemaname = public
    warehousename = mywh
    

Connecting to Snowflake Using a Named Connection

Use the -c <string> (or --connection <string>) connection parameter to specify a named connection, where <string> is the name of a connection defined in the configuration file.

For example, connect using the my_example_connection connection you created in Defining Named Connections in the Configuration File (in this topic):

$ snowsql -c my_example_connection

Recent Post

Databricks Delta table merge Example

here's some sample code that demonstrates a merge operation on a Delta table using PySpark:   from pyspark.sql import SparkSession # cre...